2024-09-16T01:42:51.658 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-16T01:42:51.683 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T01:42:51.772 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906494 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/suites/iozone}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7906494' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 8921c8ef947807b717db041d85f0761b7914d5fc subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt dry_run: true install: ceph: flavor: default sha1: 8921c8ef947807b717db041d85f0761b7914d5fc kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: ad35db06e8924912f01e32b35576297c0f1352b4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 3393 sha1: 8921c8ef947807b717db041d85f0761b7914d5fc sleep_before_teardown: 0 subset: 294/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: ad35db06e8924912f01e32b35576297c0f1352b4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi105.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPi12DqVAkE28sj9haFNPVhJpLBd8uEcqeJp6dqysd5489OiNR4ZYVBruyKPKfOIOLopGi2ldRN9Z5zsCDy+/FE= smithi137.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK5MxQY6FCxbBdoR0AbWgC56FAkm0v/gT9LN9+gU4e+5DXntp+EgOMCydVAcO5zLFp+DEZ0p3dbYYx6uGHCrPvM= smithi190.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFtlh5iF+uvSMZjuoLoXO6jvi+VydbcazDWd1TvIvlIqDt2NMPTCKv3PDPeyvT/NL3WLhS/aRSChf6MshIWkR/w= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - workunit: clients: all: - suites/iozone.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-15_21:24:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-16T01:42:51.772 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa; will attempt to use it 2024-09-16T01:42:51.773 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks 2024-09-16T01:42:51.773 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-16T01:42:51.774 INFO:teuthology.task.internal:Checking packages... 2024-09-16T01:42:51.797 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8921c8ef947807b717db041d85f0761b7914d5fc' 2024-09-16T01:42:51.797 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T01:42:51.797 INFO:teuthology.packaging:ref: None 2024-09-16T01:42:51.797 INFO:teuthology.packaging:tag: None 2024-09-16T01:42:51.797 INFO:teuthology.packaging:branch: squid 2024-09-16T01:42:51.797 INFO:teuthology.packaging:sha1: 8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T01:42:51.798 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-16T01:42:52.020 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-328.gad35db06 2024-09-16T01:42:52.022 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-16T01:42:52.027 INFO:teuthology.task.internal:no buildpackages task found 2024-09-16T01:42:52.027 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-16T01:42:52.034 INFO:teuthology.task.internal:Saving configuration 2024-09-16T01:42:52.054 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-16T01:42:52.060 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-16T01:42:52.079 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi105.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906494', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 01:37:53.711290', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPi12DqVAkE28sj9haFNPVhJpLBd8uEcqeJp6dqysd5489OiNR4ZYVBruyKPKfOIOLopGi2ldRN9Z5zsCDy+/FE='} 2024-09-16T01:42:52.098 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi137.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906494', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 01:37:53.713037', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK5MxQY6FCxbBdoR0AbWgC56FAkm0v/gT9LN9+gU4e+5DXntp+EgOMCydVAcO5zLFp+DEZ0p3dbYYx6uGHCrPvM='} 2024-09-16T01:42:52.119 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi190.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906494', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 01:37:53.709291', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFtlh5iF+uvSMZjuoLoXO6jvi+VydbcazDWd1TvIvlIqDt2NMPTCKv3PDPeyvT/NL3WLhS/aRSChf6MshIWkR/w='} 2024-09-16T01:42:52.119 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-16T01:42:52.125 INFO:teuthology.task.internal:roles: ubuntu@smithi105.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-16T01:42:52.125 INFO:teuthology.task.internal:roles: ubuntu@smithi137.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-16T01:42:52.125 INFO:teuthology.task.internal:roles: ubuntu@smithi190.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-16T01:42:52.125 INFO:teuthology.run_tasks:Running task console_log... 2024-09-16T01:42:52.196 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f743bc83be0>, signals=[15]) 2024-09-16T01:42:52.197 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-16T01:42:52.202 INFO:teuthology.task.internal:Opening connections... 2024-09-16T01:42:52.202 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi105.front.sepia.ceph.com 2024-09-16T01:42:52.204 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T01:42:52.286 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi137.front.sepia.ceph.com 2024-09-16T01:42:52.287 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T01:42:52.362 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi190.front.sepia.ceph.com 2024-09-16T01:42:52.363 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T01:42:52.437 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-16T01:42:52.442 DEBUG:teuthology.orchestra.run.smithi105:> uname -m 2024-09-16T01:42:52.461 INFO:teuthology.orchestra.run.smithi105.stdout:x86_64 2024-09-16T01:42:52.462 DEBUG:teuthology.orchestra.run.smithi105:> cat /etc/os-release 2024-09-16T01:42:52.520 INFO:teuthology.orchestra.run.smithi105.stdout:NAME="CentOS Stream" 2024-09-16T01:42:52.520 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION="9" 2024-09-16T01:42:52.520 INFO:teuthology.orchestra.run.smithi105.stdout:ID="centos" 2024-09-16T01:42:52.520 INFO:teuthology.orchestra.run.smithi105.stdout:ID_LIKE="rhel fedora" 2024-09-16T01:42:52.521 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION_ID="9" 2024-09-16T01:42:52.521 INFO:teuthology.orchestra.run.smithi105.stdout:PLATFORM_ID="platform:el9" 2024-09-16T01:42:52.521 INFO:teuthology.orchestra.run.smithi105.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T01:42:52.521 INFO:teuthology.orchestra.run.smithi105.stdout:ANSI_COLOR="0;31" 2024-09-16T01:42:52.521 INFO:teuthology.orchestra.run.smithi105.stdout:LOGO="fedora-logo-icon" 2024-09-16T01:42:52.521 INFO:teuthology.orchestra.run.smithi105.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T01:42:52.521 INFO:teuthology.orchestra.run.smithi105.stdout:HOME_URL="https://centos.org/" 2024-09-16T01:42:52.521 INFO:teuthology.orchestra.run.smithi105.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T01:42:52.521 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T01:42:52.521 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T01:42:52.522 INFO:teuthology.lock.ops:Updating smithi105.front.sepia.ceph.com on lock server 2024-09-16T01:42:52.540 DEBUG:teuthology.orchestra.run.smithi137:> uname -m 2024-09-16T01:42:52.560 INFO:teuthology.orchestra.run.smithi137.stdout:x86_64 2024-09-16T01:42:52.560 DEBUG:teuthology.orchestra.run.smithi137:> cat /etc/os-release 2024-09-16T01:42:52.617 INFO:teuthology.orchestra.run.smithi137.stdout:NAME="CentOS Stream" 2024-09-16T01:42:52.617 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION="9" 2024-09-16T01:42:52.617 INFO:teuthology.orchestra.run.smithi137.stdout:ID="centos" 2024-09-16T01:42:52.617 INFO:teuthology.orchestra.run.smithi137.stdout:ID_LIKE="rhel fedora" 2024-09-16T01:42:52.617 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION_ID="9" 2024-09-16T01:42:52.617 INFO:teuthology.orchestra.run.smithi137.stdout:PLATFORM_ID="platform:el9" 2024-09-16T01:42:52.617 INFO:teuthology.orchestra.run.smithi137.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T01:42:52.617 INFO:teuthology.orchestra.run.smithi137.stdout:ANSI_COLOR="0;31" 2024-09-16T01:42:52.617 INFO:teuthology.orchestra.run.smithi137.stdout:LOGO="fedora-logo-icon" 2024-09-16T01:42:52.617 INFO:teuthology.orchestra.run.smithi137.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T01:42:52.617 INFO:teuthology.orchestra.run.smithi137.stdout:HOME_URL="https://centos.org/" 2024-09-16T01:42:52.618 INFO:teuthology.orchestra.run.smithi137.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T01:42:52.618 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T01:42:52.618 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T01:42:52.618 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server 2024-09-16T01:42:52.632 DEBUG:teuthology.orchestra.run.smithi190:> uname -m 2024-09-16T01:42:52.650 INFO:teuthology.orchestra.run.smithi190.stdout:x86_64 2024-09-16T01:42:52.651 DEBUG:teuthology.orchestra.run.smithi190:> cat /etc/os-release 2024-09-16T01:42:52.708 INFO:teuthology.orchestra.run.smithi190.stdout:NAME="CentOS Stream" 2024-09-16T01:42:52.708 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION="9" 2024-09-16T01:42:52.708 INFO:teuthology.orchestra.run.smithi190.stdout:ID="centos" 2024-09-16T01:42:52.708 INFO:teuthology.orchestra.run.smithi190.stdout:ID_LIKE="rhel fedora" 2024-09-16T01:42:52.708 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION_ID="9" 2024-09-16T01:42:52.709 INFO:teuthology.orchestra.run.smithi190.stdout:PLATFORM_ID="platform:el9" 2024-09-16T01:42:52.709 INFO:teuthology.orchestra.run.smithi190.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T01:42:52.709 INFO:teuthology.orchestra.run.smithi190.stdout:ANSI_COLOR="0;31" 2024-09-16T01:42:52.709 INFO:teuthology.orchestra.run.smithi190.stdout:LOGO="fedora-logo-icon" 2024-09-16T01:42:52.709 INFO:teuthology.orchestra.run.smithi190.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T01:42:52.709 INFO:teuthology.orchestra.run.smithi190.stdout:HOME_URL="https://centos.org/" 2024-09-16T01:42:52.709 INFO:teuthology.orchestra.run.smithi190.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T01:42:52.709 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T01:42:52.709 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T01:42:52.710 INFO:teuthology.lock.ops:Updating smithi190.front.sepia.ceph.com on lock server 2024-09-16T01:42:52.731 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-16T01:42:52.745 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-16T01:42:52.750 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-16T01:42:52.750 DEBUG:teuthology.orchestra.run.smithi105:> test '!' -e /home/ubuntu/cephtest 2024-09-16T01:42:52.753 DEBUG:teuthology.orchestra.run.smithi137:> test '!' -e /home/ubuntu/cephtest 2024-09-16T01:42:52.756 DEBUG:teuthology.orchestra.run.smithi190:> test '!' -e /home/ubuntu/cephtest 2024-09-16T01:42:52.772 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-16T01:42:52.780 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-16T01:42:52.780 DEBUG:teuthology.orchestra.run.smithi105:> test -z $(ls -A /var/lib/ceph) 2024-09-16T01:42:52.811 DEBUG:teuthology.orchestra.run.smithi137:> test -z $(ls -A /var/lib/ceph) 2024-09-16T01:42:52.813 DEBUG:teuthology.orchestra.run.smithi190:> test -z $(ls -A /var/lib/ceph) 2024-09-16T01:42:52.922 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-16T01:42:52.985 INFO:teuthology.run_tasks:Running task kernel... 2024-09-16T01:42:53.000 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-16T01:42:53.000 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-16T01:42:53.000 DEBUG:teuthology.orchestra.run.smithi105:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T01:42:53.001 DEBUG:teuthology.orchestra.run.smithi137:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T01:42:53.001 DEBUG:teuthology.orchestra.run.smithi190:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T01:42:53.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T01:42:53.018 DEBUG:teuthology.orchestra.run.smithi137:> uname -r 2024-09-16T01:42:53.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T01:42:53.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T01:42:53.020 DEBUG:teuthology.orchestra.run.smithi105:> uname -r 2024-09-16T01:42:53.021 DEBUG:teuthology.orchestra.run.smithi190:> uname -r 2024-09-16T01:42:53.076 INFO:teuthology.orchestra.run.smithi137.stdout:5.14.0-503.el9.x86_64 2024-09-16T01:42:53.077 INFO:teuthology.task.kernel:Running kernel on smithi137: 5.14.0-503.el9.x86_64 2024-09-16T01:42:53.077 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum install -y kernel 2024-09-16T01:42:53.080 INFO:teuthology.orchestra.run.smithi190.stdout:5.14.0-503.el9.x86_64 2024-09-16T01:42:53.080 INFO:teuthology.orchestra.run.smithi105.stdout:5.14.0-503.el9.x86_64 2024-09-16T01:42:53.080 INFO:teuthology.task.kernel:Running kernel on smithi190: 5.14.0-503.el9.x86_64 2024-09-16T01:42:53.080 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum install -y kernel 2024-09-16T01:42:53.081 INFO:teuthology.task.kernel:Running kernel on smithi105: 5.14.0-503.el9.x86_64 2024-09-16T01:42:53.081 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum install -y kernel 2024-09-16T01:42:55.443 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-09-16T01:42:55.579 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.2 MB 00:00 2024-09-16T01:42:56.126 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - BaseOS 6.3 MB/s | 8.2 MB 00:01 2024-09-16T01:42:57.664 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - AppStream 21 MB/s | 20 MB 00:00 2024-09-16T01:42:57.926 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - AppStream 30 MB/s | 20 MB 00:00 2024-09-16T01:42:59.327 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - AppStream 7.4 MB/s | 20 MB 00:02 2024-09-16T01:43:02.215 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - CRB 8.5 MB/s | 6.5 MB 00:00 2024-09-16T01:43:02.660 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - CRB 7.4 MB/s | 6.5 MB 00:00 2024-09-16T01:43:03.598 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - Extras packages 118 kB/s | 19 kB 00:00 2024-09-16T01:43:03.761 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-16T01:43:04.169 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - Extras packages 100 kB/s | 19 kB 00:00 2024-09-16T01:43:04.627 INFO:teuthology.orchestra.run.smithi137.stdout:Extra Packages for Enterprise Linux 25 MB/s | 23 MB 00:00 2024-09-16T01:43:05.270 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - Extras packages 58 kB/s | 19 kB 00:00 2024-09-16T01:43:05.332 INFO:teuthology.orchestra.run.smithi190.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-16T01:43:06.600 INFO:teuthology.orchestra.run.smithi105.stdout:Extra Packages for Enterprise Linux 19 MB/s | 23 MB 00:01 2024-09-16T01:43:10.116 INFO:teuthology.orchestra.run.smithi137.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-09-16T01:43:10.821 INFO:teuthology.orchestra.run.smithi190.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-16T01:43:12.014 INFO:teuthology.orchestra.run.smithi105.stdout:lab-extras 42 kB/s | 1.7 kB 00:00 2024-09-16T01:43:12.106 INFO:teuthology.orchestra.run.smithi137.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-16T01:43:12.106 INFO:teuthology.orchestra.run.smithi137.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-16T01:43:12.193 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-09-16T01:43:12.198 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-09-16T01:43:12.198 INFO:teuthology.orchestra.run.smithi137.stdout: Package Architecture Version Repository Size 2024-09-16T01:43:12.198 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-09-16T01:43:12.198 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-09-16T01:43:12.198 INFO:teuthology.orchestra.run.smithi137.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-16T01:43:12.199 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-09-16T01:43:12.199 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-16T01:43:12.199 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-16T01:43:12.199 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-16T01:43:12.199 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-16T01:43:12.199 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-09-16T01:43:12.199 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-09-16T01:43:12.199 INFO:teuthology.orchestra.run.smithi137.stdout:Install 4 Packages 2024-09-16T01:43:12.200 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-16T01:43:12.200 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 79 M 2024-09-16T01:43:12.200 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 126 M 2024-09-16T01:43:12.200 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-09-16T01:43:12.862 INFO:teuthology.orchestra.run.smithi190.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-16T01:43:12.862 INFO:teuthology.orchestra.run.smithi190.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-16T01:43:12.951 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-09-16T01:43:12.963 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-09-16T01:43:12.963 INFO:teuthology.orchestra.run.smithi190.stdout: Package Architecture Version Repository Size 2024-09-16T01:43:12.963 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-09-16T01:43:12.964 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-09-16T01:43:12.964 INFO:teuthology.orchestra.run.smithi190.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-16T01:43:12.964 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2024-09-16T01:43:12.964 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-16T01:43:12.964 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-16T01:43:12.964 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-16T01:43:12.964 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T01:43:12.964 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-09-16T01:43:12.964 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-09-16T01:43:12.965 INFO:teuthology.orchestra.run.smithi190.stdout:Install 4 Packages 2024-09-16T01:43:12.965 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T01:43:12.965 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 79 M 2024-09-16T01:43:12.965 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 126 M 2024-09-16T01:43:12.965 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-09-16T01:43:13.117 INFO:teuthology.orchestra.run.smithi137.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 39 kB/s | 23 kB 00:00 2024-09-16T01:43:13.272 INFO:teuthology.orchestra.run.smithi190.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 211 kB/s | 23 kB 00:00 2024-09-16T01:43:13.926 INFO:teuthology.orchestra.run.smithi137.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-16T01:43:13.986 INFO:teuthology.orchestra.run.smithi105.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-16T01:43:13.986 INFO:teuthology.orchestra.run.smithi105.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-16T01:43:14.090 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2024-09-16T01:43:14.097 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2024-09-16T01:43:14.097 INFO:teuthology.orchestra.run.smithi105.stdout: Package Architecture Version Repository Size 2024-09-16T01:43:14.097 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2024-09-16T01:43:14.097 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2024-09-16T01:43:14.097 INFO:teuthology.orchestra.run.smithi105.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-16T01:43:14.097 INFO:teuthology.orchestra.run.smithi105.stdout:Installing dependencies: 2024-09-16T01:43:14.098 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-16T01:43:14.098 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-16T01:43:14.098 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-16T01:43:14.098 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:43:14.098 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2024-09-16T01:43:14.098 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2024-09-16T01:43:14.098 INFO:teuthology.orchestra.run.smithi105.stdout:Install 4 Packages 2024-09-16T01:43:14.098 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:43:14.099 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 79 M 2024-09-16T01:43:14.099 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 126 M 2024-09-16T01:43:14.099 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2024-09-16T01:43:14.307 INFO:teuthology.orchestra.run.smithi105.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 256 kB/s | 23 kB 00:00 2024-09-16T01:43:14.823 INFO:teuthology.orchestra.run.smithi105.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-09-16T01:43:15.068 INFO:teuthology.orchestra.run.smithi137.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 14 MB/s | 35 MB 00:02 2024-09-16T01:43:15.232 INFO:teuthology.orchestra.run.smithi105.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 34 MB/s | 35 MB 00:01 2024-09-16T01:43:15.491 INFO:teuthology.orchestra.run.smithi105.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 24 MB/s | 29 MB 00:01 2024-09-16T01:43:15.491 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2024-09-16T01:43:15.491 INFO:teuthology.orchestra.run.smithi105.stdout:Total 57 MB/s | 79 MB 00:01 2024-09-16T01:43:15.631 INFO:teuthology.orchestra.run.smithi190.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 6.3 MB/s | 16 MB 00:02 2024-09-16T01:43:15.651 INFO:teuthology.orchestra.run.smithi137.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 11 MB/s | 29 MB 00:02 2024-09-16T01:43:15.651 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-09-16T01:43:15.652 INFO:teuthology.orchestra.run.smithi137.stdout:Total 23 MB/s | 79 MB 00:03 2024-09-16T01:43:16.175 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2024-09-16T01:43:16.334 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-09-16T01:43:16.532 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2024-09-16T01:43:16.532 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2024-09-16T01:43:16.682 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-09-16T01:43:16.682 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-09-16T01:43:17.499 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2024-09-16T01:43:17.500 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2024-09-16T01:43:17.514 INFO:teuthology.orchestra.run.smithi190.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 6.7 MB/s | 29 MB 00:04 2024-09-16T01:43:17.664 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-09-16T01:43:17.665 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-09-16T01:43:19.930 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2024-09-16T01:43:20.098 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-09-16T01:43:20.626 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-16T01:43:20.716 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T01:43:20.918 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-16T01:43:21.010 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T01:43:23.138 INFO:teuthology.orchestra.run.smithi190.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 3.5 MB/s | 35 MB 00:09 2024-09-16T01:43:23.139 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-09-16T01:43:23.139 INFO:teuthology.orchestra.run.smithi190.stdout:Total 7.7 MB/s | 79 MB 00:10 2024-09-16T01:43:23.512 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T01:43:23.645 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T01:43:23.791 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T01:43:23.824 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-09-16T01:43:23.920 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T01:43:24.173 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-09-16T01:43:24.174 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-09-16T01:43:25.136 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-09-16T01:43:25.137 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-09-16T01:43:27.597 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-09-16T01:43:28.447 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T01:43:28.521 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-16T01:43:28.561 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T01:43:28.613 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T01:43:28.764 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T01:43:28.870 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T01:43:31.385 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T01:43:31.516 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T01:43:33.390 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T01:43:33.710 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T01:43:36.335 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T01:43:36.375 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T01:43:41.200 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T01:44:11.443 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T01:44:11.482 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-16T01:44:11.773 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T01:44:11.813 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-16T01:44:11.864 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T01:44:11.864 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-16T01:44:11.864 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T01:44:11.864 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T01:44:12.156 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T01:44:12.156 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-16T01:44:12.156 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T01:44:12.156 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T01:44:12.723 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T01:44:12.723 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:44:12.723 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2024-09-16T01:44:12.723 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-16T01:44:12.723 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-16T01:44:12.723 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-16T01:44:12.723 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-16T01:44:12.723 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:44:12.723 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2024-09-16T01:44:13.018 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T01:44:13.018 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-16T01:44:13.018 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-09-16T01:44:13.018 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-16T01:44:13.018 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-16T01:44:13.018 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-16T01:44:13.018 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-16T01:44:13.018 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-16T01:44:13.018 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-09-16T01:44:13.096 DEBUG:teuthology.orchestra.run.smithi105:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T01:44:13.185 INFO:teuthology.orchestra.run.smithi105.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T01:44:13.186 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T01:44:13.186 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-16T01:44:13.186 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T01:44:13.186 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T01:44:13.186 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-16T01:44:13.186 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi105.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T01:44:13.186 DEBUG:teuthology.orchestra.run.smithi105:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T01:44:13.244 INFO:teuthology.orchestra.run.smithi105.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T01:44:13.244 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T01:44:13.244 DEBUG:teuthology.orchestra.run.smithi105:> sudo rpm -qi grub2-tools 2024-09-16T01:44:13.284 INFO:teuthology.orchestra.run.smithi105.stdout:Name : grub2-tools 2024-09-16T01:44:13.284 INFO:teuthology.orchestra.run.smithi105.stdout:Epoch : 1 2024-09-16T01:44:13.285 INFO:teuthology.orchestra.run.smithi105.stdout:Version : 2.06 2024-09-16T01:44:13.285 INFO:teuthology.orchestra.run.smithi105.stdout:Release : 82.el9 2024-09-16T01:44:13.285 INFO:teuthology.orchestra.run.smithi105.stdout:Architecture: x86_64 2024-09-16T01:44:13.285 INFO:teuthology.orchestra.run.smithi105.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T01:44:13.285 INFO:teuthology.orchestra.run.smithi105.stdout:Group : Unspecified 2024-09-16T01:44:13.285 INFO:teuthology.orchestra.run.smithi105.stdout:Size : 8274670 2024-09-16T01:44:13.285 INFO:teuthology.orchestra.run.smithi105.stdout:License : GPLv3+ 2024-09-16T01:44:13.285 INFO:teuthology.orchestra.run.smithi105.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T01:44:13.285 INFO:teuthology.orchestra.run.smithi105.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T01:44:13.285 INFO:teuthology.orchestra.run.smithi105.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T01:44:13.285 INFO:teuthology.orchestra.run.smithi105.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T01:44:13.285 INFO:teuthology.orchestra.run.smithi105.stdout:Packager : builder@centos.org 2024-09-16T01:44:13.285 INFO:teuthology.orchestra.run.smithi105.stdout:Vendor : CentOS 2024-09-16T01:44:13.285 INFO:teuthology.orchestra.run.smithi105.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T01:44:13.286 INFO:teuthology.orchestra.run.smithi105.stdout:Summary : Support tools for GRUB. 2024-09-16T01:44:13.286 INFO:teuthology.orchestra.run.smithi105.stdout:Description : 2024-09-16T01:44:13.286 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:44:13.286 INFO:teuthology.orchestra.run.smithi105.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T01:44:13.286 INFO:teuthology.orchestra.run.smithi105.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T01:44:13.286 INFO:teuthology.orchestra.run.smithi105.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T01:44:13.286 INFO:teuthology.orchestra.run.smithi105.stdout:hardware devices. 2024-09-16T01:44:13.286 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:44:13.286 INFO:teuthology.orchestra.run.smithi105.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T01:44:13.287 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T01:44:13.287 INFO:teuthology.task.kernel:Updating grub on smithi105 to boot 5.14.0-505.el9.x86_64 2024-09-16T01:44:13.287 DEBUG:teuthology.orchestra.run.smithi105:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T01:44:13.385 DEBUG:teuthology.orchestra.run.smithi137:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T01:44:13.465 INFO:teuthology.orchestra.run.smithi137.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T01:44:13.466 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T01:44:13.466 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-16T01:44:13.466 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T01:44:13.466 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T01:44:13.466 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-16T01:44:13.466 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi137.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T01:44:13.466 DEBUG:teuthology.orchestra.run.smithi137:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T01:44:13.525 INFO:teuthology.orchestra.run.smithi137.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T01:44:13.525 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T01:44:13.525 DEBUG:teuthology.orchestra.run.smithi137:> sudo rpm -qi grub2-tools 2024-09-16T01:44:13.557 INFO:teuthology.orchestra.run.smithi137.stdout:Name : grub2-tools 2024-09-16T01:44:13.557 INFO:teuthology.orchestra.run.smithi137.stdout:Epoch : 1 2024-09-16T01:44:13.558 INFO:teuthology.orchestra.run.smithi137.stdout:Version : 2.06 2024-09-16T01:44:13.558 INFO:teuthology.orchestra.run.smithi137.stdout:Release : 82.el9 2024-09-16T01:44:13.558 INFO:teuthology.orchestra.run.smithi137.stdout:Architecture: x86_64 2024-09-16T01:44:13.558 INFO:teuthology.orchestra.run.smithi137.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T01:44:13.558 INFO:teuthology.orchestra.run.smithi137.stdout:Group : Unspecified 2024-09-16T01:44:13.558 INFO:teuthology.orchestra.run.smithi137.stdout:Size : 8274670 2024-09-16T01:44:13.558 INFO:teuthology.orchestra.run.smithi137.stdout:License : GPLv3+ 2024-09-16T01:44:13.558 INFO:teuthology.orchestra.run.smithi137.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T01:44:13.558 INFO:teuthology.orchestra.run.smithi137.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T01:44:13.558 INFO:teuthology.orchestra.run.smithi137.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T01:44:13.558 INFO:teuthology.orchestra.run.smithi137.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T01:44:13.558 INFO:teuthology.orchestra.run.smithi137.stdout:Packager : builder@centos.org 2024-09-16T01:44:13.558 INFO:teuthology.orchestra.run.smithi137.stdout:Vendor : CentOS 2024-09-16T01:44:13.558 INFO:teuthology.orchestra.run.smithi137.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T01:44:13.559 INFO:teuthology.orchestra.run.smithi137.stdout:Summary : Support tools for GRUB. 2024-09-16T01:44:13.559 INFO:teuthology.orchestra.run.smithi137.stdout:Description : 2024-09-16T01:44:13.559 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-16T01:44:13.559 INFO:teuthology.orchestra.run.smithi137.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T01:44:13.559 INFO:teuthology.orchestra.run.smithi137.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T01:44:13.559 INFO:teuthology.orchestra.run.smithi137.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T01:44:13.559 INFO:teuthology.orchestra.run.smithi137.stdout:hardware devices. 2024-09-16T01:44:13.559 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-16T01:44:13.559 INFO:teuthology.orchestra.run.smithi137.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T01:44:13.560 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T01:44:13.560 INFO:teuthology.task.kernel:Updating grub on smithi137 to boot 5.14.0-505.el9.x86_64 2024-09-16T01:44:13.560 DEBUG:teuthology.orchestra.run.smithi137:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T01:44:13.904 INFO:teuthology.orchestra.run.smithi105.stderr:Generating grub configuration file ... 2024-09-16T01:44:14.231 INFO:teuthology.orchestra.run.smithi137.stderr:Generating grub configuration file ... 2024-09-16T01:44:15.245 INFO:teuthology.orchestra.run.smithi105.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T01:44:15.279 INFO:teuthology.orchestra.run.smithi105.stderr:done 2024-09-16T01:44:15.281 DEBUG:teuthology.orchestra.run.smithi105:> mktemp 2024-09-16T01:44:15.297 INFO:teuthology.orchestra.run.smithi105.stdout:/tmp/tmp.URinrNeRS3 2024-09-16T01:44:15.297 DEBUG:teuthology.orchestra.run.smithi105:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.URinrNeRS3 2024-09-16T01:44:15.364 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0666 /tmp/tmp.URinrNeRS3 2024-09-16T01:44:15.500 DEBUG:teuthology.orchestra.remote:smithi105:/tmp/tmp.URinrNeRS3 is 6KB 2024-09-16T01:44:15.549 DEBUG:teuthology.orchestra.run.smithi105:> rm -fr /tmp/tmp.URinrNeRS3 2024-09-16T01:44:15.564 DEBUG:teuthology.orchestra.run.smithi105:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T01:44:15.626 INFO:teuthology.orchestra.run.smithi105.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T01:44:15.626 INFO:teuthology.orchestra.run.smithi105.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T01:44:15.627 INFO:teuthology.orchestra.run.smithi105.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T01:44:15.627 INFO:teuthology.orchestra.run.smithi105.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-16T01:44:15.628 DEBUG:teuthology.orchestra.run.smithi105:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-16T01:44:15.686 INFO:teuthology.orchestra.run.smithi137.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T01:44:15.703 INFO:teuthology.orchestra.run.smithi137.stderr:done 2024-09-16T01:44:15.705 DEBUG:teuthology.orchestra.run.smithi137:> mktemp 2024-09-16T01:44:15.721 INFO:teuthology.orchestra.run.smithi137.stdout:/tmp/tmp.PYJfVq9phi 2024-09-16T01:44:15.721 DEBUG:teuthology.orchestra.run.smithi137:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.PYJfVq9phi 2024-09-16T01:44:15.788 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 0666 /tmp/tmp.PYJfVq9phi 2024-09-16T01:44:15.797 DEBUG:teuthology.orchestra.run.smithi105:> sudo shutdown -r now 2024-09-16T01:44:15.936 DEBUG:teuthology.orchestra.remote:smithi137:/tmp/tmp.PYJfVq9phi is 6KB 2024-09-16T01:44:15.984 DEBUG:teuthology.orchestra.run.smithi137:> rm -fr /tmp/tmp.PYJfVq9phi 2024-09-16T01:44:15.999 DEBUG:teuthology.orchestra.run.smithi137:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T01:44:16.061 INFO:teuthology.orchestra.run.smithi137.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T01:44:16.062 INFO:teuthology.orchestra.run.smithi137.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T01:44:16.062 INFO:teuthology.orchestra.run.smithi137.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T01:44:16.062 INFO:teuthology.orchestra.run.smithi137.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-16T01:44:16.063 DEBUG:teuthology.orchestra.run.smithi137:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-16T01:44:16.213 DEBUG:teuthology.orchestra.run.smithi137:> sudo shutdown -r now 2024-09-16T01:44:19.465 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T01:44:19.504 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-16T01:44:19.816 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T01:44:19.816 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-16T01:44:19.816 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T01:44:19.816 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T01:44:20.666 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T01:44:20.666 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T01:44:20.666 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-09-16T01:44:20.666 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-16T01:44:20.667 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-16T01:44:20.667 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-16T01:44:20.667 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-16T01:44:20.667 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T01:44:20.667 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-09-16T01:44:20.990 DEBUG:teuthology.orchestra.run.smithi190:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T01:44:21.079 INFO:teuthology.orchestra.run.smithi190.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T01:44:21.079 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T01:44:21.079 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-16T01:44:21.080 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T01:44:21.080 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T01:44:21.080 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-09-16T01:44:21.080 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi190.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T01:44:21.080 DEBUG:teuthology.orchestra.run.smithi190:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T01:44:21.138 INFO:teuthology.orchestra.run.smithi190.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T01:44:21.139 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T01:44:21.139 DEBUG:teuthology.orchestra.run.smithi190:> sudo rpm -qi grub2-tools 2024-09-16T01:44:21.170 INFO:teuthology.orchestra.run.smithi190.stdout:Name : grub2-tools 2024-09-16T01:44:21.171 INFO:teuthology.orchestra.run.smithi190.stdout:Epoch : 1 2024-09-16T01:44:21.171 INFO:teuthology.orchestra.run.smithi190.stdout:Version : 2.06 2024-09-16T01:44:21.171 INFO:teuthology.orchestra.run.smithi190.stdout:Release : 82.el9 2024-09-16T01:44:21.171 INFO:teuthology.orchestra.run.smithi190.stdout:Architecture: x86_64 2024-09-16T01:44:21.171 INFO:teuthology.orchestra.run.smithi190.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T01:44:21.171 INFO:teuthology.orchestra.run.smithi190.stdout:Group : Unspecified 2024-09-16T01:44:21.171 INFO:teuthology.orchestra.run.smithi190.stdout:Size : 8274670 2024-09-16T01:44:21.171 INFO:teuthology.orchestra.run.smithi190.stdout:License : GPLv3+ 2024-09-16T01:44:21.171 INFO:teuthology.orchestra.run.smithi190.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T01:44:21.171 INFO:teuthology.orchestra.run.smithi190.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T01:44:21.171 INFO:teuthology.orchestra.run.smithi190.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T01:44:21.171 INFO:teuthology.orchestra.run.smithi190.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T01:44:21.171 INFO:teuthology.orchestra.run.smithi190.stdout:Packager : builder@centos.org 2024-09-16T01:44:21.171 INFO:teuthology.orchestra.run.smithi190.stdout:Vendor : CentOS 2024-09-16T01:44:21.172 INFO:teuthology.orchestra.run.smithi190.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T01:44:21.172 INFO:teuthology.orchestra.run.smithi190.stdout:Summary : Support tools for GRUB. 2024-09-16T01:44:21.172 INFO:teuthology.orchestra.run.smithi190.stdout:Description : 2024-09-16T01:44:21.172 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T01:44:21.172 INFO:teuthology.orchestra.run.smithi190.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T01:44:21.172 INFO:teuthology.orchestra.run.smithi190.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T01:44:21.172 INFO:teuthology.orchestra.run.smithi190.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T01:44:21.172 INFO:teuthology.orchestra.run.smithi190.stdout:hardware devices. 2024-09-16T01:44:21.172 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T01:44:21.172 INFO:teuthology.orchestra.run.smithi190.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T01:44:21.173 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T01:44:21.173 INFO:teuthology.task.kernel:Updating grub on smithi190 to boot 5.14.0-505.el9.x86_64 2024-09-16T01:44:21.173 DEBUG:teuthology.orchestra.run.smithi190:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T01:44:21.809 INFO:teuthology.orchestra.run.smithi190.stderr:Generating grub configuration file ... 2024-09-16T01:44:23.192 INFO:teuthology.orchestra.run.smithi190.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T01:44:23.220 INFO:teuthology.orchestra.run.smithi190.stderr:done 2024-09-16T01:44:23.223 DEBUG:teuthology.orchestra.run.smithi190:> mktemp 2024-09-16T01:44:23.238 INFO:teuthology.orchestra.run.smithi190.stdout:/tmp/tmp.jIoVn9SOCa 2024-09-16T01:44:23.239 DEBUG:teuthology.orchestra.run.smithi190:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.jIoVn9SOCa 2024-09-16T01:44:23.311 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0666 /tmp/tmp.jIoVn9SOCa 2024-09-16T01:44:23.472 DEBUG:teuthology.orchestra.remote:smithi190:/tmp/tmp.jIoVn9SOCa is 6KB 2024-09-16T01:44:23.501 DEBUG:teuthology.orchestra.run.smithi190:> rm -fr /tmp/tmp.jIoVn9SOCa 2024-09-16T01:44:23.525 DEBUG:teuthology.orchestra.run.smithi190:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T01:44:23.620 INFO:teuthology.orchestra.run.smithi190.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T01:44:23.621 INFO:teuthology.orchestra.run.smithi190.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T01:44:23.621 INFO:teuthology.orchestra.run.smithi190.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T01:44:23.621 INFO:teuthology.orchestra.run.smithi190.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-16T01:44:23.622 DEBUG:teuthology.orchestra.run.smithi190:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-16T01:44:23.782 DEBUG:teuthology.orchestra.run.smithi190:> sudo shutdown -r now 2024-09-16T01:44:45.821 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-16T01:44:45.822 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-09-16T01:44:45.823 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T01:44:46.217 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-16T01:44:46.217 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-09-16T01:44:46.218 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T01:44:53.792 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-09-16T01:44:53.793 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-09-16T01:44:53.794 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T01:45:45.861 DEBUG:teuthology.orchestra.remote:timed out 2024-09-16T01:45:46.221 DEBUG:teuthology.orchestra.remote:timed out 2024-09-16T01:45:53.803 DEBUG:teuthology.orchestra.remote:timed out 2024-09-16T01:45:54.863 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-09-16T01:45:54.864 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T01:45:55.221 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-09-16T01:45:55.222 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T01:45:55.257 DEBUG:teuthology.orchestra.run.smithi105:> true 2024-09-16T01:45:55.622 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-09-16T01:45:55.622 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-16T01:45:55.623 DEBUG:teuthology.orchestra.run.smithi105:> uname -r 2024-09-16T01:45:55.637 INFO:teuthology.orchestra.run.smithi105.stdout:5.14.0-505.el9.x86_64 2024-09-16T01:45:55.637 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-16T01:45:55.638 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T01:45:55.638 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T01:45:55.765 DEBUG:teuthology.orchestra.run.smithi137:> true 2024-09-16T01:45:56.127 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-09-16T01:45:56.127 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-16T01:45:56.127 DEBUG:teuthology.orchestra.run.smithi137:> uname -r 2024-09-16T01:45:56.142 INFO:teuthology.orchestra.run.smithi137.stdout:5.14.0-505.el9.x86_64 2024-09-16T01:45:56.143 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-16T01:45:56.143 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T01:45:56.143 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T01:45:56.639 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-16T01:45:56.639 DEBUG:teuthology.orchestra.run.smithi105:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T01:45:56.716 INFO:teuthology.orchestra.run.smithi105.stdout:ttyS1 2024-09-16T01:45:56.757 DEBUG:teuthology.parallel:result is None 2024-09-16T01:45:57.144 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-16T01:45:57.144 DEBUG:teuthology.orchestra.run.smithi137:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T01:45:57.221 INFO:teuthology.orchestra.run.smithi137.stdout:ttyS1 2024-09-16T01:45:57.247 DEBUG:teuthology.parallel:result is None 2024-09-16T01:46:02.804 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-09-16T01:46:02.805 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T01:46:03.177 DEBUG:teuthology.orchestra.run.smithi190:> true 2024-09-16T01:46:03.524 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-09-16T01:46:03.524 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-505.el9.x86_64"... 2024-09-16T01:46:03.524 DEBUG:teuthology.orchestra.run.smithi190:> uname -r 2024-09-16T01:46:03.538 INFO:teuthology.orchestra.run.smithi190.stdout:5.14.0-505.el9.x86_64 2024-09-16T01:46:03.539 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-16T01:46:03.539 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T01:46:03.539 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T01:46:04.540 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-09-16T01:46:04.540 DEBUG:teuthology.orchestra.run.smithi190:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T01:46:04.633 INFO:teuthology.orchestra.run.smithi190.stdout:ttyS1 2024-09-16T01:46:04.653 DEBUG:teuthology.parallel:result is None 2024-09-16T01:46:04.654 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-16T01:46:04.663 INFO:teuthology.task.internal:Creating test directory... 2024-09-16T01:46:04.663 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T01:46:04.667 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T01:46:04.670 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T01:46:04.687 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-16T01:46:04.694 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-16T01:46:04.706 INFO:teuthology.task.internal:Creating archive directory... 2024-09-16T01:46:04.706 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T01:46:04.727 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T01:46:04.729 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T01:46:05.023 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-16T01:46:05.032 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-16T01:46:05.033 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T01:46:05.035 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T01:46:05.038 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T01:46:05.062 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T01:46:05.064 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T01:46:05.067 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T01:46:05.072 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T01:46:05.076 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T01:46:05.078 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T01:46:05.080 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-16T01:46:05.134 INFO:teuthology.task.internal:Configuring sudo... 2024-09-16T01:46:05.134 DEBUG:teuthology.orchestra.run.smithi105:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T01:46:05.137 DEBUG:teuthology.orchestra.run.smithi137:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T01:46:05.139 DEBUG:teuthology.orchestra.run.smithi190:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T01:46:05.164 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-16T01:46:05.190 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-16T01:46:05.190 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T01:46:05.205 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T01:46:05.207 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T01:46:05.222 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T01:46:05.300 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T01:46:05.365 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T01:46:05.401 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T01:46:05.449 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-16T01:46:05.449 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T01:46:05.515 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T01:46:05.556 DEBUG:teuthology.orchestra.run.smithi137:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T01:46:05.614 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T01:46:05.693 DEBUG:teuthology.orchestra.run.smithi137:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T01:46:05.739 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-16T01:46:05.739 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T01:46:05.805 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T01:46:05.844 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T01:46:05.896 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T01:46:05.968 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T01:46:06.013 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-16T01:46:06.013 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T01:46:06.074 DEBUG:teuthology.orchestra.run.smithi105:> sudo service rsyslog restart 2024-09-16T01:46:06.077 DEBUG:teuthology.orchestra.run.smithi137:> sudo service rsyslog restart 2024-09-16T01:46:06.079 DEBUG:teuthology.orchestra.run.smithi190:> sudo service rsyslog restart 2024-09-16T01:46:06.131 INFO:teuthology.orchestra.run.smithi105.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T01:46:06.131 INFO:teuthology.orchestra.run.smithi137.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T01:46:06.253 INFO:teuthology.orchestra.run.smithi190.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T01:46:06.646 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-16T01:46:06.653 INFO:teuthology.task.internal:Starting timer... 2024-09-16T01:46:06.653 INFO:teuthology.run_tasks:Running task pcp... 2024-09-16T01:46:06.682 INFO:teuthology.run_tasks:Running task selinux... 2024-09-16T01:46:06.707 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-16T01:46:06.707 DEBUG:teuthology.orchestra.run.smithi105:> sudo service auditd rotate 2024-09-16T01:46:06.761 INFO:teuthology.orchestra.run.smithi105.stdout:Rotating logs: 2024-09-16T01:46:06.763 DEBUG:teuthology.orchestra.run.smithi137:> sudo service auditd rotate 2024-09-16T01:46:06.821 INFO:teuthology.orchestra.run.smithi137.stdout:Rotating logs: 2024-09-16T01:46:06.823 DEBUG:teuthology.orchestra.run.smithi190:> sudo service auditd rotate 2024-09-16T01:46:06.938 INFO:teuthology.orchestra.run.smithi190.stdout:Rotating logs: 2024-09-16T01:46:06.943 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-16T01:46:06.943 DEBUG:teuthology.orchestra.run.smithi105:> /usr/sbin/getenforce 2024-09-16T01:46:06.976 INFO:teuthology.orchestra.run.smithi105.stdout:Permissive 2024-09-16T01:46:06.976 DEBUG:teuthology.orchestra.run.smithi137:> /usr/sbin/getenforce 2024-09-16T01:46:07.005 INFO:teuthology.orchestra.run.smithi137.stdout:Permissive 2024-09-16T01:46:07.006 DEBUG:teuthology.orchestra.run.smithi190:> /usr/sbin/getenforce 2024-09-16T01:46:07.056 INFO:teuthology.orchestra.run.smithi190.stdout:Permissive 2024-09-16T01:46:07.056 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi105.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi137.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi190.front.sepia.ceph.com': 'permissive'} 2024-09-16T01:46:07.056 DEBUG:teuthology.orchestra.run.smithi105:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T01:46:07.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T01:46:07.085 DEBUG:teuthology.orchestra.run.smithi137:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T01:46:07.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T01:46:07.115 DEBUG:teuthology.orchestra.run.smithi190:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T01:46:07.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T01:46:07.184 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-16T01:46:07.185 DEBUG:teuthology.orchestra.run.smithi105:> sudo /usr/sbin/setenforce permissive 2024-09-16T01:46:07.217 DEBUG:teuthology.orchestra.run.smithi137:> sudo /usr/sbin/setenforce permissive 2024-09-16T01:46:07.253 DEBUG:teuthology.orchestra.run.smithi190:> sudo /usr/sbin/setenforce permissive 2024-09-16T01:46:07.315 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-16T01:46:07.357 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-16T01:46:07.360 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-16T01:46:08.015 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-16T01:46:09.195 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-16T01:46:09.196 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi105.front.sepia.ceph.com,smithi137.front.sepia.ceph.com,smithi190.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-16T01:50:10.189 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi105.front.sepia.ceph.com'), Remote(name='ubuntu@smithi137.front.sepia.ceph.com'), Remote(name='ubuntu@smithi190.front.sepia.ceph.com')] 2024-09-16T01:50:10.190 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-09-16T01:50:10.191 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T01:50:10.324 DEBUG:teuthology.orchestra.run.smithi105:> true 2024-09-16T01:50:10.400 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-09-16T01:50:10.401 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-09-16T01:50:10.401 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T01:50:10.485 DEBUG:teuthology.orchestra.run.smithi137:> true 2024-09-16T01:50:10.561 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-09-16T01:50:10.561 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-09-16T01:50:10.562 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T01:50:10.641 DEBUG:teuthology.orchestra.run.smithi190:> true 2024-09-16T01:50:10.722 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-09-16T01:50:10.723 INFO:teuthology.run_tasks:Running task clock... 2024-09-16T01:50:10.732 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-16T01:50:10.732 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T01:50:10.733 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T01:50:10.735 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T01:50:10.735 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T01:50:10.738 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T01:50:10.738 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T01:50:10.762 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T01:50:10.765 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T01:50:10.777 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T01:50:10.780 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T01:50:10.795 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T01:50:10.801 INFO:teuthology.orchestra.run.smithi105.stderr:sudo: ntpd: command not found 2024-09-16T01:50:10.805 INFO:teuthology.orchestra.run.smithi137.stderr:sudo: ntpd: command not found 2024-09-16T01:50:10.810 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T01:50:10.812 INFO:teuthology.orchestra.run.smithi105.stdout:506 Cannot talk to daemon 2024-09-16T01:50:10.816 INFO:teuthology.orchestra.run.smithi137.stdout:506 Cannot talk to daemon 2024-09-16T01:50:10.826 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T01:50:10.831 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T01:50:10.837 INFO:teuthology.orchestra.run.smithi190.stderr:sudo: ntpd: command not found 2024-09-16T01:50:10.840 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T01:50:10.846 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T01:50:10.849 INFO:teuthology.orchestra.run.smithi190.stdout:506 Cannot talk to daemon 2024-09-16T01:50:10.864 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T01:50:10.878 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T01:50:10.895 INFO:teuthology.orchestra.run.smithi105.stderr:bash: line 1: ntpq: command not found 2024-09-16T01:50:10.898 INFO:teuthology.orchestra.run.smithi105.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T01:50:10.898 INFO:teuthology.orchestra.run.smithi105.stdout:=============================================================================== 2024-09-16T01:50:10.898 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T01:50:10.898 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T01:50:10.898 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T01:50:10.898 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T01:50:10.904 INFO:teuthology.orchestra.run.smithi137.stderr:bash: line 1: ntpq: command not found 2024-09-16T01:50:10.907 INFO:teuthology.orchestra.run.smithi137.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T01:50:10.907 INFO:teuthology.orchestra.run.smithi137.stdout:=============================================================================== 2024-09-16T01:50:10.907 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T01:50:10.907 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T01:50:10.907 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T01:50:10.907 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T01:50:10.933 INFO:teuthology.orchestra.run.smithi190.stderr:bash: line 1: ntpq: command not found 2024-09-16T01:50:10.935 INFO:teuthology.orchestra.run.smithi190.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T01:50:10.936 INFO:teuthology.orchestra.run.smithi190.stdout:=============================================================================== 2024-09-16T01:50:10.936 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T01:50:10.936 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T01:50:10.936 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T01:50:10.936 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T01:50:10.937 INFO:teuthology.run_tasks:Running task install... 2024-09-16T01:50:10.947 DEBUG:teuthology.task.install:project ceph 2024-09-16T01:50:10.947 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'}} 2024-09-16T01:50:10.947 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'} 2024-09-16T01:50:10.947 INFO:teuthology.task.install:Using flavor: default 2024-09-16T01:50:10.953 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-16T01:50:10.953 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-16T01:50:10.953 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-16T01:50:10.954 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T01:50:10.956 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-16T01:50:10.956 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T01:50:10.957 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-16T01:50:10.957 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T01:50:11.098 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-16T01:50:11.098 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-16T01:50:11.107 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-16T01:50:11.108 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-16T01:50:11.117 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-16T01:50:11.117 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-16T01:50:11.206 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T01:50:11.206 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-16T01:50:11.206 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T01:50:11.213 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T01:50:11.213 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-16T01:50:11.213 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T01:50:11.221 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T01:50:11.221 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-16T01:50:11.221 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T01:50:11.233 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-16T01:50:11.234 DEBUG:teuthology.orchestra.run.smithi105:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T01:50:11.240 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-16T01:50:11.240 DEBUG:teuthology.orchestra.run.smithi190:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T01:50:11.247 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-16T01:50:11.247 DEBUG:teuthology.orchestra.run.smithi137:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T01:50:11.299 DEBUG:teuthology.orchestra.run.smithi105:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T01:50:11.306 DEBUG:teuthology.orchestra.run.smithi190:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T01:50:11.312 DEBUG:teuthology.orchestra.run.smithi137:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T01:50:11.375 DEBUG:teuthology.orchestra.run.smithi105:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T01:50:11.381 DEBUG:teuthology.orchestra.run.smithi190:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T01:50:11.388 DEBUG:teuthology.orchestra.run.smithi137:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T01:50:11.439 INFO:teuthology.orchestra.run.smithi105.stdout:check_obsoletes = 1 2024-09-16T01:50:11.441 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum clean all 2024-09-16T01:50:11.446 INFO:teuthology.orchestra.run.smithi190.stdout:check_obsoletes = 1 2024-09-16T01:50:11.448 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum clean all 2024-09-16T01:50:11.452 INFO:teuthology.orchestra.run.smithi137.stdout:check_obsoletes = 1 2024-09-16T01:50:11.454 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum clean all 2024-09-16T01:50:11.669 INFO:teuthology.orchestra.run.smithi105.stdout:45 files removed 2024-09-16T01:50:11.676 INFO:teuthology.orchestra.run.smithi190.stdout:45 files removed 2024-09-16T01:50:11.697 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-16T01:50:11.697 INFO:teuthology.orchestra.run.smithi137.stdout:45 files removed 2024-09-16T01:50:11.703 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-16T01:50:11.727 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-16T01:50:12.257 INFO:teuthology.orchestra.run.smithi190.stdout:ceph packages for x86_64 312 kB/s | 83 kB 00:00 2024-09-16T01:50:12.263 INFO:teuthology.orchestra.run.smithi105.stdout:ceph packages for x86_64 298 kB/s | 83 kB 00:00 2024-09-16T01:50:12.290 INFO:teuthology.orchestra.run.smithi137.stdout:ceph packages for x86_64 303 kB/s | 83 kB 00:00 2024-09-16T01:50:12.547 INFO:teuthology.orchestra.run.smithi105.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-16T01:50:12.551 INFO:teuthology.orchestra.run.smithi190.stdout:ceph noarch packages 47 kB/s | 12 kB 00:00 2024-09-16T01:50:12.574 INFO:teuthology.orchestra.run.smithi137.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-16T01:50:12.823 INFO:teuthology.orchestra.run.smithi105.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-16T01:50:12.833 INFO:teuthology.orchestra.run.smithi137.stdout:ceph source packages 8.5 kB/s | 1.9 kB 00:00 2024-09-16T01:50:12.833 INFO:teuthology.orchestra.run.smithi190.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-16T01:50:13.623 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.2 MB 00:00 2024-09-16T01:50:13.741 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - BaseOS 9.5 MB/s | 8.2 MB 00:00 2024-09-16T01:50:14.067 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - BaseOS 6.9 MB/s | 8.2 MB 00:01 2024-09-16T01:50:15.642 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - AppStream 27 MB/s | 20 MB 00:00 2024-09-16T01:50:15.699 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - AppStream 21 MB/s | 20 MB 00:00 2024-09-16T01:50:15.885 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - AppStream 28 MB/s | 20 MB 00:00 2024-09-16T01:50:20.043 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-16T01:50:20.058 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-16T01:50:21.510 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.5 MB 00:01 2024-09-16T01:50:21.618 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - Extras packages 93 kB/s | 19 kB 00:00 2024-09-16T01:50:21.853 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 19 kB 00:00 2024-09-16T01:50:22.681 INFO:teuthology.orchestra.run.smithi105.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-16T01:50:22.725 INFO:teuthology.orchestra.run.smithi137.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-16T01:50:23.228 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 19 kB 00:00 2024-09-16T01:50:24.341 INFO:teuthology.orchestra.run.smithi190.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-16T01:50:28.120 INFO:teuthology.orchestra.run.smithi105.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-09-16T01:50:28.337 INFO:teuthology.orchestra.run.smithi137.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-16T01:50:29.772 INFO:teuthology.orchestra.run.smithi190.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-09-16T01:50:30.456 INFO:teuthology.orchestra.run.smithi105.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T01:50:30.457 INFO:teuthology.orchestra.run.smithi105.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T01:50:30.470 INFO:teuthology.orchestra.run.smithi105.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-16T01:50:30.471 INFO:teuthology.orchestra.run.smithi105.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-16T01:50:30.472 INFO:teuthology.orchestra.run.smithi105.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-16T01:50:30.473 INFO:teuthology.orchestra.run.smithi105.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-16T01:50:30.475 INFO:teuthology.orchestra.run.smithi105.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-16T01:50:30.477 INFO:teuthology.orchestra.run.smithi105.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-16T01:50:30.478 INFO:teuthology.orchestra.run.smithi105.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-16T01:50:30.542 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2024-09-16T01:50:30.559 INFO:teuthology.orchestra.run.smithi105.stdout:====================================================================================== 2024-09-16T01:50:30.559 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repository Size 2024-09-16T01:50:30.559 INFO:teuthology.orchestra.run.smithi105.stdout:====================================================================================== 2024-09-16T01:50:30.560 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2024-09-16T01:50:30.560 INFO:teuthology.orchestra.run.smithi105.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-16T01:50:30.560 INFO:teuthology.orchestra.run.smithi105.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-16T01:50:30.560 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-16T01:50:30.560 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-16T01:50:30.560 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-16T01:50:30.560 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-16T01:50:30.560 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-16T01:50:30.560 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-16T01:50:30.560 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-16T01:50:30.560 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-16T01:50:30.560 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-16T01:50:30.561 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-16T01:50:30.561 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-16T01:50:30.561 INFO:teuthology.orchestra.run.smithi105.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-16T01:50:30.561 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-16T01:50:30.561 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-16T01:50:30.561 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-16T01:50:30.561 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-16T01:50:30.561 INFO:teuthology.orchestra.run.smithi105.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-16T01:50:30.561 INFO:teuthology.orchestra.run.smithi105.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-16T01:50:30.561 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-16T01:50:30.561 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-16T01:50:30.561 INFO:teuthology.orchestra.run.smithi105.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-16T01:50:30.561 INFO:teuthology.orchestra.run.smithi105.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-16T01:50:30.562 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-16T01:50:30.562 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-16T01:50:30.562 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-16T01:50:30.562 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-16T01:50:30.562 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-16T01:50:30.562 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-16T01:50:30.562 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-16T01:50:30.562 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-16T01:50:30.562 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-16T01:50:30.562 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-16T01:50:30.562 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-16T01:50:30.563 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-16T01:50:30.563 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-16T01:50:30.563 INFO:teuthology.orchestra.run.smithi105.stdout:Upgrading: 2024-09-16T01:50:30.563 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-16T01:50:30.563 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-16T01:50:30.563 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-16T01:50:30.563 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-16T01:50:30.563 INFO:teuthology.orchestra.run.smithi105.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-16T01:50:30.563 INFO:teuthology.orchestra.run.smithi105.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-16T01:50:30.563 INFO:teuthology.orchestra.run.smithi105.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-16T01:50:30.563 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-16T01:50:30.563 INFO:teuthology.orchestra.run.smithi105.stdout:Installing dependencies: 2024-09-16T01:50:30.563 INFO:teuthology.orchestra.run.smithi105.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-16T01:50:30.563 INFO:teuthology.orchestra.run.smithi105.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-16T01:50:30.564 INFO:teuthology.orchestra.run.smithi105.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T01:50:30.564 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-16T01:50:30.564 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-16T01:50:30.564 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-16T01:50:30.564 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-16T01:50:30.564 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-16T01:50:30.564 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-16T01:50:30.564 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-16T01:50:30.564 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-16T01:50:30.564 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T01:50:30.564 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T01:50:30.564 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T01:50:30.564 INFO:teuthology.orchestra.run.smithi105.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T01:50:30.565 INFO:teuthology.orchestra.run.smithi105.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T01:50:30.565 INFO:teuthology.orchestra.run.smithi105.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-16T01:50:30.565 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T01:50:30.565 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T01:50:30.565 INFO:teuthology.orchestra.run.smithi105.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-16T01:50:30.565 INFO:teuthology.orchestra.run.smithi105.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T01:50:30.565 INFO:teuthology.orchestra.run.smithi105.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T01:50:30.565 INFO:teuthology.orchestra.run.smithi105.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T01:50:30.565 INFO:teuthology.orchestra.run.smithi105.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T01:50:30.565 INFO:teuthology.orchestra.run.smithi105.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T01:50:30.565 INFO:teuthology.orchestra.run.smithi105.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-16T01:50:30.565 INFO:teuthology.orchestra.run.smithi105.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T01:50:30.565 INFO:teuthology.orchestra.run.smithi105.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-16T01:50:30.566 INFO:teuthology.orchestra.run.smithi105.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T01:50:30.566 INFO:teuthology.orchestra.run.smithi105.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-16T01:50:30.566 INFO:teuthology.orchestra.run.smithi105.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T01:50:30.566 INFO:teuthology.orchestra.run.smithi105.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-16T01:50:30.566 INFO:teuthology.orchestra.run.smithi105.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-16T01:50:30.566 INFO:teuthology.orchestra.run.smithi105.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-16T01:50:30.566 INFO:teuthology.orchestra.run.smithi105.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T01:50:30.566 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T01:50:30.566 INFO:teuthology.orchestra.run.smithi105.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T01:50:30.566 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-16T01:50:30.566 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-16T01:50:30.566 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-16T01:50:30.566 INFO:teuthology.orchestra.run.smithi105.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T01:50:30.566 INFO:teuthology.orchestra.run.smithi105.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-16T01:50:30.567 INFO:teuthology.orchestra.run.smithi105.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T01:50:30.567 INFO:teuthology.orchestra.run.smithi105.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T01:50:30.567 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T01:50:30.567 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-16T01:50:30.567 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-16T01:50:30.567 INFO:teuthology.orchestra.run.smithi105.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T01:50:30.567 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T01:50:30.567 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T01:50:30.567 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T01:50:30.567 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-16T01:50:30.567 INFO:teuthology.orchestra.run.smithi105.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-16T01:50:30.567 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T01:50:30.567 INFO:teuthology.orchestra.run.smithi105.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T01:50:30.568 INFO:teuthology.orchestra.run.smithi105.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T01:50:30.568 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-16T01:50:30.568 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-16T01:50:30.568 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T01:50:30.568 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T01:50:30.568 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-16T01:50:30.568 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T01:50:30.568 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T01:50:30.568 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T01:50:30.568 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T01:50:30.568 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T01:50:30.568 INFO:teuthology.orchestra.run.smithi105.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T01:50:30.568 INFO:teuthology.orchestra.run.smithi105.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T01:50:30.568 INFO:teuthology.orchestra.run.smithi105.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T01:50:30.569 INFO:teuthology.orchestra.run.smithi105.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T01:50:30.569 INFO:teuthology.orchestra.run.smithi105.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T01:50:30.569 INFO:teuthology.orchestra.run.smithi105.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T01:50:30.569 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T01:50:30.569 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T01:50:30.569 INFO:teuthology.orchestra.run.smithi105.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T01:50:30.569 INFO:teuthology.orchestra.run.smithi105.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T01:50:30.569 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T01:50:30.569 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T01:50:30.569 INFO:teuthology.orchestra.run.smithi105.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T01:50:30.569 INFO:teuthology.orchestra.run.smithi105.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T01:50:30.569 INFO:teuthology.orchestra.run.smithi105.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-16T01:50:30.569 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T01:50:30.569 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T01:50:30.570 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T01:50:30.570 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T01:50:30.570 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-16T01:50:30.570 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T01:50:30.570 INFO:teuthology.orchestra.run.smithi105.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T01:50:30.570 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T01:50:30.570 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T01:50:30.570 INFO:teuthology.orchestra.run.smithi105.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T01:50:30.570 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T01:50:30.570 INFO:teuthology.orchestra.run.smithi105.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T01:50:30.570 INFO:teuthology.orchestra.run.smithi105.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T01:50:30.570 INFO:teuthology.orchestra.run.smithi105.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T01:50:30.570 INFO:teuthology.orchestra.run.smithi105.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T01:50:30.570 INFO:teuthology.orchestra.run.smithi105.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T01:50:30.571 INFO:teuthology.orchestra.run.smithi105.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-16T01:50:30.571 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T01:50:30.571 INFO:teuthology.orchestra.run.smithi105.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T01:50:30.571 INFO:teuthology.orchestra.run.smithi105.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T01:50:30.571 INFO:teuthology.orchestra.run.smithi105.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T01:50:30.571 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-16T01:50:30.571 INFO:teuthology.orchestra.run.smithi105.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-16T01:50:30.571 INFO:teuthology.orchestra.run.smithi105.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T01:50:30.571 INFO:teuthology.orchestra.run.smithi105.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T01:50:30.571 INFO:teuthology.orchestra.run.smithi105.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T01:50:30.571 INFO:teuthology.orchestra.run.smithi105.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-16T01:50:30.571 INFO:teuthology.orchestra.run.smithi105.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T01:50:30.571 INFO:teuthology.orchestra.run.smithi105.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-16T01:50:30.571 INFO:teuthology.orchestra.run.smithi105.stdout:Installing weak dependencies: 2024-09-16T01:50:30.572 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-16T01:50:30.572 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:50:30.572 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2024-09-16T01:50:30.572 INFO:teuthology.orchestra.run.smithi105.stdout:====================================================================================== 2024-09-16T01:50:30.572 INFO:teuthology.orchestra.run.smithi105.stdout:Install 148 Packages 2024-09-16T01:50:30.572 INFO:teuthology.orchestra.run.smithi105.stdout:Upgrade 8 Packages 2024-09-16T01:50:30.572 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:50:30.572 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 226 M 2024-09-16T01:50:30.573 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2024-09-16T01:50:30.658 INFO:teuthology.orchestra.run.smithi137.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T01:50:30.659 INFO:teuthology.orchestra.run.smithi137.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T01:50:30.672 INFO:teuthology.orchestra.run.smithi137.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-16T01:50:30.673 INFO:teuthology.orchestra.run.smithi137.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-16T01:50:30.674 INFO:teuthology.orchestra.run.smithi137.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-16T01:50:30.675 INFO:teuthology.orchestra.run.smithi137.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-16T01:50:30.677 INFO:teuthology.orchestra.run.smithi137.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-16T01:50:30.680 INFO:teuthology.orchestra.run.smithi137.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-16T01:50:30.681 INFO:teuthology.orchestra.run.smithi137.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-16T01:50:30.745 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-09-16T01:50:30.762 INFO:teuthology.orchestra.run.smithi137.stdout:====================================================================================== 2024-09-16T01:50:30.762 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-09-16T01:50:30.762 INFO:teuthology.orchestra.run.smithi137.stdout:====================================================================================== 2024-09-16T01:50:30.762 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-09-16T01:50:30.762 INFO:teuthology.orchestra.run.smithi137.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-16T01:50:30.762 INFO:teuthology.orchestra.run.smithi137.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-16T01:50:30.762 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-16T01:50:30.762 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-16T01:50:30.762 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-16T01:50:30.762 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-16T01:50:30.763 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-16T01:50:30.763 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-16T01:50:30.763 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-16T01:50:30.763 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-16T01:50:30.763 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-16T01:50:30.763 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-16T01:50:30.763 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-16T01:50:30.763 INFO:teuthology.orchestra.run.smithi137.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-16T01:50:30.763 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-16T01:50:30.763 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-16T01:50:30.763 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-16T01:50:30.763 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-16T01:50:30.764 INFO:teuthology.orchestra.run.smithi137.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-16T01:50:30.764 INFO:teuthology.orchestra.run.smithi137.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-16T01:50:30.764 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-16T01:50:30.764 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-16T01:50:30.764 INFO:teuthology.orchestra.run.smithi137.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-16T01:50:30.764 INFO:teuthology.orchestra.run.smithi137.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-16T01:50:30.764 INFO:teuthology.orchestra.run.smithi137.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-16T01:50:30.764 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-16T01:50:30.764 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-16T01:50:30.764 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-16T01:50:30.764 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-16T01:50:30.764 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-16T01:50:30.764 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-16T01:50:30.765 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-16T01:50:30.765 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-16T01:50:30.765 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-16T01:50:30.765 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-16T01:50:30.765 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-16T01:50:30.765 INFO:teuthology.orchestra.run.smithi137.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-16T01:50:30.765 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrading: 2024-09-16T01:50:30.765 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-16T01:50:30.765 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-16T01:50:30.765 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-16T01:50:30.765 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-16T01:50:30.765 INFO:teuthology.orchestra.run.smithi137.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-16T01:50:30.765 INFO:teuthology.orchestra.run.smithi137.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-16T01:50:30.765 INFO:teuthology.orchestra.run.smithi137.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-16T01:50:30.766 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-16T01:50:30.766 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-09-16T01:50:30.766 INFO:teuthology.orchestra.run.smithi137.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-16T01:50:30.766 INFO:teuthology.orchestra.run.smithi137.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-16T01:50:30.766 INFO:teuthology.orchestra.run.smithi137.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T01:50:30.766 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-16T01:50:30.766 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-16T01:50:30.766 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-16T01:50:30.766 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-16T01:50:30.766 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-16T01:50:30.766 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-16T01:50:30.766 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-16T01:50:30.766 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-16T01:50:30.767 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T01:50:30.767 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T01:50:30.767 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T01:50:30.767 INFO:teuthology.orchestra.run.smithi137.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T01:50:30.767 INFO:teuthology.orchestra.run.smithi137.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T01:50:30.767 INFO:teuthology.orchestra.run.smithi137.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-16T01:50:30.767 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T01:50:30.767 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T01:50:30.767 INFO:teuthology.orchestra.run.smithi137.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-16T01:50:30.767 INFO:teuthology.orchestra.run.smithi137.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T01:50:30.767 INFO:teuthology.orchestra.run.smithi137.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T01:50:30.767 INFO:teuthology.orchestra.run.smithi137.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T01:50:30.767 INFO:teuthology.orchestra.run.smithi137.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T01:50:30.768 INFO:teuthology.orchestra.run.smithi137.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T01:50:30.768 INFO:teuthology.orchestra.run.smithi137.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-16T01:50:30.768 INFO:teuthology.orchestra.run.smithi137.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T01:50:30.768 INFO:teuthology.orchestra.run.smithi137.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-16T01:50:30.768 INFO:teuthology.orchestra.run.smithi137.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T01:50:30.768 INFO:teuthology.orchestra.run.smithi137.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-16T01:50:30.768 INFO:teuthology.orchestra.run.smithi137.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T01:50:30.768 INFO:teuthology.orchestra.run.smithi137.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-16T01:50:30.768 INFO:teuthology.orchestra.run.smithi137.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-16T01:50:30.768 INFO:teuthology.orchestra.run.smithi137.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-16T01:50:30.768 INFO:teuthology.orchestra.run.smithi137.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T01:50:30.768 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T01:50:30.768 INFO:teuthology.orchestra.run.smithi137.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T01:50:30.769 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-16T01:50:30.769 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-16T01:50:30.769 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-16T01:50:30.769 INFO:teuthology.orchestra.run.smithi137.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T01:50:30.769 INFO:teuthology.orchestra.run.smithi137.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-16T01:50:30.769 INFO:teuthology.orchestra.run.smithi137.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T01:50:30.769 INFO:teuthology.orchestra.run.smithi137.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T01:50:30.769 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T01:50:30.770 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-16T01:50:30.770 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-16T01:50:30.770 INFO:teuthology.orchestra.run.smithi137.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T01:50:30.770 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T01:50:30.770 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T01:50:30.770 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T01:50:30.770 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-16T01:50:30.770 INFO:teuthology.orchestra.run.smithi137.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-16T01:50:30.770 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T01:50:30.770 INFO:teuthology.orchestra.run.smithi137.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T01:50:30.770 INFO:teuthology.orchestra.run.smithi137.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T01:50:30.770 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-16T01:50:30.771 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-16T01:50:30.771 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T01:50:30.771 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T01:50:30.771 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-16T01:50:30.771 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T01:50:30.771 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T01:50:30.771 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T01:50:30.771 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T01:50:30.771 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T01:50:30.771 INFO:teuthology.orchestra.run.smithi137.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T01:50:30.771 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T01:50:30.771 INFO:teuthology.orchestra.run.smithi137.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T01:50:30.771 INFO:teuthology.orchestra.run.smithi137.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T01:50:30.772 INFO:teuthology.orchestra.run.smithi137.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T01:50:30.772 INFO:teuthology.orchestra.run.smithi137.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T01:50:30.772 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T01:50:30.772 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T01:50:30.772 INFO:teuthology.orchestra.run.smithi137.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T01:50:30.772 INFO:teuthology.orchestra.run.smithi137.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T01:50:30.772 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T01:50:30.772 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T01:50:30.772 INFO:teuthology.orchestra.run.smithi137.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T01:50:30.772 INFO:teuthology.orchestra.run.smithi137.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T01:50:30.772 INFO:teuthology.orchestra.run.smithi137.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-16T01:50:30.772 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T01:50:30.772 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T01:50:30.773 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T01:50:30.773 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T01:50:30.773 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-16T01:50:30.773 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T01:50:30.773 INFO:teuthology.orchestra.run.smithi137.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T01:50:30.773 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T01:50:30.773 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T01:50:30.773 INFO:teuthology.orchestra.run.smithi137.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T01:50:30.773 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T01:50:30.773 INFO:teuthology.orchestra.run.smithi137.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T01:50:30.773 INFO:teuthology.orchestra.run.smithi137.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T01:50:30.773 INFO:teuthology.orchestra.run.smithi137.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T01:50:30.773 INFO:teuthology.orchestra.run.smithi137.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T01:50:30.773 INFO:teuthology.orchestra.run.smithi137.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T01:50:30.774 INFO:teuthology.orchestra.run.smithi137.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-16T01:50:30.774 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T01:50:30.774 INFO:teuthology.orchestra.run.smithi137.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T01:50:30.774 INFO:teuthology.orchestra.run.smithi137.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T01:50:30.774 INFO:teuthology.orchestra.run.smithi137.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T01:50:30.774 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-16T01:50:30.774 INFO:teuthology.orchestra.run.smithi137.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-16T01:50:30.774 INFO:teuthology.orchestra.run.smithi137.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T01:50:30.774 INFO:teuthology.orchestra.run.smithi137.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T01:50:30.774 INFO:teuthology.orchestra.run.smithi137.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T01:50:30.774 INFO:teuthology.orchestra.run.smithi137.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-16T01:50:30.774 INFO:teuthology.orchestra.run.smithi137.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T01:50:30.774 INFO:teuthology.orchestra.run.smithi137.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-16T01:50:30.775 INFO:teuthology.orchestra.run.smithi137.stdout:Installing weak dependencies: 2024-09-16T01:50:30.775 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-16T01:50:30.775 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-16T01:50:30.775 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-09-16T01:50:30.775 INFO:teuthology.orchestra.run.smithi137.stdout:====================================================================================== 2024-09-16T01:50:30.775 INFO:teuthology.orchestra.run.smithi137.stdout:Install 148 Packages 2024-09-16T01:50:30.775 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrade 8 Packages 2024-09-16T01:50:30.775 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-16T01:50:30.776 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 226 M 2024-09-16T01:50:30.776 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-09-16T01:50:31.055 INFO:teuthology.orchestra.run.smithi137.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 49 kB/s | 6.5 kB 00:00 2024-09-16T01:50:31.155 INFO:teuthology.orchestra.run.smithi105.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 50 kB/s | 6.5 kB 00:00 2024-09-16T01:50:31.297 INFO:teuthology.orchestra.run.smithi137.stdout:(2/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 3.6 MB/s | 896 kB 00:00 2024-09-16T01:50:31.364 INFO:teuthology.orchestra.run.smithi137.stdout:(3/156): ceph-immutable-object-cache-19.1.1-326 2.1 MB/s | 145 kB 00:00 2024-09-16T01:50:31.548 INFO:teuthology.orchestra.run.smithi137.stdout:(4/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 8.6 MB/s | 5.4 MB 00:00 2024-09-16T01:50:31.580 INFO:teuthology.orchestra.run.smithi105.stdout:(2/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 2.1 MB/s | 896 kB 00:00 2024-09-16T01:50:31.673 INFO:teuthology.orchestra.run.smithi137.stdout:(5/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 7.7 MB/s | 2.4 MB 00:00 2024-09-16T01:50:31.697 INFO:teuthology.orchestra.run.smithi105.stdout:(3/156): ceph-immutable-object-cache-19.1.1-326 1.2 MB/s | 145 kB 00:00 2024-09-16T01:50:31.714 INFO:teuthology.orchestra.run.smithi137.stdout:(6/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 9.1 MB/s | 1.5 MB 00:00 2024-09-16T01:50:32.140 INFO:teuthology.orchestra.run.smithi190.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T01:50:32.141 INFO:teuthology.orchestra.run.smithi190.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T01:50:32.153 INFO:teuthology.orchestra.run.smithi190.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-16T01:50:32.153 INFO:teuthology.orchestra.run.smithi190.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-16T01:50:32.154 INFO:teuthology.orchestra.run.smithi190.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-16T01:50:32.155 INFO:teuthology.orchestra.run.smithi190.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-16T01:50:32.157 INFO:teuthology.orchestra.run.smithi190.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-16T01:50:32.159 INFO:teuthology.orchestra.run.smithi190.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-16T01:50:32.160 INFO:teuthology.orchestra.run.smithi190.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-16T01:50:32.224 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-09-16T01:50:32.240 INFO:teuthology.orchestra.run.smithi137.stdout:(7/156): ceph-common-19.1.1-326.g8921c8ef.el9.x 16 MB/s | 21 MB 00:01 2024-09-16T01:50:32.246 INFO:teuthology.orchestra.run.smithi190.stdout:====================================================================================== 2024-09-16T01:50:32.246 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repository Size 2024-09-16T01:50:32.247 INFO:teuthology.orchestra.run.smithi190.stdout:====================================================================================== 2024-09-16T01:50:32.247 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-09-16T01:50:32.247 INFO:teuthology.orchestra.run.smithi190.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-16T01:50:32.247 INFO:teuthology.orchestra.run.smithi190.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-16T01:50:32.247 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-16T01:50:32.247 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-16T01:50:32.247 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-16T01:50:32.247 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-16T01:50:32.247 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-16T01:50:32.247 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-16T01:50:32.247 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-16T01:50:32.247 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-16T01:50:32.247 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-16T01:50:32.248 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-16T01:50:32.248 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-16T01:50:32.248 INFO:teuthology.orchestra.run.smithi190.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-16T01:50:32.248 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-16T01:50:32.248 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-16T01:50:32.248 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-16T01:50:32.248 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-16T01:50:32.248 INFO:teuthology.orchestra.run.smithi190.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-16T01:50:32.248 INFO:teuthology.orchestra.run.smithi190.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-16T01:50:32.248 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-16T01:50:32.248 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-16T01:50:32.248 INFO:teuthology.orchestra.run.smithi190.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-16T01:50:32.249 INFO:teuthology.orchestra.run.smithi190.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-16T01:50:32.249 INFO:teuthology.orchestra.run.smithi190.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-16T01:50:32.249 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-16T01:50:32.249 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-16T01:50:32.249 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-16T01:50:32.249 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-16T01:50:32.249 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-16T01:50:32.249 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-16T01:50:32.249 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-16T01:50:32.249 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-16T01:50:32.250 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-16T01:50:32.250 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-16T01:50:32.250 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-16T01:50:32.250 INFO:teuthology.orchestra.run.smithi190.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-16T01:50:32.250 INFO:teuthology.orchestra.run.smithi190.stdout:Upgrading: 2024-09-16T01:50:32.250 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-16T01:50:32.250 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-16T01:50:32.250 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-16T01:50:32.250 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-16T01:50:32.250 INFO:teuthology.orchestra.run.smithi190.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-16T01:50:32.250 INFO:teuthology.orchestra.run.smithi190.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-16T01:50:32.250 INFO:teuthology.orchestra.run.smithi190.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-16T01:50:32.250 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-16T01:50:32.250 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2024-09-16T01:50:32.251 INFO:teuthology.orchestra.run.smithi190.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-16T01:50:32.251 INFO:teuthology.orchestra.run.smithi190.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-16T01:50:32.251 INFO:teuthology.orchestra.run.smithi190.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T01:50:32.251 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-16T01:50:32.251 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-16T01:50:32.251 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-16T01:50:32.251 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-16T01:50:32.251 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-16T01:50:32.251 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-16T01:50:32.251 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-16T01:50:32.251 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-16T01:50:32.251 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T01:50:32.251 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T01:50:32.251 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T01:50:32.252 INFO:teuthology.orchestra.run.smithi190.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T01:50:32.252 INFO:teuthology.orchestra.run.smithi190.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T01:50:32.252 INFO:teuthology.orchestra.run.smithi190.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-16T01:50:32.252 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T01:50:32.252 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T01:50:32.252 INFO:teuthology.orchestra.run.smithi190.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-16T01:50:32.252 INFO:teuthology.orchestra.run.smithi190.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T01:50:32.252 INFO:teuthology.orchestra.run.smithi190.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T01:50:32.252 INFO:teuthology.orchestra.run.smithi190.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T01:50:32.252 INFO:teuthology.orchestra.run.smithi190.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T01:50:32.252 INFO:teuthology.orchestra.run.smithi190.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T01:50:32.252 INFO:teuthology.orchestra.run.smithi190.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-16T01:50:32.252 INFO:teuthology.orchestra.run.smithi190.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T01:50:32.253 INFO:teuthology.orchestra.run.smithi190.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-16T01:50:32.253 INFO:teuthology.orchestra.run.smithi190.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T01:50:32.253 INFO:teuthology.orchestra.run.smithi190.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-16T01:50:32.253 INFO:teuthology.orchestra.run.smithi190.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T01:50:32.253 INFO:teuthology.orchestra.run.smithi190.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-16T01:50:32.253 INFO:teuthology.orchestra.run.smithi190.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-16T01:50:32.253 INFO:teuthology.orchestra.run.smithi190.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-16T01:50:32.253 INFO:teuthology.orchestra.run.smithi190.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T01:50:32.253 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T01:50:32.253 INFO:teuthology.orchestra.run.smithi190.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T01:50:32.253 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-16T01:50:32.253 INFO:teuthology.orchestra.run.smithi190.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-16T01:50:32.253 INFO:teuthology.orchestra.run.smithi190.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-16T01:50:32.253 INFO:teuthology.orchestra.run.smithi190.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T01:50:32.254 INFO:teuthology.orchestra.run.smithi190.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-16T01:50:32.254 INFO:teuthology.orchestra.run.smithi190.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T01:50:32.254 INFO:teuthology.orchestra.run.smithi190.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T01:50:32.254 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T01:50:32.254 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-16T01:50:32.254 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-16T01:50:32.254 INFO:teuthology.orchestra.run.smithi190.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T01:50:32.254 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T01:50:32.254 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T01:50:32.254 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T01:50:32.254 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-16T01:50:32.254 INFO:teuthology.orchestra.run.smithi190.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-16T01:50:32.254 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T01:50:32.254 INFO:teuthology.orchestra.run.smithi190.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T01:50:32.255 INFO:teuthology.orchestra.run.smithi190.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T01:50:32.255 INFO:teuthology.orchestra.run.smithi190.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-16T01:50:32.255 INFO:teuthology.orchestra.run.smithi190.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-16T01:50:32.255 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T01:50:32.255 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T01:50:32.255 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-16T01:50:32.255 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T01:50:32.255 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T01:50:32.255 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T01:50:32.255 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T01:50:32.255 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T01:50:32.255 INFO:teuthology.orchestra.run.smithi190.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T01:50:32.255 INFO:teuthology.orchestra.run.smithi190.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T01:50:32.255 INFO:teuthology.orchestra.run.smithi190.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T01:50:32.256 INFO:teuthology.orchestra.run.smithi190.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T01:50:32.256 INFO:teuthology.orchestra.run.smithi190.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T01:50:32.256 INFO:teuthology.orchestra.run.smithi190.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T01:50:32.256 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T01:50:32.256 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T01:50:32.256 INFO:teuthology.orchestra.run.smithi190.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T01:50:32.256 INFO:teuthology.orchestra.run.smithi190.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T01:50:32.256 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T01:50:32.256 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T01:50:32.256 INFO:teuthology.orchestra.run.smithi190.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T01:50:32.256 INFO:teuthology.orchestra.run.smithi190.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T01:50:32.256 INFO:teuthology.orchestra.run.smithi190.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-16T01:50:32.256 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T01:50:32.256 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T01:50:32.256 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T01:50:32.257 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T01:50:32.257 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-16T01:50:32.257 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T01:50:32.257 INFO:teuthology.orchestra.run.smithi190.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T01:50:32.257 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T01:50:32.257 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T01:50:32.257 INFO:teuthology.orchestra.run.smithi190.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T01:50:32.257 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T01:50:32.257 INFO:teuthology.orchestra.run.smithi190.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T01:50:32.257 INFO:teuthology.orchestra.run.smithi190.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T01:50:32.257 INFO:teuthology.orchestra.run.smithi190.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T01:50:32.257 INFO:teuthology.orchestra.run.smithi190.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T01:50:32.257 INFO:teuthology.orchestra.run.smithi190.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T01:50:32.258 INFO:teuthology.orchestra.run.smithi190.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-16T01:50:32.258 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T01:50:32.258 INFO:teuthology.orchestra.run.smithi190.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T01:50:32.258 INFO:teuthology.orchestra.run.smithi190.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T01:50:32.258 INFO:teuthology.orchestra.run.smithi190.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T01:50:32.258 INFO:teuthology.orchestra.run.smithi190.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-16T01:50:32.258 INFO:teuthology.orchestra.run.smithi190.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-16T01:50:32.258 INFO:teuthology.orchestra.run.smithi190.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T01:50:32.258 INFO:teuthology.orchestra.run.smithi190.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T01:50:32.258 INFO:teuthology.orchestra.run.smithi190.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T01:50:32.258 INFO:teuthology.orchestra.run.smithi190.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-16T01:50:32.258 INFO:teuthology.orchestra.run.smithi190.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T01:50:32.258 INFO:teuthology.orchestra.run.smithi190.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-16T01:50:32.258 INFO:teuthology.orchestra.run.smithi190.stdout:Installing weak dependencies: 2024-09-16T01:50:32.259 INFO:teuthology.orchestra.run.smithi190.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-16T01:50:32.259 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T01:50:32.259 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-09-16T01:50:32.259 INFO:teuthology.orchestra.run.smithi190.stdout:====================================================================================== 2024-09-16T01:50:32.259 INFO:teuthology.orchestra.run.smithi190.stdout:Install 148 Packages 2024-09-16T01:50:32.259 INFO:teuthology.orchestra.run.smithi190.stdout:Upgrade 8 Packages 2024-09-16T01:50:32.259 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T01:50:32.259 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 226 M 2024-09-16T01:50:32.259 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-09-16T01:50:32.317 INFO:teuthology.orchestra.run.smithi137.stdout:(8/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 7.3 MB/s | 4.7 MB 00:00 2024-09-16T01:50:32.330 INFO:teuthology.orchestra.run.smithi105.stdout:(4/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 4.1 MB/s | 5.4 MB 00:01 2024-09-16T01:50:32.359 INFO:teuthology.orchestra.run.smithi137.stdout:(9/156): ceph-selinux-19.1.1-326.g8921c8ef.el9. 607 kB/s | 25 kB 00:00 2024-09-16T01:50:32.380 INFO:teuthology.orchestra.run.smithi105.stdout:(5/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 3.5 MB/s | 2.4 MB 00:00 2024-09-16T01:50:32.706 INFO:teuthology.orchestra.run.smithi105.stdout:(6/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 4.1 MB/s | 1.5 MB 00:00 2024-09-16T01:50:32.814 INFO:teuthology.orchestra.run.smithi190.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 48 kB/s | 6.5 kB 00:00 2024-09-16T01:50:32.843 INFO:teuthology.orchestra.run.smithi137.stdout:(10/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86 15 MB/s | 17 MB 00:01 2024-09-16T01:50:32.977 INFO:teuthology.orchestra.run.smithi137.stdout:(11/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9 14 MB/s | 11 MB 00:00 2024-09-16T01:50:33.002 INFO:teuthology.orchestra.run.smithi137.stdout:(12/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 1.4 MB/s | 228 kB 00:00 2024-09-16T01:50:33.044 INFO:teuthology.orchestra.run.smithi137.stdout:(13/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 406 kB/s | 27 kB 00:00 2024-09-16T01:50:33.069 INFO:teuthology.orchestra.run.smithi137.stdout:(14/156): libcephfs-devel-19.1.1-326.g8921c8ef. 473 kB/s | 32 kB 00:00 2024-09-16T01:50:33.153 INFO:teuthology.orchestra.run.smithi137.stdout:(15/156): libcephsqlite-19.1.1-326.g8921c8ef.el 1.9 MB/s | 163 kB 00:00 2024-09-16T01:50:33.186 INFO:teuthology.orchestra.run.smithi137.stdout:(16/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 5.2 MB/s | 751 kB 00:00 2024-09-16T01:50:33.223 INFO:teuthology.orchestra.run.smithi190.stdout:(2/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 2.1 MB/s | 896 kB 00:00 2024-09-16T01:50:33.228 INFO:teuthology.orchestra.run.smithi137.stdout:(17/156): librados-devel-19.1.1-326.g8921c8ef.e 1.6 MB/s | 127 kB 00:00 2024-09-16T01:50:33.270 INFO:teuthology.orchestra.run.smithi137.stdout:(18/156): libradosstriper1-19.1.1-326.g8921c8ef 5.9 MB/s | 506 kB 00:00 2024-09-16T01:50:33.362 INFO:teuthology.orchestra.run.smithi137.stdout:(19/156): python3-ceph-argparse-19.1.1-326.g892 493 kB/s | 45 kB 00:00 2024-09-16T01:50:33.382 INFO:teuthology.orchestra.run.smithi190.stdout:(3/156): ceph-immutable-object-cache-19.1.1-326 918 kB/s | 145 kB 00:00 2024-09-16T01:50:33.389 INFO:teuthology.orchestra.run.smithi105.stdout:(7/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 4.7 MB/s | 4.7 MB 00:01 2024-09-16T01:50:33.454 INFO:teuthology.orchestra.run.smithi137.stdout:(20/156): python3-ceph-common-19.1.1-326.g8921c 1.4 MB/s | 136 kB 00:00 2024-09-16T01:50:33.474 INFO:teuthology.orchestra.run.smithi190.stdout:(4/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 6.8 MB/s | 5.4 MB 00:00 2024-09-16T01:50:33.537 INFO:teuthology.orchestra.run.smithi137.stdout:(21/156): python3-cephfs-19.1.1-326.g8921c8ef.e 1.9 MB/s | 161 kB 00:00 2024-09-16T01:50:33.659 INFO:teuthology.orchestra.run.smithi137.stdout:(22/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 12 MB/s | 5.3 MB 00:00 2024-09-16T01:50:33.688 INFO:teuthology.orchestra.run.smithi137.stdout:(23/156): python3-rados-19.1.1-326.g8921c8ef.el 2.1 MB/s | 320 kB 00:00 2024-09-16T01:50:33.707 INFO:teuthology.orchestra.run.smithi190.stdout:(5/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 6.5 MB/s | 1.5 MB 00:00 2024-09-16T01:50:33.738 INFO:teuthology.orchestra.run.smithi137.stdout:(24/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 3.7 MB/s | 300 kB 00:00 2024-09-16T01:50:33.764 INFO:teuthology.orchestra.run.smithi137.stdout:(25/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 1.3 MB/s | 99 kB 00:00 2024-09-16T01:50:33.822 INFO:teuthology.orchestra.run.smithi137.stdout:(26/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 1.0 MB/s | 85 kB 00:00 2024-09-16T01:50:34.114 INFO:teuthology.orchestra.run.smithi137.stdout:(27/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 8.9 MB/s | 3.1 MB 00:00 2024-09-16T01:50:34.206 INFO:teuthology.orchestra.run.smithi137.stdout:(28/156): ceph-grafana-dashboards-19.1.1-326.g8 348 kB/s | 32 kB 00:00 2024-09-16T01:50:34.282 INFO:teuthology.orchestra.run.smithi190.stdout:(6/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 8.2 MB/s | 4.7 MB 00:00 2024-09-16T01:50:34.290 INFO:teuthology.orchestra.run.smithi137.stdout:(29/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 1.7 MB/s | 146 kB 00:00 2024-09-16T01:50:34.448 INFO:teuthology.orchestra.run.smithi105.stdout:(8/156): ceph-common-19.1.1-326.g8921c8ef.el9.x 6.2 MB/s | 21 MB 00:03 2024-09-16T01:50:34.448 INFO:teuthology.orchestra.run.smithi137.stdout:(30/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 272 kB/s | 170 kB 00:00 2024-09-16T01:50:34.498 INFO:teuthology.orchestra.run.smithi105.stdout:(9/156): ceph-selinux-19.1.1-326.g8921c8ef.el9. 507 kB/s | 25 kB 00:00 2024-09-16T01:50:34.507 INFO:teuthology.orchestra.run.smithi137.stdout:(31/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 16 MB/s | 3.6 MB 00:00 2024-09-16T01:50:34.507 INFO:teuthology.orchestra.run.smithi190.stdout:(7/156): ceph-common-19.1.1-326.g8921c8ef.el9.x 12 MB/s | 21 MB 00:01 2024-09-16T01:50:34.558 INFO:teuthology.orchestra.run.smithi190.stdout:(8/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 2.0 MB/s | 2.4 MB 00:01 2024-09-16T01:50:34.582 INFO:teuthology.orchestra.run.smithi137.stdout:(32/156): ceph-mgr-modules-core-19.1.1-326.g892 3.3 MB/s | 249 kB 00:00 2024-09-16T01:50:34.658 INFO:teuthology.orchestra.run.smithi190.stdout:(9/156): ceph-selinux-19.1.1-326.g8921c8ef.el9. 252 kB/s | 25 kB 00:00 2024-09-16T01:50:34.682 INFO:teuthology.orchestra.run.smithi137.stdout:(33/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 494 kB/s | 49 kB 00:00 2024-09-16T01:50:34.783 INFO:teuthology.orchestra.run.smithi137.stdout:(34/156): ceph-prometheus-alerts-19.1.1-326.g89 164 kB/s | 16 kB 00:00 2024-09-16T01:50:35.150 INFO:teuthology.orchestra.run.smithi190.stdout:(10/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9 17 MB/s | 11 MB 00:00 2024-09-16T01:50:35.167 INFO:teuthology.orchestra.run.smithi137.stdout:(35/156): ceph-mgr-diskprediction-local-19.1.1- 10 MB/s | 7.4 MB 00:00 2024-09-16T01:50:35.200 INFO:teuthology.orchestra.run.smithi190.stdout:(11/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 4.5 MB/s | 228 kB 00:00 2024-09-16T01:50:35.242 INFO:teuthology.orchestra.run.smithi137.stdout:(36/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 10 MB/s | 765 kB 00:00 2024-09-16T01:50:35.250 INFO:teuthology.orchestra.run.smithi190.stdout:(12/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 542 kB/s | 27 kB 00:00 2024-09-16T01:50:35.292 INFO:teuthology.orchestra.run.smithi190.stdout:(13/156): libcephfs-devel-19.1.1-326.g8921c8ef. 763 kB/s | 32 kB 00:00 2024-09-16T01:50:35.300 INFO:teuthology.orchestra.run.smithi137.stdout:(37/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 312 kB/s | 18 kB 00:00 2024-09-16T01:50:35.332 INFO:teuthology.orchestra.run.smithi105.stdout:(10/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86 6.4 MB/s | 17 MB 00:02 2024-09-16T01:50:35.351 INFO:teuthology.orchestra.run.smithi137.stdout:(38/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 502 kB/s | 285 kB 00:00 2024-09-16T01:50:35.351 INFO:teuthology.orchestra.run.smithi190.stdout:(14/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 13 MB/s | 751 kB 00:00 2024-09-16T01:50:35.382 INFO:teuthology.orchestra.run.smithi105.stdout:(11/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 4.5 MB/s | 228 kB 00:00 2024-09-16T01:50:35.412 INFO:teuthology.orchestra.run.smithi190.stdout:(15/156): libcephsqlite-19.1.1-326.g8921c8ef.el 2.7 MB/s | 163 kB 00:00 2024-09-16T01:50:35.432 INFO:teuthology.orchestra.run.smithi105.stdout:(12/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 540 kB/s | 27 kB 00:00 2024-09-16T01:50:35.460 INFO:teuthology.orchestra.run.smithi190.stdout:(16/156): librados-devel-19.1.1-326.g8921c8ef.e 2.5 MB/s | 127 kB 00:00 2024-09-16T01:50:35.483 INFO:teuthology.orchestra.run.smithi105.stdout:(13/156): libcephfs-devel-19.1.1-326.g8921c8ef. 636 kB/s | 32 kB 00:00 2024-09-16T01:50:35.519 INFO:teuthology.orchestra.run.smithi190.stdout:(17/156): libradosstriper1-19.1.1-326.g8921c8ef 8.5 MB/s | 506 kB 00:00 2024-09-16T01:50:35.625 INFO:teuthology.orchestra.run.smithi105.stdout:(14/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 5.2 MB/s | 751 kB 00:00 2024-09-16T01:50:35.676 INFO:teuthology.orchestra.run.smithi137.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 213 kB/s | 80 kB 00:00 2024-09-16T01:50:35.700 INFO:teuthology.orchestra.run.smithi105.stdout:(15/156): libcephsqlite-19.1.1-326.g8921c8ef.el 2.1 MB/s | 163 kB 00:00 2024-09-16T01:50:35.775 INFO:teuthology.orchestra.run.smithi105.stdout:(16/156): librados-devel-19.1.1-326.g8921c8ef.e 1.7 MB/s | 127 kB 00:00 2024-09-16T01:50:35.811 INFO:teuthology.orchestra.run.smithi190.stdout:(18/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 18 MB/s | 5.3 MB 00:00 2024-09-16T01:50:35.859 INFO:teuthology.orchestra.run.smithi105.stdout:(17/156): libradosstriper1-19.1.1-326.g8921c8ef 5.9 MB/s | 506 kB 00:00 2024-09-16T01:50:35.861 INFO:teuthology.orchestra.run.smithi190.stdout:(19/156): python3-ceph-argparse-19.1.1-326.g892 902 kB/s | 45 kB 00:00 2024-09-16T01:50:35.910 INFO:teuthology.orchestra.run.smithi137.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 819 kB/s | 191 kB 00:00 2024-09-16T01:50:36.052 INFO:teuthology.orchestra.run.smithi137.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-09-16T01:50:36.053 INFO:teuthology.orchestra.run.smithi190.stdout:(20/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86 9.5 MB/s | 17 MB 00:01 2024-09-16T01:50:36.080 INFO:teuthology.orchestra.run.smithi190.stdout:(21/156): python3-ceph-common-19.1.1-326.g8921c 627 kB/s | 136 kB 00:00 2024-09-16T01:50:36.112 INFO:teuthology.orchestra.run.smithi190.stdout:(22/156): python3-cephfs-19.1.1-326.g8921c8ef.e 2.7 MB/s | 161 kB 00:00 2024-09-16T01:50:36.137 INFO:teuthology.orchestra.run.smithi190.stdout:(23/156): python3-rados-19.1.1-326.g8921c8ef.el 5.3 MB/s | 320 kB 00:00 2024-09-16T01:50:36.163 INFO:teuthology.orchestra.run.smithi190.stdout:(24/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 5.8 MB/s | 300 kB 00:00 2024-09-16T01:50:36.168 INFO:teuthology.orchestra.run.smithi137.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 980 kB/s | 801 kB 00:00 2024-09-16T01:50:36.188 INFO:teuthology.orchestra.run.smithi190.stdout:(25/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 1.9 MB/s | 99 kB 00:00 2024-09-16T01:50:36.213 INFO:teuthology.orchestra.run.smithi190.stdout:(26/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-16T01:50:36.277 INFO:teuthology.orchestra.run.smithi137.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 5.4 MB/s | 1.2 MB 00:00 2024-09-16T01:50:36.289 INFO:teuthology.orchestra.run.smithi190.stdout:(27/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 2.2 MB/s | 170 kB 00:00 2024-09-16T01:50:36.302 INFO:teuthology.orchestra.run.smithi137.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 799 kB/s | 106 kB 00:00 2024-09-16T01:50:36.381 INFO:teuthology.orchestra.run.smithi190.stdout:(28/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 16 MB/s | 3.1 MB 00:00 2024-09-16T01:50:36.401 INFO:teuthology.orchestra.run.smithi105.stdout:(18/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9 3.5 MB/s | 11 MB 00:03 2024-09-16T01:50:36.406 INFO:teuthology.orchestra.run.smithi190.stdout:(29/156): ceph-grafana-dashboards-19.1.1-326.g8 272 kB/s | 32 kB 00:00 2024-09-16T01:50:36.411 INFO:teuthology.orchestra.run.smithi137.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.0 MB/s | 135 kB 00:00 2024-09-16T01:50:36.428 INFO:teuthology.orchestra.run.smithi137.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 999 kB/s | 125 kB 00:00 2024-09-16T01:50:36.440 INFO:teuthology.orchestra.run.smithi190.stdout:(30/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 2.4 MB/s | 146 kB 00:00 2024-09-16T01:50:36.485 INFO:teuthology.orchestra.run.smithi105.stdout:(19/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 8.6 MB/s | 5.3 MB 00:00 2024-09-16T01:50:36.510 INFO:teuthology.orchestra.run.smithi105.stdout:(20/156): python3-ceph-argparse-19.1.1-326.g892 416 kB/s | 45 kB 00:00 2024-09-16T01:50:36.528 INFO:teuthology.orchestra.run.smithi137.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-09-16T01:50:36.560 INFO:teuthology.orchestra.run.smithi105.stdout:(21/156): python3-ceph-common-19.1.1-326.g8921c 1.8 MB/s | 136 kB 00:00 2024-09-16T01:50:36.627 INFO:teuthology.orchestra.run.smithi105.stdout:(22/156): python3-cephfs-19.1.1-326.g8921c8ef.e 1.3 MB/s | 161 kB 00:00 2024-09-16T01:50:36.702 INFO:teuthology.orchestra.run.smithi105.stdout:(23/156): python3-rados-19.1.1-326.g8921c8ef.el 2.2 MB/s | 320 kB 00:00 2024-09-16T01:50:36.753 INFO:teuthology.orchestra.run.smithi105.stdout:(24/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 2.3 MB/s | 300 kB 00:00 2024-09-16T01:50:36.778 INFO:teuthology.orchestra.run.smithi105.stdout:(25/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 1.3 MB/s | 99 kB 00:00 2024-09-16T01:50:36.803 INFO:teuthology.orchestra.run.smithi105.stdout:(26/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-16T01:50:36.807 INFO:teuthology.orchestra.run.smithi190.stdout:(31/156): ceph-mgr-diskprediction-local-19.1.1- 20 MB/s | 7.4 MB 00:00 2024-09-16T01:50:36.870 INFO:teuthology.orchestra.run.smithi105.stdout:(27/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 2.5 MB/s | 170 kB 00:00 2024-09-16T01:50:36.873 INFO:teuthology.orchestra.run.smithi190.stdout:(32/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 7.6 MB/s | 3.6 MB 00:00 2024-09-16T01:50:36.899 INFO:teuthology.orchestra.run.smithi190.stdout:(33/156): ceph-mgr-modules-core-19.1.1-326.g892 2.7 MB/s | 249 kB 00:00 2024-09-16T01:50:36.921 INFO:teuthology.orchestra.run.smithi105.stdout:(28/156): ceph-grafana-dashboards-19.1.1-326.g8 637 kB/s | 32 kB 00:00 2024-09-16T01:50:36.924 INFO:teuthology.orchestra.run.smithi190.stdout:(34/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 980 kB/s | 49 kB 00:00 2024-09-16T01:50:36.949 INFO:teuthology.orchestra.run.smithi190.stdout:(35/156): ceph-prometheus-alerts-19.1.1-326.g89 328 kB/s | 16 kB 00:00 2024-09-16T01:50:36.971 INFO:teuthology.orchestra.run.smithi105.stdout:(29/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 2.9 MB/s | 146 kB 00:00 2024-09-16T01:50:36.975 INFO:teuthology.orchestra.run.smithi190.stdout:(36/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 5.5 MB/s | 285 kB 00:00 2024-09-16T01:50:37.017 INFO:teuthology.orchestra.run.smithi190.stdout:(37/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 11 MB/s | 765 kB 00:00 2024-09-16T01:50:37.020 INFO:teuthology.orchestra.run.smithi137.stdout:(48/156): ceph-test-19.1.1-326.g8921c8ef.el9.x8 10 MB/s | 49 MB 00:04 2024-09-16T01:50:37.044 INFO:teuthology.orchestra.run.smithi190.stdout:(38/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 272 kB/s | 18 kB 00:00 2024-09-16T01:50:37.088 INFO:teuthology.orchestra.run.smithi105.stdout:(30/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 10 MB/s | 3.1 MB 00:00 2024-09-16T01:50:37.112 INFO:teuthology.orchestra.run.smithi137.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 1.0 MB/s | 689 kB 00:00 2024-09-16T01:50:37.162 INFO:teuthology.orchestra.run.smithi137.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 434 kB/s | 22 kB 00:00 2024-09-16T01:50:37.167 INFO:teuthology.orchestra.run.smithi190.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 531 kB/s | 80 kB 00:00 2024-09-16T01:50:37.188 INFO:teuthology.orchestra.run.smithi137.stdout:(51/156): boost-program-options-1.75.0-8.el9.x8 641 kB/s | 107 kB 00:00 2024-09-16T01:50:37.221 INFO:teuthology.orchestra.run.smithi137.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-09-16T01:50:37.246 INFO:teuthology.orchestra.run.smithi137.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 559 kB/s | 33 kB 00:00 2024-09-16T01:50:37.251 INFO:teuthology.orchestra.run.smithi190.stdout:(40/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.8 MB/s | 801 kB 00:00 2024-09-16T01:50:37.272 INFO:teuthology.orchestra.run.smithi137.stdout:(54/156): bison-3.7.4-5.el9.x86_64.rpm 1.2 MB/s | 943 kB 00:00 2024-09-16T01:50:37.276 INFO:teuthology.orchestra.run.smithi190.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.7 MB/s | 191 kB 00:00 2024-09-16T01:50:37.310 INFO:teuthology.orchestra.run.smithi190.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-09-16T01:50:37.330 INFO:teuthology.orchestra.run.smithi137.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 217 kB/s | 18 kB 00:00 2024-09-16T01:50:37.356 INFO:teuthology.orchestra.run.smithi137.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2024-09-16T01:50:37.361 INFO:teuthology.orchestra.run.smithi190.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-16T01:50:37.389 INFO:teuthology.orchestra.run.smithi137.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-16T01:50:37.414 INFO:teuthology.orchestra.run.smithi137.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 773 kB/s | 45 kB 00:00 2024-09-16T01:50:37.419 INFO:teuthology.orchestra.run.smithi190.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 8.6 MB/s | 1.2 MB 00:00 2024-09-16T01:50:37.445 INFO:teuthology.orchestra.run.smithi190.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-09-16T01:50:37.456 INFO:teuthology.orchestra.run.smithi137.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-09-16T01:50:37.470 INFO:teuthology.orchestra.run.smithi190.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-16T01:50:37.495 INFO:teuthology.orchestra.run.smithi190.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-16T01:50:37.507 INFO:teuthology.orchestra.run.smithi137.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 490 kB/s | 24 kB 00:00 2024-09-16T01:50:37.530 INFO:teuthology.orchestra.run.smithi105.stdout:(31/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 6.4 MB/s | 3.6 MB 00:00 2024-09-16T01:50:37.565 INFO:teuthology.orchestra.run.smithi137.stdout:(61/156): libtool-2.4.6-46.el9.x86_64.rpm 3.8 MB/s | 581 kB 00:00 2024-09-16T01:50:37.591 INFO:teuthology.orchestra.run.smithi137.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 568 kB/s | 47 kB 00:00 2024-09-16T01:50:37.605 INFO:teuthology.orchestra.run.smithi105.stdout:(32/156): ceph-mgr-modules-core-19.1.1-326.g892 3.3 MB/s | 249 kB 00:00 2024-09-16T01:50:37.624 INFO:teuthology.orchestra.run.smithi137.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-09-16T01:50:37.647 INFO:teuthology.orchestra.run.smithi105.stdout:(33/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 1.2 MB/s | 49 kB 00:00 2024-09-16T01:50:37.674 INFO:teuthology.orchestra.run.smithi137.stdout:(64/156): openblas-0.3.26-2.el9.x86_64.rpm 770 kB/s | 38 kB 00:00 2024-09-16T01:50:37.689 INFO:teuthology.orchestra.run.smithi105.stdout:(34/156): ceph-prometheus-alerts-19.1.1-326.g89 395 kB/s | 16 kB 00:00 2024-09-16T01:50:37.700 INFO:teuthology.orchestra.run.smithi137.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-16T01:50:37.773 INFO:teuthology.orchestra.run.smithi105.stdout:(35/156): ceph-mgr-diskprediction-local-19.1.1- 11 MB/s | 7.4 MB 00:00 2024-09-16T01:50:37.806 INFO:teuthology.orchestra.run.smithi105.stdout:(36/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 2.4 MB/s | 285 kB 00:00 2024-09-16T01:50:37.840 INFO:teuthology.orchestra.run.smithi105.stdout:(37/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 549 kB/s | 18 kB 00:00 2024-09-16T01:50:37.874 INFO:teuthology.orchestra.run.smithi105.stdout:(38/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 7.4 MB/s | 765 kB 00:00 2024-09-16T01:50:37.982 INFO:teuthology.orchestra.run.smithi105.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 562 kB/s | 80 kB 00:00 2024-09-16T01:50:38.066 INFO:teuthology.orchestra.run.smithi105.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-16T01:50:38.100 INFO:teuthology.orchestra.run.smithi105.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.5 MB/s | 801 kB 00:00 2024-09-16T01:50:38.133 INFO:teuthology.orchestra.run.smithi105.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-09-16T01:50:38.184 INFO:teuthology.orchestra.run.smithi105.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-16T01:50:38.217 INFO:teuthology.orchestra.run.smithi105.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 10 MB/s | 1.2 MB 00:00 2024-09-16T01:50:38.243 INFO:teuthology.orchestra.run.smithi105.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-16T01:50:38.268 INFO:teuthology.orchestra.run.smithi105.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-16T01:50:38.293 INFO:teuthology.orchestra.run.smithi105.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-16T01:50:38.327 INFO:teuthology.orchestra.run.smithi105.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 11 MB/s | 689 kB 00:00 2024-09-16T01:50:38.337 INFO:teuthology.orchestra.run.smithi190.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 795 kB/s | 689 kB 00:00 2024-09-16T01:50:38.377 INFO:teuthology.orchestra.run.smithi105.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-09-16T01:50:38.387 INFO:teuthology.orchestra.run.smithi190.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 1.0 MB/s | 943 kB 00:00 2024-09-16T01:50:38.402 INFO:teuthology.orchestra.run.smithi105.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 8.5 MB/s | 943 kB 00:00 2024-09-16T01:50:38.425 INFO:teuthology.orchestra.run.smithi137.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.5 MB/s | 3.0 MB 00:01 2024-09-16T01:50:38.427 INFO:teuthology.orchestra.run.smithi105.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 433 kB/s | 22 kB 00:00 2024-09-16T01:50:38.461 INFO:teuthology.orchestra.run.smithi105.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-09-16T01:50:38.486 INFO:teuthology.orchestra.run.smithi105.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 558 kB/s | 33 kB 00:00 2024-09-16T01:50:38.528 INFO:teuthology.orchestra.run.smithi105.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 437 kB/s | 18 kB 00:00 2024-09-16T01:50:38.587 INFO:teuthology.orchestra.run.smithi105.stdout:(55/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-09-16T01:50:38.612 INFO:teuthology.orchestra.run.smithi105.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2024-09-16T01:50:38.638 INFO:teuthology.orchestra.run.smithi105.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-16T01:50:38.663 INFO:teuthology.orchestra.run.smithi105.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 906 kB/s | 45 kB 00:00 2024-09-16T01:50:38.705 INFO:teuthology.orchestra.run.smithi105.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-09-16T01:50:38.739 INFO:teuthology.orchestra.run.smithi105.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 7.5 MB/s | 581 kB 00:00 2024-09-16T01:50:38.765 INFO:teuthology.orchestra.run.smithi105.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 417 kB/s | 24 kB 00:00 2024-09-16T01:50:38.789 INFO:teuthology.orchestra.run.smithi105.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-09-16T01:50:38.815 INFO:teuthology.orchestra.run.smithi105.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-16T01:50:38.848 INFO:teuthology.orchestra.run.smithi105.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-09-16T01:50:38.874 INFO:teuthology.orchestra.run.smithi105.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 659 kB/s | 38 kB 00:00 2024-09-16T01:50:38.913 INFO:teuthology.orchestra.run.smithi190.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 41 kB/s | 22 kB 00:00 2024-09-16T01:50:38.982 INFO:teuthology.orchestra.run.smithi105.stdout:(66/156): openblas-openmp-0.3.26-2.el9.x86_64.r 37 MB/s | 4.9 MB 00:00 2024-09-16T01:50:38.996 INFO:teuthology.orchestra.run.smithi190.stdout:(51/156): boost-program-options-1.75.0-8.el9.x8 162 kB/s | 107 kB 00:00 2024-09-16T01:50:39.041 INFO:teuthology.orchestra.run.smithi105.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 27 MB/s | 4.4 MB 00:00 2024-09-16T01:50:39.083 INFO:teuthology.orchestra.run.smithi105.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-16T01:50:39.125 INFO:teuthology.orchestra.run.smithi105.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 11 MB/s | 910 kB 00:00 2024-09-16T01:50:39.150 INFO:teuthology.orchestra.run.smithi105.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-16T01:50:39.209 INFO:teuthology.orchestra.run.smithi137.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 2.0 MB/s | 1.6 MB 00:00 2024-09-16T01:50:39.209 INFO:teuthology.orchestra.run.smithi105.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-09-16T01:50:39.284 INFO:teuthology.orchestra.run.smithi105.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 36 MB/s | 5.8 MB 00:00 2024-09-16T01:50:39.309 INFO:teuthology.orchestra.run.smithi105.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 874 kB/s | 87 kB 00:00 2024-09-16T01:50:39.360 INFO:teuthology.orchestra.run.smithi105.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-16T01:50:39.435 INFO:teuthology.orchestra.run.smithi105.stdout:(75/156): python3-babel-2.9.1-2.el9.noarch.rpm 40 MB/s | 6.0 MB 00:00 2024-09-16T01:50:39.460 INFO:teuthology.orchestra.run.smithi105.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-16T01:50:39.486 INFO:teuthology.orchestra.run.smithi105.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-16T01:50:39.511 INFO:teuthology.orchestra.run.smithi105.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-09-16T01:50:39.536 INFO:teuthology.orchestra.run.smithi105.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-09-16T01:50:39.555 INFO:teuthology.orchestra.run.smithi190.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 59 kB/s | 33 kB 00:00 2024-09-16T01:50:39.561 INFO:teuthology.orchestra.run.smithi105.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-16T01:50:39.587 INFO:teuthology.orchestra.run.smithi105.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 694 kB/s | 35 kB 00:00 2024-09-16T01:50:39.637 INFO:teuthology.orchestra.run.smithi105.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-09-16T01:50:39.692 INFO:teuthology.orchestra.run.smithi137.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 1.8 MB/s | 910 kB 00:00 2024-09-16T01:50:39.712 INFO:teuthology.orchestra.run.smithi105.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 5.1 MB 00:00 2024-09-16T01:50:39.722 INFO:teuthology.orchestra.run.smithi190.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 388 kB/s | 313 kB 00:00 2024-09-16T01:50:39.738 INFO:teuthology.orchestra.run.smithi105.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-09-16T01:50:39.763 INFO:teuthology.orchestra.run.smithi105.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-16T01:50:39.788 INFO:teuthology.orchestra.run.smithi105.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 840 kB/s | 42 kB 00:00 2024-09-16T01:50:39.809 INFO:teuthology.orchestra.run.smithi137.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 1.2 MB/s | 137 kB 00:00 2024-09-16T01:50:39.813 INFO:teuthology.orchestra.run.smithi105.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-09-16T01:50:39.840 INFO:teuthology.orchestra.run.smithi105.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-16T01:50:39.864 INFO:teuthology.orchestra.run.smithi105.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-16T01:50:39.890 INFO:teuthology.orchestra.run.smithi105.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-16T01:50:39.901 INFO:teuthology.orchestra.run.smithi137.stdout:(70/156): openblas-openmp-0.3.26-2.el9.x86_64.r 2.2 MB/s | 4.9 MB 00:02 2024-09-16T01:50:39.915 INFO:teuthology.orchestra.run.smithi105.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-16T01:50:39.957 INFO:teuthology.orchestra.run.smithi105.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-09-16T01:50:39.959 INFO:teuthology.orchestra.run.smithi137.stdout:(71/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 2.0 MB/s | 4.4 MB 00:02 2024-09-16T01:50:39.999 INFO:teuthology.orchestra.run.smithi105.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-09-16T01:50:40.043 INFO:teuthology.orchestra.run.smithi137.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-09-16T01:50:40.049 INFO:teuthology.orchestra.run.smithi105.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-16T01:50:40.091 INFO:teuthology.orchestra.run.smithi105.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 865 kB/s | 36 kB 00:00 2024-09-16T01:50:40.250 INFO:teuthology.orchestra.run.smithi105.stdout:(96/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 45 MB/s | 16 MB 00:00 2024-09-16T01:50:40.255 INFO:teuthology.orchestra.run.smithi190.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 34 kB/s | 18 kB 00:00 2024-09-16T01:50:40.275 INFO:teuthology.orchestra.run.smithi105.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 258 kB/s | 47 kB 00:00 2024-09-16T01:50:40.301 INFO:teuthology.orchestra.run.smithi105.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-16T01:50:40.326 INFO:teuthology.orchestra.run.smithi105.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-16T01:50:40.351 INFO:teuthology.orchestra.run.smithi105.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-16T01:50:40.376 INFO:teuthology.orchestra.run.smithi105.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-16T01:50:40.402 INFO:teuthology.orchestra.run.smithi105.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 891 kB/s | 45 kB 00:00 2024-09-16T01:50:40.443 INFO:teuthology.orchestra.run.smithi137.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 1.9 MB/s | 1.0 MB 00:00 2024-09-16T01:50:40.493 INFO:teuthology.orchestra.run.smithi137.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-16T01:50:40.636 INFO:teuthology.orchestra.run.smithi137.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 1.7 MB/s | 245 kB 00:00 2024-09-16T01:50:40.656 INFO:teuthology.orchestra.run.smithi190.stdout:(55/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.7 MB/s | 3.0 MB 00:01 2024-09-16T01:50:40.773 INFO:teuthology.orchestra.run.smithi190.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 22 kB/s | 11 kB 00:00 2024-09-16T01:50:40.794 INFO:teuthology.orchestra.run.smithi137.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 1.5 MB/s | 248 kB 00:00 2024-09-16T01:50:40.835 INFO:teuthology.orchestra.run.smithi105.stdout:(103/156): ceph-test-19.1.1-326.g8921c8ef.el9.x 7.7 MB/s | 49 MB 00:06 2024-09-16T01:50:40.844 INFO:teuthology.orchestra.run.smithi137.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 514 kB/s | 26 kB 00:00 2024-09-16T01:50:40.886 INFO:teuthology.orchestra.run.smithi105.stdout:(104/156): lua-devel-5.4.4-4.el9.x86_64.rpm 44 kB/s | 22 kB 00:00 2024-09-16T01:50:40.903 INFO:teuthology.orchestra.run.smithi137.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 328 kB/s | 19 kB 00:00 2024-09-16T01:50:40.927 INFO:teuthology.orchestra.run.smithi105.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 5.8 MB/s | 548 kB 00:00 2024-09-16T01:50:40.953 INFO:teuthology.orchestra.run.smithi105.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 841 kB/s | 19 kB 00:00 2024-09-16T01:50:40.986 INFO:teuthology.orchestra.run.smithi105.stdout:(107/156): gperftools-libs-2.9.1-3.el9.x86_64.r 3.0 MB/s | 308 kB 00:00 2024-09-16T01:50:41.003 INFO:teuthology.orchestra.run.smithi105.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.6 MB/s | 26 kB 00:00 2024-09-16T01:50:41.011 INFO:teuthology.orchestra.run.smithi137.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.6 MB/s | 172 kB 00:00 2024-09-16T01:50:41.020 INFO:teuthology.orchestra.run.smithi105.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 2.8 MB/s | 49 kB 00:00 2024-09-16T01:50:41.046 INFO:teuthology.orchestra.run.smithi105.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-16T01:50:41.062 INFO:teuthology.orchestra.run.smithi137.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 695 kB/s | 35 kB 00:00 2024-09-16T01:50:41.104 INFO:teuthology.orchestra.run.smithi105.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-09-16T01:50:41.146 INFO:teuthology.orchestra.run.smithi105.stdout:(112/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.1 MB/s | 863 kB 00:00 2024-09-16T01:50:41.197 INFO:teuthology.orchestra.run.smithi105.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-09-16T01:50:41.222 INFO:teuthology.orchestra.run.smithi105.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.8 MB/s | 43 kB 00:00 2024-09-16T01:50:41.247 INFO:teuthology.orchestra.run.smithi105.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-16T01:50:41.272 INFO:teuthology.orchestra.run.smithi105.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 567 kB/s | 14 kB 00:00 2024-09-16T01:50:41.306 INFO:teuthology.orchestra.run.smithi105.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 5.0 MB/s | 172 kB 00:00 2024-09-16T01:50:41.331 INFO:teuthology.orchestra.run.smithi105.stdout:(118/156): python3-asyncssh-2.13.2-5.el9.noarch 2.9 MB/s | 548 kB 00:00 2024-09-16T01:50:41.348 INFO:teuthology.orchestra.run.smithi190.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 79 kB/s | 45 kB 00:00 2024-09-16T01:50:41.365 INFO:teuthology.orchestra.run.smithi105.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-16T01:50:41.373 INFO:teuthology.orchestra.run.smithi190.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 224 kB/s | 160 kB 00:00 2024-09-16T01:50:41.390 INFO:teuthology.orchestra.run.smithi105.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-16T01:50:41.415 INFO:teuthology.orchestra.run.smithi105.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-16T01:50:41.449 INFO:teuthology.orchestra.run.smithi105.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-16T01:50:41.474 INFO:teuthology.orchestra.run.smithi105.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-16T01:50:41.500 INFO:teuthology.orchestra.run.smithi105.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 701 kB/s | 18 kB 00:00 2024-09-16T01:50:41.559 INFO:teuthology.orchestra.run.smithi105.stdout:(125/156): libarrow-9.0.0-11.el9.x86_64.rpm 7.3 MB/s | 4.4 MB 00:00 2024-09-16T01:50:41.584 INFO:teuthology.orchestra.run.smithi105.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 277 kB/s | 23 kB 00:00 2024-09-16T01:50:41.609 INFO:teuthology.orchestra.run.smithi105.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-16T01:50:41.635 INFO:teuthology.orchestra.run.smithi105.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-16T01:50:41.659 INFO:teuthology.orchestra.run.smithi105.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-16T01:50:41.685 INFO:teuthology.orchestra.run.smithi105.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-16T01:50:41.710 INFO:teuthology.orchestra.run.smithi105.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-16T01:50:41.753 INFO:teuthology.orchestra.run.smithi105.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 7.1 MB/s | 1.0 MB 00:00 2024-09-16T01:50:41.777 INFO:teuthology.orchestra.run.smithi105.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-09-16T01:50:41.803 INFO:teuthology.orchestra.run.smithi105.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 328 kB/s | 16 kB 00:00 2024-09-16T01:50:41.828 INFO:teuthology.orchestra.run.smithi105.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-16T01:50:41.853 INFO:teuthology.orchestra.run.smithi105.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-16T01:50:41.879 INFO:teuthology.orchestra.run.smithi105.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-09-16T01:50:41.921 INFO:teuthology.orchestra.run.smithi105.stdout:(138/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.8 MB/s | 2.0 MB 00:00 2024-09-16T01:50:41.946 INFO:teuthology.orchestra.run.smithi105.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 885 kB/s | 59 kB 00:00 2024-09-16T01:50:41.971 INFO:teuthology.orchestra.run.smithi105.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-09-16T01:50:41.997 INFO:teuthology.orchestra.run.smithi105.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-09-16T01:50:42.022 INFO:teuthology.orchestra.run.smithi105.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 985 kB/s | 74 kB 00:00 2024-09-16T01:50:42.047 INFO:teuthology.orchestra.run.smithi105.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-16T01:50:42.072 INFO:teuthology.orchestra.run.smithi105.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 801 kB/s | 20 kB 00:00 2024-09-16T01:50:42.106 INFO:teuthology.orchestra.run.smithi105.stdout:(145/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-09-16T01:50:42.131 INFO:teuthology.orchestra.run.smithi105.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-16T01:50:42.165 INFO:teuthology.orchestra.run.smithi105.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-09-16T01:50:42.216 INFO:teuthology.orchestra.run.smithi105.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-16T01:50:42.265 INFO:teuthology.orchestra.run.smithi190.stdout:(59/156): ceph-test-19.1.1-326.g8921c8ef.el9.x8 6.4 MB/s | 49 MB 00:07 2024-09-16T01:50:42.291 INFO:teuthology.orchestra.run.smithi105.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 31 MB/s | 2.3 MB 00:00 2024-09-16T01:50:42.366 INFO:teuthology.orchestra.run.smithi105.stdout:(150/156): NetworkManager-libnm-1.49.90-1.el9.x 24 MB/s | 1.8 MB 00:00 2024-09-16T01:50:42.408 INFO:teuthology.orchestra.run.smithi105.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 982 kB/s | 41 kB 00:00 2024-09-16T01:50:42.445 INFO:teuthology.orchestra.run.smithi137.stdout:(81/156): postgresql-server-13.16-1.el9.x86_64. 2.2 MB/s | 5.8 MB 00:02 2024-09-16T01:50:42.458 INFO:teuthology.orchestra.run.smithi105.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 4.9 MB/s | 249 kB 00:00 2024-09-16T01:50:42.534 INFO:teuthology.orchestra.run.smithi105.stdout:(153/156): openssl-3.2.2-6.el9.x86_64.rpm 18 MB/s | 1.4 MB 00:00 2024-09-16T01:50:42.588 INFO:teuthology.orchestra.run.smithi137.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 2.3 MB/s | 6.0 MB 00:02 2024-09-16T01:50:42.609 INFO:teuthology.orchestra.run.smithi105.stdout:(154/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 7.1 MB/s | 3.2 MB 00:00 2024-09-16T01:50:42.629 INFO:teuthology.orchestra.run.smithi137.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 250 kB 00:00 2024-09-16T01:50:42.660 INFO:teuthology.orchestra.run.smithi105.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 19 MB/s | 2.4 MB 00:00 2024-09-16T01:50:42.730 INFO:teuthology.orchestra.run.smithi137.stdout:(84/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.1 MB/s | 5.1 MB 00:01 2024-09-16T01:50:42.755 INFO:teuthology.orchestra.run.smithi137.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 616 kB/s | 77 kB 00:00 2024-09-16T01:50:42.780 INFO:teuthology.orchestra.run.smithi137.stdout:(86/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.1 MB/s | 222 kB 00:00 2024-09-16T01:50:42.805 INFO:teuthology.orchestra.run.smithi137.stdout:(87/156): python3-prettytable-0.7.2-27.el9.noar 562 kB/s | 42 kB 00:00 2024-09-16T01:50:42.815 INFO:teuthology.orchestra.run.smithi190.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 44 kB/s | 24 kB 00:00 2024-09-16T01:50:42.835 INFO:teuthology.orchestra.run.smithi105.stdout:(156/156): librados2-19.1.1-326.g8921c8ef.el9.x 4.9 MB/s | 3.4 MB 00:00 2024-09-16T01:50:42.842 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2024-09-16T01:50:42.842 INFO:teuthology.orchestra.run.smithi105.stdout:Total 18 MB/s | 226 MB 00:12 2024-09-16T01:50:42.864 INFO:teuthology.orchestra.run.smithi137.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-09-16T01:50:42.914 INFO:teuthology.orchestra.run.smithi137.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-16T01:50:42.939 INFO:teuthology.orchestra.run.smithi137.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 678 kB/s | 51 kB 00:00 2024-09-16T01:50:42.965 INFO:teuthology.orchestra.run.smithi137.stdout:(91/156): python3-protobuf-3.14.0-13.el9.noarch 1.3 MB/s | 269 kB 00:00 2024-09-16T01:50:42.990 INFO:teuthology.orchestra.run.smithi137.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 711 kB/s | 54 kB 00:00 2024-09-16T01:50:43.015 INFO:teuthology.orchestra.run.smithi137.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 828 kB/s | 42 kB 00:00 2024-09-16T01:50:43.040 INFO:teuthology.orchestra.run.smithi137.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 877 kB/s | 44 kB 00:00 2024-09-16T01:50:43.091 INFO:teuthology.orchestra.run.smithi137.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 717 kB/s | 36 kB 00:00 2024-09-16T01:50:43.149 INFO:teuthology.orchestra.run.smithi137.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 812 kB/s | 47 kB 00:00 2024-09-16T01:50:43.183 INFO:teuthology.orchestra.run.smithi137.stdout:(97/156): qatlib-24.02.0-1.el9.x86_64.rpm 1.3 MB/s | 221 kB 00:00 2024-09-16T01:50:43.234 INFO:teuthology.orchestra.run.smithi137.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-16T01:50:43.282 INFO:teuthology.orchestra.run.smithi190.stdout:(61/156): librdkafka-1.6.1-102.el9.x86_64.rpm 343 kB/s | 662 kB 00:01 2024-09-16T01:50:43.349 INFO:teuthology.orchestra.run.smithi190.stdout:(62/156): libtool-2.4.6-46.el9.x86_64.rpm 294 kB/s | 581 kB 00:01 2024-09-16T01:50:43.391 INFO:teuthology.orchestra.run.smithi190.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 82 kB/s | 47 kB 00:00 2024-09-16T01:50:43.401 INFO:teuthology.orchestra.run.smithi137.stdout:(99/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-09-16T01:50:43.426 INFO:teuthology.orchestra.run.smithi137.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 499 kB/s | 96 kB 00:00 2024-09-16T01:50:43.451 INFO:teuthology.orchestra.run.smithi137.stdout:(101/156): socat-1.7.4.1-6.el9.x86_64.rpm 1.0 MB/s | 304 kB 00:00 2024-09-16T01:50:43.476 INFO:teuthology.orchestra.run.smithi137.stdout:(102/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 847 kB/s | 64 kB 00:00 2024-09-16T01:50:43.501 INFO:teuthology.orchestra.run.smithi137.stdout:(103/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 594 kB/s | 45 kB 00:00 2024-09-16T01:50:43.552 INFO:teuthology.orchestra.run.smithi137.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-16T01:50:43.577 INFO:teuthology.orchestra.run.smithi137.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 12 MB/s | 308 kB 00:00 2024-09-16T01:50:43.602 INFO:teuthology.orchestra.run.smithi137.stdout:(106/156): lua-devel-5.4.4-4.el9.x86_64.rpm 147 kB/s | 22 kB 00:00 2024-09-16T01:50:43.628 INFO:teuthology.orchestra.run.smithi137.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 387 kB/s | 19 kB 00:00 2024-09-16T01:50:43.661 INFO:teuthology.orchestra.run.smithi137.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-16T01:50:43.687 INFO:teuthology.orchestra.run.smithi137.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-09-16T01:50:43.704 INFO:teuthology.orchestra.run.smithi137.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 4.0 MB/s | 67 kB 00:00 2024-09-16T01:50:43.729 INFO:teuthology.orchestra.run.smithi137.stdout:(111/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.3 MB/s | 863 kB 00:00 2024-09-16T01:50:43.763 INFO:teuthology.orchestra.run.smithi137.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-09-16T01:50:43.804 INFO:teuthology.orchestra.run.smithi137.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 14 MB/s | 548 kB 00:00 2024-09-16T01:50:43.973 INFO:teuthology.orchestra.run.smithi190.stdout:(64/156): openblas-0.3.26-2.el9.x86_64.rpm 69 kB/s | 38 kB 00:00 2024-09-16T01:50:43.973 INFO:teuthology.orchestra.run.smithi137.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-16T01:50:43.973 INFO:teuthology.orchestra.run.smithi137.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-16T01:50:43.973 INFO:teuthology.orchestra.run.smithi137.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 862 kB/s | 14 kB 00:00 2024-09-16T01:50:43.973 INFO:teuthology.orchestra.run.smithi137.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 5.0 MB/s | 172 kB 00:00 2024-09-16T01:50:43.974 INFO:teuthology.orchestra.run.smithi137.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 8.4 MB/s | 358 kB 00:00 2024-09-16T01:50:43.981 INFO:teuthology.orchestra.run.smithi137.stdout:(119/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.3 MB/s | 839 kB 00:00 2024-09-16T01:50:44.048 INFO:teuthology.orchestra.run.smithi137.stdout:(120/156): libarrow-9.0.0-11.el9.x86_64.rpm 9.9 MB/s | 4.4 MB 00:00 2024-09-16T01:50:44.067 INFO:teuthology.orchestra.run.smithi190.stdout:(65/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 373 kB/s | 292 kB 00:00 2024-09-16T01:50:44.073 INFO:teuthology.orchestra.run.smithi137.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-09-16T01:50:44.075 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2024-09-16T01:50:44.098 INFO:teuthology.orchestra.run.smithi137.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 1.8 MB/s | 220 kB 00:00 2024-09-16T01:50:44.123 INFO:teuthology.orchestra.run.smithi137.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-09-16T01:50:44.133 INFO:teuthology.orchestra.run.smithi190.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 240 kB/s | 188 kB 00:00 2024-09-16T01:50:44.149 INFO:teuthology.orchestra.run.smithi137.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-09-16T01:50:44.174 INFO:teuthology.orchestra.run.smithi137.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-09-16T01:50:44.208 INFO:teuthology.orchestra.run.smithi137.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 393 kB/s | 23 kB 00:00 2024-09-16T01:50:44.227 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2024-09-16T01:50:44.227 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2024-09-16T01:50:44.233 INFO:teuthology.orchestra.run.smithi137.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 331 kB/s | 19 kB 00:00 2024-09-16T01:50:44.258 INFO:teuthology.orchestra.run.smithi137.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-09-16T01:50:44.284 INFO:teuthology.orchestra.run.smithi137.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-16T01:50:44.309 INFO:teuthology.orchestra.run.smithi137.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-16T01:50:44.335 INFO:teuthology.orchestra.run.smithi137.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-16T01:50:44.368 INFO:teuthology.orchestra.run.smithi137.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-16T01:50:44.403 INFO:teuthology.orchestra.run.smithi137.stdout:(133/156): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-09-16T01:50:44.427 INFO:teuthology.orchestra.run.smithi137.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-09-16T01:50:44.453 INFO:teuthology.orchestra.run.smithi137.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-16T01:50:44.478 INFO:teuthology.orchestra.run.smithi137.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-16T01:50:44.503 INFO:teuthology.orchestra.run.smithi137.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-09-16T01:50:44.528 INFO:teuthology.orchestra.run.smithi137.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-16T01:50:44.554 INFO:teuthology.orchestra.run.smithi137.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-16T01:50:44.579 INFO:teuthology.orchestra.run.smithi137.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-09-16T01:50:44.621 INFO:teuthology.orchestra.run.smithi137.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.1 MB/s | 74 kB 00:00 2024-09-16T01:50:44.646 INFO:teuthology.orchestra.run.smithi137.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-16T01:50:44.688 INFO:teuthology.orchestra.run.smithi137.stdout:(143/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.2 MB/s | 2.0 MB 00:00 2024-09-16T01:50:44.713 INFO:teuthology.orchestra.run.smithi137.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-09-16T01:50:44.738 INFO:teuthology.orchestra.run.smithi137.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-16T01:50:44.763 INFO:teuthology.orchestra.run.smithi137.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-16T01:50:44.789 INFO:teuthology.orchestra.run.smithi137.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-09-16T01:50:44.839 INFO:teuthology.orchestra.run.smithi137.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-16T01:50:45.048 INFO:teuthology.orchestra.run.smithi137.stdout:(149/156): librados2-19.1.1-326.g8921c8ef.el9.x 12 MB/s | 3.4 MB 00:00 2024-09-16T01:50:45.050 INFO:teuthology.orchestra.run.smithi190.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 4.5 MB/s | 4.9 MB 00:01 2024-09-16T01:50:45.101 INFO:teuthology.orchestra.run.smithi190.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 1.6 MB/s | 1.6 MB 00:00 2024-09-16T01:50:45.123 INFO:teuthology.orchestra.run.smithi137.stdout:(150/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 9.5 MB/s | 3.2 MB 00:00 2024-09-16T01:50:45.176 INFO:teuthology.orchestra.run.smithi190.stdout:(69/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 4.0 MB/s | 4.4 MB 00:01 2024-09-16T01:50:45.407 INFO:teuthology.orchestra.run.smithi137.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 144 kB/s | 41 kB 00:00 2024-09-16T01:50:45.682 INFO:teuthology.orchestra.run.smithi137.stdout:(152/156): NetworkManager-1.49.90-1.el9.x86_64. 2.8 MB/s | 2.3 MB 00:00 2024-09-16T01:50:45.707 INFO:teuthology.orchestra.run.smithi137.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 829 kB/s | 249 kB 00:00 2024-09-16T01:50:45.818 INFO:teuthology.orchestra.run.smithi190.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 192 kB/s | 137 kB 00:00 2024-09-16T01:50:45.825 INFO:teuthology.orchestra.run.smithi137.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 9.5 MB/s | 1.4 MB 00:00 2024-09-16T01:50:45.875 INFO:teuthology.orchestra.run.smithi137.stdout:(155/156): NetworkManager-libnm-1.49.90-1.el9.x 2.2 MB/s | 1.8 MB 00:00 2024-09-16T01:50:45.993 INFO:teuthology.orchestra.run.smithi190.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 965 kB/s | 910 kB 00:00 2024-09-16T01:50:46.117 INFO:teuthology.orchestra.run.smithi137.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 5.8 MB/s | 2.4 MB 00:00 2024-09-16T01:50:46.123 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-09-16T01:50:46.123 INFO:teuthology.orchestra.run.smithi137.stdout:Total 15 MB/s | 226 MB 00:15 2024-09-16T01:50:46.352 INFO:teuthology.orchestra.run.smithi190.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 4.9 MB/s | 5.8 MB 00:01 2024-09-16T01:50:46.421 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2024-09-16T01:50:46.422 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2024-09-16T01:50:46.702 INFO:teuthology.orchestra.run.smithi190.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 123 kB/s | 87 kB 00:00 2024-09-16T01:50:46.861 INFO:teuthology.orchestra.run.smithi190.stdout:(74/156): protobuf-3.14.0-13.el9.x86_64.rpm 991 kB/s | 1.0 MB 00:01 2024-09-16T01:50:47.369 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-09-16T01:50:47.369 INFO:teuthology.orchestra.run.smithi190.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 92 kB/s | 61 kB 00:00 2024-09-16T01:50:47.523 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-09-16T01:50:47.523 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-09-16T01:50:47.628 INFO:teuthology.orchestra.run.smithi190.stdout:(76/156): python3-babel-2.9.1-2.el9.noarch.rpm 4.7 MB/s | 6.0 MB 00:01 2024-09-16T01:50:47.670 INFO:teuthology.orchestra.run.smithi190.stdout:(77/156): python3-devel-3.9.19-8.el9.x86_64.rpm 303 kB/s | 245 kB 00:00 2024-09-16T01:50:48.171 INFO:teuthology.orchestra.run.smithi190.stdout:(78/156): python3-jinja2-2.11.3-6.el9.noarch.rp 310 kB/s | 248 kB 00:00 2024-09-16T01:50:48.196 INFO:teuthology.orchestra.run.smithi190.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 36 kB/s | 19 kB 00:00 2024-09-16T01:50:48.221 INFO:teuthology.orchestra.run.smithi190.stdout:(80/156): python3-jsonpatch-1.21-16.el9.noarch. 44 kB/s | 26 kB 00:00 2024-09-16T01:50:48.746 INFO:teuthology.orchestra.run.smithi190.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 63 kB/s | 35 kB 00:00 2024-09-16T01:50:48.947 INFO:teuthology.orchestra.run.smithi190.stdout:(82/156): python3-mako-1.1.4-6.el9.noarch.rpm 222 kB/s | 172 kB 00:00 2024-09-16T01:50:49.322 INFO:teuthology.orchestra.run.smithi190.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.6 MB/s | 5.1 MB 00:01 2024-09-16T01:50:49.581 INFO:teuthology.orchestra.run.smithi190.stdout:(84/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 301 kB/s | 250 kB 00:00 2024-09-16T01:50:49.714 INFO:teuthology.orchestra.run.smithi190.stdout:(85/156): python3-oauthlib-3.1.1-5.el9.noarch.r 290 kB/s | 222 kB 00:00 2024-09-16T01:50:49.749 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-09-16T01:50:49.750 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-09-16T01:50:49.981 INFO:teuthology.orchestra.run.smithi190.stdout:(86/156): python3-packaging-20.9-5.el9.noarch.r 117 kB/s | 77 kB 00:00 2024-09-16T01:50:50.156 INFO:teuthology.orchestra.run.smithi190.stdout:(87/156): python3-prettytable-0.7.2-27.el9.noar 73 kB/s | 42 kB 00:00 2024-09-16T01:50:50.490 INFO:teuthology.orchestra.run.smithi190.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 347 kB/s | 269 kB 00:00 2024-09-16T01:50:50.690 INFO:teuthology.orchestra.run.smithi190.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 224 kB/s | 159 kB 00:00 2024-09-16T01:50:50.924 INFO:teuthology.orchestra.run.smithi190.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 363 kB/s | 279 kB 00:00 2024-09-16T01:50:51.090 INFO:teuthology.orchestra.run.smithi190.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 85 kB/s | 51 kB 00:00 2024-09-16T01:50:51.324 INFO:teuthology.orchestra.run.smithi190.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 85 kB/s | 54 kB 00:00 2024-09-16T01:50:51.683 INFO:teuthology.orchestra.run.smithi190.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 70 kB/s | 42 kB 00:00 2024-09-16T01:50:51.950 INFO:teuthology.orchestra.run.smithi190.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 71 kB/s | 44 kB 00:00 2024-09-16T01:50:52.534 INFO:teuthology.orchestra.run.smithi190.stdout:(95/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-09-16T01:50:52.626 INFO:teuthology.orchestra.run.smithi190.stdout:(96/156): qatlib-24.02.0-1.el9.x86_64.rpm 234 kB/s | 221 kB 00:00 2024-09-16T01:50:52.751 INFO:teuthology.orchestra.run.smithi190.stdout:(97/156): qatlib-service-24.02.0-1.el9.x86_64.r 45 kB/s | 36 kB 00:00 2024-09-16T01:50:52.930 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2024-09-16T01:50:53.032 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-16T01:50:53.070 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-16T01:50:53.132 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-16T01:50:53.173 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-16T01:50:53.176 INFO:teuthology.orchestra.run.smithi190.stdout:(98/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 74 kB/s | 47 kB 00:00 2024-09-16T01:50:53.199 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-16T01:50:53.254 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-16T01:50:53.368 INFO:teuthology.orchestra.run.smithi190.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 94 kB/s | 58 kB 00:00 2024-09-16T01:50:53.402 INFO:teuthology.orchestra.run.smithi190.stdout:(100/156): socat-1.7.4.1-6.el9.x86_64.rpm 392 kB/s | 304 kB 00:00 2024-09-16T01:50:53.590 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-16T01:50:53.717 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T01:50:53.847 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T01:50:53.852 INFO:teuthology.orchestra.run.smithi190.stdout:(101/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 142 kB/s | 96 kB 00:00 2024-09-16T01:50:53.876 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T01:50:53.967 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T01:50:53.986 INFO:teuthology.orchestra.run.smithi190.stdout:(102/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 103 kB/s | 64 kB 00:00 2024-09-16T01:50:53.997 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T01:50:54.029 INFO:teuthology.orchestra.run.smithi190.stdout:(103/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 72 kB/s | 45 kB 00:00 2024-09-16T01:50:54.086 INFO:teuthology.orchestra.run.smithi190.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-16T01:50:54.120 INFO:teuthology.orchestra.run.smithi190.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 9.0 MB/s | 308 kB 00:00 2024-09-16T01:50:54.145 INFO:teuthology.orchestra.run.smithi190.stdout:(106/156): lua-devel-5.4.4-4.el9.x86_64.rpm 76 kB/s | 22 kB 00:00 2024-09-16T01:50:54.170 INFO:teuthology.orchestra.run.smithi190.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 386 kB/s | 19 kB 00:00 2024-09-16T01:50:54.177 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-16T01:50:54.204 INFO:teuthology.orchestra.run.smithi190.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-16T01:50:54.229 INFO:teuthology.orchestra.run.smithi190.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-09-16T01:50:54.244 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-16T01:50:54.254 INFO:teuthology.orchestra.run.smithi190.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-16T01:50:54.271 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T01:50:54.288 INFO:teuthology.orchestra.run.smithi190.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2024-09-16T01:50:54.330 INFO:teuthology.orchestra.run.smithi190.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-09-16T01:50:54.375 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T01:50:54.380 INFO:teuthology.orchestra.run.smithi190.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-09-16T01:50:54.397 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T01:50:54.406 INFO:teuthology.orchestra.run.smithi190.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-16T01:50:54.431 INFO:teuthology.orchestra.run.smithi190.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-16T01:50:54.456 INFO:teuthology.orchestra.run.smithi190.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 566 kB/s | 14 kB 00:00 2024-09-16T01:50:54.460 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T01:50:54.482 INFO:teuthology.orchestra.run.smithi190.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.8 MB/s | 172 kB 00:00 2024-09-16T01:50:54.482 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T01:50:54.503 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T01:50:54.565 INFO:teuthology.orchestra.run.smithi190.stdout:(118/156): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-16T01:50:54.590 INFO:teuthology.orchestra.run.smithi190.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-09-16T01:50:54.616 INFO:teuthology.orchestra.run.smithi190.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-09-16T01:50:54.641 INFO:teuthology.orchestra.run.smithi190.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-16T01:50:54.675 INFO:teuthology.orchestra.run.smithi190.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-16T01:50:54.709 INFO:teuthology.orchestra.run.smithi190.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 320 kB/s | 11 kB 00:00 2024-09-16T01:50:54.709 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T01:50:54.734 INFO:teuthology.orchestra.run.smithi190.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 708 kB/s | 18 kB 00:00 2024-09-16T01:50:54.768 INFO:teuthology.orchestra.run.smithi190.stdout:(125/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.1 MB/s | 863 kB 00:00 2024-09-16T01:50:54.793 INFO:teuthology.orchestra.run.smithi190.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 394 kB/s | 23 kB 00:00 2024-09-16T01:50:54.852 INFO:teuthology.orchestra.run.smithi190.stdout:(127/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.7 MB/s | 2.0 MB 00:00 2024-09-16T01:50:54.877 INFO:teuthology.orchestra.run.smithi190.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 259 kB/s | 20 kB 00:00 2024-09-16T01:50:54.885 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T01:50:54.896 INFO:teuthology.orchestra.run.smithi190.stdout:(129/156): python3-jaraco-functools-3.5.0-2.el9 151 kB/s | 19 kB 00:00 2024-09-16T01:50:54.913 INFO:teuthology.orchestra.run.smithi190.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 1.3 MB/s | 46 kB 00:00 2024-09-16T01:50:54.939 INFO:teuthology.orchestra.run.smithi190.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.8 MB/s | 79 kB 00:00 2024-09-16T01:50:54.964 INFO:teuthology.orchestra.run.smithi190.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-16T01:50:54.980 INFO:teuthology.orchestra.run.smithi190.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 997 kB/s | 16 kB 00:00 2024-09-16T01:50:55.005 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-16T01:50:55.006 INFO:teuthology.orchestra.run.smithi190.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-16T01:50:55.040 INFO:teuthology.orchestra.run.smithi190.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-09-16T01:50:55.065 INFO:teuthology.orchestra.run.smithi190.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 482 kB/s | 28 kB 00:00 2024-09-16T01:50:55.090 INFO:teuthology.orchestra.run.smithi190.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-09-16T01:50:55.115 INFO:teuthology.orchestra.run.smithi190.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-16T01:50:55.118 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T01:50:55.141 INFO:teuthology.orchestra.run.smithi190.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-16T01:50:55.174 INFO:teuthology.orchestra.run.smithi190.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-09-16T01:50:55.190 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T01:50:55.199 INFO:teuthology.orchestra.run.smithi190.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.2 MB/s | 74 kB 00:00 2024-09-16T01:50:55.225 INFO:teuthology.orchestra.run.smithi190.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-16T01:50:55.258 INFO:teuthology.orchestra.run.smithi190.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-16T01:50:55.292 INFO:teuthology.orchestra.run.smithi190.stdout:(144/156): python3-kubernetes-26.1.0-2.el9.noar 2.3 MB/s | 1.0 MB 00:00 2024-09-16T01:50:55.317 INFO:teuthology.orchestra.run.smithi190.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 341 kB/s | 20 kB 00:00 2024-09-16T01:50:55.343 INFO:teuthology.orchestra.run.smithi190.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-16T01:50:55.376 INFO:teuthology.orchestra.run.smithi190.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-09-16T01:50:55.433 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T01:50:55.435 INFO:teuthology.orchestra.run.smithi190.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-16T01:50:55.541 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-16T01:50:55.635 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-16T01:50:55.677 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-16T01:50:55.702 INFO:teuthology.orchestra.run.smithi190.stdout:(149/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 9.7 MB/s | 3.2 MB 00:00 2024-09-16T01:50:55.714 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-16T01:50:55.752 INFO:teuthology.orchestra.run.smithi190.stdout:(150/156): NetworkManager-1.49.90-1.el9.x86_64. 7.3 MB/s | 2.3 MB 00:00 2024-09-16T01:50:55.779 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-16T01:50:55.819 INFO:teuthology.orchestra.run.smithi190.stdout:(151/156): librados2-19.1.1-326.g8921c8ef.el9.x 7.2 MB/s | 3.4 MB 00:00 2024-09-16T01:50:55.823 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-16T01:50:55.845 INFO:teuthology.orchestra.run.smithi190.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 443 kB/s | 41 kB 00:00 2024-09-16T01:50:55.864 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-16T01:50:55.907 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-16T01:50:55.912 INFO:teuthology.orchestra.run.smithi190.stdout:(153/156): openssl-3.2.2-6.el9.x86_64.rpm 20 MB/s | 1.4 MB 00:00 2024-09-16T01:50:55.940 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-16T01:50:55.967 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-16T01:50:56.012 INFO:teuthology.orchestra.run.smithi190.stdout:(154/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 24 MB/s | 2.4 MB 00:00 2024-09-16T01:50:56.037 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-16T01:50:56.045 INFO:teuthology.orchestra.run.smithi190.stdout:(155/156): NetworkManager-tui-1.49.90-1.el9.x86 1.1 MB/s | 249 kB 00:00 2024-09-16T01:50:56.087 INFO:teuthology.orchestra.run.smithi190.stdout:(156/156): NetworkManager-libnm-1.49.90-1.el9.x 4.7 MB/s | 1.8 MB 00:00 2024-09-16T01:50:56.090 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-16T01:50:56.093 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-09-16T01:50:56.093 INFO:teuthology.orchestra.run.smithi190.stdout:Total 9.5 MB/s | 226 MB 00:23 2024-09-16T01:50:56.121 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-16T01:50:56.164 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-16T01:50:56.248 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-16T01:50:56.291 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-16T01:50:56.335 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-16T01:50:56.363 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-16T01:50:56.388 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-09-16T01:50:56.432 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-16T01:50:56.477 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-16T01:50:56.513 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-16T01:50:56.552 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-16T01:50:56.554 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-16T01:50:56.625 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-16T01:50:56.663 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-16T01:50:56.683 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-16T01:50:56.688 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-16T01:50:56.738 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-16T01:50:56.746 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-16T01:50:57.064 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-16T01:50:57.172 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T01:50:57.279 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-16T01:50:57.294 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T01:50:57.323 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T01:50:57.327 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-09-16T01:50:57.375 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-16T01:50:57.398 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T01:50:57.413 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-16T01:50:57.437 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T01:50:57.464 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-16T01:50:57.478 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-09-16T01:50:57.478 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-09-16T01:50:57.503 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-16T01:50:57.545 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-16T01:50:57.611 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-16T01:50:57.665 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-16T01:50:57.694 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T01:50:57.709 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-16T01:50:57.780 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-16T01:50:57.781 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T01:50:57.804 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T01:50:57.835 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-16T01:50:57.867 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T01:50:57.897 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T01:50:57.907 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-16T01:50:57.921 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T01:50:57.939 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-16T01:50:57.983 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-16T01:50:58.039 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-16T01:50:58.075 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-16T01:50:58.090 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T01:50:58.118 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-16T01:50:58.160 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-16T01:50:58.271 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T01:50:58.296 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-16T01:50:58.336 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-16T01:50:58.378 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-16T01:50:58.394 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-16T01:50:58.481 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-16T01:50:58.513 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T01:50:58.579 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T01:50:58.791 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T01:50:58.835 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-16T01:50:58.908 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-16T01:50:58.998 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-16T01:50:59.035 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-16T01:50:59.069 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-16T01:50:59.177 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-16T01:50:59.252 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-16T01:50:59.263 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-16T01:50:59.292 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-16T01:50:59.303 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-16T01:50:59.328 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-16T01:50:59.351 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-16T01:50:59.394 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-16T01:50:59.425 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-16T01:50:59.444 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-16T01:50:59.536 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-16T01:50:59.586 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-16T01:50:59.605 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-16T01:50:59.621 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-16T01:50:59.637 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-16T01:50:59.650 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-16T01:50:59.688 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-16T01:50:59.690 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-09-16T01:50:59.691 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-09-16T01:50:59.694 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-16T01:50:59.731 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-16T01:50:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-16T01:50:59.772 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-16T01:50:59.792 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-16T01:50:59.804 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-16T01:50:59.827 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-16T01:50:59.848 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-16T01:50:59.860 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-16T01:50:59.876 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-16T01:50:59.902 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-16T01:50:59.936 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-16T01:50:59.945 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-16T01:50:59.991 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-16T01:51:00.039 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-16T01:51:00.065 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-16T01:51:00.100 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-16T01:51:00.132 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-16T01:51:00.206 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-16T01:51:00.235 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-16T01:51:00.260 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-16T01:51:00.291 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-16T01:51:00.783 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-16T01:51:00.863 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-16T01:51:00.902 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-16T01:51:00.945 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-16T01:51:00.983 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-16T01:51:01.012 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-16T01:51:01.025 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-16T01:51:01.055 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-16T01:51:01.088 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-16T01:51:01.189 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-16T01:51:01.189 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-16T01:51:01.235 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-16T01:51:01.244 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-16T01:51:01.281 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-16T01:51:01.298 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-16T01:51:01.368 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-16T01:51:01.371 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-16T01:51:01.401 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-16T01:51:01.403 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-16T01:51:01.439 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-16T01:51:01.486 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-16T01:51:01.522 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-16T01:51:01.565 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-16T01:51:01.595 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-16T01:51:01.608 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-16T01:51:01.632 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-16T01:51:01.710 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-16T01:51:01.741 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-16T01:51:01.784 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-16T01:51:01.825 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-16T01:51:01.926 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-16T01:51:01.959 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-16T01:51:02.019 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-16T01:51:02.050 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-16T01:51:02.074 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-16T01:51:02.161 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-16T01:51:02.317 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-16T01:51:02.357 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-16T01:51:02.480 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-16T01:51:02.535 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-16T01:51:02.572 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-16T01:51:02.608 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-16T01:51:02.645 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-16T01:51:02.685 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-16T01:51:02.706 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-16T01:51:02.741 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-16T01:51:02.774 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-16T01:51:02.816 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-16T01:51:02.875 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-16T01:51:02.961 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-16T01:51:03.005 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-16T01:51:03.036 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-16T01:51:03.138 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-16T01:51:03.196 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-16T01:51:03.734 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-16T01:51:03.830 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-16T01:51:03.858 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-16T01:51:03.891 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-16T01:51:03.907 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-16T01:51:03.960 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-16T01:51:03.993 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-16T01:51:04.091 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-16T01:51:04.148 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-16T01:51:04.193 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-16T01:51:04.202 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-16T01:51:04.245 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T01:51:04.289 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-16T01:51:04.321 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-16T01:51:04.336 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T01:51:04.367 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-16T01:51:04.412 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-16T01:51:04.507 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-16T01:51:04.544 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-16T01:51:04.623 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-16T01:51:04.905 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-16T01:51:04.940 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T01:51:05.012 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T01:51:05.042 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-16T01:51:05.268 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-16T01:51:05.383 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-16T01:51:05.951 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-09-16T01:51:06.075 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-16T01:51:06.113 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-16T01:51:06.175 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-16T01:51:06.217 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-16T01:51:06.242 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-16T01:51:06.310 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-16T01:51:06.620 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-16T01:51:06.698 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-16T01:51:06.706 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-16T01:51:06.729 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-16T01:51:06.759 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-16T01:51:06.810 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T01:51:06.932 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T01:51:06.961 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T01:51:07.052 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T01:51:07.069 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-16T01:51:07.083 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T01:51:07.122 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T01:51:07.220 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T01:51:07.251 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-16T01:51:07.257 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-16T01:51:07.296 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-16T01:51:07.307 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-16T01:51:07.332 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T01:51:07.376 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T01:51:07.414 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T01:51:07.418 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T01:51:07.441 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T01:51:07.488 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T01:51:07.545 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T01:51:07.566 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T01:51:07.762 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T01:51:07.777 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-16T01:51:07.799 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T01:51:07.871 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T01:51:07.901 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-16T01:51:07.939 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T01:51:07.969 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T01:51:08.008 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T01:51:08.058 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-16T01:51:08.100 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T01:51:08.100 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T01:51:08.100 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T01:51:08.101 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:51:08.143 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T01:51:08.166 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T01:51:08.226 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T01:51:08.454 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T01:51:08.573 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-16T01:51:08.669 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-16T01:51:08.699 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-16T01:51:08.741 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-16T01:51:08.814 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-16T01:51:08.867 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-16T01:51:08.899 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-16T01:51:08.943 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-16T01:51:08.977 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-16T01:51:09.003 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-16T01:51:09.072 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-16T01:51:09.126 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-16T01:51:09.157 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-16T01:51:09.200 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-16T01:51:09.285 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-16T01:51:09.327 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-16T01:51:09.371 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-16T01:51:09.400 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-16T01:51:09.468 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-16T01:51:09.514 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-16T01:51:09.589 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-16T01:51:09.720 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-16T01:51:09.775 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-16T01:51:10.206 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T01:51:10.239 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T01:51:10.307 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-16T01:51:10.387 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-16T01:51:10.417 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-16T01:51:10.452 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-16T01:51:10.489 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-16T01:51:10.531 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-16T01:51:10.692 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-16T01:51:10.735 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-16T01:51:10.789 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-16T01:51:10.793 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T01:51:10.834 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T01:51:10.871 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-16T01:51:10.902 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-16T01:51:10.925 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T01:51:10.925 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T01:51:10.925 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T01:51:10.926 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-16T01:51:10.938 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-16T01:51:10.961 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T01:51:10.985 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-16T01:51:11.021 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-16T01:51:11.064 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-16T01:51:11.106 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-16T01:51:11.241 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-16T01:51:11.287 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-16T01:51:11.328 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-16T01:51:11.454 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-16T01:51:11.495 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-16T01:51:11.545 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-16T01:51:11.577 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-16T01:51:11.602 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-16T01:51:11.693 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-16T01:51:11.819 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-16T01:51:11.860 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-16T01:51:11.897 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-16T01:51:11.934 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-16T01:51:11.969 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-16T01:51:12.001 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-16T01:51:12.039 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-16T01:51:12.082 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-16T01:51:12.116 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-16T01:51:12.157 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-16T01:51:12.192 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-16T01:51:12.277 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-16T01:51:12.322 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-16T01:51:12.353 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-16T01:51:12.456 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-16T01:51:12.506 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-16T01:51:13.225 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-16T01:51:13.271 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-16T01:51:13.305 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-16T01:51:13.403 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-16T01:51:13.452 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-16T01:51:13.496 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-16T01:51:13.583 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-16T01:51:13.632 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-16T01:51:13.818 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-16T01:51:13.855 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-16T01:51:13.949 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-16T01:51:14.801 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-16T01:51:14.927 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-16T01:51:16.163 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-16T01:51:16.253 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-16T01:51:16.281 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-16T01:51:16.314 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-16T01:51:16.625 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-16T01:51:16.669 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T01:51:16.759 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T01:51:16.790 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-16T01:51:16.826 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-16T01:51:17.308 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-16T01:51:17.339 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T01:51:17.410 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T01:51:17.432 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-16T01:51:19.693 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T01:51:19.728 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T01:51:19.965 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T01:51:19.965 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /sys 2024-09-16T01:51:19.965 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /proc 2024-09-16T01:51:19.965 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /mnt 2024-09-16T01:51:19.966 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /var/tmp 2024-09-16T01:51:19.966 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /home 2024-09-16T01:51:19.966 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /root 2024-09-16T01:51:19.966 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /tmp 2024-09-16T01:51:19.966 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:51:20.148 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T01:51:20.164 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T01:51:20.164 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:51:20.164 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T01:51:20.164 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:51:20.274 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T01:51:20.308 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T01:51:20.400 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T01:51:20.400 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T01:51:20.400 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T01:51:20.400 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T01:51:20.442 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T01:51:21.650 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T01:51:21.678 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T01:51:21.678 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:51:21.678 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T01:51:21.678 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T01:51:21.678 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T01:51:21.678 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:51:21.909 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T01:51:21.933 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T01:51:21.933 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:51:21.933 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T01:51:21.934 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T01:51:21.934 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T01:51:21.934 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:51:22.377 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T01:51:22.401 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T01:51:22.401 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:51:22.401 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T01:51:22.401 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T01:51:22.401 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T01:51:22.401 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:51:22.507 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-16T01:51:22.564 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-16T01:51:22.620 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-16T01:51:22.806 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-16T01:51:22.861 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-16T01:51:22.940 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-16T01:51:23.027 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-16T01:51:23.128 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T01:51:23.129 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /sys 2024-09-16T01:51:23.129 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /proc 2024-09-16T01:51:23.129 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /mnt 2024-09-16T01:51:23.129 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /var/tmp 2024-09-16T01:51:23.129 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /home 2024-09-16T01:51:23.129 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /root 2024-09-16T01:51:23.129 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /tmp 2024-09-16T01:51:23.129 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-16T01:51:23.167 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-16T01:51:23.237 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-16T01:51:23.268 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-16T01:51:23.296 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-16T01:51:23.298 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T01:51:23.314 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T01:51:23.314 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:51:23.314 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T01:51:23.314 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-16T01:51:23.744 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T01:51:23.778 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T01:51:23.809 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-16T01:51:23.872 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-16T01:51:23.901 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T01:51:24.817 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T01:51:24.845 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T01:51:24.845 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:51:24.845 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T01:51:24.845 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T01:51:24.845 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T01:51:24.845 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-16T01:51:24.943 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T01:51:24.978 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T01:51:25.085 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T01:51:25.110 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T01:51:25.110 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:51:25.110 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T01:51:25.110 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T01:51:25.110 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T01:51:25.110 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-16T01:51:25.545 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T01:51:25.569 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T01:51:25.569 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:51:25.569 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T01:51:25.570 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T01:51:25.570 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T01:51:25.570 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-16T01:51:25.675 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-16T01:51:25.732 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-16T01:51:25.787 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-16T01:51:25.943 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T01:51:25.970 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T01:51:25.973 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-16T01:51:26.021 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-16T01:51:26.072 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T01:51:26.097 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-16T01:51:26.187 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-16T01:51:26.263 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-16T01:51:26.297 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T01:51:26.322 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T01:51:26.322 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:51:26.322 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T01:51:26.322 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T01:51:26.322 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T01:51:26.322 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:51:26.330 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-16T01:51:26.373 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T01:51:26.385 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T01:51:26.395 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-16T01:51:26.436 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-16T01:51:26.463 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-16T01:51:26.483 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-16T01:51:26.538 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-16T01:51:26.909 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T01:51:26.937 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T01:51:26.968 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-16T01:51:27.031 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-16T01:51:27.060 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T01:51:27.465 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-16T01:51:27.492 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T01:51:27.516 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T01:51:27.516 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:51:27.516 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T01:51:27.516 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T01:51:27.517 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T01:51:27.517 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:51:27.561 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T01:51:27.582 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T01:51:27.583 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:51:27.583 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T01:51:27.583 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:51:27.627 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T01:51:27.650 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T01:51:27.651 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:51:27.651 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-16T01:51:27.651 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T01:51:27.651 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T01:51:27.651 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:51:27.961 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T01:51:27.984 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T01:51:27.985 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:51:27.985 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T01:51:27.985 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T01:51:27.985 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T01:51:27.985 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:51:28.101 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T01:51:28.137 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T01:51:29.099 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T01:51:29.120 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T01:51:29.223 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T01:51:29.398 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-16T01:51:29.423 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T01:51:29.447 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T01:51:29.447 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:51:29.447 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T01:51:29.447 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T01:51:29.447 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T01:51:29.447 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-16T01:51:29.498 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T01:51:29.510 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T01:51:29.608 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-16T01:51:29.659 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-16T01:51:30.583 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-16T01:51:30.617 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T01:51:30.641 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T01:51:30.641 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:51:30.641 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T01:51:30.641 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T01:51:30.641 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T01:51:30.642 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-16T01:51:30.686 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T01:51:30.708 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T01:51:30.708 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:51:30.708 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T01:51:30.708 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-16T01:51:30.760 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T01:51:30.784 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T01:51:30.784 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:51:30.784 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-16T01:51:30.785 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T01:51:30.785 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T01:51:30.785 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-16T01:51:31.094 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T01:51:31.118 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T01:51:31.118 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:51:31.118 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T01:51:31.118 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T01:51:31.119 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T01:51:31.119 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-16T01:51:32.238 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T01:51:32.238 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /sys 2024-09-16T01:51:32.239 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /proc 2024-09-16T01:51:32.239 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /mnt 2024-09-16T01:51:32.239 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /var/tmp 2024-09-16T01:51:32.239 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /home 2024-09-16T01:51:32.239 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /root 2024-09-16T01:51:32.239 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /tmp 2024-09-16T01:51:32.239 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T01:51:32.413 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T01:51:32.429 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T01:51:32.429 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:51:32.429 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T01:51:32.429 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T01:51:32.581 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-16T01:51:32.622 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-16T01:51:32.663 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-16T01:51:32.695 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-16T01:51:32.723 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-16T01:51:32.919 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-16T01:51:32.969 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-16T01:51:33.000 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-16T01:51:33.029 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-16T01:51:33.067 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-16T01:51:33.683 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T01:51:33.724 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T01:51:33.744 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T01:51:33.924 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T01:51:33.951 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T01:51:33.951 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:51:33.951 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T01:51:33.951 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T01:51:33.951 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T01:51:33.951 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T01:51:34.183 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T01:51:34.207 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T01:51:34.207 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:51:34.207 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T01:51:34.207 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T01:51:34.207 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T01:51:34.207 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T01:51:34.581 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-16T01:51:34.642 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T01:51:34.666 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T01:51:34.666 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:51:34.666 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T01:51:34.666 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T01:51:34.666 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T01:51:34.666 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T01:51:34.687 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-16T01:51:34.735 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-16T01:51:34.772 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-16T01:51:34.821 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-16T01:51:34.831 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-16T01:51:34.865 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-16T01:51:34.884 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-16T01:51:34.915 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-16T01:51:34.950 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-16T01:51:34.950 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T01:51:35.025 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T01:51:35.025 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T01:51:35.069 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-16T01:51:35.120 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-16T01:51:35.120 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T01:51:35.143 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-16T01:51:35.148 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T01:51:35.148 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T01:51:35.196 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-16T01:51:35.284 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-16T01:51:35.328 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T01:51:35.328 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T01:51:35.413 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T01:51:35.423 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-16T01:51:35.485 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T01:51:35.487 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-16T01:51:35.525 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-16T01:51:35.553 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-16T01:51:35.706 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-16T01:51:35.772 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-16T01:51:35.814 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-16T01:51:35.846 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-16T01:51:35.872 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-16T01:51:36.023 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T01:51:36.052 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T01:51:36.068 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-16T01:51:36.083 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-16T01:51:36.119 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-16T01:51:36.145 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-16T01:51:36.150 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-16T01:51:36.175 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T01:51:36.179 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-16T01:51:36.225 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-16T01:51:36.850 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T01:51:36.889 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T01:51:36.910 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T01:51:37.220 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T01:51:37.252 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T01:51:37.750 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-16T01:51:37.836 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-16T01:51:37.883 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-16T01:51:37.967 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-16T01:51:37.997 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-16T01:51:38.022 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-16T01:51:38.048 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-16T01:51:38.048 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T01:51:38.141 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T01:51:38.141 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T01:51:38.215 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T01:51:38.225 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T01:51:38.250 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-16T01:51:38.252 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T01:51:38.255 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T01:51:38.255 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T01:51:38.354 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T01:51:38.443 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T01:51:38.443 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T01:51:38.527 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T01:51:38.541 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-16T01:51:38.580 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T01:51:38.600 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T01:51:38.604 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T01:51:38.604 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:51:38.604 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T01:51:38.604 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T01:51:38.604 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T01:51:38.604 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T01:51:38.655 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T01:51:38.667 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T01:51:38.764 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-16T01:51:38.835 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-16T01:51:39.764 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-16T01:51:39.791 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T01:51:39.815 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T01:51:39.815 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:51:39.816 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T01:51:39.816 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T01:51:39.816 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T01:51:39.816 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T01:51:39.859 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T01:51:39.881 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T01:51:39.881 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:51:39.882 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T01:51:39.882 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T01:51:39.926 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T01:51:39.949 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T01:51:39.949 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:51:39.949 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-16T01:51:39.949 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T01:51:39.949 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T01:51:39.950 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T01:51:40.260 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T01:51:40.283 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T01:51:40.284 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:51:40.284 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T01:51:40.284 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T01:51:40.284 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T01:51:40.284 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T01:51:41.041 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T01:51:41.042 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-16T01:51:41.042 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-16T01:51:41.042 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-16T01:51:41.042 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-16T01:51:41.042 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-16T01:51:41.042 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-16T01:51:41.042 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-16T01:51:41.042 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T01:51:41.042 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T01:51:41.042 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T01:51:41.042 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-16T01:51:41.042 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-16T01:51:41.043 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T01:51:41.043 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-16T01:51:41.043 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-16T01:51:41.043 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-16T01:51:41.043 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-16T01:51:41.043 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-16T01:51:41.043 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-16T01:51:41.043 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-16T01:51:41.061 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-16T01:51:41.061 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-16T01:51:41.061 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-16T01:51:41.061 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-16T01:51:41.061 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-16T01:51:41.061 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-16T01:51:41.061 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-16T01:51:41.062 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-16T01:51:41.062 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-16T01:51:41.062 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-16T01:51:41.062 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-16T01:51:41.062 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-16T01:51:41.062 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-16T01:51:41.062 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-16T01:51:41.062 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-16T01:51:41.062 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-16T01:51:41.062 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-16T01:51:41.062 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-16T01:51:41.062 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-16T01:51:41.063 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-16T01:51:41.063 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-16T01:51:41.063 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-16T01:51:41.063 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-16T01:51:41.063 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-16T01:51:41.063 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-16T01:51:41.063 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-16T01:51:41.063 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-16T01:51:41.063 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-16T01:51:41.063 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-16T01:51:41.063 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-16T01:51:41.063 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-16T01:51:41.063 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-16T01:51:41.064 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-16T01:51:41.064 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-16T01:51:41.064 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-16T01:51:41.064 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-16T01:51:41.064 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-16T01:51:41.064 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-16T01:51:41.064 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-16T01:51:41.064 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-16T01:51:41.064 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-16T01:51:41.064 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-16T01:51:41.064 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-16T01:51:41.064 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-16T01:51:41.064 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-16T01:51:41.065 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-16T01:51:41.065 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-16T01:51:41.065 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-16T01:51:41.065 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-16T01:51:41.065 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-16T01:51:41.065 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-16T01:51:41.065 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-16T01:51:41.065 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-16T01:51:41.065 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-16T01:51:41.065 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-16T01:51:41.065 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-16T01:51:41.065 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-16T01:51:41.065 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-16T01:51:41.065 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-16T01:51:41.066 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-16T01:51:41.066 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-16T01:51:41.066 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-16T01:51:41.066 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-16T01:51:41.066 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-16T01:51:41.066 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-16T01:51:41.066 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-16T01:51:41.066 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-16T01:51:41.066 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-16T01:51:41.066 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-16T01:51:41.066 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-16T01:51:41.066 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-16T01:51:41.066 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-16T01:51:41.066 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-16T01:51:41.067 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-16T01:51:41.067 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-16T01:51:41.067 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-16T01:51:41.067 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-16T01:51:41.067 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-16T01:51:41.067 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-16T01:51:41.067 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-16T01:51:41.067 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-16T01:51:41.067 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-16T01:51:41.067 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-16T01:51:41.067 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-16T01:51:41.068 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-16T01:51:41.068 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-16T01:51:41.068 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-16T01:51:41.068 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-16T01:51:41.068 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-16T01:51:41.068 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-16T01:51:41.068 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-16T01:51:41.068 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-16T01:51:41.068 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-16T01:51:41.068 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-16T01:51:41.068 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-16T01:51:41.068 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-16T01:51:41.069 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-16T01:51:41.069 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-16T01:51:41.069 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-16T01:51:41.069 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-16T01:51:41.069 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-16T01:51:41.069 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-16T01:51:41.069 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-16T01:51:41.069 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-16T01:51:41.069 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-16T01:51:41.069 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-16T01:51:41.069 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-16T01:51:41.069 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-16T01:51:41.069 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-16T01:51:41.069 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-16T01:51:41.070 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-16T01:51:41.070 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-16T01:51:41.070 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-16T01:51:41.070 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-16T01:51:41.070 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-16T01:51:41.070 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-16T01:51:41.070 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-16T01:51:41.070 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-16T01:51:41.070 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-16T01:51:41.070 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-16T01:51:41.070 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-16T01:51:41.070 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-16T01:51:41.070 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-16T01:51:41.071 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-16T01:51:41.071 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-16T01:51:41.071 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-16T01:51:41.071 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-16T01:51:41.071 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-16T01:51:41.071 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-16T01:51:41.073 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-16T01:51:41.073 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-16T01:51:41.074 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-16T01:51:41.074 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-16T01:51:41.074 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-16T01:51:41.074 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-16T01:51:41.074 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-16T01:51:41.074 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-16T01:51:41.074 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-16T01:51:41.074 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-16T01:51:41.074 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-16T01:51:41.074 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-16T01:51:41.074 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-16T01:51:41.074 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-16T01:51:43.805 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T01:51:43.805 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:51:43.805 INFO:teuthology.orchestra.run.smithi105.stdout:Upgraded: 2024-09-16T01:51:43.805 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-16T01:51:43.805 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-16T01:51:43.805 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-16T01:51:43.805 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-16T01:51:43.805 INFO:teuthology.orchestra.run.smithi105.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:43.806 INFO:teuthology.orchestra.run.smithi105.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:43.806 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-16T01:51:43.806 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-16T01:51:43.806 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2024-09-16T01:51:43.806 INFO:teuthology.orchestra.run.smithi105.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-16T01:51:43.806 INFO:teuthology.orchestra.run.smithi105.stdout: automake-1.16.2-8.el9.noarch 2024-09-16T01:51:43.806 INFO:teuthology.orchestra.run.smithi105.stdout: bison-3.7.4-5.el9.x86_64 2024-09-16T01:51:43.806 INFO:teuthology.orchestra.run.smithi105.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T01:51:43.806 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:43.806 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:43.806 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:43.806 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:43.806 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:51:43.807 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:43.807 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:43.807 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:43.807 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:51:43.807 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:51:43.807 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:51:43.807 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:51:43.807 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:51:43.807 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:43.807 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:43.807 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:51:43.807 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:43.808 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:43.808 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:43.808 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:51:43.808 INFO:teuthology.orchestra.run.smithi105.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:51:43.808 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:43.808 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:43.808 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:51:43.808 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-16T01:51:43.808 INFO:teuthology.orchestra.run.smithi105.stdout: flex-2.6.4-9.el9.x86_64 2024-09-16T01:51:43.808 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T01:51:43.808 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T01:51:43.808 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T01:51:43.809 INFO:teuthology.orchestra.run.smithi105.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-16T01:51:43.809 INFO:teuthology.orchestra.run.smithi105.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T01:51:43.809 INFO:teuthology.orchestra.run.smithi105.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-16T01:51:43.809 INFO:teuthology.orchestra.run.smithi105.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-16T01:51:43.809 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T01:51:43.809 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T01:51:43.809 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:43.809 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:43.809 INFO:teuthology.orchestra.run.smithi105.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:43.809 INFO:teuthology.orchestra.run.smithi105.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T01:51:43.809 INFO:teuthology.orchestra.run.smithi105.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T01:51:43.809 INFO:teuthology.orchestra.run.smithi105.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T01:51:43.809 INFO:teuthology.orchestra.run.smithi105.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T01:51:43.809 INFO:teuthology.orchestra.run.smithi105.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T01:51:43.810 INFO:teuthology.orchestra.run.smithi105.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:43.810 INFO:teuthology.orchestra.run.smithi105.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:43.810 INFO:teuthology.orchestra.run.smithi105.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T01:51:43.810 INFO:teuthology.orchestra.run.smithi105.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:43.810 INFO:teuthology.orchestra.run.smithi105.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-16T01:51:43.810 INFO:teuthology.orchestra.run.smithi105.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T01:51:43.810 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-16T01:51:43.810 INFO:teuthology.orchestra.run.smithi105.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-16T01:51:43.810 INFO:teuthology.orchestra.run.smithi105.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T01:51:43.810 INFO:teuthology.orchestra.run.smithi105.stdout: lua-5.4.4-4.el9.x86_64 2024-09-16T01:51:43.810 INFO:teuthology.orchestra.run.smithi105.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-16T01:51:43.810 INFO:teuthology.orchestra.run.smithi105.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-16T01:51:43.811 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T01:51:43.811 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T01:51:43.811 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-16T01:51:43.811 INFO:teuthology.orchestra.run.smithi105.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T01:51:43.811 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-13.16-1.el9.x86_64 2024-09-16T01:51:43.811 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-16T01:51:43.811 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-16T01:51:43.811 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-16T01:51:43.811 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-16T01:51:43.811 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-16T01:51:43.811 INFO:teuthology.orchestra.run.smithi105.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T01:51:43.812 INFO:teuthology.orchestra.run.smithi105.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-16T01:51:43.812 INFO:teuthology.orchestra.run.smithi105.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T01:51:43.812 INFO:teuthology.orchestra.run.smithi105.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T01:51:43.812 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T01:51:43.812 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:43.812 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:43.812 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:43.812 INFO:teuthology.orchestra.run.smithi105.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T01:51:43.812 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T01:51:43.812 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T01:51:43.812 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T01:51:43.812 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-16T01:51:43.812 INFO:teuthology.orchestra.run.smithi105.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-16T01:51:43.813 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T01:51:43.813 INFO:teuthology.orchestra.run.smithi105.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T01:51:43.813 INFO:teuthology.orchestra.run.smithi105.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T01:51:43.813 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-16T01:51:43.813 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-16T01:51:43.813 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T01:51:43.813 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T01:51:43.813 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T01:51:43.813 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T01:51:43.813 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T01:51:43.813 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T01:51:43.813 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T01:51:43.813 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T01:51:43.814 INFO:teuthology.orchestra.run.smithi105.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T01:51:43.814 INFO:teuthology.orchestra.run.smithi105.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T01:51:43.814 INFO:teuthology.orchestra.run.smithi105.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T01:51:43.814 INFO:teuthology.orchestra.run.smithi105.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T01:51:43.814 INFO:teuthology.orchestra.run.smithi105.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T01:51:43.814 INFO:teuthology.orchestra.run.smithi105.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T01:51:43.814 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T01:51:43.814 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T01:51:43.814 INFO:teuthology.orchestra.run.smithi105.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T01:51:43.814 INFO:teuthology.orchestra.run.smithi105.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T01:51:43.814 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T01:51:43.814 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T01:51:43.814 INFO:teuthology.orchestra.run.smithi105.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T01:51:43.814 INFO:teuthology.orchestra.run.smithi105.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T01:51:43.815 INFO:teuthology.orchestra.run.smithi105.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-16T01:51:43.815 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T01:51:43.815 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T01:51:43.815 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T01:51:43.815 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T01:51:43.815 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-16T01:51:43.815 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T01:51:43.815 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:43.815 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:43.816 INFO:teuthology.orchestra.run.smithi105.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T01:51:43.816 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T01:51:43.816 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T01:51:43.816 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:43.816 INFO:teuthology.orchestra.run.smithi105.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T01:51:43.816 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T01:51:43.816 INFO:teuthology.orchestra.run.smithi105.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T01:51:43.816 INFO:teuthology.orchestra.run.smithi105.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T01:51:43.816 INFO:teuthology.orchestra.run.smithi105.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T01:51:43.816 INFO:teuthology.orchestra.run.smithi105.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T01:51:43.816 INFO:teuthology.orchestra.run.smithi105.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T01:51:43.816 INFO:teuthology.orchestra.run.smithi105.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-16T01:51:43.816 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T01:51:43.817 INFO:teuthology.orchestra.run.smithi105.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T01:51:43.817 INFO:teuthology.orchestra.run.smithi105.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T01:51:43.817 INFO:teuthology.orchestra.run.smithi105.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T01:51:43.817 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-16T01:51:43.817 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-16T01:51:43.817 INFO:teuthology.orchestra.run.smithi105.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-16T01:51:43.817 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:43.817 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:43.817 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:43.817 INFO:teuthology.orchestra.run.smithi105.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T01:51:43.817 INFO:teuthology.orchestra.run.smithi105.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T01:51:43.817 INFO:teuthology.orchestra.run.smithi105.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T01:51:43.817 INFO:teuthology.orchestra.run.smithi105.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-16T01:51:43.818 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-16T01:51:43.818 INFO:teuthology.orchestra.run.smithi105.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T01:51:43.818 INFO:teuthology.orchestra.run.smithi105.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-16T01:51:43.818 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:51:43.818 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2024-09-16T01:51:44.160 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T01:51:44.161 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-16T01:51:44.161 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-16T01:51:44.161 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-16T01:51:44.161 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-16T01:51:44.161 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-16T01:51:44.161 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-16T01:51:44.161 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-16T01:51:44.161 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T01:51:44.161 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T01:51:44.161 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T01:51:44.161 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-16T01:51:44.161 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-16T01:51:44.162 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T01:51:44.162 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-16T01:51:44.162 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-16T01:51:44.162 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-16T01:51:44.162 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-16T01:51:44.162 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-16T01:51:44.162 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-16T01:51:44.162 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-16T01:51:44.162 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-16T01:51:44.162 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-16T01:51:44.162 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-16T01:51:44.162 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-16T01:51:44.162 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-16T01:51:44.162 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-16T01:51:44.162 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-16T01:51:44.163 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-16T01:51:44.163 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-16T01:51:44.163 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-16T01:51:44.163 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-16T01:51:44.163 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-16T01:51:44.163 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-16T01:51:44.163 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-16T01:51:44.163 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-16T01:51:44.163 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-16T01:51:44.163 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-16T01:51:44.163 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-16T01:51:44.163 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-16T01:51:44.163 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-16T01:51:44.164 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-16T01:51:44.164 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-16T01:51:44.164 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-16T01:51:44.164 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-16T01:51:44.164 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-16T01:51:44.164 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-16T01:51:44.164 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-16T01:51:44.164 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-16T01:51:44.164 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-16T01:51:44.164 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-16T01:51:44.164 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-16T01:51:44.164 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-16T01:51:44.164 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-16T01:51:44.164 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-16T01:51:44.165 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-16T01:51:44.165 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-16T01:51:44.165 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-16T01:51:44.165 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-16T01:51:44.165 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-16T01:51:44.165 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-16T01:51:44.165 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-16T01:51:44.165 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-16T01:51:44.165 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-16T01:51:44.165 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-16T01:51:44.165 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-16T01:51:44.165 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-16T01:51:44.165 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-16T01:51:44.165 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-16T01:51:44.165 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-16T01:51:44.166 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-16T01:51:44.166 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-16T01:51:44.166 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-16T01:51:44.166 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-16T01:51:44.166 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-16T01:51:44.166 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-16T01:51:44.166 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-16T01:51:44.166 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-16T01:51:44.166 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-16T01:51:44.166 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-16T01:51:44.166 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-16T01:51:44.166 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-16T01:51:44.166 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-16T01:51:44.167 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-16T01:51:44.167 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-16T01:51:44.167 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-16T01:51:44.167 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-16T01:51:44.167 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-16T01:51:44.167 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-16T01:51:44.167 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-16T01:51:44.167 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-16T01:51:44.167 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-16T01:51:44.167 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-16T01:51:44.167 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-16T01:51:44.167 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-16T01:51:44.167 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-16T01:51:44.167 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-16T01:51:44.167 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-16T01:51:44.168 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-16T01:51:44.168 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-16T01:51:44.168 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-16T01:51:44.168 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-16T01:51:44.168 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-16T01:51:44.168 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-16T01:51:44.168 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-16T01:51:44.168 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-16T01:51:44.168 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-16T01:51:44.168 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-16T01:51:44.168 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-16T01:51:44.168 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-16T01:51:44.169 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-16T01:51:44.169 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-16T01:51:44.169 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-16T01:51:44.169 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-16T01:51:44.169 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-16T01:51:44.169 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-16T01:51:44.169 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-16T01:51:44.169 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-16T01:51:44.169 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-16T01:51:44.169 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-16T01:51:44.169 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-16T01:51:44.169 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-16T01:51:44.170 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-16T01:51:44.170 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-16T01:51:44.170 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-16T01:51:44.170 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-16T01:51:44.170 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-16T01:51:44.170 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-16T01:51:44.170 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-16T01:51:44.170 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-16T01:51:44.170 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-16T01:51:44.170 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-16T01:51:44.170 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-16T01:51:44.170 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-16T01:51:44.171 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-16T01:51:44.171 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-16T01:51:44.171 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-16T01:51:44.171 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-16T01:51:44.171 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-16T01:51:44.171 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-16T01:51:44.171 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-16T01:51:44.171 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-16T01:51:44.171 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-16T01:51:44.171 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-16T01:51:44.171 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-16T01:51:44.171 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-16T01:51:44.171 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-16T01:51:44.172 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-16T01:51:44.172 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-16T01:51:44.172 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-16T01:51:44.172 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-16T01:51:44.172 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-16T01:51:44.172 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-16T01:51:44.172 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-16T01:51:44.172 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-16T01:51:44.172 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-16T01:51:44.172 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-16T01:51:44.172 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-16T01:51:44.172 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-16T01:51:44.173 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-16T01:51:44.173 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-16T01:51:44.173 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-16T01:51:44.173 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-16T01:51:44.173 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-16T01:51:44.250 DEBUG:teuthology.parallel:result is None 2024-09-16T01:51:44.889 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-16T01:51:44.930 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-16T01:51:44.971 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-16T01:51:45.003 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-16T01:51:45.030 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-16T01:51:45.219 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-16T01:51:45.269 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-16T01:51:45.300 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-16T01:51:45.329 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-16T01:51:45.375 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-16T01:51:45.893 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T01:51:45.940 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T01:51:45.960 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T01:51:46.795 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-16T01:51:46.895 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-16T01:51:46.947 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-16T01:51:47.036 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-16T01:51:47.065 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-16T01:51:47.067 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T01:51:47.067 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-16T01:51:47.067 INFO:teuthology.orchestra.run.smithi137.stdout:Upgraded: 2024-09-16T01:51:47.067 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-16T01:51:47.067 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-16T01:51:47.067 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-16T01:51:47.067 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-16T01:51:47.067 INFO:teuthology.orchestra.run.smithi137.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:47.067 INFO:teuthology.orchestra.run.smithi137.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:47.068 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-16T01:51:47.068 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-16T01:51:47.068 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-09-16T01:51:47.068 INFO:teuthology.orchestra.run.smithi137.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-16T01:51:47.068 INFO:teuthology.orchestra.run.smithi137.stdout: automake-1.16.2-8.el9.noarch 2024-09-16T01:51:47.068 INFO:teuthology.orchestra.run.smithi137.stdout: bison-3.7.4-5.el9.x86_64 2024-09-16T01:51:47.068 INFO:teuthology.orchestra.run.smithi137.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T01:51:47.068 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:47.068 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:47.068 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:47.068 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:47.068 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:51:47.068 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:47.069 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:47.069 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:47.069 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:51:47.069 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:51:47.069 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:51:47.069 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:51:47.069 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:51:47.069 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:47.070 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:47.070 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:51:47.070 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:47.070 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:47.070 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:47.070 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:51:47.070 INFO:teuthology.orchestra.run.smithi137.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:51:47.070 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:47.070 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:47.070 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:51:47.070 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-16T01:51:47.070 INFO:teuthology.orchestra.run.smithi137.stdout: flex-2.6.4-9.el9.x86_64 2024-09-16T01:51:47.071 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T01:51:47.071 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T01:51:47.071 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T01:51:47.071 INFO:teuthology.orchestra.run.smithi137.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-16T01:51:47.071 INFO:teuthology.orchestra.run.smithi137.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T01:51:47.071 INFO:teuthology.orchestra.run.smithi137.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-16T01:51:47.071 INFO:teuthology.orchestra.run.smithi137.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-16T01:51:47.071 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T01:51:47.071 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T01:51:47.071 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:47.071 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:47.071 INFO:teuthology.orchestra.run.smithi137.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:47.071 INFO:teuthology.orchestra.run.smithi137.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T01:51:47.072 INFO:teuthology.orchestra.run.smithi137.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T01:51:47.072 INFO:teuthology.orchestra.run.smithi137.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T01:51:47.072 INFO:teuthology.orchestra.run.smithi137.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T01:51:47.072 INFO:teuthology.orchestra.run.smithi137.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T01:51:47.072 INFO:teuthology.orchestra.run.smithi137.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:47.072 INFO:teuthology.orchestra.run.smithi137.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:47.072 INFO:teuthology.orchestra.run.smithi137.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T01:51:47.072 INFO:teuthology.orchestra.run.smithi137.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:47.072 INFO:teuthology.orchestra.run.smithi137.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-16T01:51:47.072 INFO:teuthology.orchestra.run.smithi137.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T01:51:47.072 INFO:teuthology.orchestra.run.smithi137.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-16T01:51:47.072 INFO:teuthology.orchestra.run.smithi137.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-16T01:51:47.072 INFO:teuthology.orchestra.run.smithi137.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T01:51:47.073 INFO:teuthology.orchestra.run.smithi137.stdout: lua-5.4.4-4.el9.x86_64 2024-09-16T01:51:47.073 INFO:teuthology.orchestra.run.smithi137.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-16T01:51:47.073 INFO:teuthology.orchestra.run.smithi137.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-16T01:51:47.073 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T01:51:47.073 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T01:51:47.073 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-16T01:51:47.073 INFO:teuthology.orchestra.run.smithi137.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T01:51:47.073 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-13.16-1.el9.x86_64 2024-09-16T01:51:47.073 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-16T01:51:47.073 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-16T01:51:47.073 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-16T01:51:47.073 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-16T01:51:47.073 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-16T01:51:47.074 INFO:teuthology.orchestra.run.smithi137.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T01:51:47.074 INFO:teuthology.orchestra.run.smithi137.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-16T01:51:47.074 INFO:teuthology.orchestra.run.smithi137.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T01:51:47.074 INFO:teuthology.orchestra.run.smithi137.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T01:51:47.074 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T01:51:47.074 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:47.074 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:47.074 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:47.074 INFO:teuthology.orchestra.run.smithi137.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T01:51:47.074 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T01:51:47.074 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T01:51:47.074 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T01:51:47.074 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-16T01:51:47.075 INFO:teuthology.orchestra.run.smithi137.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-16T01:51:47.075 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T01:51:47.075 INFO:teuthology.orchestra.run.smithi137.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T01:51:47.075 INFO:teuthology.orchestra.run.smithi137.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T01:51:47.075 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-16T01:51:47.075 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-16T01:51:47.075 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T01:51:47.075 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T01:51:47.075 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T01:51:47.075 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T01:51:47.075 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T01:51:47.075 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T01:51:47.075 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T01:51:47.076 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T01:51:47.076 INFO:teuthology.orchestra.run.smithi137.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T01:51:47.076 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T01:51:47.076 INFO:teuthology.orchestra.run.smithi137.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T01:51:47.076 INFO:teuthology.orchestra.run.smithi137.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T01:51:47.076 INFO:teuthology.orchestra.run.smithi137.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T01:51:47.076 INFO:teuthology.orchestra.run.smithi137.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T01:51:47.076 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T01:51:47.076 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T01:51:47.076 INFO:teuthology.orchestra.run.smithi137.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T01:51:47.076 INFO:teuthology.orchestra.run.smithi137.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T01:51:47.076 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T01:51:47.076 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T01:51:47.076 INFO:teuthology.orchestra.run.smithi137.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T01:51:47.077 INFO:teuthology.orchestra.run.smithi137.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T01:51:47.077 INFO:teuthology.orchestra.run.smithi137.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-16T01:51:47.077 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T01:51:47.077 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T01:51:47.077 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T01:51:47.077 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T01:51:47.077 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-16T01:51:47.077 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T01:51:47.077 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:47.077 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:47.077 INFO:teuthology.orchestra.run.smithi137.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T01:51:47.077 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T01:51:47.077 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T01:51:47.078 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:47.078 INFO:teuthology.orchestra.run.smithi137.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T01:51:47.078 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T01:51:47.078 INFO:teuthology.orchestra.run.smithi137.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T01:51:47.078 INFO:teuthology.orchestra.run.smithi137.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T01:51:47.078 INFO:teuthology.orchestra.run.smithi137.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T01:51:47.078 INFO:teuthology.orchestra.run.smithi137.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T01:51:47.078 INFO:teuthology.orchestra.run.smithi137.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T01:51:47.078 INFO:teuthology.orchestra.run.smithi137.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-16T01:51:47.078 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T01:51:47.078 INFO:teuthology.orchestra.run.smithi137.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T01:51:47.078 INFO:teuthology.orchestra.run.smithi137.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T01:51:47.078 INFO:teuthology.orchestra.run.smithi137.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T01:51:47.078 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-16T01:51:47.079 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-16T01:51:47.079 INFO:teuthology.orchestra.run.smithi137.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-16T01:51:47.079 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:47.079 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:47.079 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:47.079 INFO:teuthology.orchestra.run.smithi137.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T01:51:47.079 INFO:teuthology.orchestra.run.smithi137.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T01:51:47.079 INFO:teuthology.orchestra.run.smithi137.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T01:51:47.079 INFO:teuthology.orchestra.run.smithi137.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-16T01:51:47.079 INFO:teuthology.orchestra.run.smithi137.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-16T01:51:47.079 INFO:teuthology.orchestra.run.smithi137.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T01:51:47.080 INFO:teuthology.orchestra.run.smithi137.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-16T01:51:47.080 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-16T01:51:47.080 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-09-16T01:51:47.090 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-16T01:51:47.125 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-16T01:51:47.125 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T01:51:47.217 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T01:51:47.217 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T01:51:47.309 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T01:51:47.335 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-16T01:51:47.340 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T01:51:47.340 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T01:51:47.492 DEBUG:teuthology.parallel:result is None 2024-09-16T01:51:47.528 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T01:51:47.528 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T01:51:47.604 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T01:51:47.668 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T01:51:53.326 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T01:51:53.326 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-16T01:51:53.326 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-16T01:51:53.326 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-16T01:51:53.326 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-16T01:51:53.326 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-16T01:51:53.326 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-16T01:51:53.326 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-16T01:51:53.327 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T01:51:53.327 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T01:51:53.327 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T01:51:53.327 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-16T01:51:53.327 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-16T01:51:53.327 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T01:51:53.327 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-16T01:51:53.327 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-16T01:51:53.327 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-16T01:51:53.327 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-16T01:51:53.327 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-16T01:51:53.327 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-16T01:51:53.327 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-16T01:51:53.328 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-16T01:51:53.328 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-16T01:51:53.328 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-16T01:51:53.328 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-16T01:51:53.328 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-16T01:51:53.328 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-16T01:51:53.328 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-16T01:51:53.328 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-16T01:51:53.328 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-16T01:51:53.328 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-16T01:51:53.328 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-16T01:51:53.328 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-16T01:51:53.328 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-16T01:51:53.329 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-16T01:51:53.329 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-16T01:51:53.329 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-16T01:51:53.329 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-16T01:51:53.329 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-16T01:51:53.329 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-16T01:51:53.329 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-16T01:51:53.329 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-16T01:51:53.329 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-16T01:51:53.329 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-16T01:51:53.329 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-16T01:51:53.329 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-16T01:51:53.329 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-16T01:51:53.330 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-16T01:51:53.330 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-16T01:51:53.330 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-16T01:51:53.330 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-16T01:51:53.330 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-16T01:51:53.330 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-16T01:51:53.330 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-16T01:51:53.330 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-16T01:51:53.330 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-16T01:51:53.330 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-16T01:51:53.330 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-16T01:51:53.330 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-16T01:51:53.330 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-16T01:51:53.330 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-16T01:51:53.331 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-16T01:51:53.331 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-16T01:51:53.331 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-16T01:51:53.331 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-16T01:51:53.331 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-16T01:51:53.331 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-16T01:51:53.331 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-16T01:51:53.331 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-16T01:51:53.331 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-16T01:51:53.331 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-16T01:51:53.331 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-16T01:51:53.331 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-16T01:51:53.331 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-16T01:51:53.332 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-16T01:51:53.332 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-16T01:51:53.332 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-16T01:51:53.332 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-16T01:51:53.332 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-16T01:51:53.332 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-16T01:51:53.332 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-16T01:51:53.332 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-16T01:51:53.332 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-16T01:51:53.332 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-16T01:51:53.332 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-16T01:51:53.332 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-16T01:51:53.332 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-16T01:51:53.333 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-16T01:51:53.333 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-16T01:51:53.333 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-16T01:51:53.333 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-16T01:51:53.333 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-16T01:51:53.333 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-16T01:51:53.333 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-16T01:51:53.333 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-16T01:51:53.333 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-16T01:51:53.333 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-16T01:51:53.333 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-16T01:51:53.333 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-16T01:51:53.333 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-16T01:51:53.334 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-16T01:51:53.334 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-16T01:51:53.334 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-16T01:51:53.334 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-16T01:51:53.334 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-16T01:51:53.334 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-16T01:51:53.334 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-16T01:51:53.334 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-16T01:51:53.334 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-16T01:51:53.334 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-16T01:51:53.334 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-16T01:51:53.334 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-16T01:51:53.334 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-16T01:51:53.334 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-16T01:51:53.335 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-16T01:51:53.335 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-16T01:51:53.335 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-16T01:51:53.335 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-16T01:51:53.335 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-16T01:51:53.335 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-16T01:51:53.335 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-16T01:51:53.335 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-16T01:51:53.335 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-16T01:51:53.335 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-16T01:51:53.335 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-16T01:51:53.335 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-16T01:51:53.335 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-16T01:51:53.336 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-16T01:51:53.336 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-16T01:51:53.336 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-16T01:51:53.336 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-16T01:51:53.336 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-16T01:51:53.336 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-16T01:51:53.336 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-16T01:51:53.336 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-16T01:51:53.336 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-16T01:51:53.336 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-16T01:51:53.336 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-16T01:51:53.336 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-16T01:51:53.336 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-16T01:51:53.337 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-16T01:51:53.337 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-16T01:51:53.337 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-16T01:51:53.337 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-16T01:51:53.337 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-16T01:51:53.337 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-16T01:51:53.337 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-16T01:51:53.337 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-16T01:51:53.337 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-16T01:51:53.337 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-16T01:51:53.337 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-16T01:51:53.337 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-16T01:51:53.337 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-16T01:51:53.338 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-16T01:51:53.338 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-16T01:51:53.338 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-16T01:51:53.338 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-16T01:51:53.338 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-16T01:51:53.338 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-16T01:51:53.338 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-16T01:51:53.338 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-16T01:51:53.338 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-16T01:51:53.338 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-16T01:51:53.338 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-16T01:51:56.303 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T01:51:56.304 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T01:51:56.304 INFO:teuthology.orchestra.run.smithi190.stdout:Upgraded: 2024-09-16T01:51:56.304 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-16T01:51:56.304 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-16T01:51:56.304 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-16T01:51:56.304 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-16T01:51:56.304 INFO:teuthology.orchestra.run.smithi190.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:56.304 INFO:teuthology.orchestra.run.smithi190.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:56.304 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-16T01:51:56.305 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-16T01:51:56.305 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-09-16T01:51:56.305 INFO:teuthology.orchestra.run.smithi190.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-16T01:51:56.305 INFO:teuthology.orchestra.run.smithi190.stdout: automake-1.16.2-8.el9.noarch 2024-09-16T01:51:56.305 INFO:teuthology.orchestra.run.smithi190.stdout: bison-3.7.4-5.el9.x86_64 2024-09-16T01:51:56.305 INFO:teuthology.orchestra.run.smithi190.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T01:51:56.305 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:56.305 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:56.305 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:56.305 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:56.305 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:51:56.305 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:56.305 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:56.306 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:56.306 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:51:56.306 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:51:56.306 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:51:56.306 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:51:56.306 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:51:56.306 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:56.307 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:56.307 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:51:56.307 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:56.307 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:56.307 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:56.307 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:51:56.307 INFO:teuthology.orchestra.run.smithi190.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:51:56.307 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:56.307 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:56.307 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:51:56.307 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-16T01:51:56.307 INFO:teuthology.orchestra.run.smithi190.stdout: flex-2.6.4-9.el9.x86_64 2024-09-16T01:51:56.308 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T01:51:56.308 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T01:51:56.308 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T01:51:56.308 INFO:teuthology.orchestra.run.smithi190.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-16T01:51:56.308 INFO:teuthology.orchestra.run.smithi190.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T01:51:56.308 INFO:teuthology.orchestra.run.smithi190.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-16T01:51:56.308 INFO:teuthology.orchestra.run.smithi190.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-16T01:51:56.308 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T01:51:56.308 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T01:51:56.308 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:56.308 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:56.308 INFO:teuthology.orchestra.run.smithi190.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:56.308 INFO:teuthology.orchestra.run.smithi190.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T01:51:56.309 INFO:teuthology.orchestra.run.smithi190.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T01:51:56.309 INFO:teuthology.orchestra.run.smithi190.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T01:51:56.309 INFO:teuthology.orchestra.run.smithi190.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T01:51:56.309 INFO:teuthology.orchestra.run.smithi190.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T01:51:56.309 INFO:teuthology.orchestra.run.smithi190.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:56.309 INFO:teuthology.orchestra.run.smithi190.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:56.309 INFO:teuthology.orchestra.run.smithi190.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T01:51:56.309 INFO:teuthology.orchestra.run.smithi190.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:56.309 INFO:teuthology.orchestra.run.smithi190.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-16T01:51:56.309 INFO:teuthology.orchestra.run.smithi190.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T01:51:56.309 INFO:teuthology.orchestra.run.smithi190.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-16T01:51:56.309 INFO:teuthology.orchestra.run.smithi190.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-16T01:51:56.309 INFO:teuthology.orchestra.run.smithi190.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T01:51:56.310 INFO:teuthology.orchestra.run.smithi190.stdout: lua-5.4.4-4.el9.x86_64 2024-09-16T01:51:56.310 INFO:teuthology.orchestra.run.smithi190.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-16T01:51:56.310 INFO:teuthology.orchestra.run.smithi190.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-16T01:51:56.310 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T01:51:56.310 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T01:51:56.310 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-16T01:51:56.310 INFO:teuthology.orchestra.run.smithi190.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T01:51:56.310 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-13.16-1.el9.x86_64 2024-09-16T01:51:56.311 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-16T01:51:56.311 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-16T01:51:56.311 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-16T01:51:56.311 INFO:teuthology.orchestra.run.smithi190.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-16T01:51:56.311 INFO:teuthology.orchestra.run.smithi190.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-16T01:51:56.311 INFO:teuthology.orchestra.run.smithi190.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T01:51:56.311 INFO:teuthology.orchestra.run.smithi190.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-16T01:51:56.311 INFO:teuthology.orchestra.run.smithi190.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T01:51:56.311 INFO:teuthology.orchestra.run.smithi190.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T01:51:56.311 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T01:51:56.311 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:56.311 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:56.311 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:56.312 INFO:teuthology.orchestra.run.smithi190.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T01:51:56.312 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T01:51:56.312 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T01:51:56.312 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T01:51:56.312 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-16T01:51:56.312 INFO:teuthology.orchestra.run.smithi190.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-16T01:51:56.312 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T01:51:56.312 INFO:teuthology.orchestra.run.smithi190.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T01:51:56.312 INFO:teuthology.orchestra.run.smithi190.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T01:51:56.312 INFO:teuthology.orchestra.run.smithi190.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-16T01:51:56.312 INFO:teuthology.orchestra.run.smithi190.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-16T01:51:56.312 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T01:51:56.312 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T01:51:56.313 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T01:51:56.313 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T01:51:56.313 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T01:51:56.313 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T01:51:56.313 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T01:51:56.313 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T01:51:56.313 INFO:teuthology.orchestra.run.smithi190.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T01:51:56.313 INFO:teuthology.orchestra.run.smithi190.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T01:51:56.313 INFO:teuthology.orchestra.run.smithi190.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T01:51:56.313 INFO:teuthology.orchestra.run.smithi190.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T01:51:56.313 INFO:teuthology.orchestra.run.smithi190.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T01:51:56.313 INFO:teuthology.orchestra.run.smithi190.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T01:51:56.313 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T01:51:56.314 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T01:51:56.314 INFO:teuthology.orchestra.run.smithi190.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T01:51:56.314 INFO:teuthology.orchestra.run.smithi190.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T01:51:56.314 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T01:51:56.314 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T01:51:56.314 INFO:teuthology.orchestra.run.smithi190.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T01:51:56.314 INFO:teuthology.orchestra.run.smithi190.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T01:51:56.314 INFO:teuthology.orchestra.run.smithi190.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-16T01:51:56.314 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T01:51:56.314 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T01:51:56.314 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T01:51:56.314 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T01:51:56.314 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-16T01:51:56.315 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T01:51:56.315 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:56.315 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:56.315 INFO:teuthology.orchestra.run.smithi190.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T01:51:56.315 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T01:51:56.315 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T01:51:56.315 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:56.315 INFO:teuthology.orchestra.run.smithi190.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T01:51:56.315 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T01:51:56.315 INFO:teuthology.orchestra.run.smithi190.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T01:51:56.315 INFO:teuthology.orchestra.run.smithi190.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T01:51:56.315 INFO:teuthology.orchestra.run.smithi190.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T01:51:56.315 INFO:teuthology.orchestra.run.smithi190.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T01:51:56.315 INFO:teuthology.orchestra.run.smithi190.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T01:51:56.316 INFO:teuthology.orchestra.run.smithi190.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-16T01:51:56.316 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T01:51:56.316 INFO:teuthology.orchestra.run.smithi190.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T01:51:56.316 INFO:teuthology.orchestra.run.smithi190.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T01:51:56.316 INFO:teuthology.orchestra.run.smithi190.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T01:51:56.316 INFO:teuthology.orchestra.run.smithi190.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-16T01:51:56.316 INFO:teuthology.orchestra.run.smithi190.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-16T01:51:56.316 INFO:teuthology.orchestra.run.smithi190.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-16T01:51:56.317 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:56.317 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:56.317 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:51:56.317 INFO:teuthology.orchestra.run.smithi190.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T01:51:56.317 INFO:teuthology.orchestra.run.smithi190.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T01:51:56.318 INFO:teuthology.orchestra.run.smithi190.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T01:51:56.318 INFO:teuthology.orchestra.run.smithi190.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-16T01:51:56.318 INFO:teuthology.orchestra.run.smithi190.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-16T01:51:56.319 INFO:teuthology.orchestra.run.smithi190.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T01:51:56.319 INFO:teuthology.orchestra.run.smithi190.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-16T01:51:56.319 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T01:51:56.319 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-09-16T01:51:56.736 DEBUG:teuthology.parallel:result is None 2024-09-16T01:51:56.737 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T01:51:56.881 DEBUG:teuthology.orchestra.run.smithi105:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T01:51:56.908 INFO:teuthology.orchestra.run.smithi105.stdout:19.1.1-326.g8921c8ef.el9 2024-09-16T01:51:56.908 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-16T01:51:56.909 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-16T01:51:56.910 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T01:51:57.053 DEBUG:teuthology.orchestra.run.smithi137:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T01:51:57.079 INFO:teuthology.orchestra.run.smithi137.stdout:19.1.1-326.g8921c8ef.el9 2024-09-16T01:51:57.080 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-16T01:51:57.080 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-16T01:51:57.082 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T01:51:57.224 DEBUG:teuthology.orchestra.run.smithi190:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T01:51:57.249 INFO:teuthology.orchestra.run.smithi190.stdout:19.1.1-326.g8921c8ef.el9 2024-09-16T01:51:57.249 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-16T01:51:57.250 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-16T01:51:57.251 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-16T01:51:57.252 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-16T01:51:57.252 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T01:51:57.285 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-16T01:51:57.285 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T01:51:57.317 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-16T01:51:57.317 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T01:51:57.348 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-16T01:51:57.348 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-16T01:51:57.349 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T01:51:57.378 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T01:51:57.447 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-16T01:51:57.447 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T01:51:57.476 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T01:51:57.544 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-16T01:51:57.545 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T01:51:57.574 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T01:51:57.642 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-16T01:51:57.642 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-16T01:51:57.642 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T01:51:57.668 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T01:51:57.733 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-16T01:51:57.733 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T01:51:57.762 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T01:51:57.829 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-16T01:51:57.830 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T01:51:57.857 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T01:51:57.926 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-16T01:51:57.926 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-16T01:51:57.926 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T01:51:57.951 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T01:51:58.017 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-16T01:51:58.017 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T01:51:58.046 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T01:51:58.113 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-16T01:51:58.113 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T01:51:58.143 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T01:51:58.211 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-16T01:51:58.342 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-09-16T01:51:58.342 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T01:51:58.342 INFO:tasks.cephadm:Cluster fsid is 423b7a2a-73ce-11ef-bceb-c7b262605968 2024-09-16T01:51:58.342 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-16T01:51:58.343 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.105', 'mon.b': '172.21.15.137', 'mon.c': '172.21.15.190'} 2024-09-16T01:51:58.343 INFO:tasks.cephadm:First mon is mon.a on smithi105 2024-09-16T01:51:58.343 INFO:tasks.cephadm:First mgr is x 2024-09-16T01:51:58.343 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-16T01:51:58.343 DEBUG:teuthology.orchestra.run.smithi105:> sudo hostname $(hostname -s) 2024-09-16T01:51:58.368 DEBUG:teuthology.orchestra.run.smithi137:> sudo hostname $(hostname -s) 2024-09-16T01:51:58.397 DEBUG:teuthology.orchestra.run.smithi190:> sudo hostname $(hostname -s) 2024-09-16T01:51:58.427 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-16T01:51:58.427 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T01:51:58.566 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56081/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-326-g8921c8ef', 'node_name': '172.21.5.31+adami01', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-326.g8921c8ef'}, 'url': 'https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-15 16:06:42.385367', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-16T01:51:58.702 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref squid, sha1 8921c8ef947807b717db041d85f0761b7914d5fc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T01:51:58.703 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm 2024-09-16T01:51:58.704 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm 2024-09-16T01:51:58.704 DEBUG:teuthology.orchestra.run.smithi105:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-16T01:51:59.073 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r--. 1 ubuntu ubuntu 784481 Sep 16 01:51 /home/ubuntu/cephtest/cephadm 2024-09-16T01:51:59.074 DEBUG:teuthology.orchestra.run.smithi137:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-16T01:51:59.431 INFO:teuthology.orchestra.run.smithi137.stdout:-rw-r--r--. 1 ubuntu ubuntu 784481 Sep 16 01:51 /home/ubuntu/cephtest/cephadm 2024-09-16T01:51:59.431 DEBUG:teuthology.orchestra.run.smithi190:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-16T01:51:59.817 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r--. 1 ubuntu ubuntu 784481 Sep 16 01:51 /home/ubuntu/cephtest/cephadm 2024-09-16T01:51:59.817 DEBUG:teuthology.orchestra.run.smithi105:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-16T01:51:59.836 DEBUG:teuthology.orchestra.run.smithi137:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-16T01:51:59.855 DEBUG:teuthology.orchestra.run.smithi190:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-16T01:51:59.890 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc on all hosts... 2024-09-16T01:51:59.890 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc pull 2024-09-16T01:51:59.893 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc pull 2024-09-16T01:51:59.898 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc pull 2024-09-16T01:52:01.480 INFO:teuthology.orchestra.run.smithi137.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-16T01:52:01.514 INFO:teuthology.orchestra.run.smithi190.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-16T01:52:01.525 INFO:teuthology.orchestra.run.smithi105.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-16T01:52:29.885 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-16T01:52:29.885 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-16T01:52:29.885 INFO:teuthology.orchestra.run.smithi137.stdout: "image_id": "f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee", 2024-09-16T01:52:29.885 INFO:teuthology.orchestra.run.smithi137.stdout: "repo_digests": [ 2024-09-16T01:52:29.886 INFO:teuthology.orchestra.run.smithi137.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80" 2024-09-16T01:52:29.886 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-09-16T01:52:29.886 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-16T01:52:33.958 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-16T01:52:33.958 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-16T01:52:33.958 INFO:teuthology.orchestra.run.smithi105.stdout: "image_id": "f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee", 2024-09-16T01:52:33.958 INFO:teuthology.orchestra.run.smithi105.stdout: "repo_digests": [ 2024-09-16T01:52:33.959 INFO:teuthology.orchestra.run.smithi105.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80" 2024-09-16T01:52:33.959 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-16T01:52:33.959 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-16T01:52:34.941 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-16T01:52:34.942 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-16T01:52:34.942 INFO:teuthology.orchestra.run.smithi190.stdout: "image_id": "f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee", 2024-09-16T01:52:34.942 INFO:teuthology.orchestra.run.smithi190.stdout: "repo_digests": [ 2024-09-16T01:52:34.942 INFO:teuthology.orchestra.run.smithi190.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80" 2024-09-16T01:52:34.942 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-16T01:52:34.942 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-16T01:52:34.968 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /etc/ceph 2024-09-16T01:52:35.000 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /etc/ceph 2024-09-16T01:52:35.034 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /etc/ceph 2024-09-16T01:52:35.061 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 777 /etc/ceph 2024-09-16T01:52:35.090 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 777 /etc/ceph 2024-09-16T01:52:35.120 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 777 /etc/ceph 2024-09-16T01:52:35.145 INFO:tasks.cephadm:Writing seed config... 2024-09-16T01:52:35.146 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-16T01:52:35.146 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-16T01:52:35.146 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-16T01:52:35.146 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-16T01:52:35.146 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-16T01:52:35.146 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-16T01:52:35.146 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-16T01:52:35.146 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-16T01:52:35.147 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-16T01:52:35.147 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-16T01:52:35.147 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-16T01:52:35.147 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-16T01:52:35.147 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-16T01:52:35.147 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-16T01:52:35.147 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-16T01:52:35.147 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-16T01:52:35.147 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-16T01:52:35.147 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-16T01:52:35.147 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-16T01:52:35.147 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-16T01:52:35.147 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-16T01:52:35.147 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-16T01:52:35.148 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-16T01:52:35.148 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-16T01:52:35.148 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-16T01:52:35.148 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-16T01:52:35.148 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-16T01:52:35.148 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-16T01:52:35.148 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-16T01:52:35.148 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-16T01:52:35.148 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-16T01:52:35.148 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-16T01:52:35.148 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-16T01:52:35.148 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-16T01:52:35.148 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-16T01:52:35.148 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-16T01:52:35.149 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-16T01:52:35.149 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-16T01:52:35.149 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-16T01:52:35.149 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-16T01:52:35.149 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-16T01:52:35.149 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-16T01:52:35.149 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-16T01:52:35.149 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-16T01:52:35.149 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-16T01:52:35.149 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-16T01:52:35.150 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-16T01:52:35.150 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-16T01:52:35.169 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 423b7a2a-73ce-11ef-bceb-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-16T01:52:35.169 DEBUG:teuthology.orchestra.run.smithi105:mon.a> sudo journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mon.a.service 2024-09-16T01:52:35.212 DEBUG:teuthology.orchestra.run.smithi105:mgr.x> sudo journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mgr.x.service 2024-09-16T01:52:35.255 INFO:tasks.cephadm:Bootstrapping... 2024-09-16T01:52:35.256 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc -v bootstrap --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.105 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-16T01:52:35.466 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2024-09-16T01:52:35.466 INFO:teuthology.orchestra.run.smithi105.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc', '-v', 'bootstrap', '--fsid', '423b7a2a-73ce-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.105', '--skip-admin-label'] 2024-09-16T01:52:35.467 INFO:teuthology.orchestra.run.smithi105.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-16T01:52:35.467 INFO:teuthology.orchestra.run.smithi105.stdout:Verifying podman|docker is present... 2024-09-16T01:52:35.489 INFO:teuthology.orchestra.run.smithi105.stdout:/bin/podman: stdout 5.2.2 2024-09-16T01:52:35.490 INFO:teuthology.orchestra.run.smithi105.stdout:Verifying lvm2 is present... 2024-09-16T01:52:35.490 INFO:teuthology.orchestra.run.smithi105.stdout:Verifying time synchronization is in place... 2024-09-16T01:52:35.497 INFO:teuthology.orchestra.run.smithi105.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-16T01:52:35.497 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-16T01:52:35.504 INFO:teuthology.orchestra.run.smithi105.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-16T01:52:35.504 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stdout inactive 2024-09-16T01:52:35.510 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stdout enabled 2024-09-16T01:52:35.517 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stdout active 2024-09-16T01:52:35.517 INFO:teuthology.orchestra.run.smithi105.stdout:Unit chronyd.service is enabled and running 2024-09-16T01:52:35.517 INFO:teuthology.orchestra.run.smithi105.stdout:Repeating the final host check... 2024-09-16T01:52:35.540 INFO:teuthology.orchestra.run.smithi105.stdout:/bin/podman: stdout 5.2.2 2024-09-16T01:52:35.540 INFO:teuthology.orchestra.run.smithi105.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-16T01:52:35.540 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl is present 2024-09-16T01:52:35.540 INFO:teuthology.orchestra.run.smithi105.stdout:lvcreate is present 2024-09-16T01:52:35.546 INFO:teuthology.orchestra.run.smithi105.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-16T01:52:35.547 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-16T01:52:35.553 INFO:teuthology.orchestra.run.smithi105.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-16T01:52:35.553 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stdout inactive 2024-09-16T01:52:35.559 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stdout enabled 2024-09-16T01:52:35.566 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stdout active 2024-09-16T01:52:35.566 INFO:teuthology.orchestra.run.smithi105.stdout:Unit chronyd.service is enabled and running 2024-09-16T01:52:35.566 INFO:teuthology.orchestra.run.smithi105.stdout:Host looks OK 2024-09-16T01:52:35.566 INFO:teuthology.orchestra.run.smithi105.stdout:Cluster fsid: 423b7a2a-73ce-11ef-bceb-c7b262605968 2024-09-16T01:52:35.566 INFO:teuthology.orchestra.run.smithi105.stdout:Acquiring lock 140511105258496 on /run/cephadm/423b7a2a-73ce-11ef-bceb-c7b262605968.lock 2024-09-16T01:52:35.566 INFO:teuthology.orchestra.run.smithi105.stdout:Lock 140511105258496 acquired on /run/cephadm/423b7a2a-73ce-11ef-bceb-c7b262605968.lock 2024-09-16T01:52:35.566 INFO:teuthology.orchestra.run.smithi105.stdout:Verifying IP 172.21.15.105 port 3300 ... 2024-09-16T01:52:35.567 INFO:teuthology.orchestra.run.smithi105.stdout:Verifying IP 172.21.15.105 port 6789 ... 2024-09-16T01:52:35.567 INFO:teuthology.orchestra.run.smithi105.stdout:Base mon IP(s) is [172.21.15.105:3300, 172.21.15.105:6789], mon addrv is [v2:172.21.15.105:3300,v1:172.21.15.105:6789] 2024-09-16T01:52:35.570 INFO:teuthology.orchestra.run.smithi105.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.105 metric 100 2024-09-16T01:52:35.570 INFO:teuthology.orchestra.run.smithi105.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.105 metric 100 2024-09-16T01:52:35.572 INFO:teuthology.orchestra.run.smithi105.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-16T01:52:35.573 INFO:teuthology.orchestra.run.smithi105.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-16T01:52:35.573 INFO:teuthology.orchestra.run.smithi105.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-16T01:52:35.575 INFO:teuthology.orchestra.run.smithi105.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-16T01:52:35.575 INFO:teuthology.orchestra.run.smithi105.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-16T01:52:35.575 INFO:teuthology.orchestra.run.smithi105.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-16T01:52:35.575 INFO:teuthology.orchestra.run.smithi105.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-16T01:52:35.575 INFO:teuthology.orchestra.run.smithi105.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d429/64 scope link noprefixroute 2024-09-16T01:52:35.575 INFO:teuthology.orchestra.run.smithi105.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-16T01:52:35.576 INFO:teuthology.orchestra.run.smithi105.stdout:Mon IP `172.21.15.105` is in CIDR network `172.21.0.0/20` 2024-09-16T01:52:35.576 INFO:teuthology.orchestra.run.smithi105.stdout:Mon IP `172.21.15.105` is in CIDR network `172.21.0.0/20` 2024-09-16T01:52:35.576 INFO:teuthology.orchestra.run.smithi105.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-16T01:52:35.576 INFO:teuthology.orchestra.run.smithi105.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-16T01:52:35.577 INFO:teuthology.orchestra.run.smithi105.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-16T01:52:36.044 INFO:teuthology.orchestra.run.smithi105.stdout:/bin/podman: stdout f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee 2024-09-16T01:52:36.044 INFO:teuthology.orchestra.run.smithi105.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-16T01:52:36.044 INFO:teuthology.orchestra.run.smithi105.stdout:/bin/podman: stderr Getting image source signatures 2024-09-16T01:52:36.044 INFO:teuthology.orchestra.run.smithi105.stdout:/bin/podman: stderr Copying blob sha256:c0574da0c3174f3dc49daa274908f97d06456bd6a3cfab87e94022c1369a83a6 2024-09-16T01:52:36.044 INFO:teuthology.orchestra.run.smithi105.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-16T01:52:36.044 INFO:teuthology.orchestra.run.smithi105.stdout:/bin/podman: stderr Copying config sha256:f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee 2024-09-16T01:52:36.045 INFO:teuthology.orchestra.run.smithi105.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-16T01:52:36.965 INFO:teuthology.orchestra.run.smithi105.stdout:ceph: stdout ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable) 2024-09-16T01:52:36.965 INFO:teuthology.orchestra.run.smithi105.stdout:Ceph version: ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable) 2024-09-16T01:52:36.965 INFO:teuthology.orchestra.run.smithi105.stdout:Extracting ceph user uid/gid from container image... 2024-09-16T01:52:37.896 INFO:teuthology.orchestra.run.smithi105.stdout:stat: stdout 167 167 2024-09-16T01:52:37.896 INFO:teuthology.orchestra.run.smithi105.stdout:Creating initial keys... 2024-09-16T01:52:38.893 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph-authtool: stdout AQBmj+dmsdyJCxAA+IZRY9F0U6yUk0KnHqYoZA== 2024-09-16T01:52:39.882 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph-authtool: stdout AQBnj+dmK3HpDRAAS3kGSzQXdF7bMOhYkKEV1g== 2024-09-16T01:52:40.932 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph-authtool: stdout AQBoj+dm8/VaDRAAf/t46GgW67Fsy9lsSYyffw== 2024-09-16T01:52:40.933 INFO:teuthology.orchestra.run.smithi105.stdout:Creating initial monmap... 2024-09-16T01:52:41.910 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-16T01:52:41.911 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-16T01:52:41.911 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 423b7a2a-73ce-11ef-bceb-c7b262605968 2024-09-16T01:52:41.911 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-16T01:52:41.911 INFO:teuthology.orchestra.run.smithi105.stdout:monmaptool for a [v2:172.21.15.105:3300,v1:172.21.15.105:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-16T01:52:41.911 INFO:teuthology.orchestra.run.smithi105.stdout:setting min_mon_release = quincy 2024-09-16T01:52:41.911 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/monmaptool: set fsid to 423b7a2a-73ce-11ef-bceb-c7b262605968 2024-09-16T01:52:41.911 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-16T01:52:41.911 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:52:41.911 INFO:teuthology.orchestra.run.smithi105.stdout:Creating mon... 2024-09-16T01:52:42.883 INFO:teuthology.orchestra.run.smithi105.stdout:create mon.a on 2024-09-16T01:52:43.105 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-16T01:52:43.257 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-16T01:52:43.459 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-423b7a2a-73ce-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-423b7a2a-73ce-11ef-bceb-c7b262605968.target. 2024-09-16T01:52:43.459 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-423b7a2a-73ce-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-423b7a2a-73ce-11ef-bceb-c7b262605968.target. 2024-09-16T01:52:43.713 INFO:teuthology.orchestra.run.smithi105.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mon.a 2024-09-16T01:52:43.713 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stderr Failed to reset failed state of unit ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-16T01:52:43.892 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-423b7a2a-73ce-11ef-bceb-c7b262605968.target.wants/ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@.service. 2024-09-16T01:52:43.946 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:43 smithi105 systemd[1]: Starting Ceph mon.a for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T01:52:44.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:44 smithi105 podman[30811]: 2024-09-16 01:52:44.031052374 +0000 UTC m=+0.015845120 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T01:52:44.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:44 smithi105 podman[30811]: 2024-09-16 01:52:44.131535163 +0000 UTC m=+0.116327916 container create c393be70099d6a1a929d7f287d781c3d4d1c5520f9bc3291e1ac5bf5b65185c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, io.buildah.version=1.37.2) 2024-09-16T01:52:44.465 INFO:teuthology.orchestra.run.smithi105.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T01:52:44.465 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stdout disabled 2024-09-16T01:52:44.473 INFO:teuthology.orchestra.run.smithi105.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T01:52:44.473 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stdout inactive 2024-09-16T01:52:44.473 INFO:teuthology.orchestra.run.smithi105.stdout:firewalld.service is not enabled 2024-09-16T01:52:44.473 INFO:teuthology.orchestra.run.smithi105.stdout:Not possible to enable service . firewalld.service is not available 2024-09-16T01:52:44.474 INFO:teuthology.orchestra.run.smithi105.stdout:Waiting for mon to start... 2024-09-16T01:52:44.474 INFO:teuthology.orchestra.run.smithi105.stdout:Waiting for mon... 2024-09-16T01:52:44.500 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:44 smithi105 podman[30811]: 2024-09-16 01:52:44.348479734 +0000 UTC m=+0.333272481 container init c393be70099d6a1a929d7f287d781c3d4d1c5520f9bc3291e1ac5bf5b65185c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mon-a, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-8921c8e) 2024-09-16T01:52:44.500 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:44 smithi105 podman[30811]: 2024-09-16 01:52:44.351780492 +0000 UTC m=+0.336573239 container start c393be70099d6a1a929d7f287d781c3d4d1c5520f9bc3291e1ac5bf5b65185c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mon-a, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-16T01:52:44.501 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:44 smithi105 ceph-mon[30829]: mkfs 423b7a2a-73ce-11ef-bceb-c7b262605968 2024-09-16T01:52:44.501 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:44 smithi105 ceph-mon[30829]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-16T01:52:44.501 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:44 smithi105 bash[30811]: c393be70099d6a1a929d7f287d781c3d4d1c5520f9bc3291e1ac5bf5b65185c9 2024-09-16T01:52:44.501 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:44 smithi105 systemd[1]: Started Ceph mon.a for 423b7a2a-73ce-11ef-bceb-c7b262605968. 2024-09-16T01:52:45.533 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:45 smithi105 ceph-mon[30829]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-16T01:52:45.533 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:45 smithi105 ceph-mon[30829]: monmap epoch 1 2024-09-16T01:52:45.533 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:45 smithi105 ceph-mon[30829]: fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 2024-09-16T01:52:45.533 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:45 smithi105 ceph-mon[30829]: last_changed 2024-09-16T01:52:41.310216+0000 2024-09-16T01:52:45.533 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:45 smithi105 ceph-mon[30829]: created 2024-09-16T01:52:41.310216+0000 2024-09-16T01:52:45.533 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:45 smithi105 ceph-mon[30829]: min_mon_release 19 (squid) 2024-09-16T01:52:45.533 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:45 smithi105 ceph-mon[30829]: election_strategy: 1 2024-09-16T01:52:45.534 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:45 smithi105 ceph-mon[30829]: 0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-16T01:52:45.534 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:45 smithi105 ceph-mon[30829]: fsmap 2024-09-16T01:52:45.534 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:45 smithi105 ceph-mon[30829]: osdmap e1: 0 total, 0 up, 0 in 2024-09-16T01:52:45.534 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:45 smithi105 ceph-mon[30829]: mgrmap e1: no daemons active 2024-09-16T01:52:45.534 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:45 smithi105 ceph-mon[30829]: from='client.? 172.21.15.105:0/1262607179' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-16T01:52:45.541 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout cluster: 2024-09-16T01:52:45.542 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout id: 423b7a2a-73ce-11ef-bceb-c7b262605968 2024-09-16T01:52:45.542 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-16T01:52:45.542 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout 2024-09-16T01:52:45.542 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout services: 2024-09-16T01:52:45.542 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.577573s) 2024-09-16T01:52:45.542 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-16T01:52:45.542 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-16T01:52:45.542 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout 2024-09-16T01:52:45.543 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout data: 2024-09-16T01:52:45.543 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-16T01:52:45.543 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-16T01:52:45.543 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-16T01:52:45.543 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout pgs: 2024-09-16T01:52:45.543 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout 2024-09-16T01:52:45.543 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.896+0000 7f99c2575640 1 Processor -- start 2024-09-16T01:52:45.543 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.896+0000 7f99c2575640 1 -- start start 2024-09-16T01:52:45.543 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.896+0000 7f99c2575640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99bc107510 0x7f99bc107910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:45.543 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.896+0000 7f99c2575640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f99bc107ee0 con 0x7f99bc107510 2024-09-16T01:52:45.543 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.897+0000 7f99bbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99bc107510 0x7f99bc107910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:45.543 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.897+0000 7f99bbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99bc107510 0x7f99bc107910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43404/0 (socket says 172.21.15.105:43404) 2024-09-16T01:52:45.543 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.897+0000 7f99bbfff640 1 -- 172.21.15.105:0/3991206231 learned_addr learned my addr 172.21.15.105:0/3991206231 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:52:45.544 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.897+0000 7f99bbfff640 1 -- 172.21.15.105:0/3991206231 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99bc1086c0 con 0x7f99bc107510 2024-09-16T01:52:45.544 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.899+0000 7f99bbfff640 1 --2- 172.21.15.105:0/3991206231 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99bc107510 0x7f99bc107910 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f99ac009820 tx=0x7f99ac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed7098fbc0d8b0e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:45.544 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.899+0000 7f99baffd640 1 -- 172.21.15.105:0/3991206231 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99ac037070 con 0x7f99bc107510 2024-09-16T01:52:45.544 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.899+0000 7f99baffd640 1 -- 172.21.15.105:0/3991206231 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99ac032440 con 0x7f99bc107510 2024-09-16T01:52:45.544 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.899+0000 7f99baffd640 1 -- 172.21.15.105:0/3991206231 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99ac032740 con 0x7f99bc107510 2024-09-16T01:52:45.544 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.900+0000 7f99c2575640 1 -- 172.21.15.105:0/3991206231 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99bc107510 msgr2=0x7f99bc107910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:45.544 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.900+0000 7f99c2575640 1 --2- 172.21.15.105:0/3991206231 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99bc107510 0x7f99bc107910 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f99ac009820 tx=0x7f99ac02a270 comp rx=0 tx=0).stop 2024-09-16T01:52:45.544 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.900+0000 7f99c2575640 1 -- 172.21.15.105:0/3991206231 shutdown_connections 2024-09-16T01:52:45.544 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.900+0000 7f99c2575640 1 --2- 172.21.15.105:0/3991206231 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99bc107510 0x7f99bc107910 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:45.544 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.900+0000 7f99c2575640 1 -- 172.21.15.105:0/3991206231 >> 172.21.15.105:0/3991206231 conn(0x7f99bc102ca0 msgr2=0x7f99bc1050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:45.544 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.900+0000 7f99c2575640 1 -- 172.21.15.105:0/3991206231 shutdown_connections 2024-09-16T01:52:45.545 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.900+0000 7f99c2575640 1 -- 172.21.15.105:0/3991206231 wait complete. 2024-09-16T01:52:45.545 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.900+0000 7f99c2575640 1 Processor -- start 2024-09-16T01:52:45.545 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.900+0000 7f99c2575640 1 -- start start 2024-09-16T01:52:45.545 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.901+0000 7f99c2575640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99bc107510 0x7f99bc19af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:45.545 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.901+0000 7f99c2575640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f99bc108c60 con 0x7f99bc107510 2024-09-16T01:52:45.545 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.901+0000 7f99bbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99bc107510 0x7f99bc19af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:45.545 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.901+0000 7f99bbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99bc107510 0x7f99bc19af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43414/0 (socket says 172.21.15.105:43414) 2024-09-16T01:52:45.545 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.901+0000 7f99bbfff640 1 -- 172.21.15.105:0/1262607179 learned_addr learned my addr 172.21.15.105:0/1262607179 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:52:45.545 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.901+0000 7f99bbfff640 1 -- 172.21.15.105:0/1262607179 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99bc19b460 con 0x7f99bc107510 2024-09-16T01:52:45.545 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.901+0000 7f99bbfff640 1 --2- 172.21.15.105:0/1262607179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99bc107510 0x7f99bc19af20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f99ac0090b0 tx=0x7f99ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:45.545 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.901+0000 7f99b97fa640 1 -- 172.21.15.105:0/1262607179 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99ac041070 con 0x7f99bc107510 2024-09-16T01:52:45.545 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.901+0000 7f99b97fa640 1 -- 172.21.15.105:0/1262607179 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99ac03c7a0 con 0x7f99bc107510 2024-09-16T01:52:45.545 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.901+0000 7f99c2575640 1 -- 172.21.15.105:0/1262607179 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99bc19b6f0 con 0x7f99bc107510 2024-09-16T01:52:45.545 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.901+0000 7f99b97fa640 1 -- 172.21.15.105:0/1262607179 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99ac037040 con 0x7f99bc107510 2024-09-16T01:52:45.546 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.902+0000 7f99c2575640 1 -- 172.21.15.105:0/1262607179 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99bc19e3e0 con 0x7f99bc107510 2024-09-16T01:52:45.546 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.902+0000 7f99b97fa640 1 -- 172.21.15.105:0/1262607179 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f99ac044070 con 0x7f99bc107510 2024-09-16T01:52:45.546 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.902+0000 7f99b97fa640 1 -- 172.21.15.105:0/1262607179 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f99ac03d3f0 con 0x7f99bc107510 2024-09-16T01:52:45.546 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.902+0000 7f99c2575640 1 -- 172.21.15.105:0/1262607179 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99bc1092b0 con 0x7f99bc107510 2024-09-16T01:52:45.546 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.905+0000 7f99b97fa640 1 -- 172.21.15.105:0/1262607179 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f99ac046450 con 0x7f99bc107510 2024-09-16T01:52:45.546 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.964+0000 7f99c2575640 1 -- 172.21.15.105:0/1262607179 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f99bc075e60 con 0x7f99bc107510 2024-09-16T01:52:45.546 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.964+0000 7f99b97fa640 1 -- 172.21.15.105:0/1262607179 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f99ac046630 con 0x7f99bc107510 2024-09-16T01:52:45.546 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.965+0000 7f99c2575640 1 -- 172.21.15.105:0/1262607179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99bc107510 msgr2=0x7f99bc19af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:45.546 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.965+0000 7f99c2575640 1 --2- 172.21.15.105:0/1262607179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99bc107510 0x7f99bc19af20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f99ac0090b0 tx=0x7f99ac0046c0 comp rx=0 tx=0).stop 2024-09-16T01:52:45.546 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.965+0000 7f99c2575640 1 -- 172.21.15.105:0/1262607179 shutdown_connections 2024-09-16T01:52:45.546 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.965+0000 7f99c2575640 1 --2- 172.21.15.105:0/1262607179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99bc107510 0x7f99bc19af20 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:45.547 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.965+0000 7f99c2575640 1 -- 172.21.15.105:0/1262607179 >> 172.21.15.105:0/1262607179 conn(0x7f99bc102ca0 msgr2=0x7f99bc1919b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:45.547 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.965+0000 7f99c2575640 1 -- 172.21.15.105:0/1262607179 shutdown_connections 2024-09-16T01:52:45.547 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:44.965+0000 7f99c2575640 1 -- 172.21.15.105:0/1262607179 wait complete. 2024-09-16T01:52:45.547 INFO:teuthology.orchestra.run.smithi105.stdout:mon is available 2024-09-16T01:52:45.547 INFO:teuthology.orchestra.run.smithi105.stdout:Assimilating anything we can from ceph.conf... 2024-09-16T01:52:46.544 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:46 smithi105 ceph-mon[30829]: from='client.? 172.21.15.105:0/119727723' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-16T01:52:46.544 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:46 smithi105 ceph-mon[30829]: from='client.? 172.21.15.105:0/119727723' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-16T01:52:46.574 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout 2024-09-16T01:52:46.581 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout [global] 2024-09-16T01:52:46.581 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout fsid = 423b7a2a-73ce-11ef-bceb-c7b262605968 2024-09-16T01:52:46.581 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.105:3300,v1:172.21.15.105:6789] 2024-09-16T01:52:46.581 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-16T01:52:46.581 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-16T01:52:46.582 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-16T01:52:46.582 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-16T01:52:46.582 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout 2024-09-16T01:52:46.582 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-16T01:52:46.582 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-16T01:52:46.582 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout 2024-09-16T01:52:46.582 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout [osd] 2024-09-16T01:52:46.582 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-16T01:52:46.582 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-16T01:52:46.582 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-16T01:52:46.582 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-16T01:52:46.582 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-16T01:52:46.582 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.912+0000 7fd423c30640 1 Processor -- start 2024-09-16T01:52:46.583 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.912+0000 7fd423c30640 1 -- start start 2024-09-16T01:52:46.583 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.912+0000 7fd423c30640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd41c1074f0 0x7fd41c1078f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:46.583 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.912+0000 7fd423c30640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd41c107ec0 con 0x7fd41c1074f0 2024-09-16T01:52:46.583 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.913+0000 7fd4219cc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd41c1074f0 0x7fd41c1078f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:46.583 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.913+0000 7fd4219cc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd41c1074f0 0x7fd41c1078f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43420/0 (socket says 172.21.15.105:43420) 2024-09-16T01:52:46.583 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.913+0000 7fd4219cc640 1 -- 172.21.15.105:0/3585722224 learned_addr learned my addr 172.21.15.105:0/3585722224 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:52:46.583 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.913+0000 7fd4219cc640 1 -- 172.21.15.105:0/3585722224 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd41c1086a0 con 0x7fd41c1074f0 2024-09-16T01:52:46.583 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.914+0000 7fd4219cc640 1 --2- 172.21.15.105:0/3585722224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd41c1074f0 0x7fd41c1078f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd414009820 tx=0x7fd41402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12eda7f895a70c66 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:46.583 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.914+0000 7fd4209ca640 1 -- 172.21.15.105:0/3585722224 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd414037070 con 0x7fd41c1074f0 2024-09-16T01:52:46.583 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.914+0000 7fd4209ca640 1 -- 172.21.15.105:0/3585722224 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd414032440 con 0x7fd41c1074f0 2024-09-16T01:52:46.583 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.914+0000 7fd4209ca640 1 -- 172.21.15.105:0/3585722224 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd414032740 con 0x7fd41c1074f0 2024-09-16T01:52:46.583 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.914+0000 7fd423c30640 1 -- 172.21.15.105:0/3585722224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd41c1074f0 msgr2=0x7fd41c1078f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:46.584 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.914+0000 7fd423c30640 1 --2- 172.21.15.105:0/3585722224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd41c1074f0 0x7fd41c1078f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd414009820 tx=0x7fd41402a270 comp rx=0 tx=0).stop 2024-09-16T01:52:46.584 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.914+0000 7fd423c30640 1 -- 172.21.15.105:0/3585722224 shutdown_connections 2024-09-16T01:52:46.584 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.914+0000 7fd423c30640 1 --2- 172.21.15.105:0/3585722224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd41c1074f0 0x7fd41c1078f0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:46.584 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.914+0000 7fd423c30640 1 -- 172.21.15.105:0/3585722224 >> 172.21.15.105:0/3585722224 conn(0x7fd41c102c40 msgr2=0x7fd41c1050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:46.584 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.914+0000 7fd423c30640 1 -- 172.21.15.105:0/3585722224 shutdown_connections 2024-09-16T01:52:46.584 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.915+0000 7fd423c30640 1 -- 172.21.15.105:0/3585722224 wait complete. 2024-09-16T01:52:46.584 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.915+0000 7fd423c30640 1 Processor -- start 2024-09-16T01:52:46.585 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.915+0000 7fd423c30640 1 -- start start 2024-09-16T01:52:46.585 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.915+0000 7fd423c30640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd41c1074f0 0x7fd41c19afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:46.585 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.915+0000 7fd423c30640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd41c108c40 con 0x7fd41c1074f0 2024-09-16T01:52:46.585 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.915+0000 7fd4219cc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd41c1074f0 0x7fd41c19afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:46.585 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.915+0000 7fd4219cc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd41c1074f0 0x7fd41c19afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43436/0 (socket says 172.21.15.105:43436) 2024-09-16T01:52:46.585 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.915+0000 7fd4219cc640 1 -- 172.21.15.105:0/119727723 learned_addr learned my addr 172.21.15.105:0/119727723 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:52:46.585 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.916+0000 7fd4219cc640 1 -- 172.21.15.105:0/119727723 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd41c19b4e0 con 0x7fd41c1074f0 2024-09-16T01:52:46.585 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.916+0000 7fd4219cc640 1 --2- 172.21.15.105:0/119727723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd41c1074f0 0x7fd41c19afa0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd4140090b0 tx=0x7fd4140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:46.585 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.916+0000 7fd41affd640 1 -- 172.21.15.105:0/119727723 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd414041070 con 0x7fd41c1074f0 2024-09-16T01:52:46.585 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.916+0000 7fd41affd640 1 -- 172.21.15.105:0/119727723 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd41403c7a0 con 0x7fd41c1074f0 2024-09-16T01:52:46.586 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.916+0000 7fd423c30640 1 -- 172.21.15.105:0/119727723 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd41c19b770 con 0x7fd41c1074f0 2024-09-16T01:52:46.586 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.916+0000 7fd41affd640 1 -- 172.21.15.105:0/119727723 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd414037040 con 0x7fd41c1074f0 2024-09-16T01:52:46.586 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.916+0000 7fd423c30640 1 -- 172.21.15.105:0/119727723 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd41c19e460 con 0x7fd41c1074f0 2024-09-16T01:52:46.586 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.916+0000 7fd41affd640 1 -- 172.21.15.105:0/119727723 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd414044070 con 0x7fd41c1074f0 2024-09-16T01:52:46.586 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.916+0000 7fd41affd640 1 -- 172.21.15.105:0/119727723 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd41403d3f0 con 0x7fd41c1074f0 2024-09-16T01:52:46.586 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.917+0000 7fd423c30640 1 -- 172.21.15.105:0/119727723 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd41c109290 con 0x7fd41c1074f0 2024-09-16T01:52:46.586 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.919+0000 7fd41affd640 1 -- 172.21.15.105:0/119727723 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fd414046450 con 0x7fd41c1074f0 2024-09-16T01:52:46.586 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.977+0000 7fd423c30640 1 -- 172.21.15.105:0/119727723 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fd41c075e60 con 0x7fd41c1074f0 2024-09-16T01:52:46.586 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.980+0000 7fd41affd640 1 -- 172.21.15.105:0/119727723 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+431 (secure 0 0 0) 0x7fd414046630 con 0x7fd41c1074f0 2024-09-16T01:52:46.586 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.980+0000 7fd41affd640 1 -- 172.21.15.105:0/119727723 <== mon.0 v2:172.21.15.105:3300/0 8 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fd414056070 con 0x7fd41c1074f0 2024-09-16T01:52:46.586 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.981+0000 7fd423c30640 1 -- 172.21.15.105:0/119727723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd41c1074f0 msgr2=0x7fd41c19afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:46.586 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.981+0000 7fd423c30640 1 --2- 172.21.15.105:0/119727723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd41c1074f0 0x7fd41c19afa0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd4140090b0 tx=0x7fd4140046c0 comp rx=0 tx=0).stop 2024-09-16T01:52:46.586 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.981+0000 7fd423c30640 1 -- 172.21.15.105:0/119727723 shutdown_connections 2024-09-16T01:52:46.586 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.981+0000 7fd423c30640 1 --2- 172.21.15.105:0/119727723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd41c1074f0 0x7fd41c19afa0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:46.586 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.981+0000 7fd423c30640 1 -- 172.21.15.105:0/119727723 >> 172.21.15.105:0/119727723 conn(0x7fd41c102c40 msgr2=0x7fd41c1919e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:46.587 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.981+0000 7fd423c30640 1 -- 172.21.15.105:0/119727723 shutdown_connections 2024-09-16T01:52:46.587 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:45.982+0000 7fd423c30640 1 -- 172.21.15.105:0/119727723 wait complete. 2024-09-16T01:52:46.587 INFO:teuthology.orchestra.run.smithi105.stdout:Generating new minimal ceph.conf... 2024-09-16T01:52:47.628 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:47 smithi105 ceph-mon[30829]: from='client.? 172.21.15.105:0/1411988515' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:52:47.635 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:46.967+0000 7f4de5bdc640 1 Processor -- start 2024-09-16T01:52:47.635 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:46.967+0000 7f4de5bdc640 1 -- start start 2024-09-16T01:52:47.635 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:46.967+0000 7f4de5bdc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4de0106380 0x7f4de0106780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:47.635 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:46.967+0000 7f4de5bdc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4de0106d50 con 0x7f4de0106380 2024-09-16T01:52:47.635 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:46.967+0000 7f4ddf7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4de0106380 0x7f4de0106780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:47.635 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:46.967+0000 7f4ddf7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4de0106380 0x7f4de0106780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43452/0 (socket says 172.21.15.105:43452) 2024-09-16T01:52:47.635 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:46.968+0000 7f4ddf7fe640 1 -- 172.21.15.105:0/3636238022 learned_addr learned my addr 172.21.15.105:0/3636238022 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:52:47.635 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:46.968+0000 7f4ddf7fe640 1 -- 172.21.15.105:0/3636238022 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de0107580 con 0x7f4de0106380 2024-09-16T01:52:47.635 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:46.968+0000 7f4ddf7fe640 1 --2- 172.21.15.105:0/3636238022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4de0106380 0x7f4de0106780 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0009e80 tx=0x7f4dd002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc423ed1d14f7398 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:47.636 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:46.969+0000 7f4dde7fc640 1 -- 172.21.15.105:0/3636238022 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4dd0036070 con 0x7f4de0106380 2024-09-16T01:52:47.636 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:46.969+0000 7f4dde7fc640 1 -- 172.21.15.105:0/3636238022 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f4dd0031890 con 0x7f4de0106380 2024-09-16T01:52:47.636 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:46.969+0000 7f4dde7fc640 1 -- 172.21.15.105:0/3636238022 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4dd0007980 con 0x7f4de0106380 2024-09-16T01:52:47.636 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:46.969+0000 7f4de5bdc640 1 -- 172.21.15.105:0/3636238022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4de0106380 msgr2=0x7f4de0106780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:47.636 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:46.969+0000 7f4de5bdc640 1 --2- 172.21.15.105:0/3636238022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4de0106380 0x7f4de0106780 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0009e80 tx=0x7f4dd002a7a0 comp rx=0 tx=0).stop 2024-09-16T01:52:47.636 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:46.969+0000 7f4de5bdc640 1 -- 172.21.15.105:0/3636238022 shutdown_connections 2024-09-16T01:52:47.636 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:46.969+0000 7f4de5bdc640 1 --2- 172.21.15.105:0/3636238022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4de0106380 0x7f4de0106780 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:47.636 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:46.969+0000 7f4de5bdc640 1 -- 172.21.15.105:0/3636238022 >> 172.21.15.105:0/3636238022 conn(0x7f4de0101b10 msgr2=0x7f4de0103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:47.637 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:46.969+0000 7f4de5bdc640 1 -- 172.21.15.105:0/3636238022 shutdown_connections 2024-09-16T01:52:47.637 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:46.969+0000 7f4de5bdc640 1 -- 172.21.15.105:0/3636238022 wait complete. 2024-09-16T01:52:47.637 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:46.970+0000 7f4de5bdc640 1 Processor -- start 2024-09-16T01:52:47.637 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:46.970+0000 7f4de5bdc640 1 -- start start 2024-09-16T01:52:47.637 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:46.970+0000 7f4de5bdc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4de0106380 0x7f4de007ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:47.637 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:46.970+0000 7f4de5bdc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4de010a930 con 0x7f4de0106380 2024-09-16T01:52:47.637 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:46.970+0000 7f4ddf7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4de0106380 0x7f4de007ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:47.637 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:46.970+0000 7f4ddf7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4de0106380 0x7f4de007ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43458/0 (socket says 172.21.15.105:43458) 2024-09-16T01:52:47.637 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:46.970+0000 7f4ddf7fe640 1 -- 172.21.15.105:0/1411988515 learned_addr learned my addr 172.21.15.105:0/1411988515 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:52:47.637 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:46.971+0000 7f4ddf7fe640 1 -- 172.21.15.105:0/1411988515 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de019e4d0 con 0x7f4de0106380 2024-09-16T01:52:47.637 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:46.971+0000 7f4ddf7fe640 1 --2- 172.21.15.105:0/1411988515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4de0106380 0x7f4de007ba70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4dd002acc0 tx=0x7f4dd0031c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:47.637 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:46.971+0000 7f4ddcff9640 1 -- 172.21.15.105:0/1411988515 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4dd0041030 con 0x7f4de0106380 2024-09-16T01:52:47.637 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:46.971+0000 7f4ddcff9640 1 -- 172.21.15.105:0/1411988515 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f4dd00073f0 con 0x7f4de0106380 2024-09-16T01:52:47.638 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:46.971+0000 7f4de5bdc640 1 -- 172.21.15.105:0/1411988515 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4de019e6a0 con 0x7f4de0106380 2024-09-16T01:52:47.638 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:46.971+0000 7f4ddcff9640 1 -- 172.21.15.105:0/1411988515 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4dd0036040 con 0x7f4de0106380 2024-09-16T01:52:47.638 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:46.971+0000 7f4de5bdc640 1 -- 172.21.15.105:0/1411988515 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4de019eb20 con 0x7f4de0106380 2024-09-16T01:52:47.638 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:46.971+0000 7f4ddcff9640 1 -- 172.21.15.105:0/1411988515 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4dd0044070 con 0x7f4de0106380 2024-09-16T01:52:47.638 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:46.971+0000 7f4ddcff9640 1 -- 172.21.15.105:0/1411988515 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4dd0042070 con 0x7f4de0106380 2024-09-16T01:52:47.638 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:46.972+0000 7f4de5bdc640 1 -- 172.21.15.105:0/1411988515 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4de010aab0 con 0x7f4de0106380 2024-09-16T01:52:47.638 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:46.974+0000 7f4ddcff9640 1 -- 172.21.15.105:0/1411988515 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f4dd004b070 con 0x7f4de0106380 2024-09-16T01:52:47.638 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:47.032+0000 7f4de5bdc640 1 -- 172.21.15.105:0/1411988515 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f4de0075d10 con 0x7f4de0106380 2024-09-16T01:52:47.638 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:47.032+0000 7f4ddcff9640 1 -- 172.21.15.105:0/1411988515 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7f4dd0046be0 con 0x7f4de0106380 2024-09-16T01:52:47.638 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:47.033+0000 7f4de5bdc640 1 -- 172.21.15.105:0/1411988515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4de0106380 msgr2=0x7f4de007ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:47.638 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:47.033+0000 7f4de5bdc640 1 --2- 172.21.15.105:0/1411988515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4de0106380 0x7f4de007ba70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4dd002acc0 tx=0x7f4dd0031c70 comp rx=0 tx=0).stop 2024-09-16T01:52:47.638 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:47.033+0000 7f4de5bdc640 1 -- 172.21.15.105:0/1411988515 shutdown_connections 2024-09-16T01:52:47.638 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:47.033+0000 7f4de5bdc640 1 --2- 172.21.15.105:0/1411988515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4de0106380 0x7f4de007ba70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:47.639 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:47.033+0000 7f4de5bdc640 1 -- 172.21.15.105:0/1411988515 >> 172.21.15.105:0/1411988515 conn(0x7f4de0101b10 msgr2=0x7f4de01028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:47.639 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:47.033+0000 7f4de5bdc640 1 -- 172.21.15.105:0/1411988515 shutdown_connections 2024-09-16T01:52:47.639 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:47.033+0000 7f4de5bdc640 1 -- 172.21.15.105:0/1411988515 wait complete. 2024-09-16T01:52:47.639 INFO:teuthology.orchestra.run.smithi105.stdout:Restarting the monitor... 2024-09-16T01:52:47.915 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:47 smithi105 systemd[1]: Stopping Ceph mon.a for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T01:52:48.170 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:47 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mon-a[30825]: 2024-09-16T01:52:47.914+0000 7f8bace24640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T01:52:48.170 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:47 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mon-a[30825]: 2024-09-16T01:52:47.914+0000 7f8bace24640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-16T01:52:48.439 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:48 smithi105 podman[31073]: 2024-09-16 01:52:48.169248697 +0000 UTC m=+0.367195020 container died c393be70099d6a1a929d7f287d781c3d4d1c5520f9bc3291e1ac5bf5b65185c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mon-a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True) 2024-09-16T01:52:48.791 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:48 smithi105 podman[31073]: 2024-09-16 01:52:48.539859157 +0000 UTC m=+0.737805480 container cleanup c393be70099d6a1a929d7f287d781c3d4d1c5520f9bc3291e1ac5bf5b65185c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mon-a, ceph=True, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-16T01:52:48.791 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:48 smithi105 bash[31073]: ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mon-a 2024-09-16T01:52:49.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:48 smithi105 podman[31085]: 2024-09-16 01:52:48.790549187 +0000 UTC m=+0.622615886 container remove c393be70099d6a1a929d7f287d781c3d4d1c5520f9bc3291e1ac5bf5b65185c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mon-a, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-16T01:52:49.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:48 smithi105 systemd[1]: ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-16T01:52:49.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:48 smithi105 systemd[1]: Stopped Ceph mon.a for 423b7a2a-73ce-11ef-bceb-c7b262605968. 2024-09-16T01:52:49.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:48 smithi105 systemd[1]: Starting Ceph mon.a for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T01:52:49.436 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 podman[31177]: 2024-09-16 01:52:49.17457193 +0000 UTC m=+0.101068519 container create 6554b6c6100439457da03be64f7dfc7f6dad18f10370ba1b0dc3f80b10cc1cf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mon-a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-16T01:52:49.436 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 podman[31177]: 2024-09-16 01:52:49.089007554 +0000 UTC m=+0.015504148 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T01:52:49.436 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 podman[31177]: 2024-09-16 01:52:49.360133311 +0000 UTC m=+0.286629899 container init 6554b6c6100439457da03be64f7dfc7f6dad18f10370ba1b0dc3f80b10cc1cf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mon-a, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-16T01:52:49.436 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 podman[31177]: 2024-09-16 01:52:49.363292746 +0000 UTC m=+0.289789337 container start 6554b6c6100439457da03be64f7dfc7f6dad18f10370ba1b0dc3f80b10cc1cf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mon-a, GIT_CLEAN=True, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T01:52:49.436 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: set uid:gid to 167:167 (ceph:ceph) 2024-09-16T01:52:49.437 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable), process ceph-mon, pid 2 2024-09-16T01:52:49.437 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: pidfile_write: ignore empty --pid-file 2024-09-16T01:52:49.437 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: load: jerasure load: lrc 2024-09-16T01:52:49.437 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: RocksDB version: 7.9.2 2024-09-16T01:52:49.437 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Git sha 0 2024-09-16T01:52:49.437 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Compile date 2024-09-15 15:04:35 2024-09-16T01:52:49.437 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: DB SUMMARY 2024-09-16T01:52:49.437 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: DB Session ID: TZAJJ6C74OYPMMPXUF5Z 2024-09-16T01:52:49.437 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: CURRENT file: CURRENT 2024-09-16T01:52:49.437 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: IDENTITY file: IDENTITY 2024-09-16T01:52:49.437 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-16T01:52:49.437 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-16T01:52:49.437 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104860 ; 2024-09-16T01:52:49.437 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.error_if_exists: 0 2024-09-16T01:52:49.438 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.create_if_missing: 0 2024-09-16T01:52:49.438 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.paranoid_checks: 1 2024-09-16T01:52:49.438 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-16T01:52:49.438 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-16T01:52:49.438 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-16T01:52:49.438 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.env: 0x560fdedc1940 2024-09-16T01:52:49.438 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.fs: PosixFileSystem 2024-09-16T01:52:49.438 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.info_log: 0x560fe010d6e0 2024-09-16T01:52:49.438 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.max_file_opening_threads: 16 2024-09-16T01:52:49.438 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.statistics: (nil) 2024-09-16T01:52:49.438 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.use_fsync: 0 2024-09-16T01:52:49.438 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.max_log_file_size: 0 2024-09-16T01:52:49.438 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-16T01:52:49.439 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-16T01:52:49.439 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.keep_log_file_num: 1000 2024-09-16T01:52:49.439 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.recycle_log_file_num: 0 2024-09-16T01:52:49.439 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.allow_fallocate: 1 2024-09-16T01:52:49.439 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.allow_mmap_reads: 0 2024-09-16T01:52:49.439 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.allow_mmap_writes: 0 2024-09-16T01:52:49.439 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.use_direct_reads: 0 2024-09-16T01:52:49.439 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-16T01:52:49.439 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.create_missing_column_families: 0 2024-09-16T01:52:49.439 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.db_log_dir: 2024-09-16T01:52:49.439 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.wal_dir: 2024-09-16T01:52:49.439 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-16T01:52:49.439 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-16T01:52:49.439 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-16T01:52:49.440 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-16T01:52:49.440 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-16T01:52:49.440 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-16T01:52:49.440 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.advise_random_on_open: 1 2024-09-16T01:52:49.440 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.db_write_buffer_size: 0 2024-09-16T01:52:49.440 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.write_buffer_manager: 0x560fe011cdc0 2024-09-16T01:52:49.440 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-16T01:52:49.440 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-16T01:52:49.440 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-16T01:52:49.440 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.rate_limiter: (nil) 2024-09-16T01:52:49.440 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-16T01:52:49.440 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.wal_recovery_mode: 2 2024-09-16T01:52:49.441 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.enable_thread_tracking: 0 2024-09-16T01:52:49.442 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.enable_pipelined_write: 0 2024-09-16T01:52:49.442 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.unordered_write: 0 2024-09-16T01:52:49.442 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-16T01:52:49.442 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-16T01:52:49.442 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-16T01:52:49.442 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-16T01:52:49.442 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.row_cache: None 2024-09-16T01:52:49.442 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.wal_filter: None 2024-09-16T01:52:49.442 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-16T01:52:49.442 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.allow_ingest_behind: 0 2024-09-16T01:52:49.442 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.two_write_queues: 0 2024-09-16T01:52:49.442 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.manual_wal_flush: 0 2024-09-16T01:52:49.443 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.wal_compression: 0 2024-09-16T01:52:49.443 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.atomic_flush: 0 2024-09-16T01:52:49.443 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-16T01:52:49.443 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-16T01:52:49.443 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-16T01:52:49.443 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.log_readahead_size: 0 2024-09-16T01:52:49.443 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-16T01:52:49.443 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.best_efforts_recovery: 0 2024-09-16T01:52:49.443 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-16T01:52:49.443 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-16T01:52:49.443 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.allow_data_in_errors: 0 2024-09-16T01:52:49.443 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.db_host_id: __hostname__ 2024-09-16T01:52:49.443 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-16T01:52:49.443 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.max_background_jobs: 2 2024-09-16T01:52:49.444 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.max_background_compactions: -1 2024-09-16T01:52:49.444 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.max_subcompactions: 1 2024-09-16T01:52:49.444 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-16T01:52:49.444 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-16T01:52:49.444 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-16T01:52:49.444 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.max_total_wal_size: 0 2024-09-16T01:52:49.444 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-16T01:52:49.444 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-16T01:52:49.444 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-16T01:52:49.444 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-16T01:52:49.444 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.max_open_files: -1 2024-09-16T01:52:49.444 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.bytes_per_sync: 0 2024-09-16T01:52:49.444 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-16T01:52:49.445 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-16T01:52:49.445 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.compaction_readahead_size: 0 2024-09-16T01:52:49.445 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.max_background_flushes: -1 2024-09-16T01:52:49.445 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Compression algorithms supported: 2024-09-16T01:52:49.445 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: kZSTD supported: 0 2024-09-16T01:52:49.445 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: kXpressCompression supported: 0 2024-09-16T01:52:49.445 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: kBZip2Compression supported: 0 2024-09-16T01:52:49.445 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-16T01:52:49.445 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: kLZ4Compression supported: 1 2024-09-16T01:52:49.445 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: kZlibCompression supported: 1 2024-09-16T01:52:49.445 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: kLZ4HCCompression supported: 1 2024-09-16T01:52:49.445 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: kSnappyCompression supported: 1 2024-09-16T01:52:49.445 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-16T01:52:49.445 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-16T01:52:49.446 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-16T01:52:49.446 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-16T01:52:49.446 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-16T01:52:49.446 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.merge_operator: 2024-09-16T01:52:49.446 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.compaction_filter: None 2024-09-16T01:52:49.446 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.compaction_filter_factory: None 2024-09-16T01:52:49.446 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.sst_partitioner_factory: None 2024-09-16T01:52:49.446 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-16T01:52:49.446 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-16T01:52:49.446 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560fe010d320) 2024-09-16T01:52:49.446 INFO:journalctl@ceph.mon.a.smithi105.stdout: cache_index_and_filter_blocks: 1 2024-09-16T01:52:49.446 INFO:journalctl@ceph.mon.a.smithi105.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-16T01:52:49.446 INFO:journalctl@ceph.mon.a.smithi105.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-16T01:52:49.446 INFO:journalctl@ceph.mon.a.smithi105.stdout: pin_top_level_index_and_filter: 1 2024-09-16T01:52:49.447 INFO:journalctl@ceph.mon.a.smithi105.stdout: index_type: 0 2024-09-16T01:52:49.447 INFO:journalctl@ceph.mon.a.smithi105.stdout: data_block_index_type: 0 2024-09-16T01:52:49.447 INFO:journalctl@ceph.mon.a.smithi105.stdout: index_shortening: 1 2024-09-16T01:52:49.447 INFO:journalctl@ceph.mon.a.smithi105.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-16T01:52:49.447 INFO:journalctl@ceph.mon.a.smithi105.stdout: checksum: 4 2024-09-16T01:52:49.447 INFO:journalctl@ceph.mon.a.smithi105.stdout: no_block_cache: 0 2024-09-16T01:52:49.447 INFO:journalctl@ceph.mon.a.smithi105.stdout: block_cache: 0x560fe01011f0 2024-09-16T01:52:49.447 INFO:journalctl@ceph.mon.a.smithi105.stdout: block_cache_name: BinnedLRUCache 2024-09-16T01:52:49.447 INFO:journalctl@ceph.mon.a.smithi105.stdout: block_cache_options: 2024-09-16T01:52:49.447 INFO:journalctl@ceph.mon.a.smithi105.stdout: capacity : 536870912 2024-09-16T01:52:49.447 INFO:journalctl@ceph.mon.a.smithi105.stdout: num_shard_bits : 4 2024-09-16T01:52:49.447 INFO:journalctl@ceph.mon.a.smithi105.stdout: strict_capacity_limit : 0 2024-09-16T01:52:49.447 INFO:journalctl@ceph.mon.a.smithi105.stdout: high_pri_pool_ratio: 0.000 2024-09-16T01:52:49.447 INFO:journalctl@ceph.mon.a.smithi105.stdout: block_cache_compressed: (nil) 2024-09-16T01:52:49.448 INFO:journalctl@ceph.mon.a.smithi105.stdout: persistent_cache: (nil) 2024-09-16T01:52:49.448 INFO:journalctl@ceph.mon.a.smithi105.stdout: block_size: 4096 2024-09-16T01:52:49.448 INFO:journalctl@ceph.mon.a.smithi105.stdout: block_size_deviation: 10 2024-09-16T01:52:49.448 INFO:journalctl@ceph.mon.a.smithi105.stdout: block_restart_interval: 16 2024-09-16T01:52:49.448 INFO:journalctl@ceph.mon.a.smithi105.stdout: index_block_restart_interval: 1 2024-09-16T01:52:49.448 INFO:journalctl@ceph.mon.a.smithi105.stdout: metadata_block_size: 4096 2024-09-16T01:52:49.448 INFO:journalctl@ceph.mon.a.smithi105.stdout: partition_filters: 0 2024-09-16T01:52:49.448 INFO:journalctl@ceph.mon.a.smithi105.stdout: use_delta_encoding: 1 2024-09-16T01:52:49.448 INFO:journalctl@ceph.mon.a.smithi105.stdout: filter_policy: bloomfilter 2024-09-16T01:52:49.448 INFO:journalctl@ceph.mon.a.smithi105.stdout: whole_key_filtering: 1 2024-09-16T01:52:49.448 INFO:journalctl@ceph.mon.a.smithi105.stdout: verify_compression: 0 2024-09-16T01:52:49.448 INFO:journalctl@ceph.mon.a.smithi105.stdout: read_amp_bytes_per_bit: 0 2024-09-16T01:52:49.448 INFO:journalctl@ceph.mon.a.smithi105.stdout: format_version: 5 2024-09-16T01:52:49.448 INFO:journalctl@ceph.mon.a.smithi105.stdout: enable_index_compression: 1 2024-09-16T01:52:49.449 INFO:journalctl@ceph.mon.a.smithi105.stdout: block_align: 0 2024-09-16T01:52:49.449 INFO:journalctl@ceph.mon.a.smithi105.stdout: max_auto_readahead_size: 262144 2024-09-16T01:52:49.449 INFO:journalctl@ceph.mon.a.smithi105.stdout: prepopulate_block_cache: 0 2024-09-16T01:52:49.449 INFO:journalctl@ceph.mon.a.smithi105.stdout: initial_auto_readahead_size: 8192 2024-09-16T01:52:49.449 INFO:journalctl@ceph.mon.a.smithi105.stdout: num_file_reads_for_auto_readahead: 2 2024-09-16T01:52:49.449 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.write_buffer_size: 33554432 2024-09-16T01:52:49.449 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.max_write_buffer_number: 2 2024-09-16T01:52:49.449 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.compression: NoCompression 2024-09-16T01:52:49.449 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.bottommost_compression: Disabled 2024-09-16T01:52:49.449 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.prefix_extractor: nullptr 2024-09-16T01:52:49.449 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-16T01:52:49.449 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.num_levels: 7 2024-09-16T01:52:49.449 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-16T01:52:49.450 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-16T01:52:49.450 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-16T01:52:49.450 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-16T01:52:49.450 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-16T01:52:49.450 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-16T01:52:49.450 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-16T01:52:49.450 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-16T01:52:49.450 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-16T01:52:49.450 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-16T01:52:49.450 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-16T01:52:49.450 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-16T01:52:49.450 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-16T01:52:49.450 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.compression_opts.level: 32767 2024-09-16T01:52:49.450 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.compression_opts.strategy: 0 2024-09-16T01:52:49.451 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-16T01:52:49.451 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-16T01:52:49.451 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-16T01:52:49.451 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-16T01:52:49.451 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.compression_opts.enabled: false 2024-09-16T01:52:49.451 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-16T01:52:49.451 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-16T01:52:49.451 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-16T01:52:49.451 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-16T01:52:49.451 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.target_file_size_base: 67108864 2024-09-16T01:52:49.451 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-16T01:52:49.451 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T01:52:49.451 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T01:52:49.451 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T01:52:49.452 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T01:52:49.452 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T01:52:49.452 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T01:52:49.452 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T01:52:49.452 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T01:52:49.452 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T01:52:49.452 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T01:52:49.452 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T01:52:49.452 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T01:52:49.452 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-16T01:52:49.452 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.arena_block_size: 1048576 2024-09-16T01:52:49.452 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T01:52:49.452 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T01:52:49.452 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T01:52:49.453 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T01:52:49.453 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T01:52:49.453 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T01:52:49.453 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T01:52:49.453 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T01:52:49.453 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T01:52:49.453 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T01:52:49.453 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T01:52:49.453 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T01:52:49.453 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T01:52:49.453 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-16T01:52:49.453 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.inplace_update_support: 0 2024-09-16T01:52:49.453 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T01:52:49.453 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T01:52:49.454 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T01:52:49.454 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T01:52:49.454 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.bloom_locality: 0 2024-09-16T01:52:49.454 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.max_successive_merges: 0 2024-09-16T01:52:49.454 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T01:52:49.454 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T01:52:49.454 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T01:52:49.454 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T01:52:49.454 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.ttl: 2592000 2024-09-16T01:52:49.454 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T01:52:49.454 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-16T01:52:49.454 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-16T01:52:49.454 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.enable_blob_files: false 2024-09-16T01:52:49.455 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.min_blob_size: 0 2024-09-16T01:52:49.455 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T01:52:49.455 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T01:52:49.455 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T01:52:49.455 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T01:52:49.455 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-16T01:52:49.455 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-16T01:52:49.455 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.blob_file_starting_level: 0 2024-09-16T01:52:49.455 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-16T01:52:49.455 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-16T01:52:49.455 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-16T01:52:49.455 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: a5341da9-2f63-4e9b-b242-8edd714a8903 2024-09-16T01:52:49.455 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726451569389405, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-16T01:52:49.456 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-16T01:52:49.456 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726451569390420, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99977, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 97865, "index_size": 304, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14177, "raw_average_key_size": 49, "raw_value_size": 90188, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726451569, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a5341da9-2f63-4e9b-b242-8edd714a8903", "db_session_id": "TZAJJ6C74OYPMMPXUF5Z", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-16T01:52:49.456 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726451569390536, "job": 1, "event": "recovery_finished"} 2024-09-16T01:52:49.456 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-16T01:52:49.456 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T01:52:49.456 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560fe012ee00 2024-09-16T01:52:49.456 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: DB pointer 0x560fe022a000 2024-09-16T01:52:49.456 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-16T01:52:49.456 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-16T01:52:49.456 INFO:journalctl@ceph.mon.a.smithi105.stdout: ** DB Stats ** 2024-09-16T01:52:49.456 INFO:journalctl@ceph.mon.a.smithi105.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T01:52:49.456 INFO:journalctl@ceph.mon.a.smithi105.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T01:52:49.456 INFO:journalctl@ceph.mon.a.smithi105.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T01:52:49.457 INFO:journalctl@ceph.mon.a.smithi105.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T01:52:49.457 INFO:journalctl@ceph.mon.a.smithi105.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T01:52:49.457 INFO:journalctl@ceph.mon.a.smithi105.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T01:52:49.457 INFO:journalctl@ceph.mon.a.smithi105.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T01:52:49.457 INFO:journalctl@ceph.mon.a.smithi105.stdout: 2024-09-16T01:52:49.457 INFO:journalctl@ceph.mon.a.smithi105.stdout: ** Compaction Stats [default] ** 2024-09-16T01:52:49.457 INFO:journalctl@ceph.mon.a.smithi105.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T01:52:49.457 INFO:journalctl@ceph.mon.a.smithi105.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-16T01:52:49.457 INFO:journalctl@ceph.mon.a.smithi105.stdout: L0 2/0 99.49 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 147.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T01:52:49.457 INFO:journalctl@ceph.mon.a.smithi105.stdout: Sum 2/0 99.49 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 147.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T01:52:49.457 INFO:journalctl@ceph.mon.a.smithi105.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 147.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T01:52:49.457 INFO:journalctl@ceph.mon.a.smithi105.stdout: 2024-09-16T01:52:49.457 INFO:journalctl@ceph.mon.a.smithi105.stdout: ** Compaction Stats [default] ** 2024-09-16T01:52:49.457 INFO:journalctl@ceph.mon.a.smithi105.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T01:52:49.458 INFO:journalctl@ceph.mon.a.smithi105.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T01:52:49.458 INFO:journalctl@ceph.mon.a.smithi105.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 147.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T01:52:49.458 INFO:journalctl@ceph.mon.a.smithi105.stdout: 2024-09-16T01:52:49.458 INFO:journalctl@ceph.mon.a.smithi105.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-16T01:52:49.458 INFO:journalctl@ceph.mon.a.smithi105.stdout: 2024-09-16T01:52:49.458 INFO:journalctl@ceph.mon.a.smithi105.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T01:52:49.458 INFO:journalctl@ceph.mon.a.smithi105.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-16T01:52:49.458 INFO:journalctl@ceph.mon.a.smithi105.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T01:52:49.458 INFO:journalctl@ceph.mon.a.smithi105.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T01:52:49.458 INFO:journalctl@ceph.mon.a.smithi105.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T01:52:49.458 INFO:journalctl@ceph.mon.a.smithi105.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T01:52:49.458 INFO:journalctl@ceph.mon.a.smithi105.stdout: Cumulative compaction: 0.00 GB write, 30.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T01:52:49.458 INFO:journalctl@ceph.mon.a.smithi105.stdout: Interval compaction: 0.00 GB write, 30.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T01:52:49.458 INFO:journalctl@ceph.mon.a.smithi105.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T01:52:49.459 INFO:journalctl@ceph.mon.a.smithi105.stdout: Block cache BinnedLRUCache@0x560fe01011f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-16T01:52:49.459 INFO:journalctl@ceph.mon.a.smithi105.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-16T01:52:49.459 INFO:journalctl@ceph.mon.a.smithi105.stdout: 2024-09-16T01:52:49.459 INFO:journalctl@ceph.mon.a.smithi105.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T01:52:49.459 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: starting mon.a rank 0 at public addrs [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] at bind addrs [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 2024-09-16T01:52:49.459 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: mon.a@-1(???) e1 preinit fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 2024-09-16T01:52:49.459 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: mon.a@-1(???).mds e1 new map 2024-09-16T01:52:49.459 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: mon.a@-1(???).mds e1 print_map 2024-09-16T01:52:49.459 INFO:journalctl@ceph.mon.a.smithi105.stdout: e1 2024-09-16T01:52:49.459 INFO:journalctl@ceph.mon.a.smithi105.stdout: btime 2024-09-16T01:52:44:387843+0000 2024-09-16T01:52:49.459 INFO:journalctl@ceph.mon.a.smithi105.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T01:52:49.459 INFO:journalctl@ceph.mon.a.smithi105.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-16T01:52:49.459 INFO:journalctl@ceph.mon.a.smithi105.stdout: legacy client fscid: -1 2024-09-16T01:52:49.460 INFO:journalctl@ceph.mon.a.smithi105.stdout: 2024-09-16T01:52:49.460 INFO:journalctl@ceph.mon.a.smithi105.stdout: No filesystems configured 2024-09-16T01:52:49.460 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-16T01:52:49.460 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T01:52:49.460 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T01:52:49.460 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T01:52:49.460 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-16T01:52:49.460 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 bash[31177]: 6554b6c6100439457da03be64f7dfc7f6dad18f10370ba1b0dc3f80b10cc1cf9 2024-09-16T01:52:49.460 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 systemd[1]: Started Ceph mon.a for 423b7a2a-73ce-11ef-bceb-c7b262605968. 2024-09-16T01:52:49.460 INFO:teuthology.orchestra.run.smithi105.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-16T01:52:49.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-16T01:52:49.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: monmap epoch 1 2024-09-16T01:52:49.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 2024-09-16T01:52:49.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: last_changed 2024-09-16T01:52:41.310216+0000 2024-09-16T01:52:49.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: created 2024-09-16T01:52:41.310216+0000 2024-09-16T01:52:49.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: min_mon_release 19 (squid) 2024-09-16T01:52:49.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: election_strategy: 1 2024-09-16T01:52:49.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: 0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-16T01:52:49.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: fsmap 2024-09-16T01:52:49.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: osdmap e1: 0 total, 0 up, 0 in 2024-09-16T01:52:49.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:49 smithi105 ceph-mon[31197]: mgrmap e1: no daemons active 2024-09-16T01:52:50.406 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.818+0000 7f04fd5eb640 1 Processor -- start 2024-09-16T01:52:50.406 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.818+0000 7f04fd5eb640 1 -- start start 2024-09-16T01:52:50.406 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.818+0000 7f04fd5eb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04f8107510 0x7f04f8107910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:50.406 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.818+0000 7f04fd5eb640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f04f8107ee0 con 0x7f04f8107510 2024-09-16T01:52:50.406 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.819+0000 7f04f6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04f8107510 0x7f04f8107910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:50.406 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.819+0000 7f04f6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04f8107510 0x7f04f8107910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50926/0 (socket says 172.21.15.105:50926) 2024-09-16T01:52:50.406 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.819+0000 7f04f6ffd640 1 -- 172.21.15.105:0/3319243452 learned_addr learned my addr 172.21.15.105:0/3319243452 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:52:50.406 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.819+0000 7f04f6ffd640 1 -- 172.21.15.105:0/3319243452 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f81086c0 con 0x7f04f8107510 2024-09-16T01:52:50.406 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.821+0000 7f04f6ffd640 1 --2- 172.21.15.105:0/3319243452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04f8107510 0x7f04f8107910 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f04e8009820 tx=0x7f04e802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9732749e866f465 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:50.407 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.821+0000 7f04f67fc640 1 -- 172.21.15.105:0/3319243452 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04e8037070 con 0x7f04f8107510 2024-09-16T01:52:50.407 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.821+0000 7f04f67fc640 1 -- 172.21.15.105:0/3319243452 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f04e80328d0 con 0x7f04f8107510 2024-09-16T01:52:50.407 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.821+0000 7f04f67fc640 1 -- 172.21.15.105:0/3319243452 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04e80307d0 con 0x7f04f8107510 2024-09-16T01:52:50.407 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.821+0000 7f04fd5eb640 1 -- 172.21.15.105:0/3319243452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04f8107510 msgr2=0x7f04f8107910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:50.407 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.821+0000 7f04fd5eb640 1 --2- 172.21.15.105:0/3319243452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04f8107510 0x7f04f8107910 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f04e8009820 tx=0x7f04e802a270 comp rx=0 tx=0).stop 2024-09-16T01:52:50.407 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.822+0000 7f04fd5eb640 1 -- 172.21.15.105:0/3319243452 shutdown_connections 2024-09-16T01:52:50.407 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.822+0000 7f04fd5eb640 1 --2- 172.21.15.105:0/3319243452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04f8107510 0x7f04f8107910 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:50.407 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.822+0000 7f04fd5eb640 1 -- 172.21.15.105:0/3319243452 >> 172.21.15.105:0/3319243452 conn(0x7f04f8102ca0 msgr2=0x7f04f81050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:50.407 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.822+0000 7f04fd5eb640 1 -- 172.21.15.105:0/3319243452 shutdown_connections 2024-09-16T01:52:50.407 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.822+0000 7f04fd5eb640 1 -- 172.21.15.105:0/3319243452 wait complete. 2024-09-16T01:52:50.407 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.822+0000 7f04fd5eb640 1 Processor -- start 2024-09-16T01:52:50.407 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.822+0000 7f04fd5eb640 1 -- start start 2024-09-16T01:52:50.407 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.822+0000 7f04fd5eb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04f8107510 0x7f04f819b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:50.407 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.822+0000 7f04fd5eb640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f04f810ba70 con 0x7f04f8107510 2024-09-16T01:52:50.408 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.823+0000 7f04f6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04f8107510 0x7f04f819b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:50.408 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.823+0000 7f04f6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04f8107510 0x7f04f819b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50930/0 (socket says 172.21.15.105:50930) 2024-09-16T01:52:50.408 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.823+0000 7f04f6ffd640 1 -- 172.21.15.105:0/1938166599 learned_addr learned my addr 172.21.15.105:0/1938166599 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:52:50.408 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.823+0000 7f04f6ffd640 1 -- 172.21.15.105:0/1938166599 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f819bca0 con 0x7f04f8107510 2024-09-16T01:52:50.408 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.823+0000 7f04f6ffd640 1 --2- 172.21.15.105:0/1938166599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04f8107510 0x7f04f819b760 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f04e8009940 tx=0x7f04e8032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:50.408 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.823+0000 7f04f4ff9640 1 -- 172.21.15.105:0/1938166599 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04e8042070 con 0x7f04f8107510 2024-09-16T01:52:50.408 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.823+0000 7f04fd5eb640 1 -- 172.21.15.105:0/1938166599 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04f819bf30 con 0x7f04f8107510 2024-09-16T01:52:50.408 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.823+0000 7f04f4ff9640 1 -- 172.21.15.105:0/1938166599 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f04e8030c80 con 0x7f04f8107510 2024-09-16T01:52:50.408 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.823+0000 7f04f4ff9640 1 -- 172.21.15.105:0/1938166599 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04e8037040 con 0x7f04f8107510 2024-09-16T01:52:50.409 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.823+0000 7f04fd5eb640 1 -- 172.21.15.105:0/1938166599 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04f819ec20 con 0x7f04f8107510 2024-09-16T01:52:50.409 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.823+0000 7f04f4ff9640 1 -- 172.21.15.105:0/1938166599 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f04e8045070 con 0x7f04f8107510 2024-09-16T01:52:50.409 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.823+0000 7f04f4ff9640 1 -- 172.21.15.105:0/1938166599 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f04e80463b0 con 0x7f04f8107510 2024-09-16T01:52:50.409 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.824+0000 7f04fd5eb640 1 -- 172.21.15.105:0/1938166599 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04f810bbf0 con 0x7f04f8107510 2024-09-16T01:52:50.409 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.826+0000 7f04f4ff9640 1 -- 172.21.15.105:0/1938166599 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f04e8047440 con 0x7f04f8107510 2024-09-16T01:52:50.409 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.882+0000 7f04fd5eb640 1 -- 172.21.15.105:0/1938166599 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f04f8107990 con 0x7f04f8107510 2024-09-16T01:52:50.409 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.887+0000 7f04f4ff9640 1 -- 172.21.15.105:0/1938166599 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f04e8047620 con 0x7f04f8107510 2024-09-16T01:52:50.409 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.888+0000 7f04fd5eb640 1 -- 172.21.15.105:0/1938166599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04f8107510 msgr2=0x7f04f819b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:50.409 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.888+0000 7f04fd5eb640 1 --2- 172.21.15.105:0/1938166599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04f8107510 0x7f04f819b760 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f04e8009940 tx=0x7f04e8032e10 comp rx=0 tx=0).stop 2024-09-16T01:52:50.409 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.888+0000 7f04fd5eb640 1 -- 172.21.15.105:0/1938166599 shutdown_connections 2024-09-16T01:52:50.409 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.888+0000 7f04fd5eb640 1 --2- 172.21.15.105:0/1938166599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04f8107510 0x7f04f819b760 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:50.409 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.888+0000 7f04fd5eb640 1 -- 172.21.15.105:0/1938166599 >> 172.21.15.105:0/1938166599 conn(0x7f04f8102ca0 msgr2=0x7f04f8103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:50.409 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.888+0000 7f04fd5eb640 1 -- 172.21.15.105:0/1938166599 shutdown_connections 2024-09-16T01:52:50.409 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:49.888+0000 7f04fd5eb640 1 -- 172.21.15.105:0/1938166599 wait complete. 2024-09-16T01:52:50.410 INFO:teuthology.orchestra.run.smithi105.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-16T01:52:50.425 INFO:teuthology.orchestra.run.smithi105.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-16T01:52:50.425 INFO:teuthology.orchestra.run.smithi105.stdout:Creating mgr... 2024-09-16T01:52:50.425 INFO:teuthology.orchestra.run.smithi105.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-16T01:52:50.425 INFO:teuthology.orchestra.run.smithi105.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-16T01:52:50.699 INFO:teuthology.orchestra.run.smithi105.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mgr.x 2024-09-16T01:52:50.699 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stderr Failed to reset failed state of unit ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mgr.x.service: Unit ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mgr.x.service not loaded. 2024-09-16T01:52:50.846 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-423b7a2a-73ce-11ef-bceb-c7b262605968.target.wants/ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@.service. 2024-09-16T01:52:51.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:50 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/1938166599' entity='client.admin' 2024-09-16T01:52:51.329 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:52:51 smithi105 podman[31409]: 2024-09-16 01:52:51.256590399 +0000 UTC m=+0.292365019 container start 774d5ede24dcf4af137ba7031885ecbbab9a6cd514d9b570d8328cae7ecac455 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x, GIT_CLEAN=True, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD) 2024-09-16T01:52:51.329 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:52:51 smithi105 bash[31409]: 774d5ede24dcf4af137ba7031885ecbbab9a6cd514d9b570d8328cae7ecac455 2024-09-16T01:52:51.342 INFO:teuthology.orchestra.run.smithi105.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T01:52:51.343 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stdout disabled 2024-09-16T01:52:51.350 INFO:teuthology.orchestra.run.smithi105.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T01:52:51.350 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stdout inactive 2024-09-16T01:52:51.350 INFO:teuthology.orchestra.run.smithi105.stdout:firewalld.service is not enabled 2024-09-16T01:52:51.350 INFO:teuthology.orchestra.run.smithi105.stdout:Not possible to enable service . firewalld.service is not available 2024-09-16T01:52:51.359 INFO:teuthology.orchestra.run.smithi105.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T01:52:51.359 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stdout disabled 2024-09-16T01:52:51.367 INFO:teuthology.orchestra.run.smithi105.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T01:52:51.367 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stdout inactive 2024-09-16T01:52:51.367 INFO:teuthology.orchestra.run.smithi105.stdout:firewalld.service is not enabled 2024-09-16T01:52:51.367 INFO:teuthology.orchestra.run.smithi105.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-16T01:52:51.367 INFO:teuthology.orchestra.run.smithi105.stdout:Waiting for mgr to start... 2024-09-16T01:52:51.367 INFO:teuthology.orchestra.run.smithi105.stdout:Waiting for mgr... 2024-09-16T01:52:51.583 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:52:51 smithi105 systemd[1]: Started Ceph mgr.x for 423b7a2a-73ce-11ef-bceb-c7b262605968. 2024-09-16T01:52:51.583 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:52:51 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:52:51.377+0000 7f991a8ab040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T01:52:51.583 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:52:51 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:52:51.436+0000 7f991a8ab040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T01:52:51.856 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:52:51 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:52:51.591+0000 7f991a8ab040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T01:52:51.856 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:52:51 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:52:51.658+0000 7f991a8ab040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T01:52:51.856 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:52:51 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:52:51.813+0000 7f991a8ab040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T01:52:52.210 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:51 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/1051259092' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T01:52:52.211 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:52:51 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:52:51.962+0000 7f991a8ab040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T01:52:52.461 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:52:52 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:52:52.281+0000 7f991a8ab040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T01:52:52.461 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:52:52 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:52:52.342+0000 7f991a8ab040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T01:52:52.463 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout 2024-09-16T01:52:52.474 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout { 2024-09-16T01:52:52.474 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "fsid": "423b7a2a-73ce-11ef-bceb-c7b262605968", 2024-09-16T01:52:52.474 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T01:52:52.475 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T01:52:52.475 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T01:52:52.475 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T01:52:52.475 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:52:52.475 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T01:52:52.475 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T01:52:52.475 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout 0 2024-09-16T01:52:52.475 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout ], 2024-09-16T01:52:52.475 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T01:52:52.475 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "a" 2024-09-16T01:52:52.475 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout ], 2024-09-16T01:52:52.475 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-16T01:52:52.475 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T01:52:52.475 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T01:52:52.476 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-16T01:52:52.476 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T01:52:52.476 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:52:52.476 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T01:52:52.476 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T01:52:52.476 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T01:52:52.476 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T01:52:52.476 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T01:52:52.476 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T01:52:52.477 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T01:52:52.477 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T01:52:52.477 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:52:52.477 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T01:52:52.477 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T01:52:52.477 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T01:52:52.477 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T01:52:52.477 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T01:52:52.477 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T01:52:52.477 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T01:52:52.477 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T01:52:52.477 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T01:52:52.477 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:52:52.477 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T01:52:52.478 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T01:52:52.478 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "btime": "2024-09-16T01:52:44:387843+0000", 2024-09-16T01:52:52.478 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T01:52:52.478 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T01:52:52.478 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:52:52.478 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T01:52:52.478 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-16T01:52:52.478 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T01:52:52.478 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T01:52:52.478 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T01:52:52.478 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T01:52:52.478 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T01:52:52.478 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout ], 2024-09-16T01:52:52.479 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T01:52:52.479 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:52:52.479 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T01:52:52.479 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T01:52:52.479 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T01:52:44.388479+0000", 2024-09-16T01:52:52.479 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T01:52:52.479 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:52:52.479 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T01:52:52.479 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout } 2024-09-16T01:52:52.480 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.786+0000 7f79470ff640 1 Processor -- start 2024-09-16T01:52:52.480 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.786+0000 7f79470ff640 1 -- start start 2024-09-16T01:52:52.480 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.786+0000 7f79470ff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7940107ef0 0x7f794010a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:52.480 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.786+0000 7f79470ff640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f794007be90 con 0x7f7940107ef0 2024-09-16T01:52:52.480 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.786+0000 7f7944e9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7940107ef0 0x7f794010a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:52.480 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.786+0000 7f7944e9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7940107ef0 0x7f794010a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50952/0 (socket says 172.21.15.105:50952) 2024-09-16T01:52:52.480 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.786+0000 7f7944e9b640 1 -- 172.21.15.105:0/3701517567 learned_addr learned my addr 172.21.15.105:0/3701517567 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:52:52.480 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.787+0000 7f7944e9b640 1 -- 172.21.15.105:0/3701517567 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f794010a820 con 0x7f7940107ef0 2024-09-16T01:52:52.480 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.787+0000 7f7944e9b640 1 --2- 172.21.15.105:0/3701517567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7940107ef0 0x7f794010a2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7934009820 tx=0x7f793402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fde218f39623e917 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:52.480 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.788+0000 7f793f7fe640 1 -- 172.21.15.105:0/3701517567 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7934037070 con 0x7f7940107ef0 2024-09-16T01:52:52.480 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.788+0000 7f793f7fe640 1 -- 172.21.15.105:0/3701517567 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f79340328d0 con 0x7f7940107ef0 2024-09-16T01:52:52.480 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.788+0000 7f793f7fe640 1 -- 172.21.15.105:0/3701517567 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f79340307d0 con 0x7f7940107ef0 2024-09-16T01:52:52.480 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.788+0000 7f79470ff640 1 -- 172.21.15.105:0/3701517567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7940107ef0 msgr2=0x7f794010a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:52.481 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.788+0000 7f79470ff640 1 --2- 172.21.15.105:0/3701517567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7940107ef0 0x7f794010a2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7934009820 tx=0x7f793402a270 comp rx=0 tx=0).stop 2024-09-16T01:52:52.481 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.788+0000 7f79470ff640 1 -- 172.21.15.105:0/3701517567 shutdown_connections 2024-09-16T01:52:52.481 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.788+0000 7f79470ff640 1 --2- 172.21.15.105:0/3701517567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7940107ef0 0x7f794010a2e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:52.481 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.788+0000 7f79470ff640 1 -- 172.21.15.105:0/3701517567 >> 172.21.15.105:0/3701517567 conn(0x7f7940101b10 msgr2=0x7f7940103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:52.481 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.788+0000 7f79470ff640 1 -- 172.21.15.105:0/3701517567 shutdown_connections 2024-09-16T01:52:52.481 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.788+0000 7f79470ff640 1 -- 172.21.15.105:0/3701517567 wait complete. 2024-09-16T01:52:52.481 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.789+0000 7f79470ff640 1 Processor -- start 2024-09-16T01:52:52.481 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.789+0000 7f79470ff640 1 -- start start 2024-09-16T01:52:52.481 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.789+0000 7f79470ff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f794019b910 0x7f794019bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:52.481 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.789+0000 7f79470ff640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f794010b860 con 0x7f794019b910 2024-09-16T01:52:52.481 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.789+0000 7f7944e9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f794019b910 0x7f794019bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:52.481 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.789+0000 7f7944e9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f794019b910 0x7f794019bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50954/0 (socket says 172.21.15.105:50954) 2024-09-16T01:52:52.481 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.789+0000 7f7944e9b640 1 -- 172.21.15.105:0/1051259092 learned_addr learned my addr 172.21.15.105:0/1051259092 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:52:52.481 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.790+0000 7f7944e9b640 1 -- 172.21.15.105:0/1051259092 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f794019c270 con 0x7f794019b910 2024-09-16T01:52:52.482 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.790+0000 7f7944e9b640 1 --2- 172.21.15.105:0/1051259092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f794019b910 0x7f794019bd30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7934009940 tx=0x7f7934032df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:52.482 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.790+0000 7f793dffb640 1 -- 172.21.15.105:0/1051259092 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7934037050 con 0x7f794019b910 2024-09-16T01:52:52.482 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.790+0000 7f79470ff640 1 -- 172.21.15.105:0/1051259092 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f794019c500 con 0x7f794019b910 2024-09-16T01:52:52.482 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.790+0000 7f793dffb640 1 -- 172.21.15.105:0/1051259092 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f7934043070 con 0x7f794019b910 2024-09-16T01:52:52.482 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.790+0000 7f793dffb640 1 -- 172.21.15.105:0/1051259092 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f793403c590 con 0x7f794019b910 2024-09-16T01:52:52.482 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.790+0000 7f79470ff640 1 -- 172.21.15.105:0/1051259092 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f794019efe0 con 0x7f794019b910 2024-09-16T01:52:52.482 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.791+0000 7f793dffb640 1 -- 172.21.15.105:0/1051259092 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f793403c730 con 0x7f794019b910 2024-09-16T01:52:52.482 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.791+0000 7f793dffb640 1 -- 172.21.15.105:0/1051259092 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f79340489f0 con 0x7f794019b910 2024-09-16T01:52:52.482 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.792+0000 7f79470ff640 1 -- 172.21.15.105:0/1051259092 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79401082d0 con 0x7f794019b910 2024-09-16T01:52:52.482 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.794+0000 7f793dffb640 1 -- 172.21.15.105:0/1051259092 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f7934030e10 con 0x7f794019b910 2024-09-16T01:52:52.482 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.854+0000 7f79470ff640 1 -- 172.21.15.105:0/1051259092 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f794002d7c0 con 0x7f794019b910 2024-09-16T01:52:52.482 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.855+0000 7f793dffb640 1 -- 172.21.15.105:0/1051259092 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f7934048460 con 0x7f794019b910 2024-09-16T01:52:52.482 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.856+0000 7f79470ff640 1 -- 172.21.15.105:0/1051259092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f794019b910 msgr2=0x7f794019bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:52.483 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.856+0000 7f79470ff640 1 --2- 172.21.15.105:0/1051259092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f794019b910 0x7f794019bd30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7934009940 tx=0x7f7934032df0 comp rx=0 tx=0).stop 2024-09-16T01:52:52.483 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.856+0000 7f79470ff640 1 -- 172.21.15.105:0/1051259092 shutdown_connections 2024-09-16T01:52:52.483 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.856+0000 7f79470ff640 1 --2- 172.21.15.105:0/1051259092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f794019b910 0x7f794019bd30 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:52.483 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.856+0000 7f79470ff640 1 -- 172.21.15.105:0/1051259092 >> 172.21.15.105:0/1051259092 conn(0x7f7940101b10 msgr2=0x7f79401028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:52.483 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.856+0000 7f79470ff640 1 -- 172.21.15.105:0/1051259092 shutdown_connections 2024-09-16T01:52:52.483 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:51.856+0000 7f79470ff640 1 -- 172.21.15.105:0/1051259092 wait complete. 2024-09-16T01:52:52.483 INFO:teuthology.orchestra.run.smithi105.stdout:mgr not available, waiting (1/15)... 2024-09-16T01:52:52.794 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:52:52 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:52:52.529+0000 7f991a8ab040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T01:52:53.794 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:52:53 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:52:53.402+0000 7f991a8ab040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T01:52:53.794 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:52:53 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:52:53.468+0000 7f991a8ab040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T01:52:54.290 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:52:54 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:52:54.175+0000 7f991a8ab040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T01:52:54.290 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:52:54 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:52:54.289+0000 7f991a8ab040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T01:52:54.544 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:52:54 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:52:54.430+0000 7f991a8ab040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T01:52:54.544 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:52:54 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:52:54.493+0000 7f991a8ab040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T01:52:54.822 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:52:54 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:52:54.653+0000 7f991a8ab040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T01:52:54.822 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:52:54 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:52:54.724+0000 7f991a8ab040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T01:52:54.822 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:52:54 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:52:54.783+0000 7f991a8ab040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T01:52:55.085 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:55 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/3975935379' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T01:52:55.085 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:52:54 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:52:54.948+0000 7f991a8ab040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T01:52:55.086 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:52:55 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:52:55.018+0000 7f991a8ab040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T01:52:55.086 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:52:55 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:52:55.084+0000 7f991a8ab040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T01:52:55.544 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:52:55 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:52:55.167+0000 7f991a8ab040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T01:52:55.545 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:52:55 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:52:55.226+0000 7f991a8ab040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T01:52:55.545 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:52:55 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:52:55.286+0000 7f991a8ab040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T01:52:55.614 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout 2024-09-16T01:52:55.614 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout { 2024-09-16T01:52:55.614 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "fsid": "423b7a2a-73ce-11ef-bceb-c7b262605968", 2024-09-16T01:52:55.614 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T01:52:55.614 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T01:52:55.614 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T01:52:55.614 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T01:52:55.614 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:52:55.614 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T01:52:55.615 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T01:52:55.615 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout 0 2024-09-16T01:52:55.615 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout ], 2024-09-16T01:52:55.615 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T01:52:55.615 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "a" 2024-09-16T01:52:55.615 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout ], 2024-09-16T01:52:55.615 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-16T01:52:55.615 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T01:52:55.615 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T01:52:55.615 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-16T01:52:55.615 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T01:52:55.615 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:52:55.615 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T01:52:55.616 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T01:52:55.616 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T01:52:55.616 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T01:52:55.616 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T01:52:55.616 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T01:52:55.616 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T01:52:55.616 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T01:52:55.616 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:52:55.616 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T01:52:55.616 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T01:52:55.616 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T01:52:55.616 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T01:52:55.617 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T01:52:55.617 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T01:52:55.617 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T01:52:55.617 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T01:52:55.617 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T01:52:55.617 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:52:55.617 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T01:52:55.617 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T01:52:55.617 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "btime": "2024-09-16T01:52:44:387843+0000", 2024-09-16T01:52:55.617 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T01:52:55.617 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T01:52:55.617 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:52:55.617 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T01:52:55.617 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-16T01:52:55.618 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T01:52:55.618 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T01:52:55.618 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T01:52:55.618 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T01:52:55.618 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T01:52:55.618 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout ], 2024-09-16T01:52:55.618 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T01:52:55.618 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:52:55.618 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T01:52:55.618 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T01:52:55.618 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T01:52:44.388479+0000", 2024-09-16T01:52:55.618 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T01:52:55.618 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:52:55.618 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T01:52:55.618 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout } 2024-09-16T01:52:55.619 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.899+0000 7f0d36ad1640 1 Processor -- start 2024-09-16T01:52:55.619 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.899+0000 7f0d36ad1640 1 -- start start 2024-09-16T01:52:55.619 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.899+0000 7f0d36ad1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d30107fd0 0x7f0d3010a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:55.619 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.899+0000 7f0d36ad1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3007bdb0 con 0x7f0d30107fd0 2024-09-16T01:52:55.619 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.899+0000 7f0d3486d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d30107fd0 0x7f0d3010a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:55.619 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.899+0000 7f0d3486d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d30107fd0 0x7f0d3010a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50964/0 (socket says 172.21.15.105:50964) 2024-09-16T01:52:55.619 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.899+0000 7f0d3486d640 1 -- 172.21.15.105:0/450448711 learned_addr learned my addr 172.21.15.105:0/450448711 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:52:55.619 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.899+0000 7f0d3486d640 1 -- 172.21.15.105:0/450448711 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d3010a900 con 0x7f0d30107fd0 2024-09-16T01:52:55.619 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.900+0000 7f0d3486d640 1 --2- 172.21.15.105:0/450448711 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d30107fd0 0x7f0d3010a3c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0d24009820 tx=0x7f0d2402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c7510d9c176dc45 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:55.619 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.900+0000 7f0d2f7fe640 1 -- 172.21.15.105:0/450448711 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d24037070 con 0x7f0d30107fd0 2024-09-16T01:52:55.619 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.900+0000 7f0d2f7fe640 1 -- 172.21.15.105:0/450448711 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0d240328d0 con 0x7f0d30107fd0 2024-09-16T01:52:55.619 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.900+0000 7f0d2f7fe640 1 -- 172.21.15.105:0/450448711 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d240307d0 con 0x7f0d30107fd0 2024-09-16T01:52:55.619 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.901+0000 7f0d36ad1640 1 -- 172.21.15.105:0/450448711 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d30107fd0 msgr2=0x7f0d3010a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:55.620 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.901+0000 7f0d36ad1640 1 --2- 172.21.15.105:0/450448711 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d30107fd0 0x7f0d3010a3c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0d24009820 tx=0x7f0d2402a270 comp rx=0 tx=0).stop 2024-09-16T01:52:55.620 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.901+0000 7f0d36ad1640 1 -- 172.21.15.105:0/450448711 shutdown_connections 2024-09-16T01:52:55.620 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.901+0000 7f0d36ad1640 1 --2- 172.21.15.105:0/450448711 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d30107fd0 0x7f0d3010a3c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:55.620 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.901+0000 7f0d36ad1640 1 -- 172.21.15.105:0/450448711 >> 172.21.15.105:0/450448711 conn(0x7f0d30101b10 msgr2=0x7f0d30103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:55.620 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.901+0000 7f0d36ad1640 1 -- 172.21.15.105:0/450448711 shutdown_connections 2024-09-16T01:52:55.620 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.901+0000 7f0d36ad1640 1 -- 172.21.15.105:0/450448711 wait complete. 2024-09-16T01:52:55.620 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.902+0000 7f0d36ad1640 1 Processor -- start 2024-09-16T01:52:55.620 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.902+0000 7f0d36ad1640 1 -- start start 2024-09-16T01:52:55.620 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.902+0000 7f0d36ad1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d30107fd0 0x7f0d3019b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:55.620 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.902+0000 7f0d36ad1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3010b820 con 0x7f0d30107fd0 2024-09-16T01:52:55.620 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.902+0000 7f0d3486d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d30107fd0 0x7f0d3019b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:55.620 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.902+0000 7f0d3486d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d30107fd0 0x7f0d3019b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50972/0 (socket says 172.21.15.105:50972) 2024-09-16T01:52:55.620 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.902+0000 7f0d3486d640 1 -- 172.21.15.105:0/3975935379 learned_addr learned my addr 172.21.15.105:0/3975935379 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:52:55.621 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.902+0000 7f0d3486d640 1 -- 172.21.15.105:0/3975935379 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d3019bcb0 con 0x7f0d30107fd0 2024-09-16T01:52:55.621 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.902+0000 7f0d3486d640 1 --2- 172.21.15.105:0/3975935379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d30107fd0 0x7f0d3019b770 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0d24030520 tx=0x7f0d24032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:55.621 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.902+0000 7f0d2dffb640 1 -- 172.21.15.105:0/3975935379 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d24042070 con 0x7f0d30107fd0 2024-09-16T01:52:55.621 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.902+0000 7f0d2dffb640 1 -- 172.21.15.105:0/3975935379 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0d24030cc0 con 0x7f0d30107fd0 2024-09-16T01:52:55.621 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.902+0000 7f0d36ad1640 1 -- 172.21.15.105:0/3975935379 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d3019bf40 con 0x7f0d30107fd0 2024-09-16T01:52:55.621 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.902+0000 7f0d2dffb640 1 -- 172.21.15.105:0/3975935379 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d24037040 con 0x7f0d30107fd0 2024-09-16T01:52:55.621 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.903+0000 7f0d36ad1640 1 -- 172.21.15.105:0/3975935379 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d3019ec30 con 0x7f0d30107fd0 2024-09-16T01:52:55.621 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.903+0000 7f0d2dffb640 1 -- 172.21.15.105:0/3975935379 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0d24045070 con 0x7f0d30107fd0 2024-09-16T01:52:55.621 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.903+0000 7f0d2dffb640 1 -- 172.21.15.105:0/3975935379 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0d240479a0 con 0x7f0d30107fd0 2024-09-16T01:52:55.621 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.904+0000 7f0d36ad1640 1 -- 172.21.15.105:0/3975935379 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d30108430 con 0x7f0d30107fd0 2024-09-16T01:52:55.621 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.906+0000 7f0d2dffb640 1 -- 172.21.15.105:0/3975935379 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f0d2403d430 con 0x7f0d30107fd0 2024-09-16T01:52:55.621 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.967+0000 7f0d36ad1640 1 -- 172.21.15.105:0/3975935379 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f0d301952d0 con 0x7f0d30107fd0 2024-09-16T01:52:55.621 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.967+0000 7f0d2dffb640 1 -- 172.21.15.105:0/3975935379 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f0d2403d610 con 0x7f0d30107fd0 2024-09-16T01:52:55.622 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.969+0000 7f0d36ad1640 1 -- 172.21.15.105:0/3975935379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d30107fd0 msgr2=0x7f0d3019b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:55.622 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.969+0000 7f0d36ad1640 1 --2- 172.21.15.105:0/3975935379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d30107fd0 0x7f0d3019b770 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0d24030520 tx=0x7f0d24032dd0 comp rx=0 tx=0).stop 2024-09-16T01:52:55.622 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.969+0000 7f0d36ad1640 1 -- 172.21.15.105:0/3975935379 shutdown_connections 2024-09-16T01:52:55.622 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.969+0000 7f0d36ad1640 1 --2- 172.21.15.105:0/3975935379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d30107fd0 0x7f0d3019b770 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:55.622 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.969+0000 7f0d36ad1640 1 -- 172.21.15.105:0/3975935379 >> 172.21.15.105:0/3975935379 conn(0x7f0d30101b10 msgr2=0x7f0d30102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:55.622 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.969+0000 7f0d36ad1640 1 -- 172.21.15.105:0/3975935379 shutdown_connections 2024-09-16T01:52:55.622 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:54.969+0000 7f0d36ad1640 1 -- 172.21.15.105:0/3975935379 wait complete. 2024-09-16T01:52:55.622 INFO:teuthology.orchestra.run.smithi105.stdout:mgr not available, waiting (2/15)... 2024-09-16T01:52:56.544 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:52:56 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:52:56.075+0000 7f991a8ab040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T01:52:56.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:56 smithi105 ceph-mon[31197]: Activating manager daemon x 2024-09-16T01:52:56.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:56 smithi105 ceph-mon[31197]: mgrmap e2: x(active, starting, since 0.00315424s) 2024-09-16T01:52:56.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:56 smithi105 ceph-mon[31197]: from='mgr.14100 172.21.15.105:0/2730918147' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T01:52:56.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:56 smithi105 ceph-mon[31197]: from='mgr.14100 172.21.15.105:0/2730918147' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T01:52:56.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:56 smithi105 ceph-mon[31197]: from='mgr.14100 172.21.15.105:0/2730918147' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T01:52:56.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:56 smithi105 ceph-mon[31197]: from='mgr.14100 172.21.15.105:0/2730918147' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:52:56.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:56 smithi105 ceph-mon[31197]: from='mgr.14100 172.21.15.105:0/2730918147' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T01:52:56.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:56 smithi105 ceph-mon[31197]: Manager daemon x is now available 2024-09-16T01:52:56.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:56 smithi105 ceph-mon[31197]: from='mgr.14100 172.21.15.105:0/2730918147' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-16T01:52:56.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:56 smithi105 ceph-mon[31197]: from='mgr.14100 172.21.15.105:0/2730918147' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-16T01:52:56.546 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:56 smithi105 ceph-mon[31197]: from='mgr.14100 172.21.15.105:0/2730918147' entity='mgr.x' 2024-09-16T01:52:56.546 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:56 smithi105 ceph-mon[31197]: from='mgr.14100 172.21.15.105:0/2730918147' entity='mgr.x' 2024-09-16T01:52:56.546 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:56 smithi105 ceph-mon[31197]: from='mgr.14100 172.21.15.105:0/2730918147' entity='mgr.x' 2024-09-16T01:52:58.266 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:58 smithi105 ceph-mon[31197]: mgrmap e3: x(active, since 1.00694s) 2024-09-16T01:52:59.544 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:59 smithi105 ceph-mon[31197]: mgrmap e4: x(active, since 2s) 2024-09-16T01:52:59.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:52:59 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2055630672' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T01:53:03.704 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout 2024-09-16T01:53:04.080 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout { 2024-09-16T01:53:04.080 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "fsid": "423b7a2a-73ce-11ef-bceb-c7b262605968", 2024-09-16T01:53:04.080 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T01:53:04.080 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T01:53:04.081 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T01:53:04.081 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T01:53:04.081 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:53:04.081 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T01:53:04.081 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T01:53:04.081 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout 0 2024-09-16T01:53:04.092 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout ], 2024-09-16T01:53:04.092 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T01:53:04.092 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "a" 2024-09-16T01:53:04.092 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout ], 2024-09-16T01:53:04.092 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-09-16T01:53:04.093 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T01:53:04.093 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T01:53:04.093 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-16T01:53:04.093 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T01:53:04.093 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:53:04.093 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T01:53:04.093 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T01:53:04.093 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T01:53:04.093 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T01:53:04.093 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T01:53:04.093 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T01:53:04.093 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T01:53:04.093 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T01:53:04.094 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:53:04.094 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T01:53:04.094 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T01:53:04.094 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T01:53:04.094 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T01:53:04.094 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T01:53:04.094 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T01:53:04.094 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T01:53:04.094 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T01:53:04.094 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T01:53:04.094 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:53:04.094 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T01:53:04.094 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T01:53:04.094 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "btime": "2024-09-16T01:52:44:387843+0000", 2024-09-16T01:53:04.095 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T01:53:04.095 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T01:53:04.095 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:53:04.095 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T01:53:04.095 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T01:53:04.095 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T01:53:04.095 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T01:53:04.095 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T01:53:04.095 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T01:53:04.095 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T01:53:04.095 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout ], 2024-09-16T01:53:04.095 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T01:53:04.095 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:53:04.095 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T01:53:04.096 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T01:53:04.096 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T01:52:44.388479+0000", 2024-09-16T01:53:04.096 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T01:53:04.096 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:53:04.096 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T01:53:04.096 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout } 2024-09-16T01:53:04.096 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.007+0000 7f50cba10640 1 Processor -- start 2024-09-16T01:53:04.096 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.008+0000 7f50cba10640 1 -- start start 2024-09-16T01:53:04.096 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.008+0000 7f50cba10640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50c4107ef0 0x7f50c410a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:04.096 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.008+0000 7f50cba10640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f50c407be90 con 0x7f50c4107ef0 2024-09-16T01:53:04.096 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.008+0000 7f50c97ac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50c4107ef0 0x7f50c410a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:04.096 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.008+0000 7f50c97ac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50c4107ef0 0x7f50c410a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51028/0 (socket says 172.21.15.105:51028) 2024-09-16T01:53:04.097 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.008+0000 7f50c97ac640 1 -- 172.21.15.105:0/1465246931 learned_addr learned my addr 172.21.15.105:0/1465246931 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:04.097 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.008+0000 7f50c97ac640 1 -- 172.21.15.105:0/1465246931 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50c410a820 con 0x7f50c4107ef0 2024-09-16T01:53:04.097 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.009+0000 7f50c97ac640 1 --2- 172.21.15.105:0/1465246931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50c4107ef0 0x7f50c410a2e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f50b8009820 tx=0x7f50b802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b877bbfa97d1155 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:04.097 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.009+0000 7f50b7fff640 1 -- 172.21.15.105:0/1465246931 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50b8037070 con 0x7f50c4107ef0 2024-09-16T01:53:04.097 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.009+0000 7f50b7fff640 1 -- 172.21.15.105:0/1465246931 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f50b80328d0 con 0x7f50c4107ef0 2024-09-16T01:53:04.097 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.009+0000 7f50b7fff640 1 -- 172.21.15.105:0/1465246931 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50b80307d0 con 0x7f50c4107ef0 2024-09-16T01:53:04.097 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.010+0000 7f50cba10640 1 -- 172.21.15.105:0/1465246931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50c4107ef0 msgr2=0x7f50c410a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:04.097 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.010+0000 7f50cba10640 1 --2- 172.21.15.105:0/1465246931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50c4107ef0 0x7f50c410a2e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f50b8009820 tx=0x7f50b802a270 comp rx=0 tx=0).stop 2024-09-16T01:53:04.097 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.010+0000 7f50cba10640 1 -- 172.21.15.105:0/1465246931 shutdown_connections 2024-09-16T01:53:04.097 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.010+0000 7f50cba10640 1 --2- 172.21.15.105:0/1465246931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50c4107ef0 0x7f50c410a2e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:04.097 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.010+0000 7f50cba10640 1 -- 172.21.15.105:0/1465246931 >> 172.21.15.105:0/1465246931 conn(0x7f50c4101b10 msgr2=0x7f50c4103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:04.097 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.010+0000 7f50cba10640 1 -- 172.21.15.105:0/1465246931 shutdown_connections 2024-09-16T01:53:04.097 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.010+0000 7f50cba10640 1 -- 172.21.15.105:0/1465246931 wait complete. 2024-09-16T01:53:04.097 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.011+0000 7f50cba10640 1 Processor -- start 2024-09-16T01:53:04.098 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.011+0000 7f50cba10640 1 -- start start 2024-09-16T01:53:04.098 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.011+0000 7f50cba10640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50c4107ef0 0x7f50c4107770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:04.098 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.011+0000 7f50cba10640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f50c410b860 con 0x7f50c4107ef0 2024-09-16T01:53:04.098 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.011+0000 7f50c97ac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50c4107ef0 0x7f50c4107770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:04.098 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.011+0000 7f50c97ac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50c4107ef0 0x7f50c4107770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51040/0 (socket says 172.21.15.105:51040) 2024-09-16T01:53:04.098 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.011+0000 7f50c97ac640 1 -- 172.21.15.105:0/2055630672 learned_addr learned my addr 172.21.15.105:0/2055630672 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:04.098 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.011+0000 7f50c97ac640 1 -- 172.21.15.105:0/2055630672 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50c4107cb0 con 0x7f50c4107ef0 2024-09-16T01:53:04.098 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.011+0000 7f50c97ac640 1 --2- 172.21.15.105:0/2055630672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50c4107ef0 0x7f50c4107770 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f50b80097f0 tx=0x7f50b8030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:04.098 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.011+0000 7f50b67fc640 1 -- 172.21.15.105:0/2055630672 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50b8037040 con 0x7f50c4107ef0 2024-09-16T01:53:04.098 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.011+0000 7f50b67fc640 1 -- 172.21.15.105:0/2055630672 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f50b803b3f0 con 0x7f50c4107ef0 2024-09-16T01:53:04.098 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.011+0000 7f50cba10640 1 -- 172.21.15.105:0/2055630672 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50c4105ea0 con 0x7f50c4107ef0 2024-09-16T01:53:04.098 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.011+0000 7f50b67fc640 1 -- 172.21.15.105:0/2055630672 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50b803a520 con 0x7f50c4107ef0 2024-09-16T01:53:04.098 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.012+0000 7f50cba10640 1 -- 172.21.15.105:0/2055630672 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50c41063e0 con 0x7f50c4107ef0 2024-09-16T01:53:04.099 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.012+0000 7f50b67fc640 1 -- 172.21.15.105:0/2055630672 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 3) ==== 46344+0+0 (secure 0 0 0) 0x7f50b8032820 con 0x7f50c4107ef0 2024-09-16T01:53:04.100 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.013+0000 7f50cba10640 1 -- 172.21.15.105:0/2055630672 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50c4108350 con 0x7f50c4107ef0 2024-09-16T01:53:04.100 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.013+0000 7f50b67fc640 1 --2- 172.21.15.105:0/2055630672 >> [v2:172.21.15.105:6800/1268348742,v1:172.21.15.105:6801/1268348742] conn(0x7f50b0039050 0x7f50b003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:04.100 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.013+0000 7f50b67fc640 1 -- 172.21.15.105:0/2055630672 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f50b806c190 con 0x7f50c4107ef0 2024-09-16T01:53:04.100 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.013+0000 7f50c8fab640 1 --2- 172.21.15.105:0/2055630672 >> [v2:172.21.15.105:6800/1268348742,v1:172.21.15.105:6801/1268348742] conn(0x7f50b0039050 0x7f50b003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:04.100 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.013+0000 7f50c8fab640 1 --2- 172.21.15.105:0/2055630672 >> [v2:172.21.15.105:6800/1268348742,v1:172.21.15.105:6801/1268348742] conn(0x7f50b0039050 0x7f50b003b510 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f50c00098c0 tx=0x7f50c0006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:04.100 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.018+0000 7f50b67fc640 1 -- 172.21.15.105:0/2055630672 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f50b80371e0 con 0x7f50c4107ef0 2024-09-16T01:53:04.100 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.084+0000 7f50b67fc640 1 -- 172.21.15.105:0/2055630672 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f50b803a710 con 0x7f50c4107ef0 2024-09-16T01:53:04.100 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.252+0000 7f50cba10640 1 -- 172.21.15.105:0/2055630672 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f50c4194f30 con 0x7f50c4107ef0 2024-09-16T01:53:04.100 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.252+0000 7f50b67fc640 1 -- 172.21.15.105:0/2055630672 <== mon.0 v2:172.21.15.105:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7f50b8042670 con 0x7f50c4107ef0 2024-09-16T01:53:04.100 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.255+0000 7f50cba10640 1 -- 172.21.15.105:0/2055630672 >> [v2:172.21.15.105:6800/1268348742,v1:172.21.15.105:6801/1268348742] conn(0x7f50b0039050 msgr2=0x7f50b003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:04.100 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.255+0000 7f50cba10640 1 --2- 172.21.15.105:0/2055630672 >> [v2:172.21.15.105:6800/1268348742,v1:172.21.15.105:6801/1268348742] conn(0x7f50b0039050 0x7f50b003b510 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f50c00098c0 tx=0x7f50c0006db0 comp rx=0 tx=0).stop 2024-09-16T01:53:04.100 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.256+0000 7f50cba10640 1 -- 172.21.15.105:0/2055630672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50c4107ef0 msgr2=0x7f50c4107770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:04.101 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.256+0000 7f50cba10640 1 --2- 172.21.15.105:0/2055630672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50c4107ef0 0x7f50c4107770 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f50b80097f0 tx=0x7f50b8030d00 comp rx=0 tx=0).stop 2024-09-16T01:53:04.101 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.256+0000 7f50cba10640 1 -- 172.21.15.105:0/2055630672 shutdown_connections 2024-09-16T01:53:04.101 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.256+0000 7f50cba10640 1 --2- 172.21.15.105:0/2055630672 >> [v2:172.21.15.105:6800/1268348742,v1:172.21.15.105:6801/1268348742] conn(0x7f50b0039050 0x7f50b003b510 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:04.101 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.256+0000 7f50cba10640 1 --2- 172.21.15.105:0/2055630672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50c4107ef0 0x7f50c4107770 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:04.101 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.256+0000 7f50cba10640 1 -- 172.21.15.105:0/2055630672 >> 172.21.15.105:0/2055630672 conn(0x7f50c4101b10 msgr2=0x7f50c4102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:04.101 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.256+0000 7f50cba10640 1 -- 172.21.15.105:0/2055630672 shutdown_connections 2024-09-16T01:53:04.101 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:52:58.256+0000 7f50cba10640 1 -- 172.21.15.105:0/2055630672 wait complete. 2024-09-16T01:53:04.101 INFO:teuthology.orchestra.run.smithi105.stdout:mgr is available 2024-09-16T01:53:07.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:06 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2668030837' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-16T01:53:10.008 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout 2024-09-16T01:53:10.016 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout [global] 2024-09-16T01:53:10.016 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout fsid = 423b7a2a-73ce-11ef-bceb-c7b262605968 2024-09-16T01:53:10.016 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.105:3300,v1:172.21.15.105:6789] 2024-09-16T01:53:10.016 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-16T01:53:10.016 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-16T01:53:10.016 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-16T01:53:10.016 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-16T01:53:10.016 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout 2024-09-16T01:53:10.016 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-16T01:53:10.016 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-16T01:53:10.016 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout 2024-09-16T01:53:10.017 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout [osd] 2024-09-16T01:53:10.017 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-16T01:53:10.017 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-16T01:53:10.017 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-16T01:53:10.017 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-16T01:53:10.017 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-16T01:53:10.017 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.761+0000 7f39f56bc640 1 Processor -- start 2024-09-16T01:53:10.017 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.761+0000 7f39f56bc640 1 -- start start 2024-09-16T01:53:10.017 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.761+0000 7f39f56bc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39f0106380 0x7f39f0106780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:10.017 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.761+0000 7f39f56bc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f39f0106d50 con 0x7f39f0106380 2024-09-16T01:53:10.017 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.762+0000 7f39eeffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39f0106380 0x7f39f0106780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:10.017 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.762+0000 7f39eeffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39f0106380 0x7f39f0106780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48246/0 (socket says 172.21.15.105:48246) 2024-09-16T01:53:10.017 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.762+0000 7f39eeffd640 1 -- 172.21.15.105:0/3863078661 learned_addr learned my addr 172.21.15.105:0/3863078661 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:10.018 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.762+0000 7f39eeffd640 1 -- 172.21.15.105:0/3863078661 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f0107580 con 0x7f39f0106380 2024-09-16T01:53:10.018 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.763+0000 7f39eeffd640 1 --2- 172.21.15.105:0/3863078661 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39f0106380 0x7f39f0106780 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f39e0009e80 tx=0x7f39e002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff06ba10292c1200 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:10.018 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.763+0000 7f39edffb640 1 -- 172.21.15.105:0/3863078661 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39e0036070 con 0x7f39f0106380 2024-09-16T01:53:10.018 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.763+0000 7f39edffb640 1 -- 172.21.15.105:0/3863078661 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f39e0031890 con 0x7f39f0106380 2024-09-16T01:53:10.018 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.763+0000 7f39edffb640 1 -- 172.21.15.105:0/3863078661 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39e0007980 con 0x7f39f0106380 2024-09-16T01:53:10.018 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.763+0000 7f39f56bc640 1 -- 172.21.15.105:0/3863078661 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39f0106380 msgr2=0x7f39f0106780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:10.018 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.763+0000 7f39f56bc640 1 --2- 172.21.15.105:0/3863078661 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39f0106380 0x7f39f0106780 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f39e0009e80 tx=0x7f39e002a7a0 comp rx=0 tx=0).stop 2024-09-16T01:53:10.018 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.763+0000 7f39f56bc640 1 -- 172.21.15.105:0/3863078661 shutdown_connections 2024-09-16T01:53:10.018 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.763+0000 7f39f56bc640 1 --2- 172.21.15.105:0/3863078661 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39f0106380 0x7f39f0106780 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:10.018 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.763+0000 7f39f56bc640 1 -- 172.21.15.105:0/3863078661 >> 172.21.15.105:0/3863078661 conn(0x7f39f0101b10 msgr2=0x7f39f0103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:10.018 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.763+0000 7f39f56bc640 1 -- 172.21.15.105:0/3863078661 shutdown_connections 2024-09-16T01:53:10.018 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.763+0000 7f39f56bc640 1 -- 172.21.15.105:0/3863078661 wait complete. 2024-09-16T01:53:10.018 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.764+0000 7f39f56bc640 1 Processor -- start 2024-09-16T01:53:10.018 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.764+0000 7f39f56bc640 1 -- start start 2024-09-16T01:53:10.019 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.764+0000 7f39f56bc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39f0106380 0x7f39f01915b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:10.019 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.764+0000 7f39f56bc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f39f010a930 con 0x7f39f0106380 2024-09-16T01:53:10.019 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.765+0000 7f39eeffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39f0106380 0x7f39f01915b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:10.019 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.765+0000 7f39eeffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39f0106380 0x7f39f01915b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48250/0 (socket says 172.21.15.105:48250) 2024-09-16T01:53:10.019 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.765+0000 7f39eeffd640 1 -- 172.21.15.105:0/2668030837 learned_addr learned my addr 172.21.15.105:0/2668030837 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:10.019 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.765+0000 7f39eeffd640 1 -- 172.21.15.105:0/2668030837 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f0191af0 con 0x7f39f0106380 2024-09-16T01:53:10.019 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.765+0000 7f39eeffd640 1 --2- 172.21.15.105:0/2668030837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39f0106380 0x7f39f01915b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f39e002acc0 tx=0x7f39e0031c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:10.019 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.765+0000 7f39d7fff640 1 -- 172.21.15.105:0/2668030837 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39e0041030 con 0x7f39f0106380 2024-09-16T01:53:10.019 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.765+0000 7f39d7fff640 1 -- 172.21.15.105:0/2668030837 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f39e00073f0 con 0x7f39f0106380 2024-09-16T01:53:10.019 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.765+0000 7f39f56bc640 1 -- 172.21.15.105:0/2668030837 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39f018fce0 con 0x7f39f0106380 2024-09-16T01:53:10.019 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.765+0000 7f39d7fff640 1 -- 172.21.15.105:0/2668030837 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39e0036040 con 0x7f39f0106380 2024-09-16T01:53:10.019 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.765+0000 7f39f56bc640 1 -- 172.21.15.105:0/2668030837 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39f0190220 con 0x7f39f0106380 2024-09-16T01:53:10.019 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.766+0000 7f39d7fff640 1 -- 172.21.15.105:0/2668030837 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f39e0044070 con 0x7f39f0106380 2024-09-16T01:53:10.020 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.766+0000 7f39d7fff640 1 --2- 172.21.15.105:0/2668030837 >> [v2:172.21.15.105:6800/1268348742,v1:172.21.15.105:6801/1268348742] conn(0x7f39d80391b0 0x7f39d803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:10.020 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.766+0000 7f39d7fff640 1 -- 172.21.15.105:0/2668030837 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f39e0042070 con 0x7f39f0106380 2024-09-16T01:53:10.020 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.766+0000 7f39ee7fc640 1 --2- 172.21.15.105:0/2668030837 >> [v2:172.21.15.105:6800/1268348742,v1:172.21.15.105:6801/1268348742] conn(0x7f39d80391b0 0x7f39d803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:10.020 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.766+0000 7f39f56bc640 1 -- 172.21.15.105:0/2668030837 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39f010aab0 con 0x7f39f0106380 2024-09-16T01:53:10.020 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.767+0000 7f39ee7fc640 1 --2- 172.21.15.105:0/2668030837 >> [v2:172.21.15.105:6800/1268348742,v1:172.21.15.105:6801/1268348742] conn(0x7f39d80391b0 0x7f39d803b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f39e40098c0 tx=0x7f39e4006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:10.020 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.771+0000 7f39d7fff640 1 -- 172.21.15.105:0/2668030837 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f39e004b070 con 0x7f39f0106380 2024-09-16T01:53:10.020 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.938+0000 7f39f56bc640 1 -- 172.21.15.105:0/2668030837 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f39f0106800 con 0x7f39f0106380 2024-09-16T01:53:10.020 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.939+0000 7f39d7fff640 1 -- 172.21.15.105:0/2668030837 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+431 (secure 0 0 0) 0x7f39e003d850 con 0x7f39f0106380 2024-09-16T01:53:10.020 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.941+0000 7f39f56bc640 1 -- 172.21.15.105:0/2668030837 >> [v2:172.21.15.105:6800/1268348742,v1:172.21.15.105:6801/1268348742] conn(0x7f39d80391b0 msgr2=0x7f39d803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:10.020 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.941+0000 7f39f56bc640 1 --2- 172.21.15.105:0/2668030837 >> [v2:172.21.15.105:6800/1268348742,v1:172.21.15.105:6801/1268348742] conn(0x7f39d80391b0 0x7f39d803b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f39e40098c0 tx=0x7f39e4006db0 comp rx=0 tx=0).stop 2024-09-16T01:53:10.020 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.941+0000 7f39f56bc640 1 -- 172.21.15.105:0/2668030837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39f0106380 msgr2=0x7f39f01915b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:10.020 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.941+0000 7f39f56bc640 1 --2- 172.21.15.105:0/2668030837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39f0106380 0x7f39f01915b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f39e002acc0 tx=0x7f39e0031c70 comp rx=0 tx=0).stop 2024-09-16T01:53:10.020 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.941+0000 7f39f56bc640 1 -- 172.21.15.105:0/2668030837 shutdown_connections 2024-09-16T01:53:10.021 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.941+0000 7f39f56bc640 1 --2- 172.21.15.105:0/2668030837 >> [v2:172.21.15.105:6800/1268348742,v1:172.21.15.105:6801/1268348742] conn(0x7f39d80391b0 0x7f39d803b670 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:10.021 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.941+0000 7f39f56bc640 1 --2- 172.21.15.105:0/2668030837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39f0106380 0x7f39f01915b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:10.021 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.941+0000 7f39f56bc640 1 -- 172.21.15.105:0/2668030837 >> 172.21.15.105:0/2668030837 conn(0x7f39f0101b10 msgr2=0x7f39f0102a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:10.021 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.941+0000 7f39f56bc640 1 -- 172.21.15.105:0/2668030837 shutdown_connections 2024-09-16T01:53:10.021 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:06.941+0000 7f39f56bc640 1 -- 172.21.15.105:0/2668030837 wait complete. 2024-09-16T01:53:10.021 INFO:teuthology.orchestra.run.smithi105.stdout:Enabling cephadm module... 2024-09-16T01:53:11.036 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:10 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2391846479' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-16T01:53:11.036 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:10 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: ignoring --setuser ceph since I am not root 2024-09-16T01:53:11.036 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:10 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: ignoring --setgroup ceph since I am not root 2024-09-16T01:53:11.036 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:10 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:10.845+0000 7f2835410040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T01:53:11.037 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:10 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:10.901+0000 7f2835410040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T01:53:11.294 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:11 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:11.052+0000 7f2835410040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T01:53:11.294 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:11 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:11.118+0000 7f2835410040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T01:53:11.295 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:11 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:11.266+0000 7f2835410040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T01:53:11.352 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.435+0000 7f133175d640 1 Processor -- start 2024-09-16T01:53:11.353 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.435+0000 7f133175d640 1 -- start start 2024-09-16T01:53:11.353 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.435+0000 7f133175d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f132c107ef0 0x7f132c10a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:11.353 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.435+0000 7f133175d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f132c07be90 con 0x7f132c107ef0 2024-09-16T01:53:11.353 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.435+0000 7f132affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f132c107ef0 0x7f132c10a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:11.353 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.435+0000 7f132affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f132c107ef0 0x7f132c10a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50006/0 (socket says 172.21.15.105:50006) 2024-09-16T01:53:11.353 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.435+0000 7f132affd640 1 -- 172.21.15.105:0/905931909 learned_addr learned my addr 172.21.15.105:0/905931909 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:11.353 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.435+0000 7f132affd640 1 -- 172.21.15.105:0/905931909 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f132c10a820 con 0x7f132c107ef0 2024-09-16T01:53:11.354 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.436+0000 7f132affd640 1 --2- 172.21.15.105:0/905931909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f132c107ef0 0x7f132c10a2e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f131c009e80 tx=0x7f131c02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b8cc16c1dabe3c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:11.354 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.436+0000 7f1329ffb640 1 -- 172.21.15.105:0/905931909 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f131c036070 con 0x7f132c107ef0 2024-09-16T01:53:11.354 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.436+0000 7f1329ffb640 1 -- 172.21.15.105:0/905931909 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f131c031890 con 0x7f132c107ef0 2024-09-16T01:53:11.354 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.436+0000 7f1329ffb640 1 -- 172.21.15.105:0/905931909 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f131c007980 con 0x7f132c107ef0 2024-09-16T01:53:11.354 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.437+0000 7f133175d640 1 -- 172.21.15.105:0/905931909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f132c107ef0 msgr2=0x7f132c10a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:11.354 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.437+0000 7f133175d640 1 --2- 172.21.15.105:0/905931909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f132c107ef0 0x7f132c10a2e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f131c009e80 tx=0x7f131c02a7a0 comp rx=0 tx=0).stop 2024-09-16T01:53:11.354 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.437+0000 7f133175d640 1 -- 172.21.15.105:0/905931909 shutdown_connections 2024-09-16T01:53:11.354 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.437+0000 7f133175d640 1 --2- 172.21.15.105:0/905931909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f132c107ef0 0x7f132c10a2e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:11.354 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.437+0000 7f133175d640 1 -- 172.21.15.105:0/905931909 >> 172.21.15.105:0/905931909 conn(0x7f132c101b10 msgr2=0x7f132c103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:11.354 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.437+0000 7f133175d640 1 -- 172.21.15.105:0/905931909 shutdown_connections 2024-09-16T01:53:11.354 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.437+0000 7f133175d640 1 -- 172.21.15.105:0/905931909 wait complete. 2024-09-16T01:53:11.354 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.438+0000 7f133175d640 1 Processor -- start 2024-09-16T01:53:11.354 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.438+0000 7f133175d640 1 -- start start 2024-09-16T01:53:11.355 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.438+0000 7f133175d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f132c107ef0 0x7f132c107770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:11.355 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.438+0000 7f133175d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f132c10b860 con 0x7f132c107ef0 2024-09-16T01:53:11.355 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.438+0000 7f132affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f132c107ef0 0x7f132c107770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:11.355 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.438+0000 7f132affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f132c107ef0 0x7f132c107770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50012/0 (socket says 172.21.15.105:50012) 2024-09-16T01:53:11.355 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.438+0000 7f132affd640 1 -- 172.21.15.105:0/2391846479 learned_addr learned my addr 172.21.15.105:0/2391846479 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:11.355 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.438+0000 7f132affd640 1 -- 172.21.15.105:0/2391846479 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f132c107cb0 con 0x7f132c107ef0 2024-09-16T01:53:11.355 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.439+0000 7f132affd640 1 --2- 172.21.15.105:0/2391846479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f132c107ef0 0x7f132c107770 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f131c009680 tx=0x7f131c031c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:11.355 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.439+0000 7f1313fff640 1 -- 172.21.15.105:0/2391846479 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f131c041020 con 0x7f132c107ef0 2024-09-16T01:53:11.355 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.439+0000 7f1313fff640 1 -- 172.21.15.105:0/2391846479 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f131c03c900 con 0x7f132c107ef0 2024-09-16T01:53:11.355 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.439+0000 7f1313fff640 1 -- 172.21.15.105:0/2391846479 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f131c036040 con 0x7f132c107ef0 2024-09-16T01:53:11.355 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.439+0000 7f133175d640 1 -- 172.21.15.105:0/2391846479 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f132c105ea0 con 0x7f132c107ef0 2024-09-16T01:53:11.355 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.439+0000 7f133175d640 1 -- 172.21.15.105:0/2391846479 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f132c1063e0 con 0x7f132c107ef0 2024-09-16T01:53:11.355 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.440+0000 7f1313fff640 1 -- 172.21.15.105:0/2391846479 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f131c044020 con 0x7f132c107ef0 2024-09-16T01:53:11.356 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.440+0000 7f133175d640 1 -- 172.21.15.105:0/2391846479 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f132c108350 con 0x7f132c107ef0 2024-09-16T01:53:11.356 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.440+0000 7f1313fff640 1 --2- 172.21.15.105:0/2391846479 >> [v2:172.21.15.105:6800/1268348742,v1:172.21.15.105:6801/1268348742] conn(0x7f1314039160 0x7f131403b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:11.356 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.440+0000 7f1313fff640 1 -- 172.21.15.105:0/2391846479 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f131c06d4e0 con 0x7f132c107ef0 2024-09-16T01:53:11.356 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.440+0000 7f132a7fc640 1 --2- 172.21.15.105:0/2391846479 >> [v2:172.21.15.105:6800/1268348742,v1:172.21.15.105:6801/1268348742] conn(0x7f1314039160 0x7f131403b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:11.356 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.441+0000 7f132a7fc640 1 --2- 172.21.15.105:0/2391846479 >> [v2:172.21.15.105:6800/1268348742,v1:172.21.15.105:6801/1268348742] conn(0x7f1314039160 0x7f131403b620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f13200098c0 tx=0x7f1320006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:11.356 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.445+0000 7f1313fff640 1 -- 172.21.15.105:0/2391846479 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f131c04b020 con 0x7f132c107ef0 2024-09-16T01:53:11.356 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.654+0000 7f133175d640 1 -- 172.21.15.105:0/2391846479 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f132c194f30 con 0x7f132c107ef0 2024-09-16T01:53:11.356 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.705+0000 7f1313fff640 1 -- 172.21.15.105:0/2391846479 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f131c045a00 con 0x7f132c107ef0 2024-09-16T01:53:11.356 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.707+0000 7f1313fff640 1 -- 172.21.15.105:0/2391846479 <== mon.0 v2:172.21.15.105:3300/0 8 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f131c046780 con 0x7f132c107ef0 2024-09-16T01:53:11.356 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.708+0000 7f133175d640 1 -- 172.21.15.105:0/2391846479 >> [v2:172.21.15.105:6800/1268348742,v1:172.21.15.105:6801/1268348742] conn(0x7f1314039160 msgr2=0x7f131403b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:11.356 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.708+0000 7f133175d640 1 --2- 172.21.15.105:0/2391846479 >> [v2:172.21.15.105:6800/1268348742,v1:172.21.15.105:6801/1268348742] conn(0x7f1314039160 0x7f131403b620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f13200098c0 tx=0x7f1320006db0 comp rx=0 tx=0).stop 2024-09-16T01:53:11.356 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.708+0000 7f133175d640 1 -- 172.21.15.105:0/2391846479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f132c107ef0 msgr2=0x7f132c107770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:11.356 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.708+0000 7f133175d640 1 --2- 172.21.15.105:0/2391846479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f132c107ef0 0x7f132c107770 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f131c009680 tx=0x7f131c031c70 comp rx=0 tx=0).stop 2024-09-16T01:53:11.357 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.708+0000 7f133175d640 1 -- 172.21.15.105:0/2391846479 shutdown_connections 2024-09-16T01:53:11.357 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.708+0000 7f133175d640 1 --2- 172.21.15.105:0/2391846479 >> [v2:172.21.15.105:6800/1268348742,v1:172.21.15.105:6801/1268348742] conn(0x7f1314039160 0x7f131403b620 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:11.357 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.708+0000 7f133175d640 1 --2- 172.21.15.105:0/2391846479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f132c107ef0 0x7f132c107770 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:11.357 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.708+0000 7f133175d640 1 -- 172.21.15.105:0/2391846479 >> 172.21.15.105:0/2391846479 conn(0x7f132c101b10 msgr2=0x7f132c102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:11.357 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.709+0000 7f133175d640 1 -- 172.21.15.105:0/2391846479 shutdown_connections 2024-09-16T01:53:11.357 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:10.709+0000 7f133175d640 1 -- 172.21.15.105:0/2391846479 wait complete. 2024-09-16T01:53:11.631 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:11 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:11.415+0000 7f2835410040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T01:53:11.916 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:11 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2391846479' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-16T01:53:11.916 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:11 smithi105 ceph-mon[31197]: mgrmap e5: x(active, since 14s) 2024-09-16T01:53:11.916 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:11 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:11.738+0000 7f2835410040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T01:53:11.916 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:11 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:11.799+0000 7f2835410040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T01:53:12.294 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:11 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:11.989+0000 7f2835410040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T01:53:13.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:12 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/455239504' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-16T01:53:13.045 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:12 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:12.868+0000 7f2835410040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T01:53:13.045 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:12 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:12.934+0000 7f2835410040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T01:53:13.061 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout { 2024-09-16T01:53:13.061 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-16T01:53:13.061 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T01:53:13.061 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-16T01:53:13.061 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-16T01:53:13.061 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout } 2024-09-16T01:53:13.061 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.707+0000 7f2945e67640 1 Processor -- start 2024-09-16T01:53:13.061 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.707+0000 7f2945e67640 1 -- start start 2024-09-16T01:53:13.061 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.707+0000 7f2945e67640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2940107ef0 0x7f294010a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:13.061 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.707+0000 7f2945e67640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f294007be90 con 0x7f2940107ef0 2024-09-16T01:53:13.062 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.707+0000 7f293f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2940107ef0 0x7f294010a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:13.062 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.707+0000 7f293f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2940107ef0 0x7f294010a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50030/0 (socket says 172.21.15.105:50030) 2024-09-16T01:53:13.062 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.707+0000 7f293f7fe640 1 -- 172.21.15.105:0/1264522310 learned_addr learned my addr 172.21.15.105:0/1264522310 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:13.062 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.707+0000 7f293f7fe640 1 -- 172.21.15.105:0/1264522310 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f294010a820 con 0x7f2940107ef0 2024-09-16T01:53:13.062 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.708+0000 7f293f7fe640 1 --2- 172.21.15.105:0/1264522310 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2940107ef0 0x7f294010a2e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2930009820 tx=0x7f293002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fa7742f76dd9f01 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:13.062 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.708+0000 7f293e7fc640 1 -- 172.21.15.105:0/1264522310 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2930037070 con 0x7f2940107ef0 2024-09-16T01:53:13.062 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.708+0000 7f293e7fc640 1 -- 172.21.15.105:0/1264522310 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f29300328d0 con 0x7f2940107ef0 2024-09-16T01:53:13.062 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.708+0000 7f293e7fc640 1 -- 172.21.15.105:0/1264522310 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29300307d0 con 0x7f2940107ef0 2024-09-16T01:53:13.062 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.709+0000 7f2945e67640 1 -- 172.21.15.105:0/1264522310 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2940107ef0 msgr2=0x7f294010a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:13.062 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.709+0000 7f2945e67640 1 --2- 172.21.15.105:0/1264522310 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2940107ef0 0x7f294010a2e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2930009820 tx=0x7f293002a270 comp rx=0 tx=0).stop 2024-09-16T01:53:13.062 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.709+0000 7f2945e67640 1 -- 172.21.15.105:0/1264522310 shutdown_connections 2024-09-16T01:53:13.062 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.709+0000 7f2945e67640 1 --2- 172.21.15.105:0/1264522310 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2940107ef0 0x7f294010a2e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:13.062 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.709+0000 7f2945e67640 1 -- 172.21.15.105:0/1264522310 >> 172.21.15.105:0/1264522310 conn(0x7f2940101b10 msgr2=0x7f2940103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:13.063 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.709+0000 7f2945e67640 1 -- 172.21.15.105:0/1264522310 shutdown_connections 2024-09-16T01:53:13.063 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.709+0000 7f2945e67640 1 -- 172.21.15.105:0/1264522310 wait complete. 2024-09-16T01:53:13.063 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.709+0000 7f2945e67640 1 Processor -- start 2024-09-16T01:53:13.063 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.710+0000 7f2945e67640 1 -- start start 2024-09-16T01:53:13.063 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.710+0000 7f2945e67640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2940107ef0 0x7f294019d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:13.063 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.710+0000 7f2945e67640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f294010b860 con 0x7f2940107ef0 2024-09-16T01:53:13.063 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.710+0000 7f293f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2940107ef0 0x7f294019d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:13.063 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.710+0000 7f293f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2940107ef0 0x7f294019d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50040/0 (socket says 172.21.15.105:50040) 2024-09-16T01:53:13.063 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.710+0000 7f293f7fe640 1 -- 172.21.15.105:0/455239504 learned_addr learned my addr 172.21.15.105:0/455239504 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:13.064 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.710+0000 7f293f7fe640 1 -- 172.21.15.105:0/455239504 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f294019dee0 con 0x7f2940107ef0 2024-09-16T01:53:13.064 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.710+0000 7f293f7fe640 1 --2- 172.21.15.105:0/455239504 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2940107ef0 0x7f294019d9a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f29300097f0 tx=0x7f2930030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:13.064 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.712+0000 7f293cff9640 1 -- 172.21.15.105:0/455239504 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2930037040 con 0x7f2940107ef0 2024-09-16T01:53:13.064 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.712+0000 7f293cff9640 1 -- 172.21.15.105:0/455239504 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f293003b3f0 con 0x7f2940107ef0 2024-09-16T01:53:13.064 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.712+0000 7f2945e67640 1 -- 172.21.15.105:0/455239504 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f294019e170 con 0x7f2940107ef0 2024-09-16T01:53:13.064 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.712+0000 7f293cff9640 1 -- 172.21.15.105:0/455239504 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f293003a520 con 0x7f2940107ef0 2024-09-16T01:53:13.064 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.712+0000 7f2945e67640 1 -- 172.21.15.105:0/455239504 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29401a0e60 con 0x7f2940107ef0 2024-09-16T01:53:13.064 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.713+0000 7f2945e67640 1 -- 172.21.15.105:0/455239504 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2940108350 con 0x7f2940107ef0 2024-09-16T01:53:13.064 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.713+0000 7f293cff9640 1 -- 172.21.15.105:0/455239504 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f293003a6c0 con 0x7f2940107ef0 2024-09-16T01:53:13.064 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.713+0000 7f293cff9640 1 --2- 172.21.15.105:0/455239504 >> [v2:172.21.15.105:6800/1268348742,v1:172.21.15.105:6801/1268348742] conn(0x7f2928039200 0x7f292803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:13.064 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.713+0000 7f293cff9640 1 -- 172.21.15.105:0/455239504 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f293006c470 con 0x7f2940107ef0 2024-09-16T01:53:13.064 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.713+0000 7f293effd640 1 -- 172.21.15.105:0/455239504 >> [v2:172.21.15.105:6800/1268348742,v1:172.21.15.105:6801/1268348742] conn(0x7f2928039200 msgr2=0x7f292803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:6800/1268348742 2024-09-16T01:53:13.064 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.713+0000 7f293effd640 1 --2- 172.21.15.105:0/455239504 >> [v2:172.21.15.105:6800/1268348742,v1:172.21.15.105:6801/1268348742] conn(0x7f2928039200 0x7f292803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T01:53:13.065 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.718+0000 7f293cff9640 1 -- 172.21.15.105:0/455239504 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2930032820 con 0x7f2940107ef0 2024-09-16T01:53:13.065 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.914+0000 7f293effd640 1 -- 172.21.15.105:0/455239504 >> [v2:172.21.15.105:6800/1268348742,v1:172.21.15.105:6801/1268348742] conn(0x7f2928039200 msgr2=0x7f292803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:6800/1268348742 2024-09-16T01:53:13.065 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.914+0000 7f293effd640 1 --2- 172.21.15.105:0/455239504 >> [v2:172.21.15.105:6800/1268348742,v1:172.21.15.105:6801/1268348742] conn(0x7f2928039200 0x7f292803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T01:53:13.065 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.930+0000 7f2945e67640 1 -- 172.21.15.105:0/455239504 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f294002d7c0 con 0x7f2940107ef0 2024-09-16T01:53:13.065 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.931+0000 7f293cff9640 1 -- 172.21.15.105:0/455239504 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f293006be90 con 0x7f2940107ef0 2024-09-16T01:53:13.065 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.933+0000 7f2945e67640 1 -- 172.21.15.105:0/455239504 >> [v2:172.21.15.105:6800/1268348742,v1:172.21.15.105:6801/1268348742] conn(0x7f2928039200 msgr2=0x7f292803b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:53:13.065 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.933+0000 7f2945e67640 1 --2- 172.21.15.105:0/455239504 >> [v2:172.21.15.105:6800/1268348742,v1:172.21.15.105:6801/1268348742] conn(0x7f2928039200 0x7f292803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:13.065 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.933+0000 7f2945e67640 1 -- 172.21.15.105:0/455239504 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2940107ef0 msgr2=0x7f294019d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:13.065 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.933+0000 7f2945e67640 1 --2- 172.21.15.105:0/455239504 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2940107ef0 0x7f294019d9a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f29300097f0 tx=0x7f2930030d00 comp rx=0 tx=0).stop 2024-09-16T01:53:13.065 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.933+0000 7f2945e67640 1 -- 172.21.15.105:0/455239504 shutdown_connections 2024-09-16T01:53:13.066 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.933+0000 7f2945e67640 1 --2- 172.21.15.105:0/455239504 >> [v2:172.21.15.105:6800/1268348742,v1:172.21.15.105:6801/1268348742] conn(0x7f2928039200 0x7f292803b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:13.066 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.933+0000 7f2945e67640 1 --2- 172.21.15.105:0/455239504 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2940107ef0 0x7f294019d9a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:13.066 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.933+0000 7f2945e67640 1 -- 172.21.15.105:0/455239504 >> 172.21.15.105:0/455239504 conn(0x7f2940101b10 msgr2=0x7f2940102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:13.066 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.933+0000 7f2945e67640 1 -- 172.21.15.105:0/455239504 shutdown_connections 2024-09-16T01:53:13.066 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:11.933+0000 7f2945e67640 1 -- 172.21.15.105:0/455239504 wait complete. 2024-09-16T01:53:13.066 INFO:teuthology.orchestra.run.smithi105.stdout:Waiting for the mgr to restart... 2024-09-16T01:53:13.067 INFO:teuthology.orchestra.run.smithi105.stdout:Waiting for mgr epoch 5... 2024-09-16T01:53:13.910 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:13 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:13.652+0000 7f2835410040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T01:53:13.911 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:13 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:13.769+0000 7f2835410040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T01:53:14.187 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:13 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:13.908+0000 7f2835410040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T01:53:14.187 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:13 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:13.969+0000 7f2835410040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T01:53:14.187 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:14 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:14.117+0000 7f2835410040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T01:53:14.187 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:14 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:14.185+0000 7f2835410040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T01:53:14.465 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:14 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:14.243+0000 7f2835410040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T01:53:14.465 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:14 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:14.399+0000 7f2835410040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T01:53:14.465 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:14 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:14.463+0000 7f2835410040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T01:53:14.729 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:14 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:14.528+0000 7f2835410040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T01:53:14.729 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:14 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:14.610+0000 7f2835410040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T01:53:14.730 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:14 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:14.669+0000 7f2835410040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T01:53:14.730 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:14 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:14.727+0000 7f2835410040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T01:53:15.794 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:15 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:15.519+0000 7f2835410040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T01:53:15.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:15 smithi105 ceph-mon[31197]: Active manager daemon x restarted 2024-09-16T01:53:15.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:15 smithi105 ceph-mon[31197]: Activating manager daemon x 2024-09-16T01:53:15.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:15 smithi105 ceph-mon[31197]: osdmap e2: 0 total, 0 up, 0 in 2024-09-16T01:53:15.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:15 smithi105 ceph-mon[31197]: mgrmap e6: x(active, starting, since 0.00402541s) 2024-09-16T01:53:15.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:15 smithi105 ceph-mon[31197]: from='mgr.14118 172.21.15.105:0/2613168790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:53:15.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:15 smithi105 ceph-mon[31197]: from='mgr.14118 172.21.15.105:0/2613168790' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T01:53:15.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:15 smithi105 ceph-mon[31197]: from='mgr.14118 172.21.15.105:0/2613168790' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T01:53:15.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:15 smithi105 ceph-mon[31197]: from='mgr.14118 172.21.15.105:0/2613168790' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T01:53:15.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:15 smithi105 ceph-mon[31197]: from='mgr.14118 172.21.15.105:0/2613168790' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T01:53:15.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:15 smithi105 ceph-mon[31197]: Manager daemon x is now available 2024-09-16T01:53:15.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:15 smithi105 ceph-mon[31197]: from='mgr.14118 172.21.15.105:0/2613168790' entity='mgr.x' 2024-09-16T01:53:15.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:15 smithi105 ceph-mon[31197]: from='mgr.14118 172.21.15.105:0/2613168790' entity='mgr.x' 2024-09-16T01:53:15.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:15 smithi105 ceph-mon[31197]: from='mgr.14118 172.21.15.105:0/2613168790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:53:15.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:15 smithi105 ceph-mon[31197]: from='mgr.14118 172.21.15.105:0/2613168790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:53:15.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:15 smithi105 ceph-mon[31197]: from='mgr.14118 172.21.15.105:0/2613168790' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-16T01:53:15.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:15 smithi105 ceph-mon[31197]: from='mgr.14118 172.21.15.105:0/2613168790' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-16T01:53:16.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:16 smithi105 ceph-mon[31197]: Found migration_current of "None". Setting to last migration. 2024-09-16T01:53:16.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:16 smithi105 ceph-mon[31197]: mgrmap e7: x(active, since 1.00574s) 2024-09-16T01:53:17.175 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout { 2024-09-16T01:53:17.176 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-16T01:53:17.176 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-16T01:53:17.176 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout } 2024-09-16T01:53:17.176 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.447+0000 7f41429d7640 1 Processor -- start 2024-09-16T01:53:17.176 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.447+0000 7f41429d7640 1 -- start start 2024-09-16T01:53:17.176 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.447+0000 7f41429d7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f413c107f30 0x7f413c10a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:17.176 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.447+0000 7f41429d7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f413c07be50 con 0x7f413c107f30 2024-09-16T01:53:17.176 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.447+0000 7f413bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f413c107f30 0x7f413c10a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:17.176 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.447+0000 7f413bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f413c107f30 0x7f413c10a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50046/0 (socket says 172.21.15.105:50046) 2024-09-16T01:53:17.176 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.447+0000 7f413bfff640 1 -- 172.21.15.105:0/2567112746 learned_addr learned my addr 172.21.15.105:0/2567112746 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:17.176 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.447+0000 7f413bfff640 1 -- 172.21.15.105:0/2567112746 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f413c10a860 con 0x7f413c107f30 2024-09-16T01:53:17.177 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.448+0000 7f413bfff640 1 --2- 172.21.15.105:0/2567112746 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f413c107f30 0x7f413c10a320 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f412c009820 tx=0x7f412c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b75a440e312d39 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:17.177 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.448+0000 7f413affd640 1 -- 172.21.15.105:0/2567112746 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f412c037070 con 0x7f413c107f30 2024-09-16T01:53:17.177 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.448+0000 7f413affd640 1 -- 172.21.15.105:0/2567112746 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f412c0328d0 con 0x7f413c107f30 2024-09-16T01:53:17.177 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.448+0000 7f413affd640 1 -- 172.21.15.105:0/2567112746 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f412c0307d0 con 0x7f413c107f30 2024-09-16T01:53:17.177 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.449+0000 7f41429d7640 1 -- 172.21.15.105:0/2567112746 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f413c107f30 msgr2=0x7f413c10a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:17.177 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.449+0000 7f41429d7640 1 --2- 172.21.15.105:0/2567112746 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f413c107f30 0x7f413c10a320 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f412c009820 tx=0x7f412c02a270 comp rx=0 tx=0).stop 2024-09-16T01:53:17.177 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.449+0000 7f41429d7640 1 -- 172.21.15.105:0/2567112746 shutdown_connections 2024-09-16T01:53:17.177 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.449+0000 7f41429d7640 1 --2- 172.21.15.105:0/2567112746 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f413c107f30 0x7f413c10a320 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:17.177 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.449+0000 7f41429d7640 1 -- 172.21.15.105:0/2567112746 >> 172.21.15.105:0/2567112746 conn(0x7f413c101b10 msgr2=0x7f413c103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:17.177 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.449+0000 7f41429d7640 1 -- 172.21.15.105:0/2567112746 shutdown_connections 2024-09-16T01:53:17.177 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.449+0000 7f41429d7640 1 -- 172.21.15.105:0/2567112746 wait complete. 2024-09-16T01:53:17.177 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.450+0000 7f41429d7640 1 Processor -- start 2024-09-16T01:53:17.177 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.450+0000 7f41429d7640 1 -- start start 2024-09-16T01:53:17.178 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.450+0000 7f41429d7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f413c107f30 0x7f413c19b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:17.178 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.450+0000 7f41429d7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f413c10b840 con 0x7f413c107f30 2024-09-16T01:53:17.178 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.450+0000 7f413bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f413c107f30 0x7f413c19b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:17.178 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.450+0000 7f413bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f413c107f30 0x7f413c19b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50060/0 (socket says 172.21.15.105:50060) 2024-09-16T01:53:17.178 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.450+0000 7f413bfff640 1 -- 172.21.15.105:0/2235736330 learned_addr learned my addr 172.21.15.105:0/2235736330 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:17.178 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.450+0000 7f413bfff640 1 -- 172.21.15.105:0/2235736330 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f413c19bca0 con 0x7f413c107f30 2024-09-16T01:53:17.178 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.451+0000 7f413bfff640 1 --2- 172.21.15.105:0/2235736330 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f413c107f30 0x7f413c19b760 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f412c030520 tx=0x7f412c032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:17.178 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.451+0000 7f41397fa640 1 -- 172.21.15.105:0/2235736330 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f412c042070 con 0x7f413c107f30 2024-09-16T01:53:17.178 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.451+0000 7f41429d7640 1 -- 172.21.15.105:0/2235736330 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f413c19bf30 con 0x7f413c107f30 2024-09-16T01:53:17.178 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.451+0000 7f41397fa640 1 -- 172.21.15.105:0/2235736330 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f412c030cc0 con 0x7f413c107f30 2024-09-16T01:53:17.178 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.451+0000 7f41397fa640 1 -- 172.21.15.105:0/2235736330 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f412c037040 con 0x7f413c107f30 2024-09-16T01:53:17.178 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.451+0000 7f41429d7640 1 -- 172.21.15.105:0/2235736330 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f413c19ec20 con 0x7f413c107f30 2024-09-16T01:53:17.178 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.452+0000 7f41397fa640 1 -- 172.21.15.105:0/2235736330 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f412c045070 con 0x7f413c107f30 2024-09-16T01:53:17.178 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.452+0000 7f41397fa640 1 --2- 172.21.15.105:0/2235736330 >> [v2:172.21.15.105:6800/1268348742,v1:172.21.15.105:6801/1268348742] conn(0x7f41240391b0 0x7f412403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:17.179 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.452+0000 7f41397fa640 1 -- 172.21.15.105:0/2235736330 --> [v2:172.21.15.105:6800/1268348742,v1:172.21.15.105:6801/1268348742] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f412403bd80 con 0x7f41240391b0 2024-09-16T01:53:17.179 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.452+0000 7f41397fa640 1 -- 172.21.15.105:0/2235736330 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f412c06c1d0 con 0x7f413c107f30 2024-09-16T01:53:17.179 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.452+0000 7f413b7fe640 1 -- 172.21.15.105:0/2235736330 >> [v2:172.21.15.105:6800/1268348742,v1:172.21.15.105:6801/1268348742] conn(0x7f41240391b0 msgr2=0x7f412403b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:6800/1268348742 2024-09-16T01:53:17.179 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.452+0000 7f413b7fe640 1 --2- 172.21.15.105:0/2235736330 >> [v2:172.21.15.105:6800/1268348742,v1:172.21.15.105:6801/1268348742] conn(0x7f41240391b0 0x7f412403b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T01:53:17.179 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.653+0000 7f413b7fe640 1 -- 172.21.15.105:0/2235736330 >> [v2:172.21.15.105:6800/1268348742,v1:172.21.15.105:6801/1268348742] conn(0x7f41240391b0 msgr2=0x7f412403b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:6800/1268348742 2024-09-16T01:53:17.179 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:13.653+0000 7f413b7fe640 1 --2- 172.21.15.105:0/2235736330 >> [v2:172.21.15.105:6800/1268348742,v1:172.21.15.105:6801/1268348742] conn(0x7f41240391b0 0x7f412403b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T01:53:17.179 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:14.054+0000 7f413b7fe640 1 -- 172.21.15.105:0/2235736330 >> [v2:172.21.15.105:6800/1268348742,v1:172.21.15.105:6801/1268348742] conn(0x7f41240391b0 msgr2=0x7f412403b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:6800/1268348742 2024-09-16T01:53:17.179 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:14.054+0000 7f413b7fe640 1 --2- 172.21.15.105:0/2235736330 >> [v2:172.21.15.105:6800/1268348742,v1:172.21.15.105:6801/1268348742] conn(0x7f41240391b0 0x7f412403b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T01:53:17.179 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:14.855+0000 7f413b7fe640 1 -- 172.21.15.105:0/2235736330 >> [v2:172.21.15.105:6800/1268348742,v1:172.21.15.105:6801/1268348742] conn(0x7f41240391b0 msgr2=0x7f412403b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:6800/1268348742 2024-09-16T01:53:17.179 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:14.855+0000 7f413b7fe640 1 --2- 172.21.15.105:0/2235736330 >> [v2:172.21.15.105:6800/1268348742,v1:172.21.15.105:6801/1268348742] conn(0x7f41240391b0 0x7f412403b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T01:53:17.179 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:15.523+0000 7f41397fa640 1 -- 172.21.15.105:0/2235736330 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mgrmap(e 6) ==== 46228+0+0 (secure 0 0 0) 0x7f412c06ad50 con 0x7f413c107f30 2024-09-16T01:53:17.179 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:15.523+0000 7f41397fa640 1 -- 172.21.15.105:0/2235736330 >> [v2:172.21.15.105:6800/1268348742,v1:172.21.15.105:6801/1268348742] conn(0x7f41240391b0 msgr2=0x7f412403b670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:53:17.180 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:15.523+0000 7f41397fa640 1 --2- 172.21.15.105:0/2235736330 >> [v2:172.21.15.105:6800/1268348742,v1:172.21.15.105:6801/1268348742] conn(0x7f41240391b0 0x7f412403b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:17.180 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:16.524+0000 7f41397fa640 1 -- 172.21.15.105:0/2235736330 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7f412c03b330 con 0x7f413c107f30 2024-09-16T01:53:17.180 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:16.524+0000 7f41397fa640 1 --2- 172.21.15.105:0/2235736330 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f412403cae0 0x7f412403eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:17.180 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:16.524+0000 7f41397fa640 1 -- 172.21.15.105:0/2235736330 --> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f412c02bba0 con 0x7f412403cae0 2024-09-16T01:53:17.180 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:16.525+0000 7f413b7fe640 1 --2- 172.21.15.105:0/2235736330 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f412403cae0 0x7f412403eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:17.180 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:16.525+0000 7f413b7fe640 1 --2- 172.21.15.105:0/2235736330 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f412403cae0 0x7f412403eed0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4130003980 tx=0x7f4130006e10 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:17.180 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:16.526+0000 7f41397fa640 1 -- 172.21.15.105:0/2235736330 <== mgr.14118 v2:172.21.15.105:6800/1469189484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f412c02bba0 con 0x7f412403cae0 2024-09-16T01:53:17.180 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:16.533+0000 7f41429d7640 1 -- 172.21.15.105:0/2235736330 --> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f413c108390 con 0x7f412403cae0 2024-09-16T01:53:17.180 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:16.533+0000 7f41397fa640 1 -- 172.21.15.105:0/2235736330 <== mgr.14118 v2:172.21.15.105:6800/1469189484 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f413c108390 con 0x7f412403cae0 2024-09-16T01:53:17.180 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:16.533+0000 7f41429d7640 1 -- 172.21.15.105:0/2235736330 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f412403cae0 msgr2=0x7f412403eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:17.180 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:16.533+0000 7f41429d7640 1 --2- 172.21.15.105:0/2235736330 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f412403cae0 0x7f412403eed0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4130003980 tx=0x7f4130006e10 comp rx=0 tx=0).stop 2024-09-16T01:53:17.180 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:16.533+0000 7f41429d7640 1 -- 172.21.15.105:0/2235736330 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f413c107f30 msgr2=0x7f413c19b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:17.182 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:16.533+0000 7f41429d7640 1 --2- 172.21.15.105:0/2235736330 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f413c107f30 0x7f413c19b760 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f412c030520 tx=0x7f412c032dd0 comp rx=0 tx=0).stop 2024-09-16T01:53:17.182 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:16.533+0000 7f41429d7640 1 -- 172.21.15.105:0/2235736330 shutdown_connections 2024-09-16T01:53:17.182 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:16.533+0000 7f41429d7640 1 --2- 172.21.15.105:0/2235736330 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f412403cae0 0x7f412403eed0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:17.182 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:16.533+0000 7f41429d7640 1 --2- 172.21.15.105:0/2235736330 >> [v2:172.21.15.105:6800/1268348742,v1:172.21.15.105:6801/1268348742] conn(0x7f41240391b0 0x7f412403b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:17.182 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:16.533+0000 7f41429d7640 1 --2- 172.21.15.105:0/2235736330 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f413c107f30 0x7f413c19b760 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:17.182 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:16.533+0000 7f41429d7640 1 -- 172.21.15.105:0/2235736330 >> 172.21.15.105:0/2235736330 conn(0x7f413c101b10 msgr2=0x7f413c102940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:17.182 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:16.533+0000 7f41429d7640 1 -- 172.21.15.105:0/2235736330 shutdown_connections 2024-09-16T01:53:17.182 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:16.533+0000 7f41429d7640 1 -- 172.21.15.105:0/2235736330 wait complete. 2024-09-16T01:53:17.182 INFO:teuthology.orchestra.run.smithi105.stdout:mgr epoch 5 is available 2024-09-16T01:53:17.182 INFO:teuthology.orchestra.run.smithi105.stdout:Setting orchestrator backend to cephadm... 2024-09-16T01:53:17.710 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:17 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:17.560+0000 7f279c76a640 -1 log_channel(cephadm) log [ERR] : [16/Sep/2024:01:53:17] ENGINE Error in HTTPServer.serve 2024-09-16T01:53:17.710 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:17 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: Traceback (most recent call last): 2024-09-16T01:53:17.710 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:17 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T01:53:17.710 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:17 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: self._connections.run(self.expiration_interval) 2024-09-16T01:53:17.710 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:17 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T01:53:17.710 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:17 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: self._run(expiration_interval) 2024-09-16T01:53:17.710 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:17 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T01:53:17.710 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:17 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T01:53:17.710 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:17 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T01:53:17.710 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:17 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T01:53:17.711 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:17 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T01:53:17.711 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:17 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: s = self.context.wrap_socket( 2024-09-16T01:53:17.711 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:17 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T01:53:17.711 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:17 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: return self.sslsocket_class._create( 2024-09-16T01:53:17.711 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:17 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T01:53:17.711 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:17 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: self.do_handshake() 2024-09-16T01:53:17.711 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:17 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T01:53:17.711 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:17 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: self._sslobj.do_handshake() 2024-09-16T01:53:17.711 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:17 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T01:53:17.711 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:17 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:18.029 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:17 smithi105 ceph-mon[31197]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-16T01:53:18.029 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:17 smithi105 ceph-mon[31197]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-16T01:53:18.030 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:17 smithi105 ceph-mon[31197]: from='mgr.14118 172.21.15.105:0/2613168790' entity='mgr.x' 2024-09-16T01:53:18.030 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:17 smithi105 ceph-mon[31197]: from='mgr.14118 172.21.15.105:0/2613168790' entity='mgr.x' 2024-09-16T01:53:18.030 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:17 smithi105 ceph-mon[31197]: from='mgr.14118 172.21.15.105:0/2613168790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:53:18.321 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.528+0000 7f75c55d8640 1 Processor -- start 2024-09-16T01:53:18.321 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.528+0000 7f75c55d8640 1 -- start start 2024-09-16T01:53:18.321 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.528+0000 7f75c55d8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75c0108b90 0x7f75c0108f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:18.322 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.528+0000 7f75c55d8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f75c01070d0 con 0x7f75c0108b90 2024-09-16T01:53:18.322 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.528+0000 7f75beffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75c0108b90 0x7f75c0108f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:18.322 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.528+0000 7f75beffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75c0108b90 0x7f75c0108f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50106/0 (socket says 172.21.15.105:50106) 2024-09-16T01:53:18.322 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.528+0000 7f75beffd640 1 -- 172.21.15.105:0/3647781222 learned_addr learned my addr 172.21.15.105:0/3647781222 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:18.322 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.528+0000 7f75beffd640 1 -- 172.21.15.105:0/3647781222 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75c0107250 con 0x7f75c0108b90 2024-09-16T01:53:18.322 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.529+0000 7f75beffd640 1 --2- 172.21.15.105:0/3647781222 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75c0108b90 0x7f75c0108f90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f75b0009820 tx=0x7f75b002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb0325092aec30b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:18.322 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.529+0000 7f75bdffb640 1 -- 172.21.15.105:0/3647781222 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75b0037070 con 0x7f75c0108b90 2024-09-16T01:53:18.322 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.529+0000 7f75bdffb640 1 -- 172.21.15.105:0/3647781222 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f75b00328d0 con 0x7f75c0108b90 2024-09-16T01:53:18.322 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.529+0000 7f75bdffb640 1 -- 172.21.15.105:0/3647781222 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75b00307d0 con 0x7f75c0108b90 2024-09-16T01:53:18.322 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.530+0000 7f75c55d8640 1 -- 172.21.15.105:0/3647781222 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75c0108b90 msgr2=0x7f75c0108f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:18.322 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.530+0000 7f75c55d8640 1 --2- 172.21.15.105:0/3647781222 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75c0108b90 0x7f75c0108f90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f75b0009820 tx=0x7f75b002a270 comp rx=0 tx=0).stop 2024-09-16T01:53:18.322 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.530+0000 7f75c55d8640 1 -- 172.21.15.105:0/3647781222 shutdown_connections 2024-09-16T01:53:18.322 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.530+0000 7f75c55d8640 1 --2- 172.21.15.105:0/3647781222 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75c0108b90 0x7f75c0108f90 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:18.322 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.530+0000 7f75c55d8640 1 -- 172.21.15.105:0/3647781222 >> 172.21.15.105:0/3647781222 conn(0x7f75c0102c20 msgr2=0x7f75c0105080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:18.323 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.530+0000 7f75c55d8640 1 -- 172.21.15.105:0/3647781222 shutdown_connections 2024-09-16T01:53:18.323 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.530+0000 7f75c55d8640 1 -- 172.21.15.105:0/3647781222 wait complete. 2024-09-16T01:53:18.323 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.530+0000 7f75c55d8640 1 Processor -- start 2024-09-16T01:53:18.323 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.531+0000 7f75c55d8640 1 -- start start 2024-09-16T01:53:18.323 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.531+0000 7f75c55d8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75c0108b90 0x7f75c019b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:18.323 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.531+0000 7f75c55d8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f75c010ba60 con 0x7f75c0108b90 2024-09-16T01:53:18.323 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.531+0000 7f75beffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75c0108b90 0x7f75c019b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:18.323 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.531+0000 7f75beffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75c0108b90 0x7f75c019b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50116/0 (socket says 172.21.15.105:50116) 2024-09-16T01:53:18.323 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.531+0000 7f75beffd640 1 -- 172.21.15.105:0/3716488986 learned_addr learned my addr 172.21.15.105:0/3716488986 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:18.323 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.531+0000 7f75beffd640 1 -- 172.21.15.105:0/3716488986 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75c019bdf0 con 0x7f75c0108b90 2024-09-16T01:53:18.323 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.531+0000 7f75beffd640 1 --2- 172.21.15.105:0/3716488986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75c0108b90 0x7f75c019b8b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f75b0009940 tx=0x7f75b0032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:18.323 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.531+0000 7f75a7fff640 1 -- 172.21.15.105:0/3716488986 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75b0042070 con 0x7f75c0108b90 2024-09-16T01:53:18.324 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.531+0000 7f75a7fff640 1 -- 172.21.15.105:0/3716488986 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f75b0030c80 con 0x7f75c0108b90 2024-09-16T01:53:18.324 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.531+0000 7f75c55d8640 1 -- 172.21.15.105:0/3716488986 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75c019c080 con 0x7f75c0108b90 2024-09-16T01:53:18.324 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.531+0000 7f75a7fff640 1 -- 172.21.15.105:0/3716488986 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75b0037040 con 0x7f75c0108b90 2024-09-16T01:53:18.324 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.531+0000 7f75c55d8640 1 -- 172.21.15.105:0/3716488986 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75c019ed70 con 0x7f75c0108b90 2024-09-16T01:53:18.324 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.532+0000 7f75a7fff640 1 -- 172.21.15.105:0/3716488986 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7f75b0045070 con 0x7f75c0108b90 2024-09-16T01:53:18.324 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.532+0000 7f75c55d8640 1 -- 172.21.15.105:0/3716488986 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75c0109010 con 0x7f75c0108b90 2024-09-16T01:53:18.324 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.532+0000 7f75a7fff640 1 --2- 172.21.15.105:0/3716488986 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f75a80390a0 0x7f75a803b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:18.324 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.532+0000 7f75a7fff640 1 -- 172.21.15.105:0/3716488986 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f75b006c2b0 con 0x7f75c0108b90 2024-09-16T01:53:18.324 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.532+0000 7f75be7fc640 1 --2- 172.21.15.105:0/3716488986 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f75a80390a0 0x7f75a803b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:18.324 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.533+0000 7f75be7fc640 1 --2- 172.21.15.105:0/3716488986 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f75a80390a0 0x7f75a803b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f75b40098c0 tx=0x7f75b4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:18.324 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.537+0000 7f75a7fff640 1 -- 172.21.15.105:0/3716488986 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f75b0045360 con 0x7f75c0108b90 2024-09-16T01:53:18.324 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.712+0000 7f75a7fff640 1 -- 172.21.15.105:0/3716488986 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f75b003b330 con 0x7f75c0108b90 2024-09-16T01:53:18.324 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.723+0000 7f75c55d8640 1 -- 172.21.15.105:0/3716488986 --> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f75c0062630 con 0x7f75a80390a0 2024-09-16T01:53:18.325 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.727+0000 7f75a7fff640 1 -- 172.21.15.105:0/3716488986 <== mgr.14118 v2:172.21.15.105:6800/1469189484 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f75c0062630 con 0x7f75a80390a0 2024-09-16T01:53:18.325 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.730+0000 7f75c55d8640 1 -- 172.21.15.105:0/3716488986 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f75a80390a0 msgr2=0x7f75a803b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:18.325 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.730+0000 7f75c55d8640 1 --2- 172.21.15.105:0/3716488986 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f75a80390a0 0x7f75a803b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f75b40098c0 tx=0x7f75b4006db0 comp rx=0 tx=0).stop 2024-09-16T01:53:18.325 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.730+0000 7f75c55d8640 1 -- 172.21.15.105:0/3716488986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75c0108b90 msgr2=0x7f75c019b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:18.325 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.730+0000 7f75c55d8640 1 --2- 172.21.15.105:0/3716488986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75c0108b90 0x7f75c019b8b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f75b0009940 tx=0x7f75b0032e10 comp rx=0 tx=0).stop 2024-09-16T01:53:18.325 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.730+0000 7f75c55d8640 1 -- 172.21.15.105:0/3716488986 shutdown_connections 2024-09-16T01:53:18.325 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.730+0000 7f75c55d8640 1 --2- 172.21.15.105:0/3716488986 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f75a80390a0 0x7f75a803b560 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:18.325 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.730+0000 7f75c55d8640 1 --2- 172.21.15.105:0/3716488986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75c0108b90 0x7f75c019b8b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:18.325 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.730+0000 7f75c55d8640 1 -- 172.21.15.105:0/3716488986 >> 172.21.15.105:0/3716488986 conn(0x7f75c0102c20 msgr2=0x7f75c0103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:18.325 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.730+0000 7f75c55d8640 1 -- 172.21.15.105:0/3716488986 shutdown_connections 2024-09-16T01:53:18.325 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:17.730+0000 7f75c55d8640 1 -- 172.21.15.105:0/3716488986 wait complete. 2024-09-16T01:53:18.875 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:18 smithi105 ceph-mon[31197]: [16/Sep/2024:01:53:17] ENGINE Bus STARTING 2024-09-16T01:53:18.875 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:18 smithi105 ceph-mon[31197]: [16/Sep/2024:01:53:17] ENGINE Serving on http://172.21.15.105:8765 2024-09-16T01:53:18.876 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:18 smithi105 ceph-mon[31197]: [16/Sep/2024:01:53:17] ENGINE Serving on https://172.21.15.105:7150 2024-09-16T01:53:18.876 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:18 smithi105 ceph-mon[31197]: [16/Sep/2024:01:53:17] ENGINE Bus STARTED 2024-09-16T01:53:18.876 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:18 smithi105 ceph-mon[31197]: [16/Sep/2024:01:53:17] ENGINE Error in HTTPServer.serve 2024-09-16T01:53:18.876 INFO:journalctl@ceph.mon.a.smithi105.stdout: Traceback (most recent call last): 2024-09-16T01:53:18.876 INFO:journalctl@ceph.mon.a.smithi105.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T01:53:18.876 INFO:journalctl@ceph.mon.a.smithi105.stdout: self._connections.run(self.expiration_interval) 2024-09-16T01:53:18.876 INFO:journalctl@ceph.mon.a.smithi105.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T01:53:18.876 INFO:journalctl@ceph.mon.a.smithi105.stdout: self._run(expiration_interval) 2024-09-16T01:53:18.876 INFO:journalctl@ceph.mon.a.smithi105.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T01:53:18.876 INFO:journalctl@ceph.mon.a.smithi105.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T01:53:18.876 INFO:journalctl@ceph.mon.a.smithi105.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T01:53:18.876 INFO:journalctl@ceph.mon.a.smithi105.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T01:53:18.876 INFO:journalctl@ceph.mon.a.smithi105.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T01:53:18.877 INFO:journalctl@ceph.mon.a.smithi105.stdout: s = self.context.wrap_socket( 2024-09-16T01:53:18.877 INFO:journalctl@ceph.mon.a.smithi105.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T01:53:18.877 INFO:journalctl@ceph.mon.a.smithi105.stdout: return self.sslsocket_class._create( 2024-09-16T01:53:18.877 INFO:journalctl@ceph.mon.a.smithi105.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T01:53:18.877 INFO:journalctl@ceph.mon.a.smithi105.stdout: self.do_handshake() 2024-09-16T01:53:18.877 INFO:journalctl@ceph.mon.a.smithi105.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T01:53:18.877 INFO:journalctl@ceph.mon.a.smithi105.stdout: self._sslobj.do_handshake() 2024-09-16T01:53:18.877 INFO:journalctl@ceph.mon.a.smithi105.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T01:53:18.877 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:18 smithi105 ceph-mon[31197]: mgrmap e8: x(active, since 2s) 2024-09-16T01:53:18.877 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:18 smithi105 ceph-mon[31197]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:53:18.877 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:18 smithi105 ceph-mon[31197]: from='mgr.14118 172.21.15.105:0/2613168790' entity='mgr.x' 2024-09-16T01:53:18.877 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:18 smithi105 ceph-mon[31197]: from='mgr.14118 172.21.15.105:0/2613168790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:53:19.507 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-16T01:53:19.508 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.690+0000 7f134aada640 1 Processor -- start 2024-09-16T01:53:19.508 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.690+0000 7f134aada640 1 -- start start 2024-09-16T01:53:19.508 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.690+0000 7f134aada640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1344106380 0x7f1344106780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:19.508 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.690+0000 7f134aada640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1344106d50 con 0x7f1344106380 2024-09-16T01:53:19.508 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.691+0000 7f1348876640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1344106380 0x7f1344106780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:19.509 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.691+0000 7f1348876640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1344106380 0x7f1344106780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59540/0 (socket says 172.21.15.105:59540) 2024-09-16T01:53:19.509 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.691+0000 7f1348876640 1 -- 172.21.15.105:0/3687879604 learned_addr learned my addr 172.21.15.105:0/3687879604 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:19.510 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.691+0000 7f1348876640 1 -- 172.21.15.105:0/3687879604 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1344107580 con 0x7f1344106380 2024-09-16T01:53:19.510 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.692+0000 7f1348876640 1 --2- 172.21.15.105:0/3687879604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1344106380 0x7f1344106780 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f133400d180 tx=0x7f133402c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90fbe9135118ba66 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:19.516 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.692+0000 7f13437fe640 1 -- 172.21.15.105:0/3687879604 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f133400b410 con 0x7f1344106380 2024-09-16T01:53:19.517 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.692+0000 7f13437fe640 1 -- 172.21.15.105:0/3687879604 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f1334009a90 con 0x7f1344106380 2024-09-16T01:53:19.517 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.692+0000 7f13437fe640 1 -- 172.21.15.105:0/3687879604 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1334033b60 con 0x7f1344106380 2024-09-16T01:53:19.517 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.692+0000 7f134aada640 1 -- 172.21.15.105:0/3687879604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1344106380 msgr2=0x7f1344106780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:19.517 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.692+0000 7f134aada640 1 --2- 172.21.15.105:0/3687879604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1344106380 0x7f1344106780 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f133400d180 tx=0x7f133402c9a0 comp rx=0 tx=0).stop 2024-09-16T01:53:19.517 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.692+0000 7f134aada640 1 -- 172.21.15.105:0/3687879604 shutdown_connections 2024-09-16T01:53:19.517 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.692+0000 7f134aada640 1 --2- 172.21.15.105:0/3687879604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1344106380 0x7f1344106780 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:19.517 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.692+0000 7f134aada640 1 -- 172.21.15.105:0/3687879604 >> 172.21.15.105:0/3687879604 conn(0x7f1344101b10 msgr2=0x7f1344103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:19.517 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.693+0000 7f134aada640 1 -- 172.21.15.105:0/3687879604 shutdown_connections 2024-09-16T01:53:19.517 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.693+0000 7f134aada640 1 -- 172.21.15.105:0/3687879604 wait complete. 2024-09-16T01:53:19.517 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.693+0000 7f134aada640 1 Processor -- start 2024-09-16T01:53:19.518 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.693+0000 7f134aada640 1 -- start start 2024-09-16T01:53:19.518 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.693+0000 7f134aada640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1344106380 0x7f134419da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:19.518 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.693+0000 7f134aada640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f134407afb0 con 0x7f1344106380 2024-09-16T01:53:19.518 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.693+0000 7f1348876640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1344106380 0x7f134419da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:19.518 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.693+0000 7f1348876640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1344106380 0x7f134419da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59556/0 (socket says 172.21.15.105:59556) 2024-09-16T01:53:19.518 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.693+0000 7f1348876640 1 -- 172.21.15.105:0/183079447 learned_addr learned my addr 172.21.15.105:0/183079447 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:19.518 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.694+0000 7f1348876640 1 -- 172.21.15.105:0/183079447 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f134419df40 con 0x7f1344106380 2024-09-16T01:53:19.518 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.694+0000 7f1348876640 1 --2- 172.21.15.105:0/183079447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1344106380 0x7f134419da00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f133400d2a0 tx=0x7f1334033b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:19.518 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.694+0000 7f1341ffb640 1 -- 172.21.15.105:0/183079447 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1334043070 con 0x7f1344106380 2024-09-16T01:53:19.518 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.694+0000 7f1341ffb640 1 -- 172.21.15.105:0/183079447 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f133403c8c0 con 0x7f1344106380 2024-09-16T01:53:19.519 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.694+0000 7f1341ffb640 1 -- 172.21.15.105:0/183079447 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f133403b6b0 con 0x7f1344106380 2024-09-16T01:53:19.519 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.694+0000 7f134aada640 1 -- 172.21.15.105:0/183079447 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f134419e1d0 con 0x7f1344106380 2024-09-16T01:53:19.519 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.694+0000 7f134aada640 1 -- 172.21.15.105:0/183079447 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13441a0ec0 con 0x7f1344106380 2024-09-16T01:53:19.519 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.695+0000 7f1341ffb640 1 -- 172.21.15.105:0/183079447 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f1334046070 con 0x7f1344106380 2024-09-16T01:53:19.519 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.695+0000 7f134aada640 1 -- 172.21.15.105:0/183079447 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1344106800 con 0x7f1344106380 2024-09-16T01:53:19.519 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.695+0000 7f1341ffb640 1 --2- 172.21.15.105:0/183079447 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f13300391b0 0x7f133003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:19.519 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.695+0000 7f1341ffb640 1 -- 172.21.15.105:0/183079447 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1334030050 con 0x7f1344106380 2024-09-16T01:53:19.519 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.695+0000 7f1343fff640 1 --2- 172.21.15.105:0/183079447 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f13300391b0 0x7f133003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:19.519 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.696+0000 7f1343fff640 1 --2- 172.21.15.105:0/183079447 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f13300391b0 0x7f133003b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f132c0098c0 tx=0x7f132c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:19.519 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.700+0000 7f1341ffb640 1 -- 172.21.15.105:0/183079447 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f13340452c0 con 0x7f1344106380 2024-09-16T01:53:19.520 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.873+0000 7f134aada640 1 -- 172.21.15.105:0/183079447 --> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f1344062630 con 0x7f13300391b0 2024-09-16T01:53:19.520 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.874+0000 7f1341ffb640 1 -- 172.21.15.105:0/183079447 <== mgr.14118 v2:172.21.15.105:6800/1469189484 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f1344062630 con 0x7f13300391b0 2024-09-16T01:53:19.520 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.876+0000 7f134aada640 1 -- 172.21.15.105:0/183079447 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f13300391b0 msgr2=0x7f133003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:19.520 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.876+0000 7f134aada640 1 --2- 172.21.15.105:0/183079447 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f13300391b0 0x7f133003b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f132c0098c0 tx=0x7f132c006db0 comp rx=0 tx=0).stop 2024-09-16T01:53:19.520 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.876+0000 7f134aada640 1 -- 172.21.15.105:0/183079447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1344106380 msgr2=0x7f134419da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:19.520 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.876+0000 7f134aada640 1 --2- 172.21.15.105:0/183079447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1344106380 0x7f134419da00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f133400d2a0 tx=0x7f1334033b80 comp rx=0 tx=0).stop 2024-09-16T01:53:19.520 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.876+0000 7f134aada640 1 -- 172.21.15.105:0/183079447 shutdown_connections 2024-09-16T01:53:19.520 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.876+0000 7f134aada640 1 --2- 172.21.15.105:0/183079447 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f13300391b0 0x7f133003b670 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:19.520 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.876+0000 7f134aada640 1 --2- 172.21.15.105:0/183079447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1344106380 0x7f134419da00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:19.521 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.876+0000 7f134aada640 1 -- 172.21.15.105:0/183079447 >> 172.21.15.105:0/183079447 conn(0x7f1344101b10 msgr2=0x7f134418fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:19.521 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.876+0000 7f134aada640 1 -- 172.21.15.105:0/183079447 shutdown_connections 2024-09-16T01:53:19.521 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:18.876+0000 7f134aada640 1 -- 172.21.15.105:0/183079447 wait complete. 2024-09-16T01:53:19.521 INFO:teuthology.orchestra.run.smithi105.stdout:Generating ssh key... 2024-09-16T01:53:20.152 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:19 smithi105 ceph-mon[31197]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:53:20.545 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:20 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: Generating public/private rsa key pair. 2024-09-16T01:53:20.545 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:20 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: Your identification has been saved in /tmp/tmps35zog7x/key 2024-09-16T01:53:20.545 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:20 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: Your public key has been saved in /tmp/tmps35zog7x/key.pub 2024-09-16T01:53:20.545 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:20 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: The key fingerprint is: 2024-09-16T01:53:20.545 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:20 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: SHA256:7BsQQbPxq22d3tLM6dmBGBZp/Z6ufLJLYmqBEwm3kgQ ceph-423b7a2a-73ce-11ef-bceb-c7b262605968 2024-09-16T01:53:20.546 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:20 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: The key's randomart image is: 2024-09-16T01:53:20.546 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:20 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: +---[RSA 3072]----+ 2024-09-16T01:53:20.546 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:20 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: | E. .= | 2024-09-16T01:53:20.546 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:20 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: | o .* o | 2024-09-16T01:53:20.546 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:20 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: | . +oo. + . | 2024-09-16T01:53:20.546 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:20 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: | o +o o . . | 2024-09-16T01:53:20.546 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:20 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: | ..oS o . | 2024-09-16T01:53:20.546 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:20 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: | o=.o + o . | 2024-09-16T01:53:20.546 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:20 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: | ..=.*+o.+ | 2024-09-16T01:53:20.546 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:20 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: | ..*.=*+.. | 2024-09-16T01:53:20.547 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:20 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: | .o .oO*o | 2024-09-16T01:53:20.547 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:20 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: +----[SHA256]-----+ 2024-09-16T01:53:20.846 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.975+0000 7fa36abdc640 1 Processor -- start 2024-09-16T01:53:20.846 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.975+0000 7fa36abdc640 1 -- start start 2024-09-16T01:53:20.846 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.975+0000 7fa36abdc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa364107ef0 0x7fa36410a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:20.846 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.975+0000 7fa36abdc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa36407be90 con 0x7fa364107ef0 2024-09-16T01:53:20.846 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.975+0000 7fa368978640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa364107ef0 0x7fa36410a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:20.846 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.975+0000 7fa368978640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa364107ef0 0x7fa36410a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59570/0 (socket says 172.21.15.105:59570) 2024-09-16T01:53:20.846 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.975+0000 7fa368978640 1 -- 172.21.15.105:0/3499665072 learned_addr learned my addr 172.21.15.105:0/3499665072 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:20.846 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.975+0000 7fa368978640 1 -- 172.21.15.105:0/3499665072 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa36410a820 con 0x7fa364107ef0 2024-09-16T01:53:20.846 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.976+0000 7fa368978640 1 --2- 172.21.15.105:0/3499665072 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa364107ef0 0x7fa36410a2e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa358009820 tx=0x7fa35802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9907e6b5478fd3b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:20.846 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.976+0000 7fa3637fe640 1 -- 172.21.15.105:0/3499665072 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa358037070 con 0x7fa364107ef0 2024-09-16T01:53:20.846 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.976+0000 7fa3637fe640 1 -- 172.21.15.105:0/3499665072 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa3580328d0 con 0x7fa364107ef0 2024-09-16T01:53:20.846 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.976+0000 7fa3637fe640 1 -- 172.21.15.105:0/3499665072 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3580307d0 con 0x7fa364107ef0 2024-09-16T01:53:20.847 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.977+0000 7fa36abdc640 1 -- 172.21.15.105:0/3499665072 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa364107ef0 msgr2=0x7fa36410a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:20.847 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.977+0000 7fa36abdc640 1 --2- 172.21.15.105:0/3499665072 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa364107ef0 0x7fa36410a2e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa358009820 tx=0x7fa35802a270 comp rx=0 tx=0).stop 2024-09-16T01:53:20.847 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.977+0000 7fa36abdc640 1 -- 172.21.15.105:0/3499665072 shutdown_connections 2024-09-16T01:53:20.847 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.977+0000 7fa36abdc640 1 --2- 172.21.15.105:0/3499665072 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa364107ef0 0x7fa36410a2e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:20.847 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.977+0000 7fa36abdc640 1 -- 172.21.15.105:0/3499665072 >> 172.21.15.105:0/3499665072 conn(0x7fa364101b10 msgr2=0x7fa364103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:20.847 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.977+0000 7fa36abdc640 1 -- 172.21.15.105:0/3499665072 shutdown_connections 2024-09-16T01:53:20.847 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.977+0000 7fa36abdc640 1 -- 172.21.15.105:0/3499665072 wait complete. 2024-09-16T01:53:20.847 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.977+0000 7fa36abdc640 1 Processor -- start 2024-09-16T01:53:20.847 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.977+0000 7fa36abdc640 1 -- start start 2024-09-16T01:53:20.847 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.978+0000 7fa36abdc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa364107ef0 0x7fa36419b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:20.847 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.978+0000 7fa36abdc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa36410b860 con 0x7fa364107ef0 2024-09-16T01:53:20.847 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.978+0000 7fa368978640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa364107ef0 0x7fa36419b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:20.847 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.978+0000 7fa368978640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa364107ef0 0x7fa36419b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59580/0 (socket says 172.21.15.105:59580) 2024-09-16T01:53:20.847 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.978+0000 7fa368978640 1 -- 172.21.15.105:0/1700921977 learned_addr learned my addr 172.21.15.105:0/1700921977 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:20.848 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.978+0000 7fa368978640 1 -- 172.21.15.105:0/1700921977 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa36419bca0 con 0x7fa364107ef0 2024-09-16T01:53:20.848 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.978+0000 7fa368978640 1 --2- 172.21.15.105:0/1700921977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa364107ef0 0x7fa36419b760 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa358030520 tx=0x7fa358032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:20.848 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.978+0000 7fa361ffb640 1 -- 172.21.15.105:0/1700921977 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa358042070 con 0x7fa364107ef0 2024-09-16T01:53:20.848 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.978+0000 7fa361ffb640 1 -- 172.21.15.105:0/1700921977 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa358030cc0 con 0x7fa364107ef0 2024-09-16T01:53:20.848 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.978+0000 7fa36abdc640 1 -- 172.21.15.105:0/1700921977 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa36419bf30 con 0x7fa364107ef0 2024-09-16T01:53:20.848 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.978+0000 7fa361ffb640 1 -- 172.21.15.105:0/1700921977 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa358037040 con 0x7fa364107ef0 2024-09-16T01:53:20.848 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.979+0000 7fa36abdc640 1 -- 172.21.15.105:0/1700921977 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa36419ec20 con 0x7fa364107ef0 2024-09-16T01:53:20.848 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.979+0000 7fa361ffb640 1 -- 172.21.15.105:0/1700921977 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fa358045070 con 0x7fa364107ef0 2024-09-16T01:53:20.848 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.980+0000 7fa361ffb640 1 --2- 172.21.15.105:0/1700921977 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7fa3540391b0 0x7fa35403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:20.848 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.980+0000 7fa361ffb640 1 -- 172.21.15.105:0/1700921977 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa35806c2e0 con 0x7fa364107ef0 2024-09-16T01:53:20.849 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.980+0000 7fa36abdc640 1 -- 172.21.15.105:0/1700921977 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa364108350 con 0x7fa364107ef0 2024-09-16T01:53:20.849 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.980+0000 7fa363fff640 1 --2- 172.21.15.105:0/1700921977 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7fa3540391b0 0x7fa35403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:20.849 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.980+0000 7fa363fff640 1 --2- 172.21.15.105:0/1700921977 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7fa3540391b0 0x7fa35403b670 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa3500098c0 tx=0x7fa350006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:20.849 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:19.985+0000 7fa361ffb640 1 -- 172.21.15.105:0/1700921977 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa358045360 con 0x7fa364107ef0 2024-09-16T01:53:20.849 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:20.150+0000 7fa36abdc640 1 -- 172.21.15.105:0/1700921977 --> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fa364062630 con 0x7fa3540391b0 2024-09-16T01:53:20.849 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:20.238+0000 7fa361ffb640 1 -- 172.21.15.105:0/1700921977 <== mgr.14118 v2:172.21.15.105:6800/1469189484 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa364062630 con 0x7fa3540391b0 2024-09-16T01:53:20.849 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:20.240+0000 7fa36abdc640 1 -- 172.21.15.105:0/1700921977 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7fa3540391b0 msgr2=0x7fa35403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:20.849 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:20.240+0000 7fa36abdc640 1 --2- 172.21.15.105:0/1700921977 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7fa3540391b0 0x7fa35403b670 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa3500098c0 tx=0x7fa350006db0 comp rx=0 tx=0).stop 2024-09-16T01:53:20.849 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:20.240+0000 7fa36abdc640 1 -- 172.21.15.105:0/1700921977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa364107ef0 msgr2=0x7fa36419b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:20.849 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:20.240+0000 7fa36abdc640 1 --2- 172.21.15.105:0/1700921977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa364107ef0 0x7fa36419b760 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa358030520 tx=0x7fa358032dd0 comp rx=0 tx=0).stop 2024-09-16T01:53:20.849 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:20.240+0000 7fa36abdc640 1 -- 172.21.15.105:0/1700921977 shutdown_connections 2024-09-16T01:53:20.849 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:20.240+0000 7fa36abdc640 1 --2- 172.21.15.105:0/1700921977 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7fa3540391b0 0x7fa35403b670 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:20.849 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:20.240+0000 7fa36abdc640 1 --2- 172.21.15.105:0/1700921977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa364107ef0 0x7fa36419b760 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:20.850 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:20.240+0000 7fa36abdc640 1 -- 172.21.15.105:0/1700921977 >> 172.21.15.105:0/1700921977 conn(0x7fa364101b10 msgr2=0x7fa364102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:20.850 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:20.240+0000 7fa36abdc640 1 -- 172.21.15.105:0/1700921977 shutdown_connections 2024-09-16T01:53:20.850 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:20.240+0000 7fa36abdc640 1 -- 172.21.15.105:0/1700921977 wait complete. 2024-09-16T01:53:21.430 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:21 smithi105 ceph-mon[31197]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:53:21.430 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:21 smithi105 ceph-mon[31197]: Generating ssh key... 2024-09-16T01:53:21.430 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:21 smithi105 ceph-mon[31197]: from='mgr.14118 172.21.15.105:0/2613168790' entity='mgr.x' 2024-09-16T01:53:21.430 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:21 smithi105 ceph-mon[31197]: from='mgr.14118 172.21.15.105:0/2613168790' entity='mgr.x' 2024-09-16T01:53:22.083 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCgouhisPdPmtGs8Do6xwgs6E7OzQkXaE4kHAxuvLRN10DtEn+mMQn1/fus//iGgiac1gacxVpPsBxCPK/94KW2JHHx/8WC/h62K7pjitpjF54qe02HK2SJ4wsAv+ZUQU29ivyWvJdDM/Q+H5KNIkA7fyUJYQ1FsT8pgICFDqaWGlRTccveiM4/tccCjfSOeNKoPd7Ble3EBpnER3AjWWn4Uawy4ovzVYZKOmMRh2kB2pMf805zZm155VGOJQVSus1Xt4Sow1bd0Tc9IsZc9FOxt++Log43YtGB6Y4hYKC4YZn/3MdffJcRmSdQ+7rHcI0Ft3YnJl4wbk36QSKSnPpJRKadK5pgF8FKkm5ULEeYpMlN9jULquYWqHUZdGb8CvfQtv7TmiYAvnoazGyohpDMLIQp3lZdyfEXMazHvy7iegbxhWp0ccaEdGxvjaq1gI8VVPxRPEMA+huO84d5G2eO8+9OpuJDT3fnRRZDz4kQpgFyOoz9g2qoCygPbi8tICE= ceph-423b7a2a-73ce-11ef-bceb-c7b262605968 2024-09-16T01:53:22.084 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.248+0000 7f301ef76640 1 Processor -- start 2024-09-16T01:53:22.084 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.248+0000 7f301ef76640 1 -- start start 2024-09-16T01:53:22.084 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.248+0000 7f301ef76640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f301807c2e0 0x7f301807c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:22.084 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.248+0000 7f301ef76640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f301807e010 con 0x7f301807c2e0 2024-09-16T01:53:22.084 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.248+0000 7f301cd12640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f301807c2e0 0x7f301807c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:22.084 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.248+0000 7f301cd12640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f301807c2e0 0x7f301807c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59582/0 (socket says 172.21.15.105:59582) 2024-09-16T01:53:22.084 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.248+0000 7f301cd12640 1 -- 172.21.15.105:0/955132470 learned_addr learned my addr 172.21.15.105:0/955132470 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:22.084 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.248+0000 7f301cd12640 1 -- 172.21.15.105:0/955132470 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f301807cc20 con 0x7f301807c2e0 2024-09-16T01:53:22.084 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.249+0000 7f301cd12640 1 --2- 172.21.15.105:0/955132470 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f301807c2e0 0x7f301807c6e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f300c009820 tx=0x7f300c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a84d62a63fdc2fcf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:22.084 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.249+0000 7f30177fe640 1 -- 172.21.15.105:0/955132470 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f300c037070 con 0x7f301807c2e0 2024-09-16T01:53:22.084 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.249+0000 7f30177fe640 1 -- 172.21.15.105:0/955132470 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f300c0328d0 con 0x7f301807c2e0 2024-09-16T01:53:22.085 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.249+0000 7f30177fe640 1 -- 172.21.15.105:0/955132470 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f300c0307d0 con 0x7f301807c2e0 2024-09-16T01:53:22.085 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.250+0000 7f301ef76640 1 -- 172.21.15.105:0/955132470 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f301807c2e0 msgr2=0x7f301807c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:22.085 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.250+0000 7f301ef76640 1 --2- 172.21.15.105:0/955132470 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f301807c2e0 0x7f301807c6e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f300c009820 tx=0x7f300c02a270 comp rx=0 tx=0).stop 2024-09-16T01:53:22.085 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.250+0000 7f301ef76640 1 -- 172.21.15.105:0/955132470 shutdown_connections 2024-09-16T01:53:22.085 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.250+0000 7f301ef76640 1 --2- 172.21.15.105:0/955132470 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f301807c2e0 0x7f301807c6e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:22.085 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.250+0000 7f301ef76640 1 -- 172.21.15.105:0/955132470 >> 172.21.15.105:0/955132470 conn(0x7f3018102c20 msgr2=0x7f3018105080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:22.085 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.250+0000 7f301ef76640 1 -- 172.21.15.105:0/955132470 shutdown_connections 2024-09-16T01:53:22.085 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.250+0000 7f301ef76640 1 -- 172.21.15.105:0/955132470 wait complete. 2024-09-16T01:53:22.085 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.251+0000 7f301ef76640 1 Processor -- start 2024-09-16T01:53:22.085 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.251+0000 7f301ef76640 1 -- start start 2024-09-16T01:53:22.085 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.251+0000 7f301ef76640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f301807c2e0 0x7f301819d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:22.085 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.251+0000 7f301ef76640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f301810db80 con 0x7f301807c2e0 2024-09-16T01:53:22.085 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.251+0000 7f301cd12640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f301807c2e0 0x7f301819d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:22.085 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.251+0000 7f301cd12640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f301807c2e0 0x7f301819d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59594/0 (socket says 172.21.15.105:59594) 2024-09-16T01:53:22.086 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.251+0000 7f301cd12640 1 -- 172.21.15.105:0/2624390954 learned_addr learned my addr 172.21.15.105:0/2624390954 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:22.086 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.251+0000 7f301cd12640 1 -- 172.21.15.105:0/2624390954 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f301819df30 con 0x7f301807c2e0 2024-09-16T01:53:22.086 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.251+0000 7f301cd12640 1 --2- 172.21.15.105:0/2624390954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f301807c2e0 0x7f301819d9f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f300c032e10 tx=0x7f300c032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:22.086 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.251+0000 7f3015ffb640 1 -- 172.21.15.105:0/2624390954 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f300c042070 con 0x7f301807c2e0 2024-09-16T01:53:22.086 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.252+0000 7f3015ffb640 1 -- 172.21.15.105:0/2624390954 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f300c030c80 con 0x7f301807c2e0 2024-09-16T01:53:22.086 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.252+0000 7f301ef76640 1 -- 172.21.15.105:0/2624390954 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f301819e1c0 con 0x7f301807c2e0 2024-09-16T01:53:22.086 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.252+0000 7f3015ffb640 1 -- 172.21.15.105:0/2624390954 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f300c037040 con 0x7f301807c2e0 2024-09-16T01:53:22.086 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.252+0000 7f301ef76640 1 -- 172.21.15.105:0/2624390954 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30181a0eb0 con 0x7f301807c2e0 2024-09-16T01:53:22.086 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.253+0000 7f3015ffb640 1 -- 172.21.15.105:0/2624390954 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f300c045070 con 0x7f301807c2e0 2024-09-16T01:53:22.086 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.253+0000 7f3015ffb640 1 --2- 172.21.15.105:0/2624390954 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f30080391b0 0x7f300803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:22.086 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.253+0000 7f301ef76640 1 -- 172.21.15.105:0/2624390954 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f301807c760 con 0x7f301807c2e0 2024-09-16T01:53:22.086 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.253+0000 7f3015ffb640 1 -- 172.21.15.105:0/2624390954 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f300c06c360 con 0x7f301807c2e0 2024-09-16T01:53:22.086 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.253+0000 7f3017fff640 1 --2- 172.21.15.105:0/2624390954 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f30080391b0 0x7f300803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:22.087 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.253+0000 7f3017fff640 1 --2- 172.21.15.105:0/2624390954 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f30080391b0 0x7f300803b670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f30040098c0 tx=0x7f3004006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:22.087 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.258+0000 7f3015ffb640 1 -- 172.21.15.105:0/2624390954 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f300c045360 con 0x7f301807c2e0 2024-09-16T01:53:22.087 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.428+0000 7f301ef76640 1 -- 172.21.15.105:0/2624390954 --> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f3018062630 con 0x7f30080391b0 2024-09-16T01:53:22.087 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.429+0000 7f3015ffb640 1 -- 172.21.15.105:0/2624390954 <== mgr.14118 v2:172.21.15.105:6800/1469189484 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f3018062630 con 0x7f30080391b0 2024-09-16T01:53:22.087 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.431+0000 7f301ef76640 1 -- 172.21.15.105:0/2624390954 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f30080391b0 msgr2=0x7f300803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:22.087 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.431+0000 7f301ef76640 1 --2- 172.21.15.105:0/2624390954 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f30080391b0 0x7f300803b670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f30040098c0 tx=0x7f3004006db0 comp rx=0 tx=0).stop 2024-09-16T01:53:22.087 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.431+0000 7f301ef76640 1 -- 172.21.15.105:0/2624390954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f301807c2e0 msgr2=0x7f301819d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:22.087 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.431+0000 7f301ef76640 1 --2- 172.21.15.105:0/2624390954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f301807c2e0 0x7f301819d9f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f300c032e10 tx=0x7f300c032e40 comp rx=0 tx=0).stop 2024-09-16T01:53:22.087 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.431+0000 7f301ef76640 1 -- 172.21.15.105:0/2624390954 shutdown_connections 2024-09-16T01:53:22.087 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.431+0000 7f301ef76640 1 --2- 172.21.15.105:0/2624390954 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f30080391b0 0x7f300803b670 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:22.087 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.431+0000 7f301ef76640 1 --2- 172.21.15.105:0/2624390954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f301807c2e0 0x7f301819d9f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:22.087 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.431+0000 7f301ef76640 1 -- 172.21.15.105:0/2624390954 >> 172.21.15.105:0/2624390954 conn(0x7f3018102c20 msgr2=0x7f3018103930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:22.087 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.431+0000 7f301ef76640 1 -- 172.21.15.105:0/2624390954 shutdown_connections 2024-09-16T01:53:22.088 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:21.431+0000 7f301ef76640 1 -- 172.21.15.105:0/2624390954 wait complete. 2024-09-16T01:53:22.088 INFO:teuthology.orchestra.run.smithi105.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-16T01:53:22.088 INFO:teuthology.orchestra.run.smithi105.stdout:Adding key to root@localhost authorized_keys... 2024-09-16T01:53:22.091 INFO:teuthology.orchestra.run.smithi105.stdout:Adding host smithi105... 2024-09-16T01:53:22.534 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:22 smithi105 ceph-mon[31197]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:53:23.484 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:23 smithi105 ceph-mon[31197]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi105", "addr": "172.21.15.105", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:53:24.444 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:24 smithi105 ceph-mon[31197]: Deploying cephadm binary to smithi105 2024-09-16T01:53:26.357 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout Added host 'smithi105' with addr '172.21.15.105' 2024-09-16T01:53:26.357 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.533+0000 7f0163a1e640 1 Processor -- start 2024-09-16T01:53:26.357 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.533+0000 7f0163a1e640 1 -- start start 2024-09-16T01:53:26.358 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.533+0000 7f0163a1e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f015c108b90 0x7f015c108f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:26.358 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.533+0000 7f0163a1e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f015c1070d0 con 0x7f015c108b90 2024-09-16T01:53:26.358 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.533+0000 7f01617ba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f015c108b90 0x7f015c108f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:26.358 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.533+0000 7f01617ba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f015c108b90 0x7f015c108f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59604/0 (socket says 172.21.15.105:59604) 2024-09-16T01:53:26.358 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.533+0000 7f01617ba640 1 -- 172.21.15.105:0/2005193571 learned_addr learned my addr 172.21.15.105:0/2005193571 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:26.358 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.534+0000 7f01617ba640 1 -- 172.21.15.105:0/2005193571 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f015c107250 con 0x7f015c108b90 2024-09-16T01:53:26.358 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.534+0000 7f01617ba640 1 --2- 172.21.15.105:0/2005193571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f015c108b90 0x7f015c108f90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0158009820 tx=0x7f015802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad487fae307de8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:26.358 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.534+0000 7f0153fff640 1 -- 172.21.15.105:0/2005193571 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0158037070 con 0x7f015c108b90 2024-09-16T01:53:26.358 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.534+0000 7f0153fff640 1 -- 172.21.15.105:0/2005193571 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f01580328d0 con 0x7f015c108b90 2024-09-16T01:53:26.358 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.534+0000 7f0153fff640 1 -- 172.21.15.105:0/2005193571 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01580307d0 con 0x7f015c108b90 2024-09-16T01:53:26.358 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.535+0000 7f0163a1e640 1 -- 172.21.15.105:0/2005193571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f015c108b90 msgr2=0x7f015c108f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:26.358 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.535+0000 7f0163a1e640 1 --2- 172.21.15.105:0/2005193571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f015c108b90 0x7f015c108f90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0158009820 tx=0x7f015802a270 comp rx=0 tx=0).stop 2024-09-16T01:53:26.358 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.535+0000 7f0163a1e640 1 -- 172.21.15.105:0/2005193571 shutdown_connections 2024-09-16T01:53:26.358 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.535+0000 7f0163a1e640 1 --2- 172.21.15.105:0/2005193571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f015c108b90 0x7f015c108f90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:26.359 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.535+0000 7f0163a1e640 1 -- 172.21.15.105:0/2005193571 >> 172.21.15.105:0/2005193571 conn(0x7f015c102c20 msgr2=0x7f015c105080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:26.359 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.535+0000 7f0163a1e640 1 -- 172.21.15.105:0/2005193571 shutdown_connections 2024-09-16T01:53:26.359 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.535+0000 7f0163a1e640 1 -- 172.21.15.105:0/2005193571 wait complete. 2024-09-16T01:53:26.359 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.536+0000 7f0163a1e640 1 Processor -- start 2024-09-16T01:53:26.359 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.536+0000 7f0163a1e640 1 -- start start 2024-09-16T01:53:26.359 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.536+0000 7f0163a1e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f015c108b90 0x7f015c19b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:26.359 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.536+0000 7f0163a1e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f015c10ba60 con 0x7f015c108b90 2024-09-16T01:53:26.359 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.536+0000 7f01617ba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f015c108b90 0x7f015c19b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:26.359 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.536+0000 7f01617ba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f015c108b90 0x7f015c19b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59620/0 (socket says 172.21.15.105:59620) 2024-09-16T01:53:26.359 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.536+0000 7f01617ba640 1 -- 172.21.15.105:0/2028909064 learned_addr learned my addr 172.21.15.105:0/2028909064 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:26.359 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.536+0000 7f01617ba640 1 -- 172.21.15.105:0/2028909064 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f015c19bdf0 con 0x7f015c108b90 2024-09-16T01:53:26.359 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.536+0000 7f01617ba640 1 --2- 172.21.15.105:0/2028909064 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f015c108b90 0x7f015c19b8b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f01580097f0 tx=0x7f0158030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:26.359 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.536+0000 7f01527fc640 1 -- 172.21.15.105:0/2028909064 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0158037040 con 0x7f015c108b90 2024-09-16T01:53:26.360 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.537+0000 7f01527fc640 1 -- 172.21.15.105:0/2028909064 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0158032bb0 con 0x7f015c108b90 2024-09-16T01:53:26.360 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.537+0000 7f01527fc640 1 -- 172.21.15.105:0/2028909064 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f015803a390 con 0x7f015c108b90 2024-09-16T01:53:26.360 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.537+0000 7f0163a1e640 1 -- 172.21.15.105:0/2028909064 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f015c19c080 con 0x7f015c108b90 2024-09-16T01:53:26.360 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.537+0000 7f0163a1e640 1 -- 172.21.15.105:0/2028909064 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f015c19ed70 con 0x7f015c108b90 2024-09-16T01:53:26.360 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.537+0000 7f01527fc640 1 -- 172.21.15.105:0/2028909064 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f015803b3f0 con 0x7f015c108b90 2024-09-16T01:53:26.360 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.538+0000 7f0163a1e640 1 -- 172.21.15.105:0/2028909064 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f015c109010 con 0x7f015c108b90 2024-09-16T01:53:26.360 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.538+0000 7f01527fc640 1 --2- 172.21.15.105:0/2028909064 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f01480391b0 0x7f014803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:26.360 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.538+0000 7f01527fc640 1 -- 172.21.15.105:0/2028909064 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f015806c480 con 0x7f015c108b90 2024-09-16T01:53:26.361 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.538+0000 7f0160fb9640 1 --2- 172.21.15.105:0/2028909064 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f01480391b0 0x7f014803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:26.361 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.538+0000 7f0160fb9640 1 --2- 172.21.15.105:0/2028909064 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f01480391b0 0x7f014803b670 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f014c0098c0 tx=0x7f014c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:26.361 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.542+0000 7f01527fc640 1 -- 172.21.15.105:0/2028909064 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f015803a5d0 con 0x7f015c108b90 2024-09-16T01:53:26.361 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:22.714+0000 7f0163a1e640 1 -- 172.21.15.105:0/2028909064 --> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi105", "addr": "172.21.15.105", "target": ["mon-mgr", ""]}) -- 0x7f015c062630 con 0x7f01480391b0 2024-09-16T01:53:26.361 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:25.867+0000 7f01527fc640 1 -- 172.21.15.105:0/2028909064 <== mgr.14118 v2:172.21.15.105:6800/1469189484 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f015c062630 con 0x7f01480391b0 2024-09-16T01:53:26.361 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:25.870+0000 7f0163a1e640 1 -- 172.21.15.105:0/2028909064 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f01480391b0 msgr2=0x7f014803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:26.361 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:25.870+0000 7f0163a1e640 1 --2- 172.21.15.105:0/2028909064 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f01480391b0 0x7f014803b670 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f014c0098c0 tx=0x7f014c006db0 comp rx=0 tx=0).stop 2024-09-16T01:53:26.361 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:25.870+0000 7f0163a1e640 1 -- 172.21.15.105:0/2028909064 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f015c108b90 msgr2=0x7f015c19b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:26.361 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:25.870+0000 7f0163a1e640 1 --2- 172.21.15.105:0/2028909064 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f015c108b90 0x7f015c19b8b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f01580097f0 tx=0x7f0158030d00 comp rx=0 tx=0).stop 2024-09-16T01:53:26.361 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:25.870+0000 7f0163a1e640 1 -- 172.21.15.105:0/2028909064 shutdown_connections 2024-09-16T01:53:26.361 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:25.870+0000 7f0163a1e640 1 --2- 172.21.15.105:0/2028909064 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f01480391b0 0x7f014803b670 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:26.361 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:25.870+0000 7f0163a1e640 1 --2- 172.21.15.105:0/2028909064 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f015c108b90 0x7f015c19b8b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:26.362 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:25.870+0000 7f0163a1e640 1 -- 172.21.15.105:0/2028909064 >> 172.21.15.105:0/2028909064 conn(0x7f015c102c20 msgr2=0x7f015c103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:26.362 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:25.870+0000 7f0163a1e640 1 -- 172.21.15.105:0/2028909064 shutdown_connections 2024-09-16T01:53:26.362 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:25.870+0000 7f0163a1e640 1 -- 172.21.15.105:0/2028909064 wait complete. 2024-09-16T01:53:26.362 INFO:teuthology.orchestra.run.smithi105.stdout:Deploying unmanaged mon service... 2024-09-16T01:53:27.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:26 smithi105 ceph-mon[31197]: from='mgr.14118 172.21.15.105:0/2613168790' entity='mgr.x' 2024-09-16T01:53:27.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:26 smithi105 ceph-mon[31197]: Added host smithi105 2024-09-16T01:53:27.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:26 smithi105 ceph-mon[31197]: from='mgr.14118 172.21.15.105:0/2613168790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:53:27.725 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-16T01:53:27.725 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.747+0000 7f73f78d5640 1 Processor -- start 2024-09-16T01:53:27.725 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.748+0000 7f73f78d5640 1 -- start start 2024-09-16T01:53:27.725 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.748+0000 7f73f78d5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73f0108b90 0x7f73f0108f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:27.725 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.748+0000 7f73f78d5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f73f01070d0 con 0x7f73f0108b90 2024-09-16T01:53:27.725 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.748+0000 7f73f5671640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73f0108b90 0x7f73f0108f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:27.725 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.748+0000 7f73f5671640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73f0108b90 0x7f73f0108f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59632/0 (socket says 172.21.15.105:59632) 2024-09-16T01:53:27.726 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.748+0000 7f73f5671640 1 -- 172.21.15.105:0/3895914334 learned_addr learned my addr 172.21.15.105:0/3895914334 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:27.726 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.748+0000 7f73f5671640 1 -- 172.21.15.105:0/3895914334 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f0107250 con 0x7f73f0108b90 2024-09-16T01:53:27.726 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.749+0000 7f73f5671640 1 --2- 172.21.15.105:0/3895914334 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73f0108b90 0x7f73f0108f90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f73ec009820 tx=0x7f73ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba8e7afd56221254 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:27.726 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.749+0000 7f73e7fff640 1 -- 172.21.15.105:0/3895914334 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73ec037070 con 0x7f73f0108b90 2024-09-16T01:53:27.726 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.749+0000 7f73e7fff640 1 -- 172.21.15.105:0/3895914334 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f73ec0328d0 con 0x7f73f0108b90 2024-09-16T01:53:27.726 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.749+0000 7f73e7fff640 1 -- 172.21.15.105:0/3895914334 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73ec0307d0 con 0x7f73f0108b90 2024-09-16T01:53:27.726 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.750+0000 7f73f78d5640 1 -- 172.21.15.105:0/3895914334 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73f0108b90 msgr2=0x7f73f0108f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:27.726 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.750+0000 7f73f78d5640 1 --2- 172.21.15.105:0/3895914334 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73f0108b90 0x7f73f0108f90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f73ec009820 tx=0x7f73ec02a270 comp rx=0 tx=0).stop 2024-09-16T01:53:27.726 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.750+0000 7f73f78d5640 1 -- 172.21.15.105:0/3895914334 shutdown_connections 2024-09-16T01:53:27.726 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.750+0000 7f73f78d5640 1 --2- 172.21.15.105:0/3895914334 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73f0108b90 0x7f73f0108f90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:27.726 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.750+0000 7f73f78d5640 1 -- 172.21.15.105:0/3895914334 >> 172.21.15.105:0/3895914334 conn(0x7f73f0102c20 msgr2=0x7f73f0105080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:27.726 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.750+0000 7f73f78d5640 1 -- 172.21.15.105:0/3895914334 shutdown_connections 2024-09-16T01:53:27.726 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.750+0000 7f73f78d5640 1 -- 172.21.15.105:0/3895914334 wait complete. 2024-09-16T01:53:27.727 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.750+0000 7f73f78d5640 1 Processor -- start 2024-09-16T01:53:27.727 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.751+0000 7f73f78d5640 1 -- start start 2024-09-16T01:53:27.727 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.751+0000 7f73f78d5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73f0108b90 0x7f73f019b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:27.727 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.751+0000 7f73f78d5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f73f010ba60 con 0x7f73f0108b90 2024-09-16T01:53:27.727 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.751+0000 7f73f5671640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73f0108b90 0x7f73f019b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:27.727 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.751+0000 7f73f5671640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73f0108b90 0x7f73f019b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59640/0 (socket says 172.21.15.105:59640) 2024-09-16T01:53:27.727 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.751+0000 7f73f5671640 1 -- 172.21.15.105:0/2689161619 learned_addr learned my addr 172.21.15.105:0/2689161619 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:27.727 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.751+0000 7f73f5671640 1 -- 172.21.15.105:0/2689161619 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f019bdf0 con 0x7f73f0108b90 2024-09-16T01:53:27.727 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.751+0000 7f73f5671640 1 --2- 172.21.15.105:0/2689161619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73f0108b90 0x7f73f019b8b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f73ec009940 tx=0x7f73ec032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:27.727 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.751+0000 7f73e67fc640 1 -- 172.21.15.105:0/2689161619 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73ec042070 con 0x7f73f0108b90 2024-09-16T01:53:27.727 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.751+0000 7f73e67fc640 1 -- 172.21.15.105:0/2689161619 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f73ec030c80 con 0x7f73f0108b90 2024-09-16T01:53:27.727 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.751+0000 7f73f78d5640 1 -- 172.21.15.105:0/2689161619 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73f019c080 con 0x7f73f0108b90 2024-09-16T01:53:27.727 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.751+0000 7f73e67fc640 1 -- 172.21.15.105:0/2689161619 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73ec037040 con 0x7f73f0108b90 2024-09-16T01:53:27.728 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.751+0000 7f73f78d5640 1 -- 172.21.15.105:0/2689161619 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73f019ed70 con 0x7f73f0108b90 2024-09-16T01:53:27.728 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.752+0000 7f73e67fc640 1 -- 172.21.15.105:0/2689161619 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f73ec045070 con 0x7f73f0108b90 2024-09-16T01:53:27.728 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.752+0000 7f73f78d5640 1 -- 172.21.15.105:0/2689161619 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73f0109010 con 0x7f73f0108b90 2024-09-16T01:53:27.728 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.753+0000 7f73e67fc640 1 --2- 172.21.15.105:0/2689161619 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f73dc0391b0 0x7f73dc03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:27.728 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.753+0000 7f73e67fc640 1 -- 172.21.15.105:0/2689161619 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f73ec06c180 con 0x7f73f0108b90 2024-09-16T01:53:27.728 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.753+0000 7f73f4e70640 1 --2- 172.21.15.105:0/2689161619 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f73dc0391b0 0x7f73dc03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:27.728 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.753+0000 7f73f4e70640 1 --2- 172.21.15.105:0/2689161619 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f73dc0391b0 0x7f73dc03b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f73e00098c0 tx=0x7f73e0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:27.728 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.757+0000 7f73e67fc640 1 -- 172.21.15.105:0/2689161619 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f73ec045360 con 0x7f73f0108b90 2024-09-16T01:53:27.728 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.976+0000 7f73f78d5640 1 -- 172.21.15.105:0/2689161619 --> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f73f0062630 con 0x7f73dc0391b0 2024-09-16T01:53:27.728 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.981+0000 7f73e67fc640 1 -- 172.21.15.105:0/2689161619 <== mgr.14118 v2:172.21.15.105:6800/1469189484 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f73f0062630 con 0x7f73dc0391b0 2024-09-16T01:53:27.728 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.983+0000 7f73f78d5640 1 -- 172.21.15.105:0/2689161619 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f73dc0391b0 msgr2=0x7f73dc03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:27.728 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.983+0000 7f73f78d5640 1 --2- 172.21.15.105:0/2689161619 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f73dc0391b0 0x7f73dc03b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f73e00098c0 tx=0x7f73e0006db0 comp rx=0 tx=0).stop 2024-09-16T01:53:27.729 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.983+0000 7f73f78d5640 1 -- 172.21.15.105:0/2689161619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73f0108b90 msgr2=0x7f73f019b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:27.729 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.983+0000 7f73f78d5640 1 --2- 172.21.15.105:0/2689161619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73f0108b90 0x7f73f019b8b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f73ec009940 tx=0x7f73ec032e10 comp rx=0 tx=0).stop 2024-09-16T01:53:27.729 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.983+0000 7f73f78d5640 1 -- 172.21.15.105:0/2689161619 shutdown_connections 2024-09-16T01:53:27.729 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.983+0000 7f73f78d5640 1 --2- 172.21.15.105:0/2689161619 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f73dc0391b0 0x7f73dc03b670 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:27.729 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.983+0000 7f73f78d5640 1 --2- 172.21.15.105:0/2689161619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73f0108b90 0x7f73f019b8b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:27.729 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.983+0000 7f73f78d5640 1 -- 172.21.15.105:0/2689161619 >> 172.21.15.105:0/2689161619 conn(0x7f73f0102c20 msgr2=0x7f73f0103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:27.729 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.983+0000 7f73f78d5640 1 -- 172.21.15.105:0/2689161619 shutdown_connections 2024-09-16T01:53:27.729 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:26.983+0000 7f73f78d5640 1 -- 172.21.15.105:0/2689161619 wait complete. 2024-09-16T01:53:27.729 INFO:teuthology.orchestra.run.smithi105.stdout:Deploying unmanaged mgr service... 2024-09-16T01:53:28.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:27 smithi105 ceph-mon[31197]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:53:28.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:27 smithi105 ceph-mon[31197]: Saving service mon spec with placement count:5 2024-09-16T01:53:28.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:27 smithi105 ceph-mon[31197]: from='mgr.14118 172.21.15.105:0/2613168790' entity='mgr.x' 2024-09-16T01:53:29.955 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-16T01:53:29.955 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.114+0000 7f21cffb1640 1 Processor -- start 2024-09-16T01:53:29.955 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.114+0000 7f21cffb1640 1 -- start start 2024-09-16T01:53:29.955 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.114+0000 7f21cffb1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21c81074f0 0x7f21c81078f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:29.955 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.114+0000 7f21cffb1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f21c8107ec0 con 0x7f21c81074f0 2024-09-16T01:53:29.955 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.115+0000 7f21cdd4d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21c81074f0 0x7f21c81078f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:29.956 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.115+0000 7f21cdd4d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21c81074f0 0x7f21c81078f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36748/0 (socket says 172.21.15.105:36748) 2024-09-16T01:53:29.956 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.115+0000 7f21cdd4d640 1 -- 172.21.15.105:0/966457999 learned_addr learned my addr 172.21.15.105:0/966457999 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:29.956 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.115+0000 7f21cdd4d640 1 -- 172.21.15.105:0/966457999 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c81086a0 con 0x7f21c81074f0 2024-09-16T01:53:29.956 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.116+0000 7f21cdd4d640 1 --2- 172.21.15.105:0/966457999 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21c81074f0 0x7f21c81078f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f21c4009820 tx=0x7f21c402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4488ace63ffcf961 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:29.956 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.116+0000 7f21ccd4b640 1 -- 172.21.15.105:0/966457999 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21c4037070 con 0x7f21c81074f0 2024-09-16T01:53:29.956 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.116+0000 7f21ccd4b640 1 -- 172.21.15.105:0/966457999 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21c40328f0 con 0x7f21c81074f0 2024-09-16T01:53:29.956 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.116+0000 7f21ccd4b640 1 -- 172.21.15.105:0/966457999 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21c4030740 con 0x7f21c81074f0 2024-09-16T01:53:29.956 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.116+0000 7f21cffb1640 1 -- 172.21.15.105:0/966457999 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21c81074f0 msgr2=0x7f21c81078f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:29.957 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.116+0000 7f21cffb1640 1 --2- 172.21.15.105:0/966457999 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21c81074f0 0x7f21c81078f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f21c4009820 tx=0x7f21c402a270 comp rx=0 tx=0).stop 2024-09-16T01:53:29.957 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.116+0000 7f21cffb1640 1 -- 172.21.15.105:0/966457999 shutdown_connections 2024-09-16T01:53:29.957 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.116+0000 7f21cffb1640 1 --2- 172.21.15.105:0/966457999 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21c81074f0 0x7f21c81078f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:29.957 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.117+0000 7f21cffb1640 1 -- 172.21.15.105:0/966457999 >> 172.21.15.105:0/966457999 conn(0x7f21c8102c40 msgr2=0x7f21c81050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:29.957 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.117+0000 7f21cffb1640 1 -- 172.21.15.105:0/966457999 shutdown_connections 2024-09-16T01:53:29.957 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.117+0000 7f21cffb1640 1 -- 172.21.15.105:0/966457999 wait complete. 2024-09-16T01:53:29.957 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.117+0000 7f21cffb1640 1 Processor -- start 2024-09-16T01:53:29.957 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.117+0000 7f21cffb1640 1 -- start start 2024-09-16T01:53:29.957 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.117+0000 7f21cffb1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21c81074f0 0x7f21c807db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:29.957 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.117+0000 7f21cffb1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f21c810ba50 con 0x7f21c81074f0 2024-09-16T01:53:29.957 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.118+0000 7f21cdd4d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21c81074f0 0x7f21c807db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:29.957 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.118+0000 7f21cdd4d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21c81074f0 0x7f21c807db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36760/0 (socket says 172.21.15.105:36760) 2024-09-16T01:53:29.957 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.118+0000 7f21cdd4d640 1 -- 172.21.15.105:0/4223578035 learned_addr learned my addr 172.21.15.105:0/4223578035 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:29.957 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.118+0000 7f21cdd4d640 1 -- 172.21.15.105:0/4223578035 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c807e090 con 0x7f21c81074f0 2024-09-16T01:53:29.958 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.118+0000 7f21cdd4d640 1 --2- 172.21.15.105:0/4223578035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21c81074f0 0x7f21c807db50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f21c4009940 tx=0x7f21c40327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:29.958 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.118+0000 7f21baffd640 1 -- 172.21.15.105:0/4223578035 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21c4042070 con 0x7f21c81074f0 2024-09-16T01:53:29.958 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.118+0000 7f21baffd640 1 -- 172.21.15.105:0/4223578035 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21c4030d00 con 0x7f21c81074f0 2024-09-16T01:53:29.958 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.118+0000 7f21baffd640 1 -- 172.21.15.105:0/4223578035 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21c4037040 con 0x7f21c81074f0 2024-09-16T01:53:29.958 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.118+0000 7f21cffb1640 1 -- 172.21.15.105:0/4223578035 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21c807c280 con 0x7f21c81074f0 2024-09-16T01:53:29.958 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.118+0000 7f21cffb1640 1 -- 172.21.15.105:0/4223578035 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21c807c7c0 con 0x7f21c81074f0 2024-09-16T01:53:29.958 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.119+0000 7f21baffd640 1 -- 172.21.15.105:0/4223578035 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f21c4045070 con 0x7f21c81074f0 2024-09-16T01:53:29.958 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.119+0000 7f21baffd640 1 --2- 172.21.15.105:0/4223578035 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f21b40391b0 0x7f21b403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:29.958 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.119+0000 7f21baffd640 1 -- 172.21.15.105:0/4223578035 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f21c406c5c0 con 0x7f21c81074f0 2024-09-16T01:53:29.958 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.119+0000 7f21cffb1640 1 -- 172.21.15.105:0/4223578035 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21c810bbd0 con 0x7f21c81074f0 2024-09-16T01:53:29.958 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.119+0000 7f21cd54c640 1 --2- 172.21.15.105:0/4223578035 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f21b40391b0 0x7f21b403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:29.958 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.120+0000 7f21cd54c640 1 --2- 172.21.15.105:0/4223578035 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f21b40391b0 0x7f21b403b670 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f21bc0098c0 tx=0x7f21bc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:29.958 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.124+0000 7f21baffd640 1 -- 172.21.15.105:0/4223578035 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f21c4045360 con 0x7f21c81074f0 2024-09-16T01:53:29.959 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.324+0000 7f21cffb1640 1 -- 172.21.15.105:0/4223578035 --> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f21c8194ef0 con 0x7f21b40391b0 2024-09-16T01:53:29.959 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.329+0000 7f21baffd640 1 -- 172.21.15.105:0/4223578035 <== mgr.14118 v2:172.21.15.105:6800/1469189484 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f21c8194ef0 con 0x7f21b40391b0 2024-09-16T01:53:29.959 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.331+0000 7f21cffb1640 1 -- 172.21.15.105:0/4223578035 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f21b40391b0 msgr2=0x7f21b403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:29.959 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.331+0000 7f21cffb1640 1 --2- 172.21.15.105:0/4223578035 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f21b40391b0 0x7f21b403b670 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f21bc0098c0 tx=0x7f21bc006db0 comp rx=0 tx=0).stop 2024-09-16T01:53:29.959 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.331+0000 7f21cffb1640 1 -- 172.21.15.105:0/4223578035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21c81074f0 msgr2=0x7f21c807db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:29.959 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.331+0000 7f21cffb1640 1 --2- 172.21.15.105:0/4223578035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21c81074f0 0x7f21c807db50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f21c4009940 tx=0x7f21c40327b0 comp rx=0 tx=0).stop 2024-09-16T01:53:29.959 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.331+0000 7f21cffb1640 1 -- 172.21.15.105:0/4223578035 shutdown_connections 2024-09-16T01:53:29.959 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.331+0000 7f21cffb1640 1 --2- 172.21.15.105:0/4223578035 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f21b40391b0 0x7f21b403b670 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:29.959 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.331+0000 7f21cffb1640 1 --2- 172.21.15.105:0/4223578035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21c81074f0 0x7f21c807db50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:29.959 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.331+0000 7f21cffb1640 1 -- 172.21.15.105:0/4223578035 >> 172.21.15.105:0/4223578035 conn(0x7f21c8102c40 msgr2=0x7f21c8103a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:29.959 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.331+0000 7f21cffb1640 1 -- 172.21.15.105:0/4223578035 shutdown_connections 2024-09-16T01:53:29.959 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:29.331+0000 7f21cffb1640 1 -- 172.21.15.105:0/4223578035 wait complete. 2024-09-16T01:53:30.223 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:29 smithi105 ceph-mon[31197]: from='mgr.14118 172.21.15.105:0/2613168790' entity='mgr.x' 2024-09-16T01:53:30.223 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:29 smithi105 ceph-mon[31197]: from='mgr.14118 172.21.15.105:0/2613168790' entity='mgr.x' 2024-09-16T01:53:30.223 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:29 smithi105 ceph-mon[31197]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:53:30.224 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:29 smithi105 ceph-mon[31197]: Saving service mgr spec with placement count:2 2024-09-16T01:53:30.224 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:29 smithi105 ceph-mon[31197]: from='mgr.14118 172.21.15.105:0/2613168790' entity='mgr.x' 2024-09-16T01:53:31.076 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.303+0000 7f9acd20c640 1 Processor -- start 2024-09-16T01:53:31.076 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.303+0000 7f9acd20c640 1 -- start start 2024-09-16T01:53:31.076 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.303+0000 7f9acd20c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ac8107f90 0x7f9ac810a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:31.076 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.303+0000 7f9acd20c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac807bdf0 con 0x7f9ac8107f90 2024-09-16T01:53:31.076 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.303+0000 7f9ac6d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ac8107f90 0x7f9ac810a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:31.076 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.303+0000 7f9ac6d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ac8107f90 0x7f9ac810a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36776/0 (socket says 172.21.15.105:36776) 2024-09-16T01:53:31.076 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.304+0000 7f9ac6d9d640 1 -- 172.21.15.105:0/3003418817 learned_addr learned my addr 172.21.15.105:0/3003418817 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:31.076 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.304+0000 7f9ac6d9d640 1 -- 172.21.15.105:0/3003418817 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ac810a8c0 con 0x7f9ac8107f90 2024-09-16T01:53:31.076 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.304+0000 7f9ac6d9d640 1 --2- 172.21.15.105:0/3003418817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ac8107f90 0x7f9ac810a380 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8009e80 tx=0x7f9ab802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed1d76c59e4f2af2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:31.076 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.305+0000 7f9ac5d9b640 1 -- 172.21.15.105:0/3003418817 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ab8036070 con 0x7f9ac8107f90 2024-09-16T01:53:31.077 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.305+0000 7f9ac5d9b640 1 -- 172.21.15.105:0/3003418817 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ab80318b0 con 0x7f9ac8107f90 2024-09-16T01:53:31.077 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.305+0000 7f9ac5d9b640 1 -- 172.21.15.105:0/3003418817 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ab8007930 con 0x7f9ac8107f90 2024-09-16T01:53:31.077 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.305+0000 7f9acd20c640 1 -- 172.21.15.105:0/3003418817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ac8107f90 msgr2=0x7f9ac810a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:31.077 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.305+0000 7f9acd20c640 1 --2- 172.21.15.105:0/3003418817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ac8107f90 0x7f9ac810a380 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8009e80 tx=0x7f9ab802a7a0 comp rx=0 tx=0).stop 2024-09-16T01:53:31.077 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.305+0000 7f9acd20c640 1 -- 172.21.15.105:0/3003418817 shutdown_connections 2024-09-16T01:53:31.077 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.305+0000 7f9acd20c640 1 --2- 172.21.15.105:0/3003418817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ac8107f90 0x7f9ac810a380 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:31.077 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.305+0000 7f9acd20c640 1 -- 172.21.15.105:0/3003418817 >> 172.21.15.105:0/3003418817 conn(0x7f9ac8101b10 msgr2=0x7f9ac8103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:31.077 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.305+0000 7f9acd20c640 1 -- 172.21.15.105:0/3003418817 shutdown_connections 2024-09-16T01:53:31.077 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.305+0000 7f9acd20c640 1 -- 172.21.15.105:0/3003418817 wait complete. 2024-09-16T01:53:31.077 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.306+0000 7f9acd20c640 1 Processor -- start 2024-09-16T01:53:31.077 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.306+0000 7f9acd20c640 1 -- start start 2024-09-16T01:53:31.077 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.306+0000 7f9acd20c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ac8107f90 0x7f9ac8107810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:31.077 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.306+0000 7f9acd20c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac810b880 con 0x7f9ac8107f90 2024-09-16T01:53:31.078 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.307+0000 7f9ac6d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ac8107f90 0x7f9ac8107810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:31.078 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.307+0000 7f9ac6d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ac8107f90 0x7f9ac8107810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36782/0 (socket says 172.21.15.105:36782) 2024-09-16T01:53:31.078 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.307+0000 7f9ac6d9d640 1 -- 172.21.15.105:0/1324295805 learned_addr learned my addr 172.21.15.105:0/1324295805 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:31.078 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.307+0000 7f9ac6d9d640 1 -- 172.21.15.105:0/1324295805 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ac8107d50 con 0x7f9ac8107f90 2024-09-16T01:53:31.078 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.307+0000 7f9ac6d9d640 1 --2- 172.21.15.105:0/1324295805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ac8107f90 0x7f9ac8107810 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8031cc0 tx=0x7f9ab8031cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:31.078 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.307+0000 7f9abffff640 1 -- 172.21.15.105:0/1324295805 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ab8036050 con 0x7f9ac8107f90 2024-09-16T01:53:31.078 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.307+0000 7f9abffff640 1 -- 172.21.15.105:0/1324295805 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ab8042070 con 0x7f9ac8107f90 2024-09-16T01:53:31.078 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.307+0000 7f9abffff640 1 -- 172.21.15.105:0/1324295805 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ab803db10 con 0x7f9ac8107f90 2024-09-16T01:53:31.078 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.307+0000 7f9acd20c640 1 -- 172.21.15.105:0/1324295805 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ac8105f40 con 0x7f9ac8107f90 2024-09-16T01:53:31.078 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.307+0000 7f9acd20c640 1 -- 172.21.15.105:0/1324295805 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ac8106480 con 0x7f9ac8107f90 2024-09-16T01:53:31.078 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.308+0000 7f9abffff640 1 -- 172.21.15.105:0/1324295805 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f9ab8047460 con 0x7f9ac8107f90 2024-09-16T01:53:31.078 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.308+0000 7f9abffff640 1 --2- 172.21.15.105:0/1324295805 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f9ab40391b0 0x7f9ab403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:31.078 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.308+0000 7f9acd20c640 1 -- 172.21.15.105:0/1324295805 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ac81083f0 con 0x7f9ac8107f90 2024-09-16T01:53:31.078 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.308+0000 7f9abffff640 1 -- 172.21.15.105:0/1324295805 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9ab806d890 con 0x7f9ac8107f90 2024-09-16T01:53:31.079 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.308+0000 7f9ac659c640 1 --2- 172.21.15.105:0/1324295805 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f9ab40391b0 0x7f9ab403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:31.079 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.309+0000 7f9ac659c640 1 --2- 172.21.15.105:0/1324295805 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f9ab40391b0 0x7f9ab403b670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9ab00098c0 tx=0x7f9ab0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:31.079 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.313+0000 7f9abffff640 1 -- 172.21.15.105:0/1324295805 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9ab8045360 con 0x7f9ac8107f90 2024-09-16T01:53:31.079 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.493+0000 7f9acd20c640 1 -- 172.21.15.105:0/1324295805 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f9ac8194e40 con 0x7f9ac8107f90 2024-09-16T01:53:31.079 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.496+0000 7f9abffff640 1 -- 172.21.15.105:0/1324295805 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f9ab8039de0 con 0x7f9ac8107f90 2024-09-16T01:53:31.079 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.498+0000 7f9acd20c640 1 -- 172.21.15.105:0/1324295805 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f9ab40391b0 msgr2=0x7f9ab403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:31.079 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.498+0000 7f9acd20c640 1 --2- 172.21.15.105:0/1324295805 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f9ab40391b0 0x7f9ab403b670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9ab00098c0 tx=0x7f9ab0006db0 comp rx=0 tx=0).stop 2024-09-16T01:53:31.079 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.498+0000 7f9acd20c640 1 -- 172.21.15.105:0/1324295805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ac8107f90 msgr2=0x7f9ac8107810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:31.079 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.498+0000 7f9acd20c640 1 --2- 172.21.15.105:0/1324295805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ac8107f90 0x7f9ac8107810 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8031cc0 tx=0x7f9ab8031cf0 comp rx=0 tx=0).stop 2024-09-16T01:53:31.079 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.498+0000 7f9acd20c640 1 -- 172.21.15.105:0/1324295805 shutdown_connections 2024-09-16T01:53:31.079 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.498+0000 7f9acd20c640 1 --2- 172.21.15.105:0/1324295805 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f9ab40391b0 0x7f9ab403b670 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:31.079 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.498+0000 7f9acd20c640 1 --2- 172.21.15.105:0/1324295805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ac8107f90 0x7f9ac8107810 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:31.079 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.498+0000 7f9acd20c640 1 -- 172.21.15.105:0/1324295805 >> 172.21.15.105:0/1324295805 conn(0x7f9ac8101b10 msgr2=0x7f9ac8102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:31.079 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.498+0000 7f9acd20c640 1 -- 172.21.15.105:0/1324295805 shutdown_connections 2024-09-16T01:53:31.080 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:30.499+0000 7f9acd20c640 1 -- 172.21.15.105:0/1324295805 wait complete. 2024-09-16T01:53:31.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:31 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/1324295805' entity='client.admin' 2024-09-16T01:53:32.920 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:32 smithi105 ceph-mon[31197]: from='mgr.14118 172.21.15.105:0/2613168790' entity='mgr.x' 2024-09-16T01:53:32.920 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:32 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/3614250347' entity='client.admin' 2024-09-16T01:53:32.921 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.144+0000 7f0abd05e640 1 Processor -- start 2024-09-16T01:53:32.926 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.144+0000 7f0abd05e640 1 -- start start 2024-09-16T01:53:32.927 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.144+0000 7f0abd05e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ab8107fb0 0x7f0ab810a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:32.927 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.144+0000 7f0abd05e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab807be00 con 0x7f0ab8107fb0 2024-09-16T01:53:32.927 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.144+0000 7f0ab6d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ab8107fb0 0x7f0ab810a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:32.927 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.144+0000 7f0ab6d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ab8107fb0 0x7f0ab810a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36796/0 (socket says 172.21.15.105:36796) 2024-09-16T01:53:32.927 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.144+0000 7f0ab6d9d640 1 -- 172.21.15.105:0/947962452 learned_addr learned my addr 172.21.15.105:0/947962452 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:32.927 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.145+0000 7f0ab6d9d640 1 -- 172.21.15.105:0/947962452 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ab810a8e0 con 0x7f0ab8107fb0 2024-09-16T01:53:32.927 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.146+0000 7f0ab6d9d640 1 --2- 172.21.15.105:0/947962452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ab8107fb0 0x7f0ab810a3a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8009820 tx=0x7f0aa802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5d3577934dd3d52 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:32.927 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.146+0000 7f0ab659c640 1 -- 172.21.15.105:0/947962452 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0aa8037070 con 0x7f0ab8107fb0 2024-09-16T01:53:32.927 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.146+0000 7f0ab659c640 1 -- 172.21.15.105:0/947962452 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0aa80328f0 con 0x7f0ab8107fb0 2024-09-16T01:53:32.927 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.146+0000 7f0ab659c640 1 -- 172.21.15.105:0/947962452 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0aa8030740 con 0x7f0ab8107fb0 2024-09-16T01:53:32.927 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.146+0000 7f0abd05e640 1 -- 172.21.15.105:0/947962452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ab8107fb0 msgr2=0x7f0ab810a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:32.927 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.146+0000 7f0abd05e640 1 --2- 172.21.15.105:0/947962452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ab8107fb0 0x7f0ab810a3a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8009820 tx=0x7f0aa802a270 comp rx=0 tx=0).stop 2024-09-16T01:53:32.928 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.146+0000 7f0abd05e640 1 -- 172.21.15.105:0/947962452 shutdown_connections 2024-09-16T01:53:32.928 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.146+0000 7f0abd05e640 1 --2- 172.21.15.105:0/947962452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ab8107fb0 0x7f0ab810a3a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:32.928 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.146+0000 7f0abd05e640 1 -- 172.21.15.105:0/947962452 >> 172.21.15.105:0/947962452 conn(0x7f0ab8101bb0 msgr2=0x7f0ab8103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:32.928 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.146+0000 7f0abd05e640 1 -- 172.21.15.105:0/947962452 shutdown_connections 2024-09-16T01:53:32.928 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.146+0000 7f0abd05e640 1 -- 172.21.15.105:0/947962452 wait complete. 2024-09-16T01:53:32.928 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.147+0000 7f0abd05e640 1 Processor -- start 2024-09-16T01:53:32.928 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.147+0000 7f0abd05e640 1 -- start start 2024-09-16T01:53:32.928 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.147+0000 7f0abd05e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ab8107fb0 0x7f0ab819b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:32.928 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.147+0000 7f0abd05e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab810b8a0 con 0x7f0ab8107fb0 2024-09-16T01:53:32.928 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.147+0000 7f0ab6d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ab8107fb0 0x7f0ab819b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:32.928 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.147+0000 7f0ab6d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ab8107fb0 0x7f0ab819b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36800/0 (socket says 172.21.15.105:36800) 2024-09-16T01:53:32.928 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.147+0000 7f0ab6d9d640 1 -- 172.21.15.105:0/3614250347 learned_addr learned my addr 172.21.15.105:0/3614250347 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:32.928 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.148+0000 7f0ab6d9d640 1 -- 172.21.15.105:0/3614250347 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ab819bc40 con 0x7f0ab8107fb0 2024-09-16T01:53:32.928 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.148+0000 7f0ab6d9d640 1 --2- 172.21.15.105:0/3614250347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ab8107fb0 0x7f0ab819b700 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8030cc0 tx=0x7f0aa8030cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:32.929 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.148+0000 7f0ab4d99640 1 -- 172.21.15.105:0/3614250347 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0aa8042070 con 0x7f0ab8107fb0 2024-09-16T01:53:32.929 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.148+0000 7f0ab4d99640 1 -- 172.21.15.105:0/3614250347 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0aa8030e60 con 0x7f0ab8107fb0 2024-09-16T01:53:32.929 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.148+0000 7f0abd05e640 1 -- 172.21.15.105:0/3614250347 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ab819bed0 con 0x7f0ab8107fb0 2024-09-16T01:53:32.929 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.148+0000 7f0ab4d99640 1 -- 172.21.15.105:0/3614250347 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0aa8037040 con 0x7f0ab8107fb0 2024-09-16T01:53:32.929 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.148+0000 7f0abd05e640 1 -- 172.21.15.105:0/3614250347 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ab819ebc0 con 0x7f0ab8107fb0 2024-09-16T01:53:32.929 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.149+0000 7f0ab4d99640 1 -- 172.21.15.105:0/3614250347 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f0aa8045070 con 0x7f0ab8107fb0 2024-09-16T01:53:32.929 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.149+0000 7f0abd05e640 1 -- 172.21.15.105:0/3614250347 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ab8108410 con 0x7f0ab8107fb0 2024-09-16T01:53:32.929 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.149+0000 7f0ab4d99640 1 --2- 172.21.15.105:0/3614250347 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f0aa4039200 0x7f0aa403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:32.929 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.149+0000 7f0ab4d99640 1 -- 172.21.15.105:0/3614250347 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0aa806c2d0 con 0x7f0ab8107fb0 2024-09-16T01:53:32.929 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.149+0000 7f0aaffff640 1 --2- 172.21.15.105:0/3614250347 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f0aa4039200 0x7f0aa403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:32.929 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.150+0000 7f0aaffff640 1 --2- 172.21.15.105:0/3614250347 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f0aa4039200 0x7f0aa403b6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0aa00098c0 tx=0x7f0aa0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:32.929 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.154+0000 7f0ab4d99640 1 -- 172.21.15.105:0/3614250347 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0aa8045360 con 0x7f0ab8107fb0 2024-09-16T01:53:32.929 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.330+0000 7f0abd05e640 1 -- 172.21.15.105:0/3614250347 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f0ab802d7c0 con 0x7f0ab8107fb0 2024-09-16T01:53:32.930 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.333+0000 7f0ab4d99640 1 -- 172.21.15.105:0/3614250347 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f0aa802ed90 con 0x7f0ab8107fb0 2024-09-16T01:53:32.930 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.335+0000 7f0abd05e640 1 -- 172.21.15.105:0/3614250347 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f0aa4039200 msgr2=0x7f0aa403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:32.930 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.335+0000 7f0abd05e640 1 --2- 172.21.15.105:0/3614250347 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f0aa4039200 0x7f0aa403b6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0aa00098c0 tx=0x7f0aa0006db0 comp rx=0 tx=0).stop 2024-09-16T01:53:32.930 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.335+0000 7f0abd05e640 1 -- 172.21.15.105:0/3614250347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ab8107fb0 msgr2=0x7f0ab819b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:32.930 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.335+0000 7f0abd05e640 1 --2- 172.21.15.105:0/3614250347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ab8107fb0 0x7f0ab819b700 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8030cc0 tx=0x7f0aa8030cf0 comp rx=0 tx=0).stop 2024-09-16T01:53:32.930 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.335+0000 7f0abd05e640 1 -- 172.21.15.105:0/3614250347 shutdown_connections 2024-09-16T01:53:32.930 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.335+0000 7f0abd05e640 1 --2- 172.21.15.105:0/3614250347 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f0aa4039200 0x7f0aa403b6c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:32.930 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.335+0000 7f0abd05e640 1 --2- 172.21.15.105:0/3614250347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ab8107fb0 0x7f0ab819b700 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:32.930 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.335+0000 7f0abd05e640 1 -- 172.21.15.105:0/3614250347 >> 172.21.15.105:0/3614250347 conn(0x7f0ab8101bb0 msgr2=0x7f0ab8102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:32.930 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.336+0000 7f0abd05e640 1 -- 172.21.15.105:0/3614250347 shutdown_connections 2024-09-16T01:53:32.930 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:32.336+0000 7f0abd05e640 1 -- 172.21.15.105:0/3614250347 wait complete. 2024-09-16T01:53:32.930 INFO:teuthology.orchestra.run.smithi105.stdout:Enabling the dashboard module... 2024-09-16T01:53:34.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:34 smithi105 ceph-mon[31197]: from='mgr.14118 172.21.15.105:0/2613168790' entity='mgr.x' 2024-09-16T01:53:34.869 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:34 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/3725980284' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-16T01:53:34.870 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:34 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: ignoring --setuser ceph since I am not root 2024-09-16T01:53:34.870 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:34 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: ignoring --setgroup ceph since I am not root 2024-09-16T01:53:34.870 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:34 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:34.602+0000 7f2281be0040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T01:53:34.870 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:34 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:34.660+0000 7f2281be0040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T01:53:35.114 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:34 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:34.810+0000 7f2281be0040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T01:53:35.114 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:34 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:34.877+0000 7f2281be0040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T01:53:35.114 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:35 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:35.027+0000 7f2281be0040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T01:53:35.121 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.326+0000 7fa765ee7640 1 Processor -- start 2024-09-16T01:53:35.121 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.326+0000 7fa765ee7640 1 -- start start 2024-09-16T01:53:35.121 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.326+0000 7fa765ee7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa76007de30 0x7fa76007c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:35.121 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.326+0000 7fa765ee7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa76007c820 con 0x7fa76007de30 2024-09-16T01:53:35.121 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.326+0000 7fa75f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa76007de30 0x7fa76007c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:35.121 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.326+0000 7fa75f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa76007de30 0x7fa76007c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36806/0 (socket says 172.21.15.105:36806) 2024-09-16T01:53:35.121 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.326+0000 7fa75f7fe640 1 -- 172.21.15.105:0/2083234048 learned_addr learned my addr 172.21.15.105:0/2083234048 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:35.121 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.326+0000 7fa75f7fe640 1 -- 172.21.15.105:0/2083234048 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa76007c9a0 con 0x7fa76007de30 2024-09-16T01:53:35.122 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.327+0000 7fa75f7fe640 1 --2- 172.21.15.105:0/2083234048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa76007de30 0x7fa76007c2e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa750009820 tx=0x7fa75002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8efd34e56745190e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:35.122 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.327+0000 7fa75e7fc640 1 -- 172.21.15.105:0/2083234048 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa750037070 con 0x7fa76007de30 2024-09-16T01:53:35.122 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.327+0000 7fa75e7fc640 1 -- 172.21.15.105:0/2083234048 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7500328f0 con 0x7fa76007de30 2024-09-16T01:53:35.122 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.327+0000 7fa75e7fc640 1 -- 172.21.15.105:0/2083234048 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7500307c0 con 0x7fa76007de30 2024-09-16T01:53:35.122 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.328+0000 7fa765ee7640 1 -- 172.21.15.105:0/2083234048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa76007de30 msgr2=0x7fa76007c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:35.122 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.328+0000 7fa765ee7640 1 --2- 172.21.15.105:0/2083234048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa76007de30 0x7fa76007c2e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa750009820 tx=0x7fa75002a300 comp rx=0 tx=0).stop 2024-09-16T01:53:35.122 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.328+0000 7fa765ee7640 1 -- 172.21.15.105:0/2083234048 shutdown_connections 2024-09-16T01:53:35.122 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.328+0000 7fa765ee7640 1 --2- 172.21.15.105:0/2083234048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa76007de30 0x7fa76007c2e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:35.122 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.328+0000 7fa765ee7640 1 -- 172.21.15.105:0/2083234048 >> 172.21.15.105:0/2083234048 conn(0x7fa760102ca0 msgr2=0x7fa7601050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:35.122 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.328+0000 7fa765ee7640 1 -- 172.21.15.105:0/2083234048 shutdown_connections 2024-09-16T01:53:35.122 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.328+0000 7fa765ee7640 1 -- 172.21.15.105:0/2083234048 wait complete. 2024-09-16T01:53:35.122 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.329+0000 7fa765ee7640 1 Processor -- start 2024-09-16T01:53:35.122 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.329+0000 7fa765ee7640 1 -- start start 2024-09-16T01:53:35.122 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.329+0000 7fa765ee7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa76007de30 0x7fa76019d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:35.123 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.329+0000 7fa765ee7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa76010dac0 con 0x7fa76007de30 2024-09-16T01:53:35.123 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.329+0000 7fa75f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa76007de30 0x7fa76019d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:35.123 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.329+0000 7fa75f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa76007de30 0x7fa76019d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36812/0 (socket says 172.21.15.105:36812) 2024-09-16T01:53:35.123 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.329+0000 7fa75f7fe640 1 -- 172.21.15.105:0/3725980284 learned_addr learned my addr 172.21.15.105:0/3725980284 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:35.123 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.329+0000 7fa75f7fe640 1 -- 172.21.15.105:0/3725980284 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa76019de70 con 0x7fa76007de30 2024-09-16T01:53:35.123 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.329+0000 7fa75f7fe640 1 --2- 172.21.15.105:0/3725980284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa76007de30 0x7fa76019d930 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa7500304d0 tx=0x7fa750032cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:35.123 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.330+0000 7fa75cff9640 1 -- 172.21.15.105:0/3725980284 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa750037040 con 0x7fa76007de30 2024-09-16T01:53:35.123 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.330+0000 7fa75cff9640 1 -- 172.21.15.105:0/3725980284 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa750040070 con 0x7fa76007de30 2024-09-16T01:53:35.123 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.330+0000 7fa765ee7640 1 -- 172.21.15.105:0/3725980284 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa76019e100 con 0x7fa76007de30 2024-09-16T01:53:35.123 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.330+0000 7fa75cff9640 1 -- 172.21.15.105:0/3725980284 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa75003be40 con 0x7fa76007de30 2024-09-16T01:53:35.123 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.330+0000 7fa765ee7640 1 -- 172.21.15.105:0/3725980284 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7601a0df0 con 0x7fa76007de30 2024-09-16T01:53:35.123 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.330+0000 7fa75cff9640 1 -- 172.21.15.105:0/3725980284 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fa750045460 con 0x7fa76007de30 2024-09-16T01:53:35.123 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.331+0000 7fa765ee7640 1 -- 172.21.15.105:0/3725980284 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa76007c360 con 0x7fa76007de30 2024-09-16T01:53:35.123 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.331+0000 7fa75cff9640 1 --2- 172.21.15.105:0/3725980284 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7fa7480391b0 0x7fa74803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:35.124 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.331+0000 7fa75cff9640 1 -- 172.21.15.105:0/3725980284 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa75006c5d0 con 0x7fa76007de30 2024-09-16T01:53:35.124 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.331+0000 7fa75effd640 1 --2- 172.21.15.105:0/3725980284 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7fa7480391b0 0x7fa74803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:35.124 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.331+0000 7fa75effd640 1 --2- 172.21.15.105:0/3725980284 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7fa7480391b0 0x7fa74803b670 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa7540098c0 tx=0x7fa754006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:35.124 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.335+0000 7fa75cff9640 1 -- 172.21.15.105:0/3725980284 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa750030dc0 con 0x7fa76007de30 2024-09-16T01:53:35.124 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:33.553+0000 7fa765ee7640 1 -- 172.21.15.105:0/3725980284 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fa76002d7c0 con 0x7fa76007de30 2024-09-16T01:53:35.124 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:34.454+0000 7fa75cff9640 1 -- 172.21.15.105:0/3725980284 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fa750045850 con 0x7fa76007de30 2024-09-16T01:53:35.124 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:34.455+0000 7fa75cff9640 1 -- 172.21.15.105:0/3725980284 <== mon.0 v2:172.21.15.105:3300/0 8 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7fa75003a420 con 0x7fa76007de30 2024-09-16T01:53:35.124 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:34.457+0000 7fa765ee7640 1 -- 172.21.15.105:0/3725980284 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7fa7480391b0 msgr2=0x7fa74803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:35.124 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:34.457+0000 7fa765ee7640 1 --2- 172.21.15.105:0/3725980284 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7fa7480391b0 0x7fa74803b670 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa7540098c0 tx=0x7fa754006db0 comp rx=0 tx=0).stop 2024-09-16T01:53:35.124 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:34.457+0000 7fa765ee7640 1 -- 172.21.15.105:0/3725980284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa76007de30 msgr2=0x7fa76019d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:35.124 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:34.457+0000 7fa765ee7640 1 --2- 172.21.15.105:0/3725980284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa76007de30 0x7fa76019d930 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa7500304d0 tx=0x7fa750032cd0 comp rx=0 tx=0).stop 2024-09-16T01:53:35.124 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:34.457+0000 7fa765ee7640 1 -- 172.21.15.105:0/3725980284 shutdown_connections 2024-09-16T01:53:35.124 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:34.457+0000 7fa765ee7640 1 --2- 172.21.15.105:0/3725980284 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7fa7480391b0 0x7fa74803b670 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:35.125 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:34.457+0000 7fa765ee7640 1 --2- 172.21.15.105:0/3725980284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa76007de30 0x7fa76019d930 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:35.125 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:34.457+0000 7fa765ee7640 1 -- 172.21.15.105:0/3725980284 >> 172.21.15.105:0/3725980284 conn(0x7fa760102ca0 msgr2=0x7fa760103940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:35.125 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:34.457+0000 7fa765ee7640 1 -- 172.21.15.105:0/3725980284 shutdown_connections 2024-09-16T01:53:35.125 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:34.457+0000 7fa765ee7640 1 -- 172.21.15.105:0/3725980284 wait complete. 2024-09-16T01:53:35.369 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:35 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:35.172+0000 7f2281be0040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T01:53:35.701 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:35 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/3725980284' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-16T01:53:35.701 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:35 smithi105 ceph-mon[31197]: mgrmap e9: x(active, since 18s) 2024-09-16T01:53:35.701 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:35 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:35.490+0000 7f2281be0040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T01:53:35.702 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:35 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:35.552+0000 7f2281be0040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T01:53:35.987 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:35 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:35.743+0000 7f2281be0040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T01:53:36.614 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:36 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/873388120' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-16T01:53:36.616 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout { 2024-09-16T01:53:36.624 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-16T01:53:36.624 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T01:53:36.626 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-16T01:53:36.626 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-16T01:53:36.626 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout } 2024-09-16T01:53:36.626 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.779+0000 7fa7c88f8640 1 Processor -- start 2024-09-16T01:53:36.626 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.779+0000 7fa7c88f8640 1 -- start start 2024-09-16T01:53:36.627 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.780+0000 7fa7c88f8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7c4105770 0x7fa7c4105b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:36.627 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.780+0000 7fa7c88f8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c4106140 con 0x7fa7c4105770 2024-09-16T01:53:36.627 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.780+0000 7fa7c259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7c4105770 0x7fa7c4105b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:36.627 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.780+0000 7fa7c259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7c4105770 0x7fa7c4105b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36836/0 (socket says 172.21.15.105:36836) 2024-09-16T01:53:36.627 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.780+0000 7fa7c259c640 1 -- 172.21.15.105:0/2677942672 learned_addr learned my addr 172.21.15.105:0/2677942672 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:36.627 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.780+0000 7fa7c259c640 1 -- 172.21.15.105:0/2677942672 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c4108750 con 0x7fa7c4105770 2024-09-16T01:53:36.628 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.781+0000 7fa7c259c640 1 --2- 172.21.15.105:0/2677942672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7c4105770 0x7fa7c4105b70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4009820 tx=0x7fa7b402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2382f45c47246d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:36.628 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.781+0000 7fa7c159a640 1 -- 172.21.15.105:0/2677942672 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7b4037070 con 0x7fa7c4105770 2024-09-16T01:53:36.628 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.781+0000 7fa7c159a640 1 -- 172.21.15.105:0/2677942672 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7b40328f0 con 0x7fa7c4105770 2024-09-16T01:53:36.628 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.781+0000 7fa7c159a640 1 -- 172.21.15.105:0/2677942672 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7b4030740 con 0x7fa7c4105770 2024-09-16T01:53:36.628 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.782+0000 7fa7c88f8640 1 -- 172.21.15.105:0/2677942672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7c4105770 msgr2=0x7fa7c4105b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:36.629 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.782+0000 7fa7c88f8640 1 --2- 172.21.15.105:0/2677942672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7c4105770 0x7fa7c4105b70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4009820 tx=0x7fa7b402a270 comp rx=0 tx=0).stop 2024-09-16T01:53:36.629 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.782+0000 7fa7c88f8640 1 -- 172.21.15.105:0/2677942672 shutdown_connections 2024-09-16T01:53:36.629 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.782+0000 7fa7c88f8640 1 --2- 172.21.15.105:0/2677942672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7c4105770 0x7fa7c4105b70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:36.629 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.782+0000 7fa7c88f8640 1 -- 172.21.15.105:0/2677942672 >> 172.21.15.105:0/2677942672 conn(0x7fa7c4101280 msgr2=0x7fa7c4103700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:36.629 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.782+0000 7fa7c88f8640 1 -- 172.21.15.105:0/2677942672 shutdown_connections 2024-09-16T01:53:36.629 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.782+0000 7fa7c88f8640 1 -- 172.21.15.105:0/2677942672 wait complete. 2024-09-16T01:53:36.630 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.783+0000 7fa7c88f8640 1 Processor -- start 2024-09-16T01:53:36.630 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.783+0000 7fa7c88f8640 1 -- start start 2024-09-16T01:53:36.630 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.783+0000 7fa7c88f8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7c4105770 0x7fa7c4107fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:36.630 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.783+0000 7fa7c88f8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c410b980 con 0x7fa7c4105770 2024-09-16T01:53:36.630 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.783+0000 7fa7c259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7c4105770 0x7fa7c4107fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:36.630 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.783+0000 7fa7c259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7c4105770 0x7fa7c4107fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36852/0 (socket says 172.21.15.105:36852) 2024-09-16T01:53:36.631 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.783+0000 7fa7c259c640 1 -- 172.21.15.105:0/873388120 learned_addr learned my addr 172.21.15.105:0/873388120 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:36.631 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.783+0000 7fa7c259c640 1 -- 172.21.15.105:0/873388120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c4108510 con 0x7fa7c4105770 2024-09-16T01:53:36.631 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.784+0000 7fa7c259c640 1 --2- 172.21.15.105:0/873388120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7c4105770 0x7fa7c4107fd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4009940 tx=0x7fa7b4030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:36.631 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.784+0000 7fa7b37fe640 1 -- 172.21.15.105:0/873388120 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7b4042070 con 0x7fa7c4105770 2024-09-16T01:53:36.631 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.784+0000 7fa7c88f8640 1 -- 172.21.15.105:0/873388120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7c4106700 con 0x7fa7c4105770 2024-09-16T01:53:36.631 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.784+0000 7fa7b37fe640 1 -- 172.21.15.105:0/873388120 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7b4030e30 con 0x7fa7c4105770 2024-09-16T01:53:36.632 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.784+0000 7fa7b37fe640 1 -- 172.21.15.105:0/873388120 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7b4037040 con 0x7fa7c4105770 2024-09-16T01:53:36.632 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.784+0000 7fa7c88f8640 1 -- 172.21.15.105:0/873388120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7c4106c40 con 0x7fa7c4105770 2024-09-16T01:53:36.632 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.785+0000 7fa7c88f8640 1 -- 172.21.15.105:0/873388120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7c4105bf0 con 0x7fa7c4105770 2024-09-16T01:53:36.633 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.785+0000 7fa7b37fe640 1 -- 172.21.15.105:0/873388120 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7fa7b4045070 con 0x7fa7c4105770 2024-09-16T01:53:36.633 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.785+0000 7fa7b37fe640 1 --2- 172.21.15.105:0/873388120 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7fa7ac039200 0x7fa7ac03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:36.633 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.785+0000 7fa7b37fe640 1 -- 172.21.15.105:0/873388120 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa7b406c3a0 con 0x7fa7c4105770 2024-09-16T01:53:36.634 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.785+0000 7fa7c1d9b640 1 -- 172.21.15.105:0/873388120 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7fa7ac039200 msgr2=0x7fa7ac03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:6800/1469189484 2024-09-16T01:53:36.634 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.785+0000 7fa7c1d9b640 1 --2- 172.21.15.105:0/873388120 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7fa7ac039200 0x7fa7ac03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T01:53:36.634 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.790+0000 7fa7b37fe640 1 -- 172.21.15.105:0/873388120 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa7b4045520 con 0x7fa7c4105770 2024-09-16T01:53:36.634 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.986+0000 7fa7c1d9b640 1 -- 172.21.15.105:0/873388120 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7fa7ac039200 msgr2=0x7fa7ac03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:6800/1469189484 2024-09-16T01:53:36.634 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:35.986+0000 7fa7c1d9b640 1 --2- 172.21.15.105:0/873388120 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7fa7ac039200 0x7fa7ac03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T01:53:36.634 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:36.006+0000 7fa7c88f8640 1 -- 172.21.15.105:0/873388120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fa7c4194ec0 con 0x7fa7c4105770 2024-09-16T01:53:36.635 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:36.007+0000 7fa7b37fe640 1 -- 172.21.15.105:0/873388120 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fa7b403add0 con 0x7fa7c4105770 2024-09-16T01:53:36.635 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:36.009+0000 7fa7c88f8640 1 -- 172.21.15.105:0/873388120 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7fa7ac039200 msgr2=0x7fa7ac03b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:53:36.635 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:36.009+0000 7fa7c88f8640 1 --2- 172.21.15.105:0/873388120 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7fa7ac039200 0x7fa7ac03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:36.635 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:36.009+0000 7fa7c88f8640 1 -- 172.21.15.105:0/873388120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7c4105770 msgr2=0x7fa7c4107fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:36.635 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:36.009+0000 7fa7c88f8640 1 --2- 172.21.15.105:0/873388120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7c4105770 0x7fa7c4107fd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4009940 tx=0x7fa7b4030cc0 comp rx=0 tx=0).stop 2024-09-16T01:53:36.635 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:36.009+0000 7fa7c88f8640 1 -- 172.21.15.105:0/873388120 shutdown_connections 2024-09-16T01:53:36.636 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:36.009+0000 7fa7c88f8640 1 --2- 172.21.15.105:0/873388120 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7fa7ac039200 0x7fa7ac03b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:36.636 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:36.009+0000 7fa7c88f8640 1 --2- 172.21.15.105:0/873388120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7c4105770 0x7fa7c4107fd0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:36.636 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:36.009+0000 7fa7c88f8640 1 -- 172.21.15.105:0/873388120 >> 172.21.15.105:0/873388120 conn(0x7fa7c4101280 msgr2=0x7fa7c4102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:36.636 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:36.009+0000 7fa7c88f8640 1 -- 172.21.15.105:0/873388120 shutdown_connections 2024-09-16T01:53:36.636 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:36.009+0000 7fa7c88f8640 1 -- 172.21.15.105:0/873388120 wait complete. 2024-09-16T01:53:36.636 INFO:teuthology.orchestra.run.smithi105.stdout:Waiting for the mgr to restart... 2024-09-16T01:53:36.637 INFO:teuthology.orchestra.run.smithi105.stdout:Waiting for mgr epoch 9... 2024-09-16T01:53:36.950 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:36 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:36.681+0000 7f2281be0040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T01:53:36.950 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:36 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:36.752+0000 7f2281be0040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T01:53:37.544 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:37 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:37.484+0000 7f2281be0040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T01:53:37.953 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:37 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:37.598+0000 7f2281be0040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T01:53:37.953 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:37 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:37.742+0000 7f2281be0040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T01:53:37.953 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:37 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:37.803+0000 7f2281be0040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T01:53:37.953 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:37 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:37.951+0000 7f2281be0040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T01:53:38.239 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:38 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:38.020+0000 7f2281be0040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T01:53:38.239 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:38 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:38.079+0000 7f2281be0040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T01:53:38.239 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:38 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:38.237+0000 7f2281be0040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T01:53:38.513 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:38 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:38.302+0000 7f2281be0040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T01:53:38.513 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:38 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:38.369+0000 7f2281be0040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T01:53:38.513 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:38 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:38.452+0000 7f2281be0040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T01:53:38.513 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:38 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:38.511+0000 7f2281be0040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T01:53:38.794 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:38 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:38.571+0000 7f2281be0040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T01:53:39.794 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:39 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:39.349+0000 7f2281be0040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T01:53:39.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:39 smithi105 ceph-mon[31197]: Active manager daemon x restarted 2024-09-16T01:53:39.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:39 smithi105 ceph-mon[31197]: Activating manager daemon x 2024-09-16T01:53:39.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:39 smithi105 ceph-mon[31197]: osdmap e3: 0 total, 0 up, 0 in 2024-09-16T01:53:39.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:39 smithi105 ceph-mon[31197]: mgrmap e10: x(active, starting, since 0.00346806s) 2024-09-16T01:53:39.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:39 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:53:39.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:39 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T01:53:39.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:39 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T01:53:39.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:39 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T01:53:39.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:39 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T01:53:39.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:39 smithi105 ceph-mon[31197]: Manager daemon x is now available 2024-09-16T01:53:39.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:39 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:53:39.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:39 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-16T01:53:39.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:39 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-16T01:53:40.545 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:40 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:40.369+0000 7f21ecef4640 -1 log_channel(cephadm) log [ERR] : [16/Sep/2024:01:53:40] ENGINE Error in HTTPServer.serve 2024-09-16T01:53:40.546 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:40 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: Traceback (most recent call last): 2024-09-16T01:53:40.546 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:40 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T01:53:40.546 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:40 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: self._connections.run(self.expiration_interval) 2024-09-16T01:53:40.546 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:40 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T01:53:40.546 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:40 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: self._run(expiration_interval) 2024-09-16T01:53:40.546 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:40 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T01:53:40.546 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:40 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T01:53:40.546 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:40 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T01:53:40.546 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:40 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T01:53:40.546 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:40 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T01:53:40.547 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:40 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: s = self.context.wrap_socket( 2024-09-16T01:53:40.547 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:40 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T01:53:40.547 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:40 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: return self.sslsocket_class._create( 2024-09-16T01:53:40.547 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:40 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T01:53:40.547 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:40 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: self.do_handshake() 2024-09-16T01:53:40.547 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:40 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T01:53:40.547 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:40 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: self._sslobj.do_handshake() 2024-09-16T01:53:40.547 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:40 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T01:53:40.547 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 01:53:40 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T01:53:41.102 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout { 2024-09-16T01:53:41.102 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-16T01:53:41.102 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-16T01:53:41.102 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout } 2024-09-16T01:53:41.102 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.028+0000 7f26b4490640 1 Processor -- start 2024-09-16T01:53:41.102 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.028+0000 7f26b4490640 1 -- start start 2024-09-16T01:53:41.102 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.028+0000 7f26b4490640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26ac107f10 0x7f26ac10a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:41.102 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.028+0000 7f26b4490640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f26ac07be50 con 0x7f26ac107f10 2024-09-16T01:53:41.102 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.029+0000 7f26b222c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26ac107f10 0x7f26ac10a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:41.102 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.029+0000 7f26b222c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26ac107f10 0x7f26ac10a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36854/0 (socket says 172.21.15.105:36854) 2024-09-16T01:53:41.102 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.029+0000 7f26b222c640 1 -- 172.21.15.105:0/256078598 learned_addr learned my addr 172.21.15.105:0/256078598 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:41.103 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.029+0000 7f26b222c640 1 -- 172.21.15.105:0/256078598 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26ac10a840 con 0x7f26ac107f10 2024-09-16T01:53:41.103 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.030+0000 7f26b222c640 1 --2- 172.21.15.105:0/256078598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26ac107f10 0x7f26ac10a300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f26a8009820 tx=0x7f26a802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=accb83cd1748beb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:41.103 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.030+0000 7f26b122a640 1 -- 172.21.15.105:0/256078598 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26a8037070 con 0x7f26ac107f10 2024-09-16T01:53:41.103 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.030+0000 7f26b122a640 1 -- 172.21.15.105:0/256078598 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26a80328f0 con 0x7f26ac107f10 2024-09-16T01:53:41.103 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.030+0000 7f26b122a640 1 -- 172.21.15.105:0/256078598 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26a8030740 con 0x7f26ac107f10 2024-09-16T01:53:41.103 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.031+0000 7f26b4490640 1 -- 172.21.15.105:0/256078598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26ac107f10 msgr2=0x7f26ac10a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:41.103 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.031+0000 7f26b4490640 1 --2- 172.21.15.105:0/256078598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26ac107f10 0x7f26ac10a300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f26a8009820 tx=0x7f26a802a270 comp rx=0 tx=0).stop 2024-09-16T01:53:41.103 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.031+0000 7f26b4490640 1 -- 172.21.15.105:0/256078598 shutdown_connections 2024-09-16T01:53:41.103 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.031+0000 7f26b4490640 1 --2- 172.21.15.105:0/256078598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26ac107f10 0x7f26ac10a300 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:41.103 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.031+0000 7f26b4490640 1 -- 172.21.15.105:0/256078598 >> 172.21.15.105:0/256078598 conn(0x7f26ac101b10 msgr2=0x7f26ac103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:41.103 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.032+0000 7f26b4490640 1 -- 172.21.15.105:0/256078598 shutdown_connections 2024-09-16T01:53:41.103 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.032+0000 7f26b4490640 1 -- 172.21.15.105:0/256078598 wait complete. 2024-09-16T01:53:41.103 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.032+0000 7f26b4490640 1 Processor -- start 2024-09-16T01:53:41.104 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.032+0000 7f26b4490640 1 -- start start 2024-09-16T01:53:41.104 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.033+0000 7f26b4490640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26ac107f10 0x7f26ac19b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:41.104 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.033+0000 7f26b4490640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f26ac10b860 con 0x7f26ac107f10 2024-09-16T01:53:41.104 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.033+0000 7f26b222c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26ac107f10 0x7f26ac19b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:41.104 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.033+0000 7f26b222c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26ac107f10 0x7f26ac19b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36868/0 (socket says 172.21.15.105:36868) 2024-09-16T01:53:41.104 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.033+0000 7f26b222c640 1 -- 172.21.15.105:0/2132428010 learned_addr learned my addr 172.21.15.105:0/2132428010 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:41.104 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.033+0000 7f26b222c640 1 -- 172.21.15.105:0/2132428010 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26ac19bc70 con 0x7f26ac107f10 2024-09-16T01:53:41.104 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.033+0000 7f26b222c640 1 --2- 172.21.15.105:0/2132428010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26ac107f10 0x7f26ac19b730 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f26a80097f0 tx=0x7f26a8030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:41.104 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.033+0000 7f26a37fe640 1 -- 172.21.15.105:0/2132428010 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26a8042070 con 0x7f26ac107f10 2024-09-16T01:53:41.104 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.033+0000 7f26b4490640 1 -- 172.21.15.105:0/2132428010 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26ac19bf00 con 0x7f26ac107f10 2024-09-16T01:53:41.104 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.033+0000 7f26a37fe640 1 -- 172.21.15.105:0/2132428010 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26a8030d80 con 0x7f26ac107f10 2024-09-16T01:53:41.104 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.033+0000 7f26a37fe640 1 -- 172.21.15.105:0/2132428010 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26a8037040 con 0x7f26ac107f10 2024-09-16T01:53:41.104 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.034+0000 7f26b4490640 1 -- 172.21.15.105:0/2132428010 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26ac19ebf0 con 0x7f26ac107f10 2024-09-16T01:53:41.105 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.034+0000 7f26a37fe640 1 -- 172.21.15.105:0/2132428010 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f26a8045070 con 0x7f26ac107f10 2024-09-16T01:53:41.105 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.035+0000 7f26a37fe640 1 --2- 172.21.15.105:0/2132428010 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f2698039200 0x7f269803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:41.105 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.035+0000 7f26b4490640 1 -- 172.21.15.105:0/2132428010 --> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f26ac108370 con 0x7f2698039200 2024-09-16T01:53:41.105 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.035+0000 7f26a37fe640 1 -- 172.21.15.105:0/2132428010 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f26a806d310 con 0x7f26ac107f10 2024-09-16T01:53:41.105 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.035+0000 7f26b1a2b640 1 -- 172.21.15.105:0/2132428010 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f2698039200 msgr2=0x7f269803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:6800/1469189484 2024-09-16T01:53:41.105 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.035+0000 7f26b1a2b640 1 --2- 172.21.15.105:0/2132428010 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f2698039200 0x7f269803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T01:53:41.105 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.236+0000 7f26b1a2b640 1 -- 172.21.15.105:0/2132428010 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f2698039200 msgr2=0x7f269803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:6800/1469189484 2024-09-16T01:53:41.105 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.236+0000 7f26b1a2b640 1 --2- 172.21.15.105:0/2132428010 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f2698039200 0x7f269803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T01:53:41.105 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.637+0000 7f26b1a2b640 1 -- 172.21.15.105:0/2132428010 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f2698039200 msgr2=0x7f269803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:6800/1469189484 2024-09-16T01:53:41.105 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:37.637+0000 7f26b1a2b640 1 --2- 172.21.15.105:0/2132428010 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f2698039200 0x7f269803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T01:53:41.105 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:38.438+0000 7f26b1a2b640 1 -- 172.21.15.105:0/2132428010 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f2698039200 msgr2=0x7f269803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:6800/1469189484 2024-09-16T01:53:41.105 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:38.438+0000 7f26b1a2b640 1 --2- 172.21.15.105:0/2132428010 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f2698039200 0x7f269803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T01:53:41.105 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:39.353+0000 7f26a37fe640 1 -- 172.21.15.105:0/2132428010 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mgrmap(e 10) ==== 46241+0+0 (secure 0 0 0) 0x7f26a806bd40 con 0x7f26ac107f10 2024-09-16T01:53:41.106 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:39.353+0000 7f26a37fe640 1 -- 172.21.15.105:0/2132428010 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f2698039200 msgr2=0x7f269803b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:53:41.106 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:39.353+0000 7f26a37fe640 1 --2- 172.21.15.105:0/2132428010 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f2698039200 0x7f269803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:41.106 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:40.354+0000 7f26a37fe640 1 -- 172.21.15.105:0/2132428010 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mgrmap(e 11) ==== 46368+0+0 (secure 0 0 0) 0x7f26a803b330 con 0x7f26ac107f10 2024-09-16T01:53:41.106 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:40.354+0000 7f26a37fe640 1 --2- 172.21.15.105:0/2132428010 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f269803cad0 0x7f269803eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:41.106 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:40.354+0000 7f26a37fe640 1 -- 172.21.15.105:0/2132428010 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f26a802fec0 con 0x7f269803cad0 2024-09-16T01:53:41.106 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:40.355+0000 7f26b1a2b640 1 --2- 172.21.15.105:0/2132428010 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f269803cad0 0x7f269803eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:41.106 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:40.355+0000 7f26b1a2b640 1 --2- 172.21.15.105:0/2132428010 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f269803cad0 0x7f269803eec0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f269c003980 tx=0x7f269c006e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:41.106 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:40.356+0000 7f26a37fe640 1 -- 172.21.15.105:0/2132428010 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f26a802fec0 con 0x7f269803cad0 2024-09-16T01:53:41.106 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:40.363+0000 7f26b4490640 1 -- 172.21.15.105:0/2132428010 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f26ac075d10 con 0x7f269803cad0 2024-09-16T01:53:41.106 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:40.364+0000 7f26a37fe640 1 -- 172.21.15.105:0/2132428010 <== mgr.14150 v2:172.21.15.105:6800/275626334 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f26ac075d10 con 0x7f269803cad0 2024-09-16T01:53:41.106 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:40.364+0000 7f26b4490640 1 -- 172.21.15.105:0/2132428010 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f269803cad0 msgr2=0x7f269803eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:41.106 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:40.364+0000 7f26b4490640 1 --2- 172.21.15.105:0/2132428010 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f269803cad0 0x7f269803eec0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f269c003980 tx=0x7f269c006e10 comp rx=0 tx=0).stop 2024-09-16T01:53:41.106 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:40.364+0000 7f26b4490640 1 -- 172.21.15.105:0/2132428010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26ac107f10 msgr2=0x7f26ac19b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:41.106 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:40.364+0000 7f26b4490640 1 --2- 172.21.15.105:0/2132428010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26ac107f10 0x7f26ac19b730 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f26a80097f0 tx=0x7f26a8030c70 comp rx=0 tx=0).stop 2024-09-16T01:53:41.107 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:40.364+0000 7f26b4490640 1 -- 172.21.15.105:0/2132428010 shutdown_connections 2024-09-16T01:53:41.107 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:40.364+0000 7f26b4490640 1 --2- 172.21.15.105:0/2132428010 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f269803cad0 0x7f269803eec0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:41.107 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:40.364+0000 7f26b4490640 1 --2- 172.21.15.105:0/2132428010 >> [v2:172.21.15.105:6800/1469189484,v1:172.21.15.105:6801/1469189484] conn(0x7f2698039200 0x7f269803b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:41.107 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:40.364+0000 7f26b4490640 1 --2- 172.21.15.105:0/2132428010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26ac107f10 0x7f26ac19b730 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:41.107 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:40.364+0000 7f26b4490640 1 -- 172.21.15.105:0/2132428010 >> 172.21.15.105:0/2132428010 conn(0x7f26ac101b10 msgr2=0x7f26ac102870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:41.107 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:40.364+0000 7f26b4490640 1 -- 172.21.15.105:0/2132428010 shutdown_connections 2024-09-16T01:53:41.107 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:40.364+0000 7f26b4490640 1 -- 172.21.15.105:0/2132428010 wait complete. 2024-09-16T01:53:41.107 INFO:teuthology.orchestra.run.smithi105.stdout:mgr epoch 9 is available 2024-09-16T01:53:41.107 INFO:teuthology.orchestra.run.smithi105.stdout:Generating a dashboard self-signed certificate... 2024-09-16T01:53:41.648 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:41 smithi105 ceph-mon[31197]: [16/Sep/2024:01:53:40] ENGINE Bus STARTING 2024-09-16T01:53:41.648 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:41 smithi105 ceph-mon[31197]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-16T01:53:41.648 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:41 smithi105 ceph-mon[31197]: mgrmap e11: x(active, since 1.00553s) 2024-09-16T01:53:41.648 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:41 smithi105 ceph-mon[31197]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-16T01:53:41.648 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:41 smithi105 ceph-mon[31197]: [16/Sep/2024:01:53:40] ENGINE Serving on https://172.21.15.105:7150 2024-09-16T01:53:41.648 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:41 smithi105 ceph-mon[31197]: [16/Sep/2024:01:53:40] ENGINE Error in HTTPServer.serve 2024-09-16T01:53:41.648 INFO:journalctl@ceph.mon.a.smithi105.stdout: Traceback (most recent call last): 2024-09-16T01:53:41.648 INFO:journalctl@ceph.mon.a.smithi105.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T01:53:41.648 INFO:journalctl@ceph.mon.a.smithi105.stdout: self._connections.run(self.expiration_interval) 2024-09-16T01:53:41.649 INFO:journalctl@ceph.mon.a.smithi105.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T01:53:41.649 INFO:journalctl@ceph.mon.a.smithi105.stdout: self._run(expiration_interval) 2024-09-16T01:53:41.649 INFO:journalctl@ceph.mon.a.smithi105.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T01:53:41.649 INFO:journalctl@ceph.mon.a.smithi105.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T01:53:41.649 INFO:journalctl@ceph.mon.a.smithi105.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T01:53:41.649 INFO:journalctl@ceph.mon.a.smithi105.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T01:53:41.649 INFO:journalctl@ceph.mon.a.smithi105.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T01:53:41.649 INFO:journalctl@ceph.mon.a.smithi105.stdout: s = self.context.wrap_socket( 2024-09-16T01:53:41.649 INFO:journalctl@ceph.mon.a.smithi105.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T01:53:41.649 INFO:journalctl@ceph.mon.a.smithi105.stdout: return self.sslsocket_class._create( 2024-09-16T01:53:41.649 INFO:journalctl@ceph.mon.a.smithi105.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T01:53:41.649 INFO:journalctl@ceph.mon.a.smithi105.stdout: self.do_handshake() 2024-09-16T01:53:41.649 INFO:journalctl@ceph.mon.a.smithi105.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T01:53:41.650 INFO:journalctl@ceph.mon.a.smithi105.stdout: self._sslobj.do_handshake() 2024-09-16T01:53:41.650 INFO:journalctl@ceph.mon.a.smithi105.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T01:53:41.650 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:41 smithi105 ceph-mon[31197]: [16/Sep/2024:01:53:40] ENGINE Serving on http://172.21.15.105:8765 2024-09-16T01:53:41.650 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:41 smithi105 ceph-mon[31197]: [16/Sep/2024:01:53:40] ENGINE Bus STARTED 2024-09-16T01:53:42.370 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-16T01:53:42.370 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.443+0000 7fe23e4ee640 1 Processor -- start 2024-09-16T01:53:42.370 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.443+0000 7fe23e4ee640 1 -- start start 2024-09-16T01:53:42.370 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.444+0000 7fe23e4ee640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe238106380 0x7fe238106780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:42.370 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.444+0000 7fe23e4ee640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe238106d50 con 0x7fe238106380 2024-09-16T01:53:42.370 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.444+0000 7fe237fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe238106380 0x7fe238106780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:42.371 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.444+0000 7fe237fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe238106380 0x7fe238106780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42498/0 (socket says 172.21.15.105:42498) 2024-09-16T01:53:42.371 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.444+0000 7fe237fff640 1 -- 172.21.15.105:0/1346667650 learned_addr learned my addr 172.21.15.105:0/1346667650 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:42.371 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.444+0000 7fe237fff640 1 -- 172.21.15.105:0/1346667650 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe238107580 con 0x7fe238106380 2024-09-16T01:53:42.371 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.445+0000 7fe237fff640 1 --2- 172.21.15.105:0/1346667650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe238106380 0x7fe238106780 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe228009820 tx=0x7fe22802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2492b2052380ad42 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:42.371 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.445+0000 7fe2377fe640 1 -- 172.21.15.105:0/1346667650 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe228037070 con 0x7fe238106380 2024-09-16T01:53:42.371 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.445+0000 7fe2377fe640 1 -- 172.21.15.105:0/1346667650 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2280328f0 con 0x7fe238106380 2024-09-16T01:53:42.371 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.445+0000 7fe2377fe640 1 -- 172.21.15.105:0/1346667650 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe228030740 con 0x7fe238106380 2024-09-16T01:53:42.371 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.445+0000 7fe23e4ee640 1 -- 172.21.15.105:0/1346667650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe238106380 msgr2=0x7fe238106780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:42.371 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.445+0000 7fe23e4ee640 1 --2- 172.21.15.105:0/1346667650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe238106380 0x7fe238106780 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe228009820 tx=0x7fe22802a270 comp rx=0 tx=0).stop 2024-09-16T01:53:42.371 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.446+0000 7fe23e4ee640 1 -- 172.21.15.105:0/1346667650 shutdown_connections 2024-09-16T01:53:42.371 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.446+0000 7fe23e4ee640 1 --2- 172.21.15.105:0/1346667650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe238106380 0x7fe238106780 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:42.372 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.446+0000 7fe23e4ee640 1 -- 172.21.15.105:0/1346667650 >> 172.21.15.105:0/1346667650 conn(0x7fe238101b10 msgr2=0x7fe238103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:42.372 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.446+0000 7fe23e4ee640 1 -- 172.21.15.105:0/1346667650 shutdown_connections 2024-09-16T01:53:42.372 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.446+0000 7fe23e4ee640 1 -- 172.21.15.105:0/1346667650 wait complete. 2024-09-16T01:53:42.372 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.446+0000 7fe23e4ee640 1 Processor -- start 2024-09-16T01:53:42.372 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.446+0000 7fe23e4ee640 1 -- start start 2024-09-16T01:53:42.372 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.446+0000 7fe23e4ee640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe238106380 0x7fe23819b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:42.372 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.446+0000 7fe23e4ee640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe23810a930 con 0x7fe238106380 2024-09-16T01:53:42.372 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.447+0000 7fe237fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe238106380 0x7fe23819b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:42.372 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.447+0000 7fe237fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe238106380 0x7fe23819b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42512/0 (socket says 172.21.15.105:42512) 2024-09-16T01:53:42.372 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.447+0000 7fe237fff640 1 -- 172.21.15.105:0/2221027033 learned_addr learned my addr 172.21.15.105:0/2221027033 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:42.373 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.447+0000 7fe237fff640 1 -- 172.21.15.105:0/2221027033 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe23819bd50 con 0x7fe238106380 2024-09-16T01:53:42.373 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.447+0000 7fe237fff640 1 --2- 172.21.15.105:0/2221027033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe238106380 0x7fe23819b810 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe228009940 tx=0x7fe2280327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:42.373 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.447+0000 7fe235ffb640 1 -- 172.21.15.105:0/2221027033 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe228042070 con 0x7fe238106380 2024-09-16T01:53:42.373 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.447+0000 7fe235ffb640 1 -- 172.21.15.105:0/2221027033 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe228030d00 con 0x7fe238106380 2024-09-16T01:53:42.373 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.447+0000 7fe23e4ee640 1 -- 172.21.15.105:0/2221027033 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe23819bfe0 con 0x7fe238106380 2024-09-16T01:53:42.373 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.447+0000 7fe235ffb640 1 -- 172.21.15.105:0/2221027033 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe228037040 con 0x7fe238106380 2024-09-16T01:53:42.373 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.447+0000 7fe23e4ee640 1 -- 172.21.15.105:0/2221027033 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe23819ecd0 con 0x7fe238106380 2024-09-16T01:53:42.373 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.448+0000 7fe235ffb640 1 -- 172.21.15.105:0/2221027033 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7fe228045070 con 0x7fe238106380 2024-09-16T01:53:42.373 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.448+0000 7fe23e4ee640 1 -- 172.21.15.105:0/2221027033 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe23810aab0 con 0x7fe238106380 2024-09-16T01:53:42.373 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.448+0000 7fe235ffb640 1 --2- 172.21.15.105:0/2221027033 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe224039200 0x7fe22403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:42.373 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.448+0000 7fe235ffb640 1 -- 172.21.15.105:0/2221027033 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe22806c480 con 0x7fe238106380 2024-09-16T01:53:42.374 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.448+0000 7fe22ffff640 1 --2- 172.21.15.105:0/2221027033 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe224039200 0x7fe22403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:42.374 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.449+0000 7fe22ffff640 1 --2- 172.21.15.105:0/2221027033 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe224039200 0x7fe22403b6c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe2200098c0 tx=0x7fe220006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:42.374 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.453+0000 7fe235ffb640 1 -- 172.21.15.105:0/2221027033 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe228045360 con 0x7fe238106380 2024-09-16T01:53:42.374 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.646+0000 7fe23e4ee640 1 -- 172.21.15.105:0/2221027033 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fe238062630 con 0x7fe224039200 2024-09-16T01:53:42.374 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.745+0000 7fe235ffb640 1 -- 172.21.15.105:0/2221027033 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fe238062630 con 0x7fe224039200 2024-09-16T01:53:42.374 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.747+0000 7fe23e4ee640 1 -- 172.21.15.105:0/2221027033 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe224039200 msgr2=0x7fe22403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:42.374 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.747+0000 7fe23e4ee640 1 --2- 172.21.15.105:0/2221027033 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe224039200 0x7fe22403b6c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe2200098c0 tx=0x7fe220006db0 comp rx=0 tx=0).stop 2024-09-16T01:53:42.374 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.747+0000 7fe23e4ee640 1 -- 172.21.15.105:0/2221027033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe238106380 msgr2=0x7fe23819b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:42.374 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.747+0000 7fe23e4ee640 1 --2- 172.21.15.105:0/2221027033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe238106380 0x7fe23819b810 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe228009940 tx=0x7fe2280327b0 comp rx=0 tx=0).stop 2024-09-16T01:53:42.374 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.747+0000 7fe23e4ee640 1 -- 172.21.15.105:0/2221027033 shutdown_connections 2024-09-16T01:53:42.374 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.747+0000 7fe23e4ee640 1 --2- 172.21.15.105:0/2221027033 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe224039200 0x7fe22403b6c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:42.375 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.747+0000 7fe23e4ee640 1 --2- 172.21.15.105:0/2221027033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe238106380 0x7fe23819b810 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:42.375 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.747+0000 7fe23e4ee640 1 -- 172.21.15.105:0/2221027033 >> 172.21.15.105:0/2221027033 conn(0x7fe238101b10 msgr2=0x7fe238102ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:42.375 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.747+0000 7fe23e4ee640 1 -- 172.21.15.105:0/2221027033 shutdown_connections 2024-09-16T01:53:42.375 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:41.747+0000 7fe23e4ee640 1 -- 172.21.15.105:0/2221027033 wait complete. 2024-09-16T01:53:42.375 INFO:teuthology.orchestra.run.smithi105.stdout:Creating initial admin user... 2024-09-16T01:53:42.487 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:42 smithi105 ceph-mon[31197]: mgrmap e12: x(active, since 2s) 2024-09-16T01:53:42.487 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:42 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:53:42.487 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:42 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:53:42.488 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:42 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:53:43.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:43 smithi105 ceph-mon[31197]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:53:43.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:43 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:53:43.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:43 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:53:44.198 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$5sEwEkeCPgERiYmxpSJ89.3z9BJRyhmBjj7La0I0KqvhV46tRZqyu", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726451623, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-16T01:53:44.198 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.787+0000 7f18c03d2640 1 Processor -- start 2024-09-16T01:53:44.198 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.787+0000 7f18c03d2640 1 -- start start 2024-09-16T01:53:44.198 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.788+0000 7f18c03d2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18b81074d0 0x7f18b81078d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:44.198 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.788+0000 7f18c03d2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f18b8107ea0 con 0x7f18b81074d0 2024-09-16T01:53:44.198 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.788+0000 7f18be16e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18b81074d0 0x7f18b81078d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:44.198 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.788+0000 7f18be16e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18b81074d0 0x7f18b81078d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42516/0 (socket says 172.21.15.105:42516) 2024-09-16T01:53:44.198 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.788+0000 7f18be16e640 1 -- 172.21.15.105:0/2958679575 learned_addr learned my addr 172.21.15.105:0/2958679575 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:44.198 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.788+0000 7f18be16e640 1 -- 172.21.15.105:0/2958679575 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b81086d0 con 0x7f18b81074d0 2024-09-16T01:53:44.199 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.789+0000 7f18be16e640 1 --2- 172.21.15.105:0/2958679575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18b81074d0 0x7f18b81078d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f18b4009820 tx=0x7f18b402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae0941810b46a57 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:44.199 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.789+0000 7f18bd16c640 1 -- 172.21.15.105:0/2958679575 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18b4037070 con 0x7f18b81074d0 2024-09-16T01:53:44.199 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.789+0000 7f18bd16c640 1 -- 172.21.15.105:0/2958679575 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18b40328f0 con 0x7f18b81074d0 2024-09-16T01:53:44.199 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.789+0000 7f18bd16c640 1 -- 172.21.15.105:0/2958679575 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18b4030740 con 0x7f18b81074d0 2024-09-16T01:53:44.199 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.789+0000 7f18c03d2640 1 -- 172.21.15.105:0/2958679575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18b81074d0 msgr2=0x7f18b81078d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:44.199 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.789+0000 7f18c03d2640 1 --2- 172.21.15.105:0/2958679575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18b81074d0 0x7f18b81078d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f18b4009820 tx=0x7f18b402a270 comp rx=0 tx=0).stop 2024-09-16T01:53:44.199 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.789+0000 7f18c03d2640 1 -- 172.21.15.105:0/2958679575 shutdown_connections 2024-09-16T01:53:44.199 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.789+0000 7f18c03d2640 1 --2- 172.21.15.105:0/2958679575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18b81074d0 0x7f18b81078d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:44.199 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.789+0000 7f18c03d2640 1 -- 172.21.15.105:0/2958679575 >> 172.21.15.105:0/2958679575 conn(0x7f18b8102c20 msgr2=0x7f18b81050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:44.199 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.790+0000 7f18c03d2640 1 -- 172.21.15.105:0/2958679575 shutdown_connections 2024-09-16T01:53:44.199 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.790+0000 7f18c03d2640 1 -- 172.21.15.105:0/2958679575 wait complete. 2024-09-16T01:53:44.199 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.790+0000 7f18c03d2640 1 Processor -- start 2024-09-16T01:53:44.199 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.790+0000 7f18c03d2640 1 -- start start 2024-09-16T01:53:44.199 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.790+0000 7f18c03d2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18b81074d0 0x7f18b819b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:44.200 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.790+0000 7f18c03d2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f18b810ba80 con 0x7f18b81074d0 2024-09-16T01:53:44.200 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.790+0000 7f18be16e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18b81074d0 0x7f18b819b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:44.200 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.790+0000 7f18be16e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18b81074d0 0x7f18b819b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42522/0 (socket says 172.21.15.105:42522) 2024-09-16T01:53:44.200 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.790+0000 7f18be16e640 1 -- 172.21.15.105:0/2506873800 learned_addr learned my addr 172.21.15.105:0/2506873800 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:44.200 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.791+0000 7f18be16e640 1 -- 172.21.15.105:0/2506873800 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b819bce0 con 0x7f18b81074d0 2024-09-16T01:53:44.200 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.791+0000 7f18be16e640 1 --2- 172.21.15.105:0/2506873800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18b81074d0 0x7f18b819b7a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f18b4009940 tx=0x7f18b4030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:44.200 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.791+0000 7f18af7fe640 1 -- 172.21.15.105:0/2506873800 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18b4042070 con 0x7f18b81074d0 2024-09-16T01:53:44.200 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.791+0000 7f18af7fe640 1 -- 172.21.15.105:0/2506873800 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18b4030e30 con 0x7f18b81074d0 2024-09-16T01:53:44.200 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.791+0000 7f18c03d2640 1 -- 172.21.15.105:0/2506873800 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b819bf70 con 0x7f18b81074d0 2024-09-16T01:53:44.200 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.791+0000 7f18af7fe640 1 -- 172.21.15.105:0/2506873800 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18b4037040 con 0x7f18b81074d0 2024-09-16T01:53:44.200 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.791+0000 7f18c03d2640 1 -- 172.21.15.105:0/2506873800 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b819ec60 con 0x7f18b81074d0 2024-09-16T01:53:44.200 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.792+0000 7f18af7fe640 1 -- 172.21.15.105:0/2506873800 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f18b4045070 con 0x7f18b81074d0 2024-09-16T01:53:44.200 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.792+0000 7f18c03d2640 1 -- 172.21.15.105:0/2506873800 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18b810bc00 con 0x7f18b81074d0 2024-09-16T01:53:44.201 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.792+0000 7f18af7fe640 1 --2- 172.21.15.105:0/2506873800 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f18a4039200 0x7f18a403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:44.201 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.792+0000 7f18af7fe640 1 -- 172.21.15.105:0/2506873800 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f18b406c5d0 con 0x7f18b81074d0 2024-09-16T01:53:44.201 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.792+0000 7f18bd96d640 1 --2- 172.21.15.105:0/2506873800 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f18a4039200 0x7f18a403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:44.201 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.793+0000 7f18bd96d640 1 --2- 172.21.15.105:0/2506873800 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f18a4039200 0x7f18a403b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f18a80098c0 tx=0x7f18a8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:44.201 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.797+0000 7f18af7fe640 1 -- 172.21.15.105:0/2506873800 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f18b403add0 con 0x7f18b81074d0 2024-09-16T01:53:44.201 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:42.998+0000 7f18c03d2640 1 -- 172.21.15.105:0/2506873800 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f18b8193c90 con 0x7f18a4039200 2024-09-16T01:53:44.201 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:43.224+0000 7f18af7fe640 1 -- 172.21.15.105:0/2506873800 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f18b8193c90 con 0x7f18a4039200 2024-09-16T01:53:44.201 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:43.226+0000 7f18c03d2640 1 -- 172.21.15.105:0/2506873800 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f18a4039200 msgr2=0x7f18a403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:44.201 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:43.226+0000 7f18c03d2640 1 --2- 172.21.15.105:0/2506873800 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f18a4039200 0x7f18a403b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f18a80098c0 tx=0x7f18a8006db0 comp rx=0 tx=0).stop 2024-09-16T01:53:44.201 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:43.227+0000 7f18c03d2640 1 -- 172.21.15.105:0/2506873800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18b81074d0 msgr2=0x7f18b819b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:44.201 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:43.227+0000 7f18c03d2640 1 --2- 172.21.15.105:0/2506873800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18b81074d0 0x7f18b819b7a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f18b4009940 tx=0x7f18b4030cc0 comp rx=0 tx=0).stop 2024-09-16T01:53:44.201 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:43.227+0000 7f18c03d2640 1 -- 172.21.15.105:0/2506873800 shutdown_connections 2024-09-16T01:53:44.201 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:43.227+0000 7f18c03d2640 1 --2- 172.21.15.105:0/2506873800 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f18a4039200 0x7f18a403b6c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:44.202 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:43.227+0000 7f18c03d2640 1 --2- 172.21.15.105:0/2506873800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18b81074d0 0x7f18b819b7a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:44.202 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:43.227+0000 7f18c03d2640 1 -- 172.21.15.105:0/2506873800 >> 172.21.15.105:0/2506873800 conn(0x7f18b8102c20 msgr2=0x7f18b81038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:44.202 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:43.227+0000 7f18c03d2640 1 -- 172.21.15.105:0/2506873800 shutdown_connections 2024-09-16T01:53:44.202 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:43.227+0000 7f18c03d2640 1 -- 172.21.15.105:0/2506873800 wait complete. 2024-09-16T01:53:44.202 INFO:teuthology.orchestra.run.smithi105.stdout:Fetching dashboard port number... 2024-09-16T01:53:44.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:44 smithi105 ceph-mon[31197]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:53:45.544 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:45 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/1850672014' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-16T01:53:45.830 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout 8443 2024-09-16T01:53:45.830 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.038+0000 7f7473ac4640 1 Processor -- start 2024-09-16T01:53:45.830 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.039+0000 7f7473ac4640 1 -- start start 2024-09-16T01:53:45.830 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.039+0000 7f7473ac4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f746c108b90 0x7f746c108f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:45.830 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.039+0000 7f7473ac4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f746c1070d0 con 0x7f746c108b90 2024-09-16T01:53:45.831 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.039+0000 7f7471860640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f746c108b90 0x7f746c108f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:45.831 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.039+0000 7f7471860640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f746c108b90 0x7f746c108f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42528/0 (socket says 172.21.15.105:42528) 2024-09-16T01:53:45.831 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.039+0000 7f7471860640 1 -- 172.21.15.105:0/713665998 learned_addr learned my addr 172.21.15.105:0/713665998 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:45.831 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.039+0000 7f7471860640 1 -- 172.21.15.105:0/713665998 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f746c107250 con 0x7f746c108b90 2024-09-16T01:53:45.831 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.040+0000 7f7471860640 1 --2- 172.21.15.105:0/713665998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f746c108b90 0x7f746c108f90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7460009820 tx=0x7f746002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb62ee7faaa97ce server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:45.831 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.040+0000 7f747085e640 1 -- 172.21.15.105:0/713665998 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7460037070 con 0x7f746c108b90 2024-09-16T01:53:45.831 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.040+0000 7f747085e640 1 -- 172.21.15.105:0/713665998 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74600328f0 con 0x7f746c108b90 2024-09-16T01:53:45.831 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.040+0000 7f747085e640 1 -- 172.21.15.105:0/713665998 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7460030740 con 0x7f746c108b90 2024-09-16T01:53:45.831 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.041+0000 7f7473ac4640 1 -- 172.21.15.105:0/713665998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f746c108b90 msgr2=0x7f746c108f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:45.831 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.041+0000 7f7473ac4640 1 --2- 172.21.15.105:0/713665998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f746c108b90 0x7f746c108f90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7460009820 tx=0x7f746002a270 comp rx=0 tx=0).stop 2024-09-16T01:53:45.831 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.041+0000 7f7473ac4640 1 -- 172.21.15.105:0/713665998 shutdown_connections 2024-09-16T01:53:45.831 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.041+0000 7f7473ac4640 1 --2- 172.21.15.105:0/713665998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f746c108b90 0x7f746c108f90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:45.832 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.041+0000 7f7473ac4640 1 -- 172.21.15.105:0/713665998 >> 172.21.15.105:0/713665998 conn(0x7f746c102c20 msgr2=0x7f746c105080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:45.832 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.041+0000 7f7473ac4640 1 -- 172.21.15.105:0/713665998 shutdown_connections 2024-09-16T01:53:45.832 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.041+0000 7f7473ac4640 1 -- 172.21.15.105:0/713665998 wait complete. 2024-09-16T01:53:45.832 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.041+0000 7f7473ac4640 1 Processor -- start 2024-09-16T01:53:45.832 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.041+0000 7f7473ac4640 1 -- start start 2024-09-16T01:53:45.832 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.042+0000 7f7473ac4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f746c108b90 0x7f746c19b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:45.832 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.042+0000 7f7473ac4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f746c10ba60 con 0x7f746c108b90 2024-09-16T01:53:45.832 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.042+0000 7f7471860640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f746c108b90 0x7f746c19b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:45.832 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.042+0000 7f7471860640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f746c108b90 0x7f746c19b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42532/0 (socket says 172.21.15.105:42532) 2024-09-16T01:53:45.832 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.042+0000 7f7471860640 1 -- 172.21.15.105:0/1850672014 learned_addr learned my addr 172.21.15.105:0/1850672014 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:45.832 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.042+0000 7f7471860640 1 -- 172.21.15.105:0/1850672014 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f746c19bdd0 con 0x7f746c108b90 2024-09-16T01:53:45.832 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.042+0000 7f7471860640 1 --2- 172.21.15.105:0/1850672014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f746c108b90 0x7f746c19b890 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7460009940 tx=0x7f7460030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:45.832 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.042+0000 7f745effd640 1 -- 172.21.15.105:0/1850672014 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7460042070 con 0x7f746c108b90 2024-09-16T01:53:45.832 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.042+0000 7f745effd640 1 -- 172.21.15.105:0/1850672014 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7460030e30 con 0x7f746c108b90 2024-09-16T01:53:45.833 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.042+0000 7f7473ac4640 1 -- 172.21.15.105:0/1850672014 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f746c19c060 con 0x7f746c108b90 2024-09-16T01:53:45.833 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.042+0000 7f745effd640 1 -- 172.21.15.105:0/1850672014 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7460037040 con 0x7f746c108b90 2024-09-16T01:53:45.833 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.042+0000 7f7473ac4640 1 -- 172.21.15.105:0/1850672014 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f746c19ed50 con 0x7f746c108b90 2024-09-16T01:53:45.833 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.043+0000 7f745effd640 1 -- 172.21.15.105:0/1850672014 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f7460045070 con 0x7f746c108b90 2024-09-16T01:53:45.833 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.043+0000 7f7473ac4640 1 -- 172.21.15.105:0/1850672014 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f746c109010 con 0x7f746c108b90 2024-09-16T01:53:45.833 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.043+0000 7f745effd640 1 --2- 172.21.15.105:0/1850672014 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7458039200 0x7f745803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:45.833 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.043+0000 7f745effd640 1 -- 172.21.15.105:0/1850672014 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f746006c420 con 0x7f746c108b90 2024-09-16T01:53:45.833 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.044+0000 7f747105f640 1 --2- 172.21.15.105:0/1850672014 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7458039200 0x7f745803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:45.833 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.044+0000 7f747105f640 1 --2- 172.21.15.105:0/1850672014 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7458039200 0x7f745803b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f74680098c0 tx=0x7f7468006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:45.833 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.048+0000 7f745effd640 1 -- 172.21.15.105:0/1850672014 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f746004b170 con 0x7f746c108b90 2024-09-16T01:53:45.833 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.212+0000 7f7473ac4640 1 -- 172.21.15.105:0/1850672014 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f746c075e10 con 0x7f746c108b90 2024-09-16T01:53:45.833 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.213+0000 7f745effd640 1 -- 172.21.15.105:0/1850672014 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f746002fd90 con 0x7f746c108b90 2024-09-16T01:53:45.833 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.215+0000 7f7473ac4640 1 -- 172.21.15.105:0/1850672014 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7458039200 msgr2=0x7f745803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:45.834 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.215+0000 7f7473ac4640 1 --2- 172.21.15.105:0/1850672014 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7458039200 0x7f745803b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f74680098c0 tx=0x7f7468006db0 comp rx=0 tx=0).stop 2024-09-16T01:53:45.834 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.215+0000 7f7473ac4640 1 -- 172.21.15.105:0/1850672014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f746c108b90 msgr2=0x7f746c19b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:45.834 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.215+0000 7f7473ac4640 1 --2- 172.21.15.105:0/1850672014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f746c108b90 0x7f746c19b890 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7460009940 tx=0x7f7460030cc0 comp rx=0 tx=0).stop 2024-09-16T01:53:45.834 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.215+0000 7f7473ac4640 1 -- 172.21.15.105:0/1850672014 shutdown_connections 2024-09-16T01:53:45.834 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.215+0000 7f7473ac4640 1 --2- 172.21.15.105:0/1850672014 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7458039200 0x7f745803b6c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:45.834 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.215+0000 7f7473ac4640 1 --2- 172.21.15.105:0/1850672014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f746c108b90 0x7f746c19b890 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:45.834 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.215+0000 7f7473ac4640 1 -- 172.21.15.105:0/1850672014 >> 172.21.15.105:0/1850672014 conn(0x7f746c102c20 msgr2=0x7f746c103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:45.834 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.215+0000 7f7473ac4640 1 -- 172.21.15.105:0/1850672014 shutdown_connections 2024-09-16T01:53:45.834 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:45.215+0000 7f7473ac4640 1 -- 172.21.15.105:0/1850672014 wait complete. 2024-09-16T01:53:45.837 INFO:teuthology.orchestra.run.smithi105.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T01:53:45.838 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stdout disabled 2024-09-16T01:53:45.845 INFO:teuthology.orchestra.run.smithi105.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T01:53:45.845 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stdout inactive 2024-09-16T01:53:45.845 INFO:teuthology.orchestra.run.smithi105.stdout:firewalld.service is not enabled 2024-09-16T01:53:45.846 INFO:teuthology.orchestra.run.smithi105.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-16T01:53:45.847 INFO:teuthology.orchestra.run.smithi105.stdout:Ceph Dashboard is now available at: 2024-09-16T01:53:45.847 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:53:45.847 INFO:teuthology.orchestra.run.smithi105.stdout: URL: https://smithi105.front.sepia.ceph.com:8443/ 2024-09-16T01:53:45.847 INFO:teuthology.orchestra.run.smithi105.stdout: User: admin 2024-09-16T01:53:45.847 INFO:teuthology.orchestra.run.smithi105.stdout: Password: 1ra6ulx75p 2024-09-16T01:53:45.847 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:53:45.848 INFO:teuthology.orchestra.run.smithi105.stdout:Saving cluster configuration to /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/config directory 2024-09-16T01:53:46.788 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:46 smithi105 ceph-mon[31197]: mgrmap e13: x(active, since 6s) 2024-09-16T01:53:47.129 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.269+0000 7fe750942640 1 Processor -- start 2024-09-16T01:53:47.129 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.269+0000 7fe750942640 1 -- start start 2024-09-16T01:53:47.129 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.270+0000 7fe750942640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe74c107510 0x7fe74c107910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:47.129 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.270+0000 7fe750942640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c107ee0 con 0x7fe74c107510 2024-09-16T01:53:47.129 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.270+0000 7fe74a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe74c107510 0x7fe74c107910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:47.129 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.270+0000 7fe74a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe74c107510 0x7fe74c107910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42534/0 (socket says 172.21.15.105:42534) 2024-09-16T01:53:47.130 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.270+0000 7fe74a59c640 1 -- 172.21.15.105:0/1276560473 learned_addr learned my addr 172.21.15.105:0/1276560473 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:47.130 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.270+0000 7fe74a59c640 1 -- 172.21.15.105:0/1276560473 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe74c1086c0 con 0x7fe74c107510 2024-09-16T01:53:47.130 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.271+0000 7fe74a59c640 1 --2- 172.21.15.105:0/1276560473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe74c107510 0x7fe74c107910 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe73c009820 tx=0x7fe73c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ffff9b81cc8afd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:47.130 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.271+0000 7fe74959a640 1 -- 172.21.15.105:0/1276560473 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe73c037070 con 0x7fe74c107510 2024-09-16T01:53:47.130 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.271+0000 7fe74959a640 1 -- 172.21.15.105:0/1276560473 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe73c0328f0 con 0x7fe74c107510 2024-09-16T01:53:47.130 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.271+0000 7fe74959a640 1 -- 172.21.15.105:0/1276560473 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe73c030740 con 0x7fe74c107510 2024-09-16T01:53:47.130 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.271+0000 7fe750942640 1 -- 172.21.15.105:0/1276560473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe74c107510 msgr2=0x7fe74c107910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:47.130 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.271+0000 7fe750942640 1 --2- 172.21.15.105:0/1276560473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe74c107510 0x7fe74c107910 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe73c009820 tx=0x7fe73c02a270 comp rx=0 tx=0).stop 2024-09-16T01:53:47.130 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.272+0000 7fe750942640 1 -- 172.21.15.105:0/1276560473 shutdown_connections 2024-09-16T01:53:47.130 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.272+0000 7fe750942640 1 --2- 172.21.15.105:0/1276560473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe74c107510 0x7fe74c107910 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:47.130 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.272+0000 7fe750942640 1 -- 172.21.15.105:0/1276560473 >> 172.21.15.105:0/1276560473 conn(0x7fe74c102ca0 msgr2=0x7fe74c1050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:47.130 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.272+0000 7fe750942640 1 -- 172.21.15.105:0/1276560473 shutdown_connections 2024-09-16T01:53:47.130 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.272+0000 7fe750942640 1 -- 172.21.15.105:0/1276560473 wait complete. 2024-09-16T01:53:47.131 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.272+0000 7fe750942640 1 Processor -- start 2024-09-16T01:53:47.131 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.272+0000 7fe750942640 1 -- start start 2024-09-16T01:53:47.131 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.272+0000 7fe750942640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe74c107510 0x7fe74c19d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:47.131 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.272+0000 7fe750942640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c10ba70 con 0x7fe74c107510 2024-09-16T01:53:47.131 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.273+0000 7fe74a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe74c107510 0x7fe74c19d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:47.131 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.273+0000 7fe74a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe74c107510 0x7fe74c19d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42538/0 (socket says 172.21.15.105:42538) 2024-09-16T01:53:47.131 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.273+0000 7fe74a59c640 1 -- 172.21.15.105:0/2129011790 learned_addr learned my addr 172.21.15.105:0/2129011790 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:47.131 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.273+0000 7fe74a59c640 1 -- 172.21.15.105:0/2129011790 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe74c19deb0 con 0x7fe74c107510 2024-09-16T01:53:47.131 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.273+0000 7fe74a59c640 1 --2- 172.21.15.105:0/2129011790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe74c107510 0x7fe74c19d970 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe73c0097f0 tx=0x7fe73c030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:47.131 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.273+0000 7fe7437fe640 1 -- 172.21.15.105:0/2129011790 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe73c042070 con 0x7fe74c107510 2024-09-16T01:53:47.131 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.273+0000 7fe7437fe640 1 -- 172.21.15.105:0/2129011790 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe73c030d80 con 0x7fe74c107510 2024-09-16T01:53:47.131 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.273+0000 7fe750942640 1 -- 172.21.15.105:0/2129011790 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe74c19e140 con 0x7fe74c107510 2024-09-16T01:53:47.131 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.273+0000 7fe7437fe640 1 -- 172.21.15.105:0/2129011790 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe73c037040 con 0x7fe74c107510 2024-09-16T01:53:47.132 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.273+0000 7fe750942640 1 -- 172.21.15.105:0/2129011790 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe74c1a0e30 con 0x7fe74c107510 2024-09-16T01:53:47.132 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.274+0000 7fe7437fe640 1 -- 172.21.15.105:0/2129011790 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fe73c045070 con 0x7fe74c107510 2024-09-16T01:53:47.132 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.274+0000 7fe750942640 1 -- 172.21.15.105:0/2129011790 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe74c10bbf0 con 0x7fe74c107510 2024-09-16T01:53:47.132 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.274+0000 7fe7437fe640 1 --2- 172.21.15.105:0/2129011790 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe738039300 0x7fe73803b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:47.132 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.274+0000 7fe7437fe640 1 -- 172.21.15.105:0/2129011790 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe73c06c620 con 0x7fe74c107510 2024-09-16T01:53:47.132 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.274+0000 7fe749d9b640 1 --2- 172.21.15.105:0/2129011790 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe738039300 0x7fe73803b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:47.132 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.275+0000 7fe749d9b640 1 --2- 172.21.15.105:0/2129011790 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe738039300 0x7fe73803b7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe7340098c0 tx=0x7fe734006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:47.132 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.279+0000 7fe7437fe640 1 -- 172.21.15.105:0/2129011790 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe73c045360 con 0x7fe74c107510 2024-09-16T01:53:47.132 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.531+0000 7fe750942640 1 -- 172.21.15.105:0/2129011790 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fe74c107990 con 0x7fe74c107510 2024-09-16T01:53:47.132 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.534+0000 7fe7437fe640 1 -- 172.21.15.105:0/2129011790 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fe73c03ada0 con 0x7fe74c107510 2024-09-16T01:53:47.132 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-16T01:53:47.132 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.536+0000 7fe750942640 1 -- 172.21.15.105:0/2129011790 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe738039300 msgr2=0x7fe73803b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:47.132 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.536+0000 7fe750942640 1 --2- 172.21.15.105:0/2129011790 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe738039300 0x7fe73803b7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe7340098c0 tx=0x7fe734006db0 comp rx=0 tx=0).stop 2024-09-16T01:53:47.133 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.536+0000 7fe750942640 1 -- 172.21.15.105:0/2129011790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe74c107510 msgr2=0x7fe74c19d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:47.133 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.536+0000 7fe750942640 1 --2- 172.21.15.105:0/2129011790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe74c107510 0x7fe74c19d970 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe73c0097f0 tx=0x7fe73c030c70 comp rx=0 tx=0).stop 2024-09-16T01:53:47.133 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.536+0000 7fe750942640 1 -- 172.21.15.105:0/2129011790 shutdown_connections 2024-09-16T01:53:47.133 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.536+0000 7fe750942640 1 --2- 172.21.15.105:0/2129011790 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe738039300 0x7fe73803b7c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:47.133 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.536+0000 7fe750942640 1 --2- 172.21.15.105:0/2129011790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe74c107510 0x7fe74c19d970 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:47.133 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.536+0000 7fe750942640 1 -- 172.21.15.105:0/2129011790 >> 172.21.15.105:0/2129011790 conn(0x7fe74c102ca0 msgr2=0x7fe74c103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:47.133 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.536+0000 7fe750942640 1 -- 172.21.15.105:0/2129011790 shutdown_connections 2024-09-16T01:53:47.133 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-09-16T01:53:46.536+0000 7fe750942640 1 -- 172.21.15.105:0/2129011790 wait complete. 2024-09-16T01:53:47.133 INFO:teuthology.orchestra.run.smithi105.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-16T01:53:47.133 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:53:47.133 INFO:teuthology.orchestra.run.smithi105.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-16T01:53:47.133 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:53:47.133 INFO:teuthology.orchestra.run.smithi105.stdout:Or, if you are only running a single cluster on this host: 2024-09-16T01:53:47.134 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:53:47.134 INFO:teuthology.orchestra.run.smithi105.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-16T01:53:47.134 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:53:47.134 INFO:teuthology.orchestra.run.smithi105.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-16T01:53:47.134 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:53:47.134 INFO:teuthology.orchestra.run.smithi105.stdout: ceph telemetry on 2024-09-16T01:53:47.134 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:53:47.134 INFO:teuthology.orchestra.run.smithi105.stdout:For more information see: 2024-09-16T01:53:47.134 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:53:47.134 INFO:teuthology.orchestra.run.smithi105.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-16T01:53:47.134 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:53:47.134 INFO:teuthology.orchestra.run.smithi105.stdout:Bootstrap complete. 2024-09-16T01:53:47.170 INFO:tasks.cephadm:Fetching config... 2024-09-16T01:53:47.171 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-16T01:53:47.171 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-16T01:53:47.187 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-16T01:53:47.187 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-16T01:53:47.188 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-16T01:53:47.244 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-16T01:53:47.244 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-16T01:53:47.244 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-16T01:53:47.308 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-16T01:53:47.309 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-16T01:53:47.309 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-16T01:53:47.364 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-16T01:53:47.365 DEBUG:teuthology.orchestra.run.smithi105:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCgouhisPdPmtGs8Do6xwgs6E7OzQkXaE4kHAxuvLRN10DtEn+mMQn1/fus//iGgiac1gacxVpPsBxCPK/94KW2JHHx/8WC/h62K7pjitpjF54qe02HK2SJ4wsAv+ZUQU29ivyWvJdDM/Q+H5KNIkA7fyUJYQ1FsT8pgICFDqaWGlRTccveiM4/tccCjfSOeNKoPd7Ble3EBpnER3AjWWn4Uawy4ovzVYZKOmMRh2kB2pMf805zZm155VGOJQVSus1Xt4Sow1bd0Tc9IsZc9FOxt++Log43YtGB6Y4hYKC4YZn/3MdffJcRmSdQ+7rHcI0Ft3YnJl4wbk36QSKSnPpJRKadK5pgF8FKkm5ULEeYpMlN9jULquYWqHUZdGb8CvfQtv7TmiYAvnoazGyohpDMLIQp3lZdyfEXMazHvy7iegbxhWp0ccaEdGxvjaq1gI8VVPxRPEMA+huO84d5G2eO8+9OpuJDT3fnRRZDz4kQpgFyOoz9g2qoCygPbi8tICE= ceph-423b7a2a-73ce-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-16T01:53:47.441 INFO:teuthology.orchestra.run.smithi105.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCgouhisPdPmtGs8Do6xwgs6E7OzQkXaE4kHAxuvLRN10DtEn+mMQn1/fus//iGgiac1gacxVpPsBxCPK/94KW2JHHx/8WC/h62K7pjitpjF54qe02HK2SJ4wsAv+ZUQU29ivyWvJdDM/Q+H5KNIkA7fyUJYQ1FsT8pgICFDqaWGlRTccveiM4/tccCjfSOeNKoPd7Ble3EBpnER3AjWWn4Uawy4ovzVYZKOmMRh2kB2pMf805zZm155VGOJQVSus1Xt4Sow1bd0Tc9IsZc9FOxt++Log43YtGB6Y4hYKC4YZn/3MdffJcRmSdQ+7rHcI0Ft3YnJl4wbk36QSKSnPpJRKadK5pgF8FKkm5ULEeYpMlN9jULquYWqHUZdGb8CvfQtv7TmiYAvnoazGyohpDMLIQp3lZdyfEXMazHvy7iegbxhWp0ccaEdGxvjaq1gI8VVPxRPEMA+huO84d5G2eO8+9OpuJDT3fnRRZDz4kQpgFyOoz9g2qoCygPbi8tICE= ceph-423b7a2a-73ce-11ef-bceb-c7b262605968 2024-09-16T01:53:47.450 DEBUG:teuthology.orchestra.run.smithi137:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCgouhisPdPmtGs8Do6xwgs6E7OzQkXaE4kHAxuvLRN10DtEn+mMQn1/fus//iGgiac1gacxVpPsBxCPK/94KW2JHHx/8WC/h62K7pjitpjF54qe02HK2SJ4wsAv+ZUQU29ivyWvJdDM/Q+H5KNIkA7fyUJYQ1FsT8pgICFDqaWGlRTccveiM4/tccCjfSOeNKoPd7Ble3EBpnER3AjWWn4Uawy4ovzVYZKOmMRh2kB2pMf805zZm155VGOJQVSus1Xt4Sow1bd0Tc9IsZc9FOxt++Log43YtGB6Y4hYKC4YZn/3MdffJcRmSdQ+7rHcI0Ft3YnJl4wbk36QSKSnPpJRKadK5pgF8FKkm5ULEeYpMlN9jULquYWqHUZdGb8CvfQtv7TmiYAvnoazGyohpDMLIQp3lZdyfEXMazHvy7iegbxhWp0ccaEdGxvjaq1gI8VVPxRPEMA+huO84d5G2eO8+9OpuJDT3fnRRZDz4kQpgFyOoz9g2qoCygPbi8tICE= ceph-423b7a2a-73ce-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-16T01:53:47.505 INFO:teuthology.orchestra.run.smithi137.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCgouhisPdPmtGs8Do6xwgs6E7OzQkXaE4kHAxuvLRN10DtEn+mMQn1/fus//iGgiac1gacxVpPsBxCPK/94KW2JHHx/8WC/h62K7pjitpjF54qe02HK2SJ4wsAv+ZUQU29ivyWvJdDM/Q+H5KNIkA7fyUJYQ1FsT8pgICFDqaWGlRTccveiM4/tccCjfSOeNKoPd7Ble3EBpnER3AjWWn4Uawy4ovzVYZKOmMRh2kB2pMf805zZm155VGOJQVSus1Xt4Sow1bd0Tc9IsZc9FOxt++Log43YtGB6Y4hYKC4YZn/3MdffJcRmSdQ+7rHcI0Ft3YnJl4wbk36QSKSnPpJRKadK5pgF8FKkm5ULEeYpMlN9jULquYWqHUZdGb8CvfQtv7TmiYAvnoazGyohpDMLIQp3lZdyfEXMazHvy7iegbxhWp0ccaEdGxvjaq1gI8VVPxRPEMA+huO84d5G2eO8+9OpuJDT3fnRRZDz4kQpgFyOoz9g2qoCygPbi8tICE= ceph-423b7a2a-73ce-11ef-bceb-c7b262605968 2024-09-16T01:53:47.518 DEBUG:teuthology.orchestra.run.smithi190:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCgouhisPdPmtGs8Do6xwgs6E7OzQkXaE4kHAxuvLRN10DtEn+mMQn1/fus//iGgiac1gacxVpPsBxCPK/94KW2JHHx/8WC/h62K7pjitpjF54qe02HK2SJ4wsAv+ZUQU29ivyWvJdDM/Q+H5KNIkA7fyUJYQ1FsT8pgICFDqaWGlRTccveiM4/tccCjfSOeNKoPd7Ble3EBpnER3AjWWn4Uawy4ovzVYZKOmMRh2kB2pMf805zZm155VGOJQVSus1Xt4Sow1bd0Tc9IsZc9FOxt++Log43YtGB6Y4hYKC4YZn/3MdffJcRmSdQ+7rHcI0Ft3YnJl4wbk36QSKSnPpJRKadK5pgF8FKkm5ULEeYpMlN9jULquYWqHUZdGb8CvfQtv7TmiYAvnoazGyohpDMLIQp3lZdyfEXMazHvy7iegbxhWp0ccaEdGxvjaq1gI8VVPxRPEMA+huO84d5G2eO8+9OpuJDT3fnRRZDz4kQpgFyOoz9g2qoCygPbi8tICE= ceph-423b7a2a-73ce-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-16T01:53:47.569 INFO:teuthology.orchestra.run.smithi190.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCgouhisPdPmtGs8Do6xwgs6E7OzQkXaE4kHAxuvLRN10DtEn+mMQn1/fus//iGgiac1gacxVpPsBxCPK/94KW2JHHx/8WC/h62K7pjitpjF54qe02HK2SJ4wsAv+ZUQU29ivyWvJdDM/Q+H5KNIkA7fyUJYQ1FsT8pgICFDqaWGlRTccveiM4/tccCjfSOeNKoPd7Ble3EBpnER3AjWWn4Uawy4ovzVYZKOmMRh2kB2pMf805zZm155VGOJQVSus1Xt4Sow1bd0Tc9IsZc9FOxt++Log43YtGB6Y4hYKC4YZn/3MdffJcRmSdQ+7rHcI0Ft3YnJl4wbk36QSKSnPpJRKadK5pgF8FKkm5ULEeYpMlN9jULquYWqHUZdGb8CvfQtv7TmiYAvnoazGyohpDMLIQp3lZdyfEXMazHvy7iegbxhWp0ccaEdGxvjaq1gI8VVPxRPEMA+huO84d5G2eO8+9OpuJDT3fnRRZDz4kQpgFyOoz9g2qoCygPbi8tICE= ceph-423b7a2a-73ce-11ef-bceb-c7b262605968 2024-09-16T01:53:47.582 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-16T01:53:47.692 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:47 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2129011790' entity='client.admin' 2024-09-16T01:53:47.829 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:53:48.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.309+0000 7f73b3186640 1 -- 172.21.15.105:0/1875019156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73ac102160 msgr2=0x7f73ac104550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:48.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.309+0000 7f73b3186640 1 --2- 172.21.15.105:0/1875019156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73ac102160 0x7f73ac104550 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f73a0009880 tx=0x7f73a002a5d0 comp rx=0 tx=0).stop 2024-09-16T01:53:48.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.309+0000 7f73b3186640 1 -- 172.21.15.105:0/1875019156 shutdown_connections 2024-09-16T01:53:48.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.309+0000 7f73b3186640 1 --2- 172.21.15.105:0/1875019156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73ac102160 0x7f73ac104550 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:48.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.309+0000 7f73b3186640 1 -- 172.21.15.105:0/1875019156 >> 172.21.15.105:0/1875019156 conn(0x7f73ac0fbca0 msgr2=0x7f73ac0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:48.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.309+0000 7f73b3186640 1 -- 172.21.15.105:0/1875019156 shutdown_connections 2024-09-16T01:53:48.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.309+0000 7f73b3186640 1 -- 172.21.15.105:0/1875019156 wait complete. 2024-09-16T01:53:48.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.310+0000 7f73b3186640 1 Processor -- start 2024-09-16T01:53:48.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.310+0000 7f73b3186640 1 -- start start 2024-09-16T01:53:48.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.310+0000 7f73b3186640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73ac102160 0x7f73ac196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:48.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.310+0000 7f73b3186640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac1075e0 con 0x7f73ac102160 2024-09-16T01:53:48.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.311+0000 7f73b0f22640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73ac102160 0x7f73ac196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:48.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.311+0000 7f73b0f22640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73ac102160 0x7f73ac196c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50776/0 (socket says 172.21.15.105:50776) 2024-09-16T01:53:48.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.311+0000 7f73b0f22640 1 -- 172.21.15.105:0/2056446100 learned_addr learned my addr 172.21.15.105:0/2056446100 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:48.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.312+0000 7f73b0f22640 1 -- 172.21.15.105:0/2056446100 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73ac1971d0 con 0x7f73ac102160 2024-09-16T01:53:48.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.312+0000 7f73b0f22640 1 --2- 172.21.15.105:0/2056446100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73ac102160 0x7f73ac196c90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f73a0009230 tx=0x7f73a00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:48.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.312+0000 7f73a9ffb640 1 -- 172.21.15.105:0/2056446100 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73a003f070 con 0x7f73ac102160 2024-09-16T01:53:48.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.312+0000 7f73a9ffb640 1 -- 172.21.15.105:0/2056446100 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73a0003aa0 con 0x7f73ac102160 2024-09-16T01:53:48.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.312+0000 7f73b3186640 1 -- 172.21.15.105:0/2056446100 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73ac197460 con 0x7f73ac102160 2024-09-16T01:53:48.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.312+0000 7f73a9ffb640 1 -- 172.21.15.105:0/2056446100 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73a0035030 con 0x7f73ac102160 2024-09-16T01:53:48.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.312+0000 7f73b3186640 1 -- 172.21.15.105:0/2056446100 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73ac19a150 con 0x7f73ac102160 2024-09-16T01:53:48.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.313+0000 7f73a9ffb640 1 -- 172.21.15.105:0/2056446100 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f73a0042070 con 0x7f73ac102160 2024-09-16T01:53:48.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.313+0000 7f73a9ffb640 1 --2- 172.21.15.105:0/2056446100 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f739c039300 0x7f739c03b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:48.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.313+0000 7f73b3186640 1 -- 172.21.15.105:0/2056446100 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73ac107d60 con 0x7f73ac102160 2024-09-16T01:53:48.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.313+0000 7f73a9ffb640 1 -- 172.21.15.105:0/2056446100 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f73a006dfd0 con 0x7f73ac102160 2024-09-16T01:53:48.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.314+0000 7f73abfff640 1 --2- 172.21.15.105:0/2056446100 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f739c039300 0x7f739c03b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:48.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.314+0000 7f73abfff640 1 --2- 172.21.15.105:0/2056446100 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f739c039300 0x7f739c03b7c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f73980098c0 tx=0x7f7398006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:48.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.318+0000 7f73a9ffb640 1 -- 172.21.15.105:0/2056446100 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f73a002f300 con 0x7f73ac102160 2024-09-16T01:53:48.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.493+0000 7f73b3186640 1 -- 172.21.15.105:0/2056446100 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f73ac02d830 con 0x7f73ac102160 2024-09-16T01:53:48.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.495+0000 7f73a9ffb640 1 -- 172.21.15.105:0/2056446100 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f73a0039350 con 0x7f73ac102160 2024-09-16T01:53:48.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.497+0000 7f73b3186640 1 -- 172.21.15.105:0/2056446100 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f739c039300 msgr2=0x7f739c03b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:48.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.497+0000 7f73b3186640 1 --2- 172.21.15.105:0/2056446100 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f739c039300 0x7f739c03b7c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f73980098c0 tx=0x7f7398006db0 comp rx=0 tx=0).stop 2024-09-16T01:53:48.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.497+0000 7f73b3186640 1 -- 172.21.15.105:0/2056446100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73ac102160 msgr2=0x7f73ac196c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:48.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.497+0000 7f73b3186640 1 --2- 172.21.15.105:0/2056446100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73ac102160 0x7f73ac196c90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f73a0009230 tx=0x7f73a00048f0 comp rx=0 tx=0).stop 2024-09-16T01:53:48.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.498+0000 7f73b3186640 1 -- 172.21.15.105:0/2056446100 shutdown_connections 2024-09-16T01:53:48.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.498+0000 7f73b3186640 1 --2- 172.21.15.105:0/2056446100 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f739c039300 0x7f739c03b7c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:48.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.498+0000 7f73b3186640 1 --2- 172.21.15.105:0/2056446100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73ac102160 0x7f73ac196c90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:48.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.498+0000 7f73b3186640 1 -- 172.21.15.105:0/2056446100 >> 172.21.15.105:0/2056446100 conn(0x7f73ac0fbca0 msgr2=0x7f73ac0fc730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:48.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.498+0000 7f73b3186640 1 -- 172.21.15.105:0/2056446100 shutdown_connections 2024-09-16T01:53:48.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:48.498+0000 7f73b3186640 1 -- 172.21.15.105:0/2056446100 wait complete. 2024-09-16T01:53:49.497 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-16T01:53:49.497 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-16T01:53:49.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:49 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2056446100' entity='client.admin' 2024-09-16T01:53:49.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:49 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:53:49.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:49 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:53:49.745 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:49 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi105", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:53:49.745 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:49 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:53:49.745 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:49 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:53:49.745 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:49 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:53:49.969 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:53:50.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.427+0000 7fca58ff9640 1 -- 172.21.15.105:0/2796690203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca54100530 msgr2=0x7fca54100930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:50.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.427+0000 7fca58ff9640 1 --2- 172.21.15.105:0/2796690203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca54100530 0x7fca54100930 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fca44009880 tx=0x7fca4402a5d0 comp rx=0 tx=0).stop 2024-09-16T01:53:50.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.427+0000 7fca58ff9640 1 -- 172.21.15.105:0/2796690203 shutdown_connections 2024-09-16T01:53:50.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.427+0000 7fca58ff9640 1 --2- 172.21.15.105:0/2796690203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca54100530 0x7fca54100930 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:50.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.427+0000 7fca58ff9640 1 -- 172.21.15.105:0/2796690203 >> 172.21.15.105:0/2796690203 conn(0x7fca540fbca0 msgr2=0x7fca540fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:50.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.427+0000 7fca58ff9640 1 -- 172.21.15.105:0/2796690203 shutdown_connections 2024-09-16T01:53:50.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.427+0000 7fca58ff9640 1 -- 172.21.15.105:0/2796690203 wait complete. 2024-09-16T01:53:50.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.428+0000 7fca58ff9640 1 Processor -- start 2024-09-16T01:53:50.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.428+0000 7fca58ff9640 1 -- start start 2024-09-16T01:53:50.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.428+0000 7fca58ff9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca54100530 0x7fca54195c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:50.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.428+0000 7fca58ff9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fca541064f0 con 0x7fca54100530 2024-09-16T01:53:50.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.428+0000 7fca5259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca54100530 0x7fca54195c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:50.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.428+0000 7fca5259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca54100530 0x7fca54195c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50794/0 (socket says 172.21.15.105:50794) 2024-09-16T01:53:50.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.428+0000 7fca5259c640 1 -- 172.21.15.105:0/2476876093 learned_addr learned my addr 172.21.15.105:0/2476876093 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:50.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.429+0000 7fca5259c640 1 -- 172.21.15.105:0/2476876093 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca54196140 con 0x7fca54100530 2024-09-16T01:53:50.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.429+0000 7fca5259c640 1 --2- 172.21.15.105:0/2476876093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca54100530 0x7fca54195c00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fca44009230 tx=0x7fca440048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:50.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.429+0000 7fca4b7fe640 1 -- 172.21.15.105:0/2476876093 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca4403f070 con 0x7fca54100530 2024-09-16T01:53:50.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.429+0000 7fca58ff9640 1 -- 172.21.15.105:0/2476876093 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca541963d0 con 0x7fca54100530 2024-09-16T01:53:50.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.429+0000 7fca4b7fe640 1 -- 172.21.15.105:0/2476876093 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca44003aa0 con 0x7fca54100530 2024-09-16T01:53:50.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.429+0000 7fca4b7fe640 1 -- 172.21.15.105:0/2476876093 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca44035030 con 0x7fca54100530 2024-09-16T01:53:50.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.429+0000 7fca58ff9640 1 -- 172.21.15.105:0/2476876093 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca541990c0 con 0x7fca54100530 2024-09-16T01:53:50.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.430+0000 7fca4b7fe640 1 -- 172.21.15.105:0/2476876093 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fca44042070 con 0x7fca54100530 2024-09-16T01:53:50.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.430+0000 7fca4b7fe640 1 --2- 172.21.15.105:0/2476876093 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fca400392a0 0x7fca4003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:50.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.430+0000 7fca58ff9640 1 -- 172.21.15.105:0/2476876093 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca54106c70 con 0x7fca54100530 2024-09-16T01:53:50.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.430+0000 7fca4b7fe640 1 -- 172.21.15.105:0/2476876093 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fca4406e090 con 0x7fca54100530 2024-09-16T01:53:50.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.430+0000 7fca51d9b640 1 --2- 172.21.15.105:0/2476876093 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fca400392a0 0x7fca4003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:50.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.431+0000 7fca51d9b640 1 --2- 172.21.15.105:0/2476876093 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fca400392a0 0x7fca4003b760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fca3c0098c0 tx=0x7fca3c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:50.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.435+0000 7fca4b7fe640 1 -- 172.21.15.105:0/2476876093 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fca4402f300 con 0x7fca54100530 2024-09-16T01:53:50.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.611+0000 7fca58ff9640 1 -- 172.21.15.105:0/2476876093 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fca5402d830 con 0x7fca400392a0 2024-09-16T01:53:50.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.615+0000 7fca4b7fe640 1 -- 172.21.15.105:0/2476876093 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fca5402d830 con 0x7fca400392a0 2024-09-16T01:53:50.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.617+0000 7fca58ff9640 1 -- 172.21.15.105:0/2476876093 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fca400392a0 msgr2=0x7fca4003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:50.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.617+0000 7fca58ff9640 1 --2- 172.21.15.105:0/2476876093 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fca400392a0 0x7fca4003b760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fca3c0098c0 tx=0x7fca3c006db0 comp rx=0 tx=0).stop 2024-09-16T01:53:50.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.617+0000 7fca58ff9640 1 -- 172.21.15.105:0/2476876093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca54100530 msgr2=0x7fca54195c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:50.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.617+0000 7fca58ff9640 1 --2- 172.21.15.105:0/2476876093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca54100530 0x7fca54195c00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fca44009230 tx=0x7fca440048f0 comp rx=0 tx=0).stop 2024-09-16T01:53:50.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.617+0000 7fca58ff9640 1 -- 172.21.15.105:0/2476876093 shutdown_connections 2024-09-16T01:53:50.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.617+0000 7fca58ff9640 1 --2- 172.21.15.105:0/2476876093 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fca400392a0 0x7fca4003b760 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:50.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.617+0000 7fca58ff9640 1 --2- 172.21.15.105:0/2476876093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca54100530 0x7fca54195c00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:50.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.617+0000 7fca58ff9640 1 -- 172.21.15.105:0/2476876093 >> 172.21.15.105:0/2476876093 conn(0x7fca540fbca0 msgr2=0x7fca540fc750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:50.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.617+0000 7fca58ff9640 1 -- 172.21.15.105:0/2476876093 shutdown_connections 2024-09-16T01:53:50.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:50.617+0000 7fca58ff9640 1 -- 172.21.15.105:0/2476876093 wait complete. 2024-09-16T01:53:51.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:51 smithi105 ceph-mon[31197]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:53:51.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:51 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:53:51.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:51 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:53:51.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:51 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:53:51.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:51 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:53:51.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:51 smithi105 ceph-mon[31197]: Updating smithi105:/etc/ceph/ceph.conf 2024-09-16T01:53:51.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:51 smithi105 ceph-mon[31197]: Updating smithi105:/var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:53:51.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:51 smithi105 ceph-mon[31197]: Updating smithi105:/etc/ceph/ceph.client.admin.keyring 2024-09-16T01:53:51.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:51 smithi105 ceph-mon[31197]: Updating smithi105:/var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T01:53:51.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:51 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:53:51.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:51 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:53:51.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:51 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:53:51.881 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi137 2024-09-16T01:53:51.881 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-16T01:53:51.881 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/etc/ceph/ceph.conf 2024-09-16T01:53:51.900 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-16T01:53:51.900 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T01:53:51.958 INFO:tasks.cephadm:Adding host smithi137 to orchestrator... 2024-09-16T01:53:51.958 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph orch host add smithi137 2024-09-16T01:53:52.196 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:53:52.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:52.655+0000 7f3065735640 1 -- 172.21.15.105:0/2589854331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30601027e0 msgr2=0x7f3060102bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:52.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:52.655+0000 7f3065735640 1 --2- 172.21.15.105:0/2589854331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30601027e0 0x7f3060102bc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3050009880 tx=0x7f305002a670 comp rx=0 tx=0).stop 2024-09-16T01:53:52.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:52.655+0000 7f3065735640 1 -- 172.21.15.105:0/2589854331 shutdown_connections 2024-09-16T01:53:52.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:52.655+0000 7f3065735640 1 --2- 172.21.15.105:0/2589854331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30601027e0 0x7f3060102bc0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:52.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:52.655+0000 7f3065735640 1 -- 172.21.15.105:0/2589854331 >> 172.21.15.105:0/2589854331 conn(0x7f30600fc1a0 msgr2=0x7f30600fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:52.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:52.655+0000 7f3065735640 1 -- 172.21.15.105:0/2589854331 shutdown_connections 2024-09-16T01:53:52.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:52.655+0000 7f3065735640 1 -- 172.21.15.105:0/2589854331 wait complete. 2024-09-16T01:53:52.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:52.656+0000 7f3065735640 1 Processor -- start 2024-09-16T01:53:52.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:52.656+0000 7f3065735640 1 -- start start 2024-09-16T01:53:52.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:52.656+0000 7f3065735640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30601027e0 0x7f3060100440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:52.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:52.656+0000 7f3065735640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f306010bb90 con 0x7f30601027e0 2024-09-16T01:53:52.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:52.656+0000 7f305effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30601027e0 0x7f3060100440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:52.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:52.656+0000 7f305effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30601027e0 0x7f3060100440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50816/0 (socket says 172.21.15.105:50816) 2024-09-16T01:53:52.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:52.656+0000 7f305effd640 1 -- 172.21.15.105:0/4035932235 learned_addr learned my addr 172.21.15.105:0/4035932235 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:52.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:52.656+0000 7f305effd640 1 -- 172.21.15.105:0/4035932235 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3060102330 con 0x7f30601027e0 2024-09-16T01:53:52.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:52.657+0000 7f305effd640 1 --2- 172.21.15.105:0/4035932235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30601027e0 0x7f3060100440 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3050009230 tx=0x7f30500048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:52.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:52.657+0000 7f3047fff640 1 -- 172.21.15.105:0/4035932235 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f305003f070 con 0x7f30601027e0 2024-09-16T01:53:52.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:52.657+0000 7f3047fff640 1 -- 172.21.15.105:0/4035932235 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3050004d10 con 0x7f30601027e0 2024-09-16T01:53:52.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:52.657+0000 7f3065735640 1 -- 172.21.15.105:0/4035932235 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3060100980 con 0x7f30601027e0 2024-09-16T01:53:52.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:52.657+0000 7f3047fff640 1 -- 172.21.15.105:0/4035932235 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3050035030 con 0x7f30601027e0 2024-09-16T01:53:52.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:52.657+0000 7f3065735640 1 -- 172.21.15.105:0/4035932235 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3060100e40 con 0x7f30601027e0 2024-09-16T01:53:52.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:52.658+0000 7f3047fff640 1 -- 172.21.15.105:0/4035932235 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f3050042070 con 0x7f30601027e0 2024-09-16T01:53:52.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:52.658+0000 7f3047fff640 1 --2- 172.21.15.105:0/4035932235 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f30480392a0 0x7f304803b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:52.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:52.658+0000 7f3065735640 1 -- 172.21.15.105:0/4035932235 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f306006a760 con 0x7f30601027e0 2024-09-16T01:53:52.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:52.658+0000 7f3047fff640 1 -- 172.21.15.105:0/4035932235 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f305006cfe0 con 0x7f30601027e0 2024-09-16T01:53:52.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:52.658+0000 7f305e7fc640 1 --2- 172.21.15.105:0/4035932235 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f30480392a0 0x7f304803b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:52.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:52.658+0000 7f305e7fc640 1 --2- 172.21.15.105:0/4035932235 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f30480392a0 0x7f304803b760 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f30540098c0 tx=0x7f3054006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:52.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:52.662+0000 7f3047fff640 1 -- 172.21.15.105:0/4035932235 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3050039350 con 0x7f30601027e0 2024-09-16T01:53:52.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:52.834+0000 7f3065735640 1 -- 172.21.15.105:0/4035932235 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi137", "target": ["mon-mgr", ""]}) -- 0x7f3060101970 con 0x7f30480392a0 2024-09-16T01:53:54.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:53 smithi105 ceph-mon[31197]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi137", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:53:55.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:54 smithi105 ceph-mon[31197]: Deploying cephadm binary to smithi137 2024-09-16T01:53:55.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:55.994+0000 7f3047fff640 1 -- 172.21.15.105:0/4035932235 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f3060101970 con 0x7f30480392a0 2024-09-16T01:53:55.997 INFO:teuthology.orchestra.run.smithi105.stdout:Added host 'smithi137' with addr '172.21.15.137' 2024-09-16T01:53:55.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:55.996+0000 7f3065735640 1 -- 172.21.15.105:0/4035932235 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f30480392a0 msgr2=0x7f304803b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:55.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:55.996+0000 7f3065735640 1 --2- 172.21.15.105:0/4035932235 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f30480392a0 0x7f304803b760 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f30540098c0 tx=0x7f3054006db0 comp rx=0 tx=0).stop 2024-09-16T01:53:55.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:55.997+0000 7f3065735640 1 -- 172.21.15.105:0/4035932235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30601027e0 msgr2=0x7f3060100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:56.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:55.997+0000 7f3065735640 1 --2- 172.21.15.105:0/4035932235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30601027e0 0x7f3060100440 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3050009230 tx=0x7f30500048f0 comp rx=0 tx=0).stop 2024-09-16T01:53:56.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:55.997+0000 7f3065735640 1 -- 172.21.15.105:0/4035932235 shutdown_connections 2024-09-16T01:53:56.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:55.997+0000 7f3065735640 1 --2- 172.21.15.105:0/4035932235 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f30480392a0 0x7f304803b760 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:56.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:55.997+0000 7f3065735640 1 --2- 172.21.15.105:0/4035932235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30601027e0 0x7f3060100440 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:56.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:55.997+0000 7f3065735640 1 -- 172.21.15.105:0/4035932235 >> 172.21.15.105:0/4035932235 conn(0x7f30600fc1a0 msgr2=0x7f30601055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:56.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:55.997+0000 7f3065735640 1 -- 172.21.15.105:0/4035932235 shutdown_connections 2024-09-16T01:53:56.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:55.997+0000 7f3065735640 1 -- 172.21.15.105:0/4035932235 wait complete. 2024-09-16T01:53:56.640 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-16T01:53:56.879 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:53:57.246 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:56 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:53:57.246 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:56 smithi105 ceph-mon[31197]: Added host smithi137 2024-09-16T01:53:57.246 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:56 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:53:57.246 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:56 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:53:57.246 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:56 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:53:57.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.324+0000 7f4c91ce7640 1 -- 172.21.15.105:0/2652922863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c8c107660 msgr2=0x7f4c8c107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:57.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.324+0000 7f4c91ce7640 1 --2- 172.21.15.105:0/2652922863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c8c107660 0x7f4c8c107a40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c009880 tx=0x7f4c7c02a5d0 comp rx=0 tx=0).stop 2024-09-16T01:53:57.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.324+0000 7f4c91ce7640 1 -- 172.21.15.105:0/2652922863 shutdown_connections 2024-09-16T01:53:57.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.324+0000 7f4c91ce7640 1 --2- 172.21.15.105:0/2652922863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c8c107660 0x7f4c8c107a40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:57.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.324+0000 7f4c91ce7640 1 -- 172.21.15.105:0/2652922863 >> 172.21.15.105:0/2652922863 conn(0x7f4c8c0fd2b0 msgr2=0x7f4c8c0ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:57.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.324+0000 7f4c91ce7640 1 -- 172.21.15.105:0/2652922863 shutdown_connections 2024-09-16T01:53:57.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.324+0000 7f4c91ce7640 1 -- 172.21.15.105:0/2652922863 wait complete. 2024-09-16T01:53:57.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.325+0000 7f4c91ce7640 1 Processor -- start 2024-09-16T01:53:57.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.325+0000 7f4c91ce7640 1 -- start start 2024-09-16T01:53:57.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.325+0000 7f4c91ce7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c8c107660 0x7f4c8c197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:57.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.325+0000 7f4c91ce7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c10bb20 con 0x7f4c8c107660 2024-09-16T01:53:57.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.325+0000 7f4c8b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c8c107660 0x7f4c8c197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:57.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.325+0000 7f4c8b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c8c107660 0x7f4c8c197000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50840/0 (socket says 172.21.15.105:50840) 2024-09-16T01:53:57.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.325+0000 7f4c8b7fe640 1 -- 172.21.15.105:0/2934424183 learned_addr learned my addr 172.21.15.105:0/2934424183 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:57.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.326+0000 7f4c8b7fe640 1 -- 172.21.15.105:0/2934424183 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c8c197540 con 0x7f4c8c107660 2024-09-16T01:53:57.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.326+0000 7f4c8b7fe640 1 --2- 172.21.15.105:0/2934424183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c8c107660 0x7f4c8c197000 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c009230 tx=0x7f4c7c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:57.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.326+0000 7f4c88ff9640 1 -- 172.21.15.105:0/2934424183 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c7c03f070 con 0x7f4c8c107660 2024-09-16T01:53:57.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.326+0000 7f4c88ff9640 1 -- 172.21.15.105:0/2934424183 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c7c004d10 con 0x7f4c8c107660 2024-09-16T01:53:57.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.326+0000 7f4c91ce7640 1 -- 172.21.15.105:0/2934424183 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c8c197830 con 0x7f4c8c107660 2024-09-16T01:53:57.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.326+0000 7f4c88ff9640 1 -- 172.21.15.105:0/2934424183 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c7c035030 con 0x7f4c8c107660 2024-09-16T01:53:57.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.326+0000 7f4c91ce7640 1 -- 172.21.15.105:0/2934424183 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c8c19b8d0 con 0x7f4c8c107660 2024-09-16T01:53:57.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.327+0000 7f4c88ff9640 1 -- 172.21.15.105:0/2934424183 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f4c7c042070 con 0x7f4c8c107660 2024-09-16T01:53:57.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.327+0000 7f4c91ce7640 1 -- 172.21.15.105:0/2934424183 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c8c102d10 con 0x7f4c8c107660 2024-09-16T01:53:57.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.327+0000 7f4c88ff9640 1 --2- 172.21.15.105:0/2934424183 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f4c740392a0 0x7f4c7403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:57.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.327+0000 7f4c88ff9640 1 -- 172.21.15.105:0/2934424183 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4c7c06d130 con 0x7f4c8c107660 2024-09-16T01:53:57.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.327+0000 7f4c8affd640 1 --2- 172.21.15.105:0/2934424183 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f4c740392a0 0x7f4c7403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:57.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.328+0000 7f4c8affd640 1 --2- 172.21.15.105:0/2934424183 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f4c740392a0 0x7f4c7403b760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4c800098c0 tx=0x7f4c80006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:57.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.332+0000 7f4c88ff9640 1 -- 172.21.15.105:0/2934424183 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4c7c003b10 con 0x7f4c8c107660 2024-09-16T01:53:57.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.504+0000 7f4c91ce7640 1 -- 172.21.15.105:0/2934424183 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4c8c198360 con 0x7f4c740392a0 2024-09-16T01:53:57.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.505+0000 7f4c88ff9640 1 -- 172.21.15.105:0/2934424183 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7f4c8c198360 con 0x7f4c740392a0 2024-09-16T01:53:57.507 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:53:57.507 INFO:teuthology.orchestra.run.smithi105.stdout:[{"addr": "172.21.15.105", "hostname": "smithi105", "labels": [], "status": ""}, {"addr": "172.21.15.137", "hostname": "smithi137", "labels": [], "status": ""}] 2024-09-16T01:53:57.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.507+0000 7f4c91ce7640 1 -- 172.21.15.105:0/2934424183 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f4c740392a0 msgr2=0x7f4c7403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:57.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.507+0000 7f4c91ce7640 1 --2- 172.21.15.105:0/2934424183 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f4c740392a0 0x7f4c7403b760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4c800098c0 tx=0x7f4c80006db0 comp rx=0 tx=0).stop 2024-09-16T01:53:57.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.507+0000 7f4c91ce7640 1 -- 172.21.15.105:0/2934424183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c8c107660 msgr2=0x7f4c8c197000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:57.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.507+0000 7f4c91ce7640 1 --2- 172.21.15.105:0/2934424183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c8c107660 0x7f4c8c197000 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c009230 tx=0x7f4c7c0048f0 comp rx=0 tx=0).stop 2024-09-16T01:53:57.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.507+0000 7f4c91ce7640 1 -- 172.21.15.105:0/2934424183 shutdown_connections 2024-09-16T01:53:57.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.507+0000 7f4c91ce7640 1 --2- 172.21.15.105:0/2934424183 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f4c740392a0 0x7f4c7403b760 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:57.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.507+0000 7f4c91ce7640 1 --2- 172.21.15.105:0/2934424183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c8c107660 0x7f4c8c197000 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:57.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.507+0000 7f4c91ce7640 1 -- 172.21.15.105:0/2934424183 >> 172.21.15.105:0/2934424183 conn(0x7f4c8c0fd2b0 msgr2=0x7f4c8c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:57.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.507+0000 7f4c91ce7640 1 -- 172.21.15.105:0/2934424183 shutdown_connections 2024-09-16T01:53:57.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:57.507+0000 7f4c91ce7640 1 -- 172.21.15.105:0/2934424183 wait complete. 2024-09-16T01:53:58.067 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi190 2024-09-16T01:53:58.067 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-16T01:53:58.067 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/etc/ceph/ceph.conf 2024-09-16T01:53:58.087 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-16T01:53:58.087 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T01:53:58.145 INFO:tasks.cephadm:Adding host smithi190 to orchestrator... 2024-09-16T01:53:58.145 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph orch host add smithi190 2024-09-16T01:53:58.381 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:53:58.618 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:58 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:53:58.619 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:53:58 smithi105 ceph-mon[31197]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T01:53:58.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:58.815+0000 7fb2aa1c7640 1 -- 172.21.15.105:0/1316842296 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2a4100990 msgr2=0x7fb2a4100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:58.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:58.815+0000 7fb2aa1c7640 1 --2- 172.21.15.105:0/1316842296 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2a4100990 0x7fb2a4100d70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb294009c20 tx=0x7fb29402aa90 comp rx=0 tx=0).stop 2024-09-16T01:53:58.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:58.815+0000 7fb2aa1c7640 1 -- 172.21.15.105:0/1316842296 shutdown_connections 2024-09-16T01:53:58.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:58.815+0000 7fb2aa1c7640 1 --2- 172.21.15.105:0/1316842296 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2a4100990 0x7fb2a4100d70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:58.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:58.815+0000 7fb2aa1c7640 1 -- 172.21.15.105:0/1316842296 >> 172.21.15.105:0/1316842296 conn(0x7fb2a4076250 msgr2=0x7fb2a4076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:58.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:58.815+0000 7fb2aa1c7640 1 -- 172.21.15.105:0/1316842296 shutdown_connections 2024-09-16T01:53:58.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:58.815+0000 7fb2aa1c7640 1 -- 172.21.15.105:0/1316842296 wait complete. 2024-09-16T01:53:58.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:58.815+0000 7fb2aa1c7640 1 Processor -- start 2024-09-16T01:53:58.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:58.816+0000 7fb2aa1c7640 1 -- start start 2024-09-16T01:53:58.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:58.816+0000 7fb2aa1c7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2a4100990 0x7fb2a4197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:58.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:58.816+0000 7fb2aa1c7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a410aa40 con 0x7fb2a4100990 2024-09-16T01:53:58.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:58.816+0000 7fb2a37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2a4100990 0x7fb2a4197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:58.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:58.816+0000 7fb2a37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2a4100990 0x7fb2a4197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45492/0 (socket says 172.21.15.105:45492) 2024-09-16T01:53:58.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:58.816+0000 7fb2a37fe640 1 -- 172.21.15.105:0/32314227 learned_addr learned my addr 172.21.15.105:0/32314227 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:53:58.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:58.816+0000 7fb2a37fe640 1 -- 172.21.15.105:0/32314227 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a4198420 con 0x7fb2a4100990 2024-09-16T01:53:58.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:58.816+0000 7fb2a37fe640 1 --2- 172.21.15.105:0/32314227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2a4100990 0x7fb2a4197ee0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb294005e30 tx=0x7fb29402b010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:58.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:58.816+0000 7fb2a0ff9640 1 -- 172.21.15.105:0/32314227 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb294047070 con 0x7fb2a4100990 2024-09-16T01:53:58.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:58.816+0000 7fb2a0ff9640 1 -- 172.21.15.105:0/32314227 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb294003e20 con 0x7fb2a4100990 2024-09-16T01:53:58.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:58.816+0000 7fb2aa1c7640 1 -- 172.21.15.105:0/32314227 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2a41986b0 con 0x7fb2a4100990 2024-09-16T01:53:58.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:58.816+0000 7fb2a0ff9640 1 -- 172.21.15.105:0/32314227 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb294033050 con 0x7fb2a4100990 2024-09-16T01:53:58.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:58.817+0000 7fb2aa1c7640 1 -- 172.21.15.105:0/32314227 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2a4198b70 con 0x7fb2a4100990 2024-09-16T01:53:58.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:58.817+0000 7fb2a0ff9640 1 -- 172.21.15.105:0/32314227 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fb29404a070 con 0x7fb2a4100990 2024-09-16T01:53:58.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:58.817+0000 7fb2aa1c7640 1 -- 172.21.15.105:0/32314227 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2a4101c40 con 0x7fb2a4100990 2024-09-16T01:53:58.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:58.818+0000 7fb2a0ff9640 1 --2- 172.21.15.105:0/32314227 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb28c0392a0 0x7fb28c03b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:58.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:58.818+0000 7fb2a0ff9640 1 -- 172.21.15.105:0/32314227 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb29403e030 con 0x7fb2a4100990 2024-09-16T01:53:58.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:58.818+0000 7fb2a2ffd640 1 --2- 172.21.15.105:0/32314227 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb28c0392a0 0x7fb28c03b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:58.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:58.818+0000 7fb2a2ffd640 1 --2- 172.21.15.105:0/32314227 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb28c0392a0 0x7fb28c03b760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb2980098c0 tx=0x7fb298006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:58.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:58.822+0000 7fb2a0ff9640 1 -- 172.21.15.105:0/32314227 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb29402edf0 con 0x7fb2a4100990 2024-09-16T01:53:59.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:53:59.013+0000 7fb2aa1c7640 1 -- 172.21.15.105:0/32314227 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi190", "target": ["mon-mgr", ""]}) -- 0x7fb2a4100df0 con 0x7fb28c0392a0 2024-09-16T01:54:00.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:00 smithi105 ceph-mon[31197]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi190", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:54:00.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:00 smithi105 ceph-mon[31197]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:00.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:00 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:00.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:00 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:00.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:00 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:00.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:00 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:00.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:00 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi137", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:54:00.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:00 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:00.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:00 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:54:01.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:01 smithi105 ceph-mon[31197]: Deploying cephadm binary to smithi190 2024-09-16T01:54:01.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:01 smithi105 ceph-mon[31197]: Updating smithi137:/etc/ceph/ceph.conf 2024-09-16T01:54:01.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:01 smithi105 ceph-mon[31197]: Updating smithi137:/var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:54:02.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:02.143+0000 7fb2a0ff9640 1 -- 172.21.15.105:0/32314227 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fb2a4100df0 con 0x7fb28c0392a0 2024-09-16T01:54:02.145 INFO:teuthology.orchestra.run.smithi105.stdout:Added host 'smithi190' with addr '172.21.15.190' 2024-09-16T01:54:02.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:02.145+0000 7fb2aa1c7640 1 -- 172.21.15.105:0/32314227 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb28c0392a0 msgr2=0x7fb28c03b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:02.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:02.145+0000 7fb2aa1c7640 1 --2- 172.21.15.105:0/32314227 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb28c0392a0 0x7fb28c03b760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb2980098c0 tx=0x7fb298006db0 comp rx=0 tx=0).stop 2024-09-16T01:54:02.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:02.145+0000 7fb2aa1c7640 1 -- 172.21.15.105:0/32314227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2a4100990 msgr2=0x7fb2a4197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:02.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:02.145+0000 7fb2aa1c7640 1 --2- 172.21.15.105:0/32314227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2a4100990 0x7fb2a4197ee0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb294005e30 tx=0x7fb29402b010 comp rx=0 tx=0).stop 2024-09-16T01:54:02.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:02.145+0000 7fb2aa1c7640 1 -- 172.21.15.105:0/32314227 shutdown_connections 2024-09-16T01:54:02.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:02.145+0000 7fb2aa1c7640 1 --2- 172.21.15.105:0/32314227 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb28c0392a0 0x7fb28c03b760 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:02.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:02.145+0000 7fb2aa1c7640 1 --2- 172.21.15.105:0/32314227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2a4100990 0x7fb2a4197ee0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:02.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:02.145+0000 7fb2aa1c7640 1 -- 172.21.15.105:0/32314227 >> 172.21.15.105:0/32314227 conn(0x7fb2a4076250 msgr2=0x7fb2a41020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:02.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:02.146+0000 7fb2aa1c7640 1 -- 172.21.15.105:0/32314227 shutdown_connections 2024-09-16T01:54:02.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:02.146+0000 7fb2aa1c7640 1 -- 172.21.15.105:0/32314227 wait complete. 2024-09-16T01:54:02.439 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:02 smithi105 ceph-mon[31197]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-09-16T01:54:02.439 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:02 smithi105 ceph-mon[31197]: Updating smithi137:/var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T01:54:02.439 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:02 smithi105 ceph-mon[31197]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:02.439 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:02 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:02.439 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:02 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:02.439 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:02 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:02.439 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:02 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:02.439 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:02 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:54:02.736 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-16T01:54:02.974 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:54:03.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.414+0000 7f34b914c640 1 -- 172.21.15.105:0/3141727750 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34b4106590 msgr2=0x7f34b4106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:03.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.414+0000 7f34b914c640 1 --2- 172.21.15.105:0/3141727750 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34b4106590 0x7f34b4106970 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f34a4009880 tx=0x7f34a402a5d0 comp rx=0 tx=0).stop 2024-09-16T01:54:03.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.414+0000 7f34b914c640 1 -- 172.21.15.105:0/3141727750 shutdown_connections 2024-09-16T01:54:03.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.414+0000 7f34b914c640 1 --2- 172.21.15.105:0/3141727750 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34b4106590 0x7f34b4106970 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:03.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.414+0000 7f34b914c640 1 -- 172.21.15.105:0/3141727750 >> 172.21.15.105:0/3141727750 conn(0x7f34b40fc1a0 msgr2=0x7f34b40fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:03.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.414+0000 7f34b914c640 1 -- 172.21.15.105:0/3141727750 shutdown_connections 2024-09-16T01:54:03.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.414+0000 7f34b914c640 1 -- 172.21.15.105:0/3141727750 wait complete. 2024-09-16T01:54:03.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.414+0000 7f34b914c640 1 Processor -- start 2024-09-16T01:54:03.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.415+0000 7f34b914c640 1 -- start start 2024-09-16T01:54:03.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.415+0000 7f34b914c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34b4106590 0x7f34b40fc650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:03.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.415+0000 7f34b914c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f34b410aa70 con 0x7f34b4106590 2024-09-16T01:54:03.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.415+0000 7f34b2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34b4106590 0x7f34b40fc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:03.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.415+0000 7f34b2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34b4106590 0x7f34b40fc650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45514/0 (socket says 172.21.15.105:45514) 2024-09-16T01:54:03.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.415+0000 7f34b2d9d640 1 -- 172.21.15.105:0/1499247780 learned_addr learned my addr 172.21.15.105:0/1499247780 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:54:03.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.415+0000 7f34b2d9d640 1 -- 172.21.15.105:0/1499247780 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b40fe540 con 0x7f34b4106590 2024-09-16T01:54:03.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.415+0000 7f34b2d9d640 1 --2- 172.21.15.105:0/1499247780 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34b4106590 0x7f34b40fc650 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f34a4009230 tx=0x7f34a40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:03.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.416+0000 7f34abfff640 1 -- 172.21.15.105:0/1499247780 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f34a403f070 con 0x7f34b4106590 2024-09-16T01:54:03.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.416+0000 7f34abfff640 1 -- 172.21.15.105:0/1499247780 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f34a4004d10 con 0x7f34b4106590 2024-09-16T01:54:03.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.416+0000 7f34b914c640 1 -- 172.21.15.105:0/1499247780 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34b40fcb90 con 0x7f34b4106590 2024-09-16T01:54:03.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.416+0000 7f34abfff640 1 -- 172.21.15.105:0/1499247780 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f34a4035030 con 0x7f34b4106590 2024-09-16T01:54:03.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.416+0000 7f34b914c640 1 -- 172.21.15.105:0/1499247780 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34b40fd050 con 0x7f34b4106590 2024-09-16T01:54:03.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.416+0000 7f34abfff640 1 -- 172.21.15.105:0/1499247780 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f34a4042070 con 0x7f34b4106590 2024-09-16T01:54:03.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.417+0000 7f34b914c640 1 -- 172.21.15.105:0/1499247780 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34b4101c00 con 0x7f34b4106590 2024-09-16T01:54:03.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.417+0000 7f34abfff640 1 --2- 172.21.15.105:0/1499247780 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f34a00392a0 0x7f34a003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:03.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.417+0000 7f34abfff640 1 -- 172.21.15.105:0/1499247780 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f34a406cfe0 con 0x7f34b4106590 2024-09-16T01:54:03.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.417+0000 7f34b259c640 1 --2- 172.21.15.105:0/1499247780 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f34a00392a0 0x7f34a003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:03.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.417+0000 7f34b259c640 1 --2- 172.21.15.105:0/1499247780 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f34a00392a0 0x7f34a003b760 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f349c0098c0 tx=0x7f349c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:03.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.422+0000 7f34abfff640 1 -- 172.21.15.105:0/1499247780 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f34a4039350 con 0x7f34b4106590 2024-09-16T01:54:03.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.594+0000 7f34b914c640 1 -- 172.21.15.105:0/1499247780 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f34b40fdb80 con 0x7f34a00392a0 2024-09-16T01:54:03.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.595+0000 7f34abfff640 1 -- 172.21.15.105:0/1499247780 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7f34b40fdb80 con 0x7f34a00392a0 2024-09-16T01:54:03.597 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:54:03.597 INFO:teuthology.orchestra.run.smithi105.stdout:[{"addr": "172.21.15.105", "hostname": "smithi105", "labels": [], "status": ""}, {"addr": "172.21.15.137", "hostname": "smithi137", "labels": [], "status": ""}, {"addr": "172.21.15.190", "hostname": "smithi190", "labels": [], "status": ""}] 2024-09-16T01:54:03.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.597+0000 7f34b914c640 1 -- 172.21.15.105:0/1499247780 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f34a00392a0 msgr2=0x7f34a003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:03.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.597+0000 7f34b914c640 1 --2- 172.21.15.105:0/1499247780 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f34a00392a0 0x7f34a003b760 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f349c0098c0 tx=0x7f349c006db0 comp rx=0 tx=0).stop 2024-09-16T01:54:03.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.597+0000 7f34b914c640 1 -- 172.21.15.105:0/1499247780 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34b4106590 msgr2=0x7f34b40fc650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:03.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.597+0000 7f34b914c640 1 --2- 172.21.15.105:0/1499247780 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34b4106590 0x7f34b40fc650 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f34a4009230 tx=0x7f34a40048f0 comp rx=0 tx=0).stop 2024-09-16T01:54:03.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.597+0000 7f34b914c640 1 -- 172.21.15.105:0/1499247780 shutdown_connections 2024-09-16T01:54:03.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.597+0000 7f34b914c640 1 --2- 172.21.15.105:0/1499247780 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f34a00392a0 0x7f34a003b760 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:03.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.597+0000 7f34b914c640 1 --2- 172.21.15.105:0/1499247780 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34b4106590 0x7f34b40fc650 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:03.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.597+0000 7f34b914c640 1 -- 172.21.15.105:0/1499247780 >> 172.21.15.105:0/1499247780 conn(0x7f34b40fc1a0 msgr2=0x7f34b4069ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:03.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.597+0000 7f34b914c640 1 -- 172.21.15.105:0/1499247780 shutdown_connections 2024-09-16T01:54:03.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:03.597+0000 7f34b914c640 1 -- 172.21.15.105:0/1499247780 wait complete. 2024-09-16T01:54:03.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:03 smithi105 ceph-mon[31197]: Added host smithi190 2024-09-16T01:54:03.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:03 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:03.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:03 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:04.192 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-16T01:54:04.192 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-16T01:54:04.429 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:54:04.749 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:04 smithi105 ceph-mon[31197]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:04.749 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:04 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:04.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:04.828+0000 7f5e20021640 1 -- 172.21.15.105:0/4178094931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e18107690 msgr2=0x7f5e18107a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:04.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:04.828+0000 7f5e20021640 1 --2- 172.21.15.105:0/4178094931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e18107690 0x7f5e18107a70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5e08009c20 tx=0x7f5e0802aa90 comp rx=0 tx=0).stop 2024-09-16T01:54:04.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:04.829+0000 7f5e20021640 1 -- 172.21.15.105:0/4178094931 shutdown_connections 2024-09-16T01:54:04.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:04.829+0000 7f5e20021640 1 --2- 172.21.15.105:0/4178094931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e18107690 0x7f5e18107a70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:04.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:04.829+0000 7f5e20021640 1 -- 172.21.15.105:0/4178094931 >> 172.21.15.105:0/4178094931 conn(0x7f5e18076250 msgr2=0x7f5e18076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:04.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:04.829+0000 7f5e20021640 1 -- 172.21.15.105:0/4178094931 shutdown_connections 2024-09-16T01:54:04.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:04.829+0000 7f5e20021640 1 -- 172.21.15.105:0/4178094931 wait complete. 2024-09-16T01:54:04.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:04.829+0000 7f5e20021640 1 Processor -- start 2024-09-16T01:54:04.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:04.829+0000 7f5e20021640 1 -- start start 2024-09-16T01:54:04.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:04.829+0000 7f5e20021640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e18107690 0x7f5e18199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:04.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:04.829+0000 7f5e20021640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1810bb90 con 0x7f5e18107690 2024-09-16T01:54:04.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:04.829+0000 7f5e1ddbd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e18107690 0x7f5e18199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:04.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:04.830+0000 7f5e1ddbd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e18107690 0x7f5e18199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45534/0 (socket says 172.21.15.105:45534) 2024-09-16T01:54:04.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:04.830+0000 7f5e1ddbd640 1 -- 172.21.15.105:0/1096036964 learned_addr learned my addr 172.21.15.105:0/1096036964 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:54:04.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:04.830+0000 7f5e1ddbd640 1 -- 172.21.15.105:0/1096036964 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e18199840 con 0x7f5e18107690 2024-09-16T01:54:04.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:04.830+0000 7f5e1ddbd640 1 --2- 172.21.15.105:0/1096036964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e18107690 0x7f5e18199300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5e08005e30 tx=0x7f5e0802b010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:04.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:04.830+0000 7f5e16ffd640 1 -- 172.21.15.105:0/1096036964 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e08047070 con 0x7f5e18107690 2024-09-16T01:54:04.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:04.830+0000 7f5e20021640 1 -- 172.21.15.105:0/1096036964 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e18199ad0 con 0x7f5e18107690 2024-09-16T01:54:04.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:04.830+0000 7f5e16ffd640 1 -- 172.21.15.105:0/1096036964 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e08003e20 con 0x7f5e18107690 2024-09-16T01:54:04.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:04.830+0000 7f5e16ffd640 1 -- 172.21.15.105:0/1096036964 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e08033050 con 0x7f5e18107690 2024-09-16T01:54:04.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:04.830+0000 7f5e20021640 1 -- 172.21.15.105:0/1096036964 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e18199ed0 con 0x7f5e18107690 2024-09-16T01:54:04.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:04.831+0000 7f5e20021640 1 -- 172.21.15.105:0/1096036964 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e18102d30 con 0x7f5e18107690 2024-09-16T01:54:04.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:04.831+0000 7f5e16ffd640 1 -- 172.21.15.105:0/1096036964 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f5e0804a070 con 0x7f5e18107690 2024-09-16T01:54:04.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:04.831+0000 7f5e16ffd640 1 --2- 172.21.15.105:0/1096036964 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5e000392a0 0x7f5e0003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:04.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:04.831+0000 7f5e16ffd640 1 -- 172.21.15.105:0/1096036964 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5e0803e030 con 0x7f5e18107690 2024-09-16T01:54:04.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:04.831+0000 7f5e1d5bc640 1 --2- 172.21.15.105:0/1096036964 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5e000392a0 0x7f5e0003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:04.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:04.832+0000 7f5e1d5bc640 1 --2- 172.21.15.105:0/1096036964 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5e000392a0 0x7f5e0003b760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c0098c0 tx=0x7f5e0c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:04.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:04.836+0000 7f5e16ffd640 1 -- 172.21.15.105:0/1096036964 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5e0802edf0 con 0x7f5e18107690 2024-09-16T01:54:05.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:05.004+0000 7f5e20021640 1 -- 172.21.15.105:0/1096036964 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f5e1819a6f0 con 0x7f5e18107690 2024-09-16T01:54:05.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:05.497+0000 7f5e16ffd640 1 -- 172.21.15.105:0/1096036964 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f5e1819a6f0 con 0x7f5e18107690 2024-09-16T01:54:05.499 INFO:teuthology.orchestra.run.smithi105.stderr:adjusted tunables profile to default 2024-09-16T01:54:05.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:05.499+0000 7f5e20021640 1 -- 172.21.15.105:0/1096036964 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5e000392a0 msgr2=0x7f5e0003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:05.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:05.499+0000 7f5e20021640 1 --2- 172.21.15.105:0/1096036964 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5e000392a0 0x7f5e0003b760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c0098c0 tx=0x7f5e0c006db0 comp rx=0 tx=0).stop 2024-09-16T01:54:05.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:05.499+0000 7f5e20021640 1 -- 172.21.15.105:0/1096036964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e18107690 msgr2=0x7f5e18199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:05.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:05.499+0000 7f5e20021640 1 --2- 172.21.15.105:0/1096036964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e18107690 0x7f5e18199300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5e08005e30 tx=0x7f5e0802b010 comp rx=0 tx=0).stop 2024-09-16T01:54:05.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:05.499+0000 7f5e20021640 1 -- 172.21.15.105:0/1096036964 shutdown_connections 2024-09-16T01:54:05.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:05.499+0000 7f5e20021640 1 --2- 172.21.15.105:0/1096036964 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5e000392a0 0x7f5e0003b760 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:05.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:05.499+0000 7f5e20021640 1 --2- 172.21.15.105:0/1096036964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e18107690 0x7f5e18199300 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:05.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:05.499+0000 7f5e20021640 1 -- 172.21.15.105:0/1096036964 >> 172.21.15.105:0/1096036964 conn(0x7f5e18076250 msgr2=0x7f5e180fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:05.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:05.499+0000 7f5e20021640 1 -- 172.21.15.105:0/1096036964 shutdown_connections 2024-09-16T01:54:05.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:05.499+0000 7f5e20021640 1 -- 172.21.15.105:0/1096036964 wait complete. 2024-09-16T01:54:05.782 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:05 smithi105 ceph-mon[31197]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T01:54:05.782 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:05 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/1096036964' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-16T01:54:06.115 INFO:tasks.cephadm:Adding mon.a on smithi105 2024-09-16T01:54:06.115 INFO:tasks.cephadm:Adding mon.b on smithi137 2024-09-16T01:54:06.115 INFO:tasks.cephadm:Adding mon.c on smithi190 2024-09-16T01:54:06.115 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi105:172.21.15.105=a;smithi137:172.21.15.137=b;smithi190:172.21.15.190=c' 2024-09-16T01:54:06.356 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T01:54:06.401 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T01:54:06.750 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:06 smithi105 ceph-mon[31197]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:06.750 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:06 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/1096036964' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-16T01:54:06.750 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:06 smithi105 ceph-mon[31197]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T01:54:06.750 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:06 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:06.750 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:06 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:06.750 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:06 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:06.750 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:06 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:06.750 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:06 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi190", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:54:06.750 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:06 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:06.750 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:06 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:54:06.940 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:06.913+0000 7ff06f2ef640 1 -- 172.21.15.190:0/3822655624 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0680ff8e0 msgr2=0x7ff0680ffce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:06.940 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:06.913+0000 7ff06f2ef640 1 --2- 172.21.15.190:0/3822655624 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0680ff8e0 0x7ff0680ffce0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff064009880 tx=0x7ff06402a5d0 comp rx=0 tx=0).stop 2024-09-16T01:54:06.940 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:06.913+0000 7ff06f2ef640 1 -- 172.21.15.190:0/3822655624 shutdown_connections 2024-09-16T01:54:06.940 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:06.913+0000 7ff06f2ef640 1 --2- 172.21.15.190:0/3822655624 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0680ff8e0 0x7ff0680ffce0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:06.940 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:06.913+0000 7ff06f2ef640 1 -- 172.21.15.190:0/3822655624 >> 172.21.15.190:0/3822655624 conn(0x7ff0680fb430 msgr2=0x7ff0680fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:06.940 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:06.913+0000 7ff06f2ef640 1 -- 172.21.15.190:0/3822655624 shutdown_connections 2024-09-16T01:54:06.940 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:06.913+0000 7ff06f2ef640 1 -- 172.21.15.190:0/3822655624 wait complete. 2024-09-16T01:54:06.941 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:06.914+0000 7ff06f2ef640 1 Processor -- start 2024-09-16T01:54:06.941 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:06.914+0000 7ff06f2ef640 1 -- start start 2024-09-16T01:54:06.941 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:06.914+0000 7ff06f2ef640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0680ff8e0 0x7ff068196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:06.941 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:06.914+0000 7ff06f2ef640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff0681075e0 con 0x7ff0680ff8e0 2024-09-16T01:54:06.941 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:06.915+0000 7ff06d08b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0680ff8e0 0x7ff068196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:06.941 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:06.915+0000 7ff06d08b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0680ff8e0 0x7ff068196cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.190:40364/0 (socket says 172.21.15.190:40364) 2024-09-16T01:54:06.941 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:06.915+0000 7ff06d08b640 1 -- 172.21.15.190:0/1671322463 learned_addr learned my addr 172.21.15.190:0/1671322463 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-09-16T01:54:06.941 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:06.915+0000 7ff06d08b640 1 -- 172.21.15.190:0/1671322463 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff068197230 con 0x7ff0680ff8e0 2024-09-16T01:54:06.941 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:06.915+0000 7ff06d08b640 1 --2- 172.21.15.190:0/1671322463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0680ff8e0 0x7ff068196cf0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff064005ec0 tx=0x7ff0640048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:06.941 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:06.915+0000 7ff05e7fc640 1 -- 172.21.15.190:0/1671322463 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff06403f070 con 0x7ff0680ff8e0 2024-09-16T01:54:06.941 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:06.915+0000 7ff05e7fc640 1 -- 172.21.15.190:0/1671322463 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff064003aa0 con 0x7ff0680ff8e0 2024-09-16T01:54:06.941 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:06.915+0000 7ff05e7fc640 1 -- 172.21.15.190:0/1671322463 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff064035030 con 0x7ff0680ff8e0 2024-09-16T01:54:06.941 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:06.915+0000 7ff06f2ef640 1 -- 172.21.15.190:0/1671322463 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0681974c0 con 0x7ff0680ff8e0 2024-09-16T01:54:06.942 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:06.916+0000 7ff06f2ef640 1 -- 172.21.15.190:0/1671322463 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff06819a1a0 con 0x7ff0680ff8e0 2024-09-16T01:54:06.942 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:06.916+0000 7ff05e7fc640 1 -- 172.21.15.190:0/1671322463 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7ff064042070 con 0x7ff0680ff8e0 2024-09-16T01:54:06.942 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:06.916+0000 7ff05e7fc640 1 --2- 172.21.15.190:0/1671322463 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff0540392a0 0x7ff05403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:06.942 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:06.916+0000 7ff06f2ef640 1 -- 172.21.15.190:0/1671322463 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff068107d60 con 0x7ff0680ff8e0 2024-09-16T01:54:06.942 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:06.917+0000 7ff05e7fc640 1 -- 172.21.15.190:0/1671322463 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff06406d350 con 0x7ff0680ff8e0 2024-09-16T01:54:06.942 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:06.917+0000 7ff06c88a640 1 --2- 172.21.15.190:0/1671322463 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff0540392a0 0x7ff05403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:06.942 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:06.917+0000 7ff06c88a640 1 --2- 172.21.15.190:0/1671322463 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff0540392a0 0x7ff05403b760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff0580098c0 tx=0x7ff058006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:06.942 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:06.921+0000 7ff05e7fc640 1 -- 172.21.15.190:0/1671322463 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff0640433b0 con 0x7ff0680ff8e0 2024-09-16T01:54:07.112 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:07.110+0000 7ff06f2ef640 1 -- 172.21.15.190:0/1671322463 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi105:172.21.15.105=a;smithi137:172.21.15.137=b;smithi190:172.21.15.190=c", "target": ["mon-mgr", ""]}) -- 0x7ff06802d8f0 con 0x7ff0540392a0 2024-09-16T01:54:07.117 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:07.115+0000 7ff05e7fc640 1 -- 172.21.15.190:0/1671322463 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff06802d8f0 con 0x7ff0540392a0 2024-09-16T01:54:07.117 INFO:teuthology.orchestra.run.smithi190.stdout:Scheduled mon update... 2024-09-16T01:54:07.120 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:07.117+0000 7ff06f2ef640 1 -- 172.21.15.190:0/1671322463 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff0540392a0 msgr2=0x7ff05403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:07.120 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:07.117+0000 7ff06f2ef640 1 --2- 172.21.15.190:0/1671322463 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff0540392a0 0x7ff05403b760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff0580098c0 tx=0x7ff058006db0 comp rx=0 tx=0).stop 2024-09-16T01:54:07.120 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:07.117+0000 7ff06f2ef640 1 -- 172.21.15.190:0/1671322463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0680ff8e0 msgr2=0x7ff068196cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:07.120 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:07.117+0000 7ff06f2ef640 1 --2- 172.21.15.190:0/1671322463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0680ff8e0 0x7ff068196cf0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff064005ec0 tx=0x7ff0640048f0 comp rx=0 tx=0).stop 2024-09-16T01:54:07.120 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:07.117+0000 7ff06f2ef640 1 -- 172.21.15.190:0/1671322463 shutdown_connections 2024-09-16T01:54:07.120 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:07.117+0000 7ff06f2ef640 1 --2- 172.21.15.190:0/1671322463 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff0540392a0 0x7ff05403b760 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:07.120 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:07.117+0000 7ff06f2ef640 1 --2- 172.21.15.190:0/1671322463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0680ff8e0 0x7ff068196cf0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:07.120 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:07.117+0000 7ff06f2ef640 1 -- 172.21.15.190:0/1671322463 >> 172.21.15.190:0/1671322463 conn(0x7ff0680fb430 msgr2=0x7ff0680fbf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:07.121 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:07.117+0000 7ff06f2ef640 1 -- 172.21.15.190:0/1671322463 shutdown_connections 2024-09-16T01:54:07.121 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:07.117+0000 7ff06f2ef640 1 -- 172.21.15.190:0/1671322463 wait complete. 2024-09-16T01:54:07.747 DEBUG:teuthology.orchestra.run.smithi137:mon.b> sudo journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mon.b.service 2024-09-16T01:54:07.750 DEBUG:teuthology.orchestra.run.smithi190:mon.c> sudo journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mon.c.service 2024-09-16T01:54:07.753 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T01:54:07.753 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T01:54:07.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:07 smithi105 ceph-mon[31197]: Updating smithi190:/etc/ceph/ceph.conf 2024-09-16T01:54:07.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:07 smithi105 ceph-mon[31197]: Updating smithi190:/var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:54:07.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:07 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:07.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:07 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:07.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:07 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:07.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:07 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:07.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:07 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:54:07.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:07 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:08.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:08 smithi105 ceph-mon[31197]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2024-09-16T01:54:08.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:08 smithi105 ceph-mon[31197]: Updating smithi190:/var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T01:54:08.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:08 smithi105 ceph-mon[31197]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi105:172.21.15.105=a;smithi137:172.21.15.137=b;smithi190:172.21.15.190=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:54:08.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:08 smithi105 ceph-mon[31197]: Saving service mon spec with placement smithi105:172.21.15.105=a;smithi137:172.21.15.137=b;smithi190:172.21.15.190=c;count:3 2024-09-16T01:54:08.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:08 smithi105 ceph-mon[31197]: Deploying daemon mon.c on smithi190 2024-09-16T01:54:08.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:08 smithi105 ceph-mon[31197]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:08.864 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:54:09.541 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.539+0000 7febde1d4640 1 -- 172.21.15.190:0/881167471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febd8107660 msgr2=0x7febd8107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:09.541 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.539+0000 7febde1d4640 1 --2- 172.21.15.190:0/881167471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febd8107660 0x7febd8107a40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7febc8009880 tx=0x7febc802a5d0 comp rx=0 tx=0).stop 2024-09-16T01:54:09.541 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.539+0000 7febde1d4640 1 -- 172.21.15.190:0/881167471 shutdown_connections 2024-09-16T01:54:09.541 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.539+0000 7febde1d4640 1 --2- 172.21.15.190:0/881167471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febd8107660 0x7febd8107a40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:09.542 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.539+0000 7febde1d4640 1 -- 172.21.15.190:0/881167471 >> 172.21.15.190:0/881167471 conn(0x7febd80fd2b0 msgr2=0x7febd80ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:09.542 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.539+0000 7febde1d4640 1 -- 172.21.15.190:0/881167471 shutdown_connections 2024-09-16T01:54:09.542 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.539+0000 7febde1d4640 1 -- 172.21.15.190:0/881167471 wait complete. 2024-09-16T01:54:09.542 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.540+0000 7febde1d4640 1 Processor -- start 2024-09-16T01:54:09.542 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.540+0000 7febde1d4640 1 -- start start 2024-09-16T01:54:09.542 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.540+0000 7febde1d4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febd8107660 0x7febd8196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:09.542 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.540+0000 7febde1d4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7febd810bb20 con 0x7febd8107660 2024-09-16T01:54:09.543 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.540+0000 7febd77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febd8107660 0x7febd8196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:09.543 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.540+0000 7febd77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febd8107660 0x7febd8196fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.190:40392/0 (socket says 172.21.15.190:40392) 2024-09-16T01:54:09.543 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.540+0000 7febd77fe640 1 -- 172.21.15.190:0/2216825255 learned_addr learned my addr 172.21.15.190:0/2216825255 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-09-16T01:54:09.543 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.541+0000 7febd77fe640 1 -- 172.21.15.190:0/2216825255 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd8197500 con 0x7febd8107660 2024-09-16T01:54:09.543 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.541+0000 7febd77fe640 1 --2- 172.21.15.190:0/2216825255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febd8107660 0x7febd8196fc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7febc8009230 tx=0x7febc80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:09.543 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.541+0000 7febd4ff9640 1 -- 172.21.15.190:0/2216825255 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febc803f070 con 0x7febd8107660 2024-09-16T01:54:09.543 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.541+0000 7febd4ff9640 1 -- 172.21.15.190:0/2216825255 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febc8004d10 con 0x7febd8107660 2024-09-16T01:54:09.543 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.541+0000 7febd4ff9640 1 -- 172.21.15.190:0/2216825255 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febc8035030 con 0x7febd8107660 2024-09-16T01:54:09.544 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.541+0000 7febde1d4640 1 -- 172.21.15.190:0/2216825255 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febd81977f0 con 0x7febd8107660 2024-09-16T01:54:09.544 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.541+0000 7febde1d4640 1 -- 172.21.15.190:0/2216825255 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febd819b890 con 0x7febd8107660 2024-09-16T01:54:09.544 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.542+0000 7febd4ff9640 1 -- 172.21.15.190:0/2216825255 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7febc8042070 con 0x7febd8107660 2024-09-16T01:54:09.544 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.543+0000 7febde1d4640 1 -- 172.21.15.190:0/2216825255 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febd8102d10 con 0x7febd8107660 2024-09-16T01:54:09.545 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.543+0000 7febd4ff9640 1 --2- 172.21.15.190:0/2216825255 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7febc00392a0 0x7febc003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:09.545 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.543+0000 7febd4ff9640 1 -- 172.21.15.190:0/2216825255 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7febc806d140 con 0x7febd8107660 2024-09-16T01:54:09.545 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.543+0000 7febd6ffd640 1 --2- 172.21.15.190:0/2216825255 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7febc00392a0 0x7febc003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:09.545 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.544+0000 7febd6ffd640 1 --2- 172.21.15.190:0/2216825255 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7febc00392a0 0x7febc003b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7febcc0098c0 tx=0x7febcc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:09.549 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.548+0000 7febd4ff9640 1 -- 172.21.15.190:0/2216825255 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7febc80303a0 con 0x7febd8107660 2024-09-16T01:54:09.787 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.786+0000 7febde1d4640 1 -- 172.21.15.190:0/2216825255 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7febd8198320 con 0x7febd8107660 2024-09-16T01:54:09.788 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.787+0000 7febd4ff9640 1 -- 172.21.15.190:0/2216825255 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7febc8042360 con 0x7febd8107660 2024-09-16T01:54:09.788 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T01:54:09.788 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":1,"fsid":"423b7a2a-73ce-11ef-bceb-c7b262605968","modified":"2024-09-16T01:52:41.310216Z","created":"2024-09-16T01:52:41.310216Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T01:54:09.788 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 1 2024-09-16T01:54:09.790 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.789+0000 7febde1d4640 1 -- 172.21.15.190:0/2216825255 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7febc00392a0 msgr2=0x7febc003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:09.790 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.789+0000 7febde1d4640 1 --2- 172.21.15.190:0/2216825255 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7febc00392a0 0x7febc003b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7febcc0098c0 tx=0x7febcc006db0 comp rx=0 tx=0).stop 2024-09-16T01:54:09.790 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.789+0000 7febde1d4640 1 -- 172.21.15.190:0/2216825255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febd8107660 msgr2=0x7febd8196fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:09.790 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.789+0000 7febde1d4640 1 --2- 172.21.15.190:0/2216825255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febd8107660 0x7febd8196fc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7febc8009230 tx=0x7febc80048f0 comp rx=0 tx=0).stop 2024-09-16T01:54:09.791 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.789+0000 7febde1d4640 1 -- 172.21.15.190:0/2216825255 shutdown_connections 2024-09-16T01:54:09.791 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.789+0000 7febde1d4640 1 --2- 172.21.15.190:0/2216825255 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7febc00392a0 0x7febc003b760 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:09.791 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.789+0000 7febde1d4640 1 --2- 172.21.15.190:0/2216825255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febd8107660 0x7febd8196fc0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:09.791 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.789+0000 7febde1d4640 1 -- 172.21.15.190:0/2216825255 >> 172.21.15.190:0/2216825255 conn(0x7febd80fd2b0 msgr2=0x7febd80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:09.791 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.789+0000 7febde1d4640 1 -- 172.21.15.190:0/2216825255 shutdown_connections 2024-09-16T01:54:09.791 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:09.789+0000 7febde1d4640 1 -- 172.21.15.190:0/2216825255 wait complete. 2024-09-16T01:54:10.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:10 smithi105 ceph-mon[31197]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:10.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:10 smithi105 ceph-mon[31197]: from='client.? 172.21.15.190:0/2216825255' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T01:54:11.771 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T01:54:11.772 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T01:54:12.061 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T01:54:12.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:11 smithi190 systemd[1]: Started Ceph mon.c for 423b7a2a-73ce-11ef-bceb-c7b262605968. 2024-09-16T01:54:16.991 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:16.989+0000 7f801d0a9640 1 -- 172.21.15.190:0/4032905221 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8000003530 msgr2=0x7f80000059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:16.991 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:16.989+0000 7f801d0a9640 1 --2- 172.21.15.190:0/4032905221 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8000003530 0x7f80000059c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f80181016a0 tx=0x7f800802c490 comp rx=0 tx=0).stop 2024-09-16T01:54:16.991 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:16.989+0000 7f801d0a9640 1 -- 172.21.15.190:0/4032905221 shutdown_connections 2024-09-16T01:54:16.991 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:16.989+0000 7f801d0a9640 1 --2- 172.21.15.190:0/4032905221 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8000003530 0x7f80000059c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:16.991 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:16.989+0000 7f801d0a9640 1 -- 172.21.15.190:0/4032905221 >> 172.21.15.190:0/4032905221 conn(0x7f80180fc220 msgr2=0x7f80180fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:16.991 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:16.989+0000 7f801d0a9640 1 -- 172.21.15.190:0/4032905221 shutdown_connections 2024-09-16T01:54:16.991 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:16.989+0000 7f801d0a9640 1 -- 172.21.15.190:0/4032905221 wait complete. 2024-09-16T01:54:16.992 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:16.990+0000 7f801d0a9640 1 Processor -- start 2024-09-16T01:54:16.993 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:16.991+0000 7f801d0a9640 1 -- start start 2024-09-16T01:54:16.993 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:16.991+0000 7f801d0a9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8000003530 0x7f80181971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:16.993 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:16.991+0000 7f801d0a9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80181976f0 0x7f801819ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:16.993 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:16.991+0000 7f801d0a9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8018108520 con 0x7f8000003530 2024-09-16T01:54:16.993 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:16.991+0000 7f801d0a9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f80181083a0 con 0x7f80181976f0 2024-09-16T01:54:16.993 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:16.991+0000 7f801659c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80181976f0 0x7f801819ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:16.993 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:16.991+0000 7f801659c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80181976f0 0x7f801819ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:54558/0 (socket says 172.21.15.190:54558) 2024-09-16T01:54:16.993 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:16.991+0000 7f801659c640 1 -- 172.21.15.190:0/1730982480 learned_addr learned my addr 172.21.15.190:0/1730982480 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-09-16T01:54:16.993 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:16.991+0000 7f801659c640 1 -- 172.21.15.190:0/1730982480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80181976f0 msgr2=0x7f801819ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-16T01:54:16.993 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:16.991+0000 7f801659c640 1 -- 172.21.15.190:0/1730982480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80181976f0 msgr2=0x7f801819ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T01:54:16.993 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:16.991+0000 7f801659c640 1 --2- 172.21.15.190:0/1730982480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80181976f0 0x7f801819ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T01:54:16.993 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:16.991+0000 7f801659c640 1 --2- 172.21.15.190:0/1730982480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80181976f0 0x7f801819ba80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T01:54:16.994 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:16.991+0000 7f8016d9d640 1 --2- 172.21.15.190:0/1730982480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8000003530 0x7f80181971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:16.995 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:16.992+0000 7f8016d9d640 1 -- 172.21.15.190:0/1730982480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80181976f0 msgr2=0x7f801819ba80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:54:16.995 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:16.992+0000 7f8016d9d640 1 --2- 172.21.15.190:0/1730982480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80181976f0 0x7f801819ba80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:16.995 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:16.992+0000 7f8016d9d640 1 -- 172.21.15.190:0/1730982480 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8018197d10 con 0x7f8000003530 2024-09-16T01:54:16.995 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:16.992+0000 7f8016d9d640 1 --2- 172.21.15.190:0/1730982480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8000003530 0x7f80181971b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8008036040 tx=0x7f8008032a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:16.995 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:16.992+0000 7f7ffffff640 1 -- 172.21.15.190:0/1730982480 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f8008038040 con 0x7f8000003530 2024-09-16T01:54:16.995 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:16.992+0000 7f7ffffff640 1 -- 172.21.15.190:0/1730982480 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8008005040 con 0x7f8000003530 2024-09-16T01:54:16.995 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:16.992+0000 7f801d0a9640 1 -- 172.21.15.190:0/1730982480 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f801819bfc0 con 0x7f8000003530 2024-09-16T01:54:16.995 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:16.992+0000 7f7ffffff640 1 -- 172.21.15.190:0/1730982480 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f8008004930 con 0x7f8000003530 2024-09-16T01:54:16.995 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:16.992+0000 7f801d0a9640 1 -- 172.21.15.190:0/1730982480 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f801819c580 con 0x7f8000003530 2024-09-16T01:54:16.996 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:16.993+0000 7f7ffffff640 1 -- 172.21.15.190:0/1730982480 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f8008043020 con 0x7f8000003530 2024-09-16T01:54:16.996 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:16.993+0000 7f7ffffff640 1 --2- 172.21.15.190:0/1730982480 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f8004039040 0x7f800403b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:16.996 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:16.993+0000 7f7ffffff640 1 -- 172.21.15.190:0/1730982480 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f800806f6e0 con 0x7f8000003530 2024-09-16T01:54:16.996 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:16.993+0000 7f801d0a9640 1 -- 172.21.15.190:0/1730982480 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8018101ca0 con 0x7f8000003530 2024-09-16T01:54:16.996 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:16.994+0000 7f801659c640 1 --2- 172.21.15.190:0/1730982480 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f8004039040 0x7f800403b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:16.996 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:16.994+0000 7f801659c640 1 --2- 172.21.15.190:0/1730982480 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f8004039040 0x7f800403b500 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f800c009100 tx=0x7f800c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:17.000 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:16.999+0000 7f7ffffff640 1 -- 172.21.15.190:0/1730982480 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f800804a020 con 0x7f8000003530 2024-09-16T01:54:17.234 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:17.233+0000 7f801d0a9640 1 -- 172.21.15.190:0/1730982480 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f8018109650 con 0x7f8000003530 2024-09-16T01:54:17.235 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:17.233+0000 7f7ffffff640 1 -- 172.21.15.190:0/1730982480 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f800804c970 con 0x7f8000003530 2024-09-16T01:54:17.235 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:16 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:54:17.235 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:16 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:54:17.235 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:16 smithi190 ceph-mon[34762]: mon.a calling monitor election 2024-09-16T01:54:17.235 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:16 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:54:17.235 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:16 smithi190 ceph-mon[34762]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:17.235 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:16 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:54:17.235 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:16 smithi190 ceph-mon[34762]: mon.c calling monitor election 2024-09-16T01:54:17.236 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:16 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:54:17.236 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:16 smithi190 ceph-mon[34762]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:17.236 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:16 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:54:17.236 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:16 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:54:17.236 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:16 smithi190 ceph-mon[34762]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-16T01:54:17.236 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:16 smithi190 ceph-mon[34762]: monmap epoch 2 2024-09-16T01:54:17.236 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:16 smithi190 ceph-mon[34762]: fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 2024-09-16T01:54:17.236 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:16 smithi190 ceph-mon[34762]: last_changed 2024-09-16T01:54:11.891519+0000 2024-09-16T01:54:17.236 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:16 smithi190 ceph-mon[34762]: created 2024-09-16T01:52:41.310216+0000 2024-09-16T01:54:17.236 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:16 smithi190 ceph-mon[34762]: min_mon_release 19 (squid) 2024-09-16T01:54:17.236 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:16 smithi190 ceph-mon[34762]: election_strategy: 1 2024-09-16T01:54:17.236 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:16 smithi190 ceph-mon[34762]: 0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-16T01:54:17.236 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:16 smithi190 ceph-mon[34762]: 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.c 2024-09-16T01:54:17.236 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:16 smithi190 ceph-mon[34762]: fsmap 2024-09-16T01:54:17.237 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:16 smithi190 ceph-mon[34762]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T01:54:17.237 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:16 smithi190 ceph-mon[34762]: mgrmap e13: x(active, since 37s) 2024-09-16T01:54:17.237 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:16 smithi190 ceph-mon[34762]: overall HEALTH_OK 2024-09-16T01:54:17.237 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:16 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:17.237 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:16 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:17.237 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:16 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:17.237 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:16 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:54:17.237 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:16 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:17.237 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T01:54:17.237 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":2,"fsid":"423b7a2a-73ce-11ef-bceb-c7b262605968","modified":"2024-09-16T01:54:11.891519Z","created":"2024-09-16T01:52:41.310216Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-16T01:54:17.238 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 2 2024-09-16T01:54:17.238 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:17.236+0000 7f801d0a9640 1 -- 172.21.15.190:0/1730982480 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f8004039040 msgr2=0x7f800403b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:17.238 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:17.236+0000 7f801d0a9640 1 --2- 172.21.15.190:0/1730982480 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f8004039040 0x7f800403b500 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f800c009100 tx=0x7f800c008040 comp rx=0 tx=0).stop 2024-09-16T01:54:17.238 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:17.236+0000 7f801d0a9640 1 -- 172.21.15.190:0/1730982480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8000003530 msgr2=0x7f80181971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:17.238 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:17.236+0000 7f801d0a9640 1 --2- 172.21.15.190:0/1730982480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8000003530 0x7f80181971b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8008036040 tx=0x7f8008032a70 comp rx=0 tx=0).stop 2024-09-16T01:54:17.239 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:17.236+0000 7f801d0a9640 1 -- 172.21.15.190:0/1730982480 shutdown_connections 2024-09-16T01:54:17.239 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:17.236+0000 7f801d0a9640 1 --2- 172.21.15.190:0/1730982480 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f8004039040 0x7f800403b500 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:17.239 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:17.236+0000 7f801d0a9640 1 --2- 172.21.15.190:0/1730982480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80181976f0 0x7f801819ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:17.239 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:17.236+0000 7f801d0a9640 1 --2- 172.21.15.190:0/1730982480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8000003530 0x7f80181971b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:17.239 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:17.236+0000 7f801d0a9640 1 -- 172.21.15.190:0/1730982480 >> 172.21.15.190:0/1730982480 conn(0x7f80180fc220 msgr2=0x7f801806a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:17.239 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:17.236+0000 7f801d0a9640 1 -- 172.21.15.190:0/1730982480 shutdown_connections 2024-09-16T01:54:17.239 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:17.236+0000 7f801d0a9640 1 -- 172.21.15.190:0/1730982480 wait complete. 2024-09-16T01:54:17.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:16 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:54:17.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:16 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:54:17.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:16 smithi105 ceph-mon[31197]: mon.a calling monitor election 2024-09-16T01:54:17.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:16 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:54:17.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:16 smithi105 ceph-mon[31197]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:17.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:16 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:54:17.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:16 smithi105 ceph-mon[31197]: mon.c calling monitor election 2024-09-16T01:54:17.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:16 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:54:17.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:16 smithi105 ceph-mon[31197]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:17.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:16 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:54:17.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:16 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:54:17.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:16 smithi105 ceph-mon[31197]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-16T01:54:17.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:16 smithi105 ceph-mon[31197]: monmap epoch 2 2024-09-16T01:54:17.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:16 smithi105 ceph-mon[31197]: fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 2024-09-16T01:54:17.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:16 smithi105 ceph-mon[31197]: last_changed 2024-09-16T01:54:11.891519+0000 2024-09-16T01:54:17.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:16 smithi105 ceph-mon[31197]: created 2024-09-16T01:52:41.310216+0000 2024-09-16T01:54:17.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:16 smithi105 ceph-mon[31197]: min_mon_release 19 (squid) 2024-09-16T01:54:17.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:16 smithi105 ceph-mon[31197]: election_strategy: 1 2024-09-16T01:54:17.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:16 smithi105 ceph-mon[31197]: 0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-16T01:54:17.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:16 smithi105 ceph-mon[31197]: 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.c 2024-09-16T01:54:17.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:16 smithi105 ceph-mon[31197]: fsmap 2024-09-16T01:54:17.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:16 smithi105 ceph-mon[31197]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T01:54:17.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:16 smithi105 ceph-mon[31197]: mgrmap e13: x(active, since 37s) 2024-09-16T01:54:17.297 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:16 smithi105 ceph-mon[31197]: overall HEALTH_OK 2024-09-16T01:54:17.297 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:16 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:17.297 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:16 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:17.297 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:16 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:17.297 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:16 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:54:17.297 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:16 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:18.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:17 smithi190 ceph-mon[34762]: Deploying daemon mon.b on smithi137 2024-09-16T01:54:18.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:17 smithi190 ceph-mon[34762]: from='client.? 172.21.15.190:0/1730982480' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T01:54:18.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:17 smithi190 ceph-mon[34762]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:18.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:17 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:54:18.294 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:17 smithi105 ceph-mon[31197]: Deploying daemon mon.b on smithi137 2024-09-16T01:54:18.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:17 smithi105 ceph-mon[31197]: from='client.? 172.21.15.190:0/1730982480' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T01:54:18.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:17 smithi105 ceph-mon[31197]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:18.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:17 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:54:18.793 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T01:54:18.794 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T01:54:19.049 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T01:54:19.519 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.505+0000 7f32d39a2640 1 -- 172.21.15.190:0/1162113157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32cc106650 msgr2=0x7f32cc106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:19.519 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.505+0000 7f32d39a2640 1 --2- 172.21.15.190:0/1162113157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32cc106650 0x7f32cc106a30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f32bc009880 tx=0x7f32bc02a5d0 comp rx=0 tx=0).stop 2024-09-16T01:54:19.519 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.505+0000 7f32d39a2640 1 -- 172.21.15.190:0/1162113157 shutdown_connections 2024-09-16T01:54:19.519 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.505+0000 7f32d39a2640 1 --2- 172.21.15.190:0/1162113157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32cc106650 0x7f32cc106a30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:19.519 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.505+0000 7f32d39a2640 1 -- 172.21.15.190:0/1162113157 >> 172.21.15.190:0/1162113157 conn(0x7f32cc0fc280 msgr2=0x7f32cc0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:19.520 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.505+0000 7f32d39a2640 1 -- 172.21.15.190:0/1162113157 shutdown_connections 2024-09-16T01:54:19.520 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.505+0000 7f32d39a2640 1 -- 172.21.15.190:0/1162113157 wait complete. 2024-09-16T01:54:19.520 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.506+0000 7f32d39a2640 1 Processor -- start 2024-09-16T01:54:19.520 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.506+0000 7f32d39a2640 1 -- start start 2024-09-16T01:54:19.520 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.506+0000 7f32d39a2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32cc106650 0x7f32cc196160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:19.520 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.506+0000 7f32d39a2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32cc1966a0 0x7f32cc19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:19.520 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.506+0000 7f32d39a2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc10ab30 con 0x7f32cc106650 2024-09-16T01:54:19.520 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.506+0000 7f32d39a2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc10a9b0 con 0x7f32cc1966a0 2024-09-16T01:54:19.520 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.506+0000 7f32d0f3d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32cc1966a0 0x7f32cc19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:19.520 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.506+0000 7f32d173e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32cc106650 0x7f32cc196160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:19.520 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.506+0000 7f32d0f3d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32cc1966a0 0x7f32cc19aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:54572/0 (socket says 172.21.15.190:54572) 2024-09-16T01:54:19.520 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.506+0000 7f32d0f3d640 1 -- 172.21.15.190:0/4055394671 learned_addr learned my addr 172.21.15.190:0/4055394671 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-09-16T01:54:19.521 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.506+0000 7f32d0f3d640 1 -- 172.21.15.190:0/4055394671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32cc1966a0 msgr2=0x7f32cc19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-16T01:54:19.521 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.506+0000 7f32d0f3d640 1 -- 172.21.15.190:0/4055394671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32cc1966a0 msgr2=0x7f32cc19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T01:54:19.521 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.506+0000 7f32d0f3d640 1 --2- 172.21.15.190:0/4055394671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32cc1966a0 0x7f32cc19aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T01:54:19.521 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.506+0000 7f32d0f3d640 1 --2- 172.21.15.190:0/4055394671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32cc1966a0 0x7f32cc19aa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T01:54:19.521 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.507+0000 7f32d173e640 1 -- 172.21.15.190:0/4055394671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32cc1966a0 msgr2=0x7f32cc19aa30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:54:19.521 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.507+0000 7f32d173e640 1 --2- 172.21.15.190:0/4055394671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32cc1966a0 0x7f32cc19aa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:19.521 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.507+0000 7f32d173e640 1 -- 172.21.15.190:0/4055394671 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32cc196cc0 con 0x7f32cc106650 2024-09-16T01:54:19.521 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.507+0000 7f32d173e640 1 --2- 172.21.15.190:0/4055394671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32cc106650 0x7f32cc196160 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f32bc000c00 tx=0x7f32bc004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:19.521 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.507+0000 7f32c27fc640 1 -- 172.21.15.190:0/4055394671 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f32bc035030 con 0x7f32cc106650 2024-09-16T01:54:19.521 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.507+0000 7f32d39a2640 1 -- 172.21.15.190:0/4055394671 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32cc19af70 con 0x7f32cc106650 2024-09-16T01:54:19.521 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.507+0000 7f32c27fc640 1 -- 172.21.15.190:0/4055394671 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32bc0053a0 con 0x7f32cc106650 2024-09-16T01:54:19.521 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.507+0000 7f32c27fc640 1 -- 172.21.15.190:0/4055394671 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f32bc031830 con 0x7f32cc106650 2024-09-16T01:54:19.522 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.507+0000 7f32d39a2640 1 -- 172.21.15.190:0/4055394671 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32cc19b4d0 con 0x7f32cc106650 2024-09-16T01:54:19.522 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.508+0000 7f32d39a2640 1 -- 172.21.15.190:0/4055394671 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32cc19b840 con 0x7f32cc106650 2024-09-16T01:54:19.522 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.508+0000 7f32c27fc640 1 -- 172.21.15.190:0/4055394671 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f32bc042070 con 0x7f32cc106650 2024-09-16T01:54:19.522 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.508+0000 7f32c27fc640 1 --2- 172.21.15.190:0/4055394671 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f32b8039040 0x7f32b803b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:19.522 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.508+0000 7f32c27fc640 1 -- 172.21.15.190:0/4055394671 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f32bc06d070 con 0x7f32cc106650 2024-09-16T01:54:19.522 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.508+0000 7f32d0f3d640 1 --2- 172.21.15.190:0/4055394671 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f32b8039040 0x7f32b803b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:19.522 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.509+0000 7f32d0f3d640 1 --2- 172.21.15.190:0/4055394671 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f32b8039040 0x7f32b803b500 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f32c8009100 tx=0x7f32c8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:19.522 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.513+0000 7f32c27fc640 1 -- 172.21.15.190:0/4055394671 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f32bc0372d0 con 0x7f32cc106650 2024-09-16T01:54:19.745 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.744+0000 7f32d39a2640 1 -- 172.21.15.190:0/4055394671 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f32cc1974c0 con 0x7f32cc106650 2024-09-16T01:54:19.746 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.744+0000 7f32c27fc640 1 -- 172.21.15.190:0/4055394671 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f32bc036ca0 con 0x7f32cc106650 2024-09-16T01:54:19.746 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T01:54:19.746 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":2,"fsid":"423b7a2a-73ce-11ef-bceb-c7b262605968","modified":"2024-09-16T01:54:11.891519Z","created":"2024-09-16T01:52:41.310216Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-16T01:54:19.746 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 2 2024-09-16T01:54:19.748 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.746+0000 7f32d39a2640 1 -- 172.21.15.190:0/4055394671 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f32b8039040 msgr2=0x7f32b803b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:19.748 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.746+0000 7f32d39a2640 1 --2- 172.21.15.190:0/4055394671 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f32b8039040 0x7f32b803b500 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f32c8009100 tx=0x7f32c8008040 comp rx=0 tx=0).stop 2024-09-16T01:54:19.748 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.746+0000 7f32d39a2640 1 -- 172.21.15.190:0/4055394671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32cc106650 msgr2=0x7f32cc196160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:19.748 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.746+0000 7f32d39a2640 1 --2- 172.21.15.190:0/4055394671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32cc106650 0x7f32cc196160 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f32bc000c00 tx=0x7f32bc004e40 comp rx=0 tx=0).stop 2024-09-16T01:54:19.748 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.746+0000 7f32d39a2640 1 -- 172.21.15.190:0/4055394671 shutdown_connections 2024-09-16T01:54:19.748 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.746+0000 7f32d39a2640 1 --2- 172.21.15.190:0/4055394671 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f32b8039040 0x7f32b803b500 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:19.748 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.746+0000 7f32d39a2640 1 --2- 172.21.15.190:0/4055394671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32cc1966a0 0x7f32cc19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:19.748 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.746+0000 7f32d39a2640 1 --2- 172.21.15.190:0/4055394671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32cc106650 0x7f32cc196160 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:19.749 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.746+0000 7f32d39a2640 1 -- 172.21.15.190:0/4055394671 >> 172.21.15.190:0/4055394671 conn(0x7f32cc0fc280 msgr2=0x7f32cc0fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:19.749 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.747+0000 7f32d39a2640 1 -- 172.21.15.190:0/4055394671 shutdown_connections 2024-09-16T01:54:19.749 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:19.747+0000 7f32d39a2640 1 -- 172.21.15.190:0/4055394671 wait complete. 2024-09-16T01:54:20.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:20 smithi190 ceph-mon[34762]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:20.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:20 smithi190 ceph-mon[34762]: from='client.? 172.21.15.190:0/4055394671' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T01:54:20.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:20 smithi105 ceph-mon[31197]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:20.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:20 smithi105 ceph-mon[31197]: from='client.? 172.21.15.190:0/4055394671' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T01:54:20.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:20 smithi137 systemd[1]: Started Ceph mon.b for 423b7a2a-73ce-11ef-bceb-c7b262605968. 2024-09-16T01:54:21.336 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T01:54:21.337 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T01:54:21.587 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T01:54:26.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:25 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:54:26.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:25 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:54:26.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:25 smithi105 ceph-mon[31197]: mon.a calling monitor election 2024-09-16T01:54:26.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:25 smithi105 ceph-mon[31197]: mon.c calling monitor election 2024-09-16T01:54:26.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:25 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:54:26.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:25 smithi105 ceph-mon[31197]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:26.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:25 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:54:26.046 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:25 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:54:26.046 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:25 smithi105 ceph-mon[31197]: mon.b calling monitor election 2024-09-16T01:54:26.046 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:25 smithi105 ceph-mon[31197]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:26.046 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:25 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:54:26.046 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:25 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:54:26.046 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:25 smithi105 ceph-mon[31197]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:26.046 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:25 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:54:26.046 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:25 smithi105 ceph-mon[31197]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-16T01:54:26.046 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:25 smithi105 ceph-mon[31197]: monmap epoch 3 2024-09-16T01:54:26.046 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:25 smithi105 ceph-mon[31197]: fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 2024-09-16T01:54:26.046 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:25 smithi105 ceph-mon[31197]: last_changed 2024-09-16T01:54:20.675126+0000 2024-09-16T01:54:26.046 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:25 smithi105 ceph-mon[31197]: created 2024-09-16T01:52:41.310216+0000 2024-09-16T01:54:26.047 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:25 smithi105 ceph-mon[31197]: min_mon_release 19 (squid) 2024-09-16T01:54:26.047 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:25 smithi105 ceph-mon[31197]: election_strategy: 1 2024-09-16T01:54:26.047 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:25 smithi105 ceph-mon[31197]: 0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-16T01:54:26.047 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:25 smithi105 ceph-mon[31197]: 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.c 2024-09-16T01:54:26.047 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:25 smithi105 ceph-mon[31197]: 2: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.b 2024-09-16T01:54:26.047 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:25 smithi105 ceph-mon[31197]: fsmap 2024-09-16T01:54:26.047 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:25 smithi105 ceph-mon[31197]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T01:54:26.047 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:25 smithi105 ceph-mon[31197]: mgrmap e13: x(active, since 46s) 2024-09-16T01:54:26.047 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:25 smithi105 ceph-mon[31197]: overall HEALTH_OK 2024-09-16T01:54:26.047 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:25 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:26.047 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:25 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:26.047 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:25 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:26.047 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:25 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:26.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:25 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:54:26.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:25 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:54:26.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:25 smithi190 ceph-mon[34762]: mon.a calling monitor election 2024-09-16T01:54:26.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:25 smithi190 ceph-mon[34762]: mon.c calling monitor election 2024-09-16T01:54:26.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:25 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:54:26.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:25 smithi190 ceph-mon[34762]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:26.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:25 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:54:26.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:25 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:54:26.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:25 smithi190 ceph-mon[34762]: mon.b calling monitor election 2024-09-16T01:54:26.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:25 smithi190 ceph-mon[34762]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:26.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:25 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:54:26.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:25 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:54:26.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:25 smithi190 ceph-mon[34762]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:26.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:25 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:54:26.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:25 smithi190 ceph-mon[34762]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-16T01:54:26.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:25 smithi190 ceph-mon[34762]: monmap epoch 3 2024-09-16T01:54:26.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:25 smithi190 ceph-mon[34762]: fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 2024-09-16T01:54:26.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:25 smithi190 ceph-mon[34762]: last_changed 2024-09-16T01:54:20.675126+0000 2024-09-16T01:54:26.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:25 smithi190 ceph-mon[34762]: created 2024-09-16T01:52:41.310216+0000 2024-09-16T01:54:26.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:25 smithi190 ceph-mon[34762]: min_mon_release 19 (squid) 2024-09-16T01:54:26.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:25 smithi190 ceph-mon[34762]: election_strategy: 1 2024-09-16T01:54:26.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:25 smithi190 ceph-mon[34762]: 0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-16T01:54:26.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:25 smithi190 ceph-mon[34762]: 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.c 2024-09-16T01:54:26.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:25 smithi190 ceph-mon[34762]: 2: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.b 2024-09-16T01:54:26.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:25 smithi190 ceph-mon[34762]: fsmap 2024-09-16T01:54:26.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:25 smithi190 ceph-mon[34762]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T01:54:26.095 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:25 smithi190 ceph-mon[34762]: mgrmap e13: x(active, since 46s) 2024-09-16T01:54:26.095 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:25 smithi190 ceph-mon[34762]: overall HEALTH_OK 2024-09-16T01:54:26.095 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:25 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:26.095 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:25 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:26.095 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:25 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:26.095 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:25 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:26.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:25 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:54:26.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:25 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:54:26.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:25 smithi137 ceph-mon[34099]: mon.a calling monitor election 2024-09-16T01:54:26.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:25 smithi137 ceph-mon[34099]: mon.c calling monitor election 2024-09-16T01:54:26.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:25 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:54:26.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:25 smithi137 ceph-mon[34099]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:26.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:25 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:54:26.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:25 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:54:26.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:25 smithi137 ceph-mon[34099]: mon.b calling monitor election 2024-09-16T01:54:26.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:25 smithi137 ceph-mon[34099]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:26.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:25 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:54:26.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:25 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:54:26.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:25 smithi137 ceph-mon[34099]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:26.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:25 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:54:26.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:25 smithi137 ceph-mon[34099]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-16T01:54:26.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:25 smithi137 ceph-mon[34099]: monmap epoch 3 2024-09-16T01:54:26.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:25 smithi137 ceph-mon[34099]: fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 2024-09-16T01:54:26.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:25 smithi137 ceph-mon[34099]: last_changed 2024-09-16T01:54:20.675126+0000 2024-09-16T01:54:26.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:25 smithi137 ceph-mon[34099]: created 2024-09-16T01:52:41.310216+0000 2024-09-16T01:54:26.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:25 smithi137 ceph-mon[34099]: min_mon_release 19 (squid) 2024-09-16T01:54:26.173 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:25 smithi137 ceph-mon[34099]: election_strategy: 1 2024-09-16T01:54:26.173 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:25 smithi137 ceph-mon[34099]: 0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-16T01:54:26.173 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:25 smithi137 ceph-mon[34099]: 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.c 2024-09-16T01:54:26.173 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:25 smithi137 ceph-mon[34099]: 2: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.b 2024-09-16T01:54:26.173 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:25 smithi137 ceph-mon[34099]: fsmap 2024-09-16T01:54:26.173 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:25 smithi137 ceph-mon[34099]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T01:54:26.173 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:25 smithi137 ceph-mon[34099]: mgrmap e13: x(active, since 46s) 2024-09-16T01:54:26.173 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:25 smithi137 ceph-mon[34099]: overall HEALTH_OK 2024-09-16T01:54:26.173 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:25 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:26.173 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:25 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:26.174 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:25 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:26.174 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:25 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:26.400 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.398+0000 7f42f7a75640 1 -- 172.21.15.190:0/1448589078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42dc003530 msgr2=0x7f42dc0059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:26.400 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.398+0000 7f42f7a75640 1 --2- 172.21.15.190:0/1448589078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42dc003530 0x7f42dc0059c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f42f0102b00 tx=0x7f42ec02c490 comp rx=0 tx=0).stop 2024-09-16T01:54:26.400 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.398+0000 7f42f7a75640 1 -- 172.21.15.190:0/1448589078 shutdown_connections 2024-09-16T01:54:26.400 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.398+0000 7f42f7a75640 1 --2- 172.21.15.190:0/1448589078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42dc003530 0x7f42dc0059c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:26.400 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.398+0000 7f42f7a75640 1 -- 172.21.15.190:0/1448589078 >> 172.21.15.190:0/1448589078 conn(0x7f42f00fd330 msgr2=0x7f42f00ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:26.401 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.398+0000 7f42f7a75640 1 -- 172.21.15.190:0/1448589078 shutdown_connections 2024-09-16T01:54:26.401 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.398+0000 7f42f7a75640 1 -- 172.21.15.190:0/1448589078 wait complete. 2024-09-16T01:54:26.403 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.399+0000 7f42f7a75640 1 Processor -- start 2024-09-16T01:54:26.403 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.399+0000 7f42f7a75640 1 -- start start 2024-09-16T01:54:26.403 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.399+0000 7f42f7a75640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42dc003530 0x7f42f0197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:26.403 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.399+0000 7f42f7a75640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42f0197890 0x7f42f019bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:26.403 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.399+0000 7f42f7a75640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42f019c160 0x7f42f019c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:26.403 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.399+0000 7f42f7a75640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f42f01095e0 con 0x7f42f0197890 2024-09-16T01:54:26.403 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.399+0000 7f42f7a75640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f42f0109460 con 0x7f42f019c160 2024-09-16T01:54:26.404 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.399+0000 7f42f7a75640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f42f0109760 con 0x7f42dc003530 2024-09-16T01:54:26.404 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.400+0000 7f42f5811640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42dc003530 0x7f42f0197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:26.404 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.400+0000 7f42f5811640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42dc003530 0x7f42f0197350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:58818/0 (socket says 172.21.15.190:58818) 2024-09-16T01:54:26.404 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.400+0000 7f42f5811640 1 -- 172.21.15.190:0/975693951 learned_addr learned my addr 172.21.15.190:0/975693951 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-09-16T01:54:26.404 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.400+0000 7f42f5811640 1 -- 172.21.15.190:0/975693951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42dc003530 msgr2=0x7f42f0197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-16T01:54:26.404 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.400+0000 7f42f5811640 1 -- 172.21.15.190:0/975693951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42dc003530 msgr2=0x7f42f0197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T01:54:26.404 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.400+0000 7f42f5811640 1 --2- 172.21.15.190:0/975693951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42dc003530 0x7f42f0197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T01:54:26.404 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.400+0000 7f42f5010640 1 --2- 172.21.15.190:0/975693951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42f0197890 0x7f42f019bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:26.404 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.400+0000 7f42f5811640 1 --2- 172.21.15.190:0/975693951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42dc003530 0x7f42f0197350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T01:54:26.404 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.400+0000 7f42f6012640 1 --2- 172.21.15.190:0/975693951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42f019c160 0x7f42f019c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:26.404 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.400+0000 7f42f6012640 1 -- 172.21.15.190:0/975693951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42f019c160 msgr2=0x7f42f019c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-16T01:54:26.404 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.400+0000 7f42f6012640 1 -- 172.21.15.190:0/975693951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42f019c160 msgr2=0x7f42f019c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T01:54:26.404 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.400+0000 7f42f6012640 1 --2- 172.21.15.190:0/975693951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42f019c160 0x7f42f019c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T01:54:26.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.400+0000 7f42f6012640 1 --2- 172.21.15.190:0/975693951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42f019c160 0x7f42f019c610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T01:54:26.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.400+0000 7f42f5010640 1 -- 172.21.15.190:0/975693951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42dc003530 msgr2=0x7f42f0197350 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:54:26.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.400+0000 7f42f5010640 1 --2- 172.21.15.190:0/975693951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42dc003530 0x7f42f0197350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:26.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.400+0000 7f42f5010640 1 -- 172.21.15.190:0/975693951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42f019c160 msgr2=0x7f42f019c610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:54:26.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.400+0000 7f42f5010640 1 --2- 172.21.15.190:0/975693951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42f019c160 0x7f42f019c610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:26.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.400+0000 7f42f5010640 1 -- 172.21.15.190:0/975693951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f01a3ed0 con 0x7f42f0197890 2024-09-16T01:54:26.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.400+0000 7f42f5010640 1 --2- 172.21.15.190:0/975693951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42f0197890 0x7f42f019bc20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f42e000b450 tx=0x7f42e000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:26.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.400+0000 7f42e6ffd640 1 -- 172.21.15.190:0/975693951 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e0004230 con 0x7f42f0197890 2024-09-16T01:54:26.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.400+0000 7f42e6ffd640 1 -- 172.21.15.190:0/975693951 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42e00043d0 con 0x7f42f0197890 2024-09-16T01:54:26.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.400+0000 7f42f7a75640 1 -- 172.21.15.190:0/975693951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42f01a4100 con 0x7f42f0197890 2024-09-16T01:54:26.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.400+0000 7f42e6ffd640 1 -- 172.21.15.190:0/975693951 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e0003be0 con 0x7f42f0197890 2024-09-16T01:54:26.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.400+0000 7f42f7a75640 1 -- 172.21.15.190:0/975693951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42f01a4610 con 0x7f42f0197890 2024-09-16T01:54:26.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.401+0000 7f42e6ffd640 1 -- 172.21.15.190:0/975693951 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f42e000f050 con 0x7f42f0197890 2024-09-16T01:54:26.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.401+0000 7f42e6ffd640 1 --2- 172.21.15.190:0/975693951 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f42d4039730 0x7f42d403bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:26.406 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.402+0000 7f42e6ffd640 1 -- 172.21.15.190:0/975693951 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f42e004c430 con 0x7f42f0197890 2024-09-16T01:54:26.406 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.402+0000 7f42f7a75640 1 -- 172.21.15.190:0/975693951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42f010a710 con 0x7f42f0197890 2024-09-16T01:54:26.406 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.402+0000 7f42f5811640 1 --2- 172.21.15.190:0/975693951 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f42d4039730 0x7f42d403bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:26.406 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.402+0000 7f42f5811640 1 --2- 172.21.15.190:0/975693951 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f42d4039730 0x7f42d403bbf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f42ec012c10 tx=0x7f42ec001e70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:26.408 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.407+0000 7f42e6ffd640 1 -- 172.21.15.190:0/975693951 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f42e000d040 con 0x7f42f0197890 2024-09-16T01:54:26.683 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.682+0000 7f42f7a75640 1 -- 172.21.15.190:0/975693951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f42f0190d40 con 0x7f42f0197890 2024-09-16T01:54:26.684 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.682+0000 7f42e6ffd640 1 -- 172.21.15.190:0/975693951 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1289 (secure 0 0 0) 0x7f42e0003e10 con 0x7f42f0197890 2024-09-16T01:54:26.684 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T01:54:26.684 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":3,"fsid":"423b7a2a-73ce-11ef-bceb-c7b262605968","modified":"2024-09-16T01:54:20.675126Z","created":"2024-09-16T01:52:41.310216Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:3300","nonce":0},{"type":"v1","addr":"172.21.15.137:6789","nonce":0}]},"addr":"172.21.15.137:6789/0","public_addr":"172.21.15.137:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-16T01:54:26.685 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 3 2024-09-16T01:54:26.686 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.685+0000 7f42f7a75640 1 -- 172.21.15.190:0/975693951 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f42d4039730 msgr2=0x7f42d403bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:26.687 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.685+0000 7f42f7a75640 1 --2- 172.21.15.190:0/975693951 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f42d4039730 0x7f42d403bbf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f42ec012c10 tx=0x7f42ec001e70 comp rx=0 tx=0).stop 2024-09-16T01:54:26.687 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.685+0000 7f42f7a75640 1 -- 172.21.15.190:0/975693951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42f0197890 msgr2=0x7f42f019bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:26.687 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.685+0000 7f42f7a75640 1 --2- 172.21.15.190:0/975693951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42f0197890 0x7f42f019bc20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f42e000b450 tx=0x7f42e000b910 comp rx=0 tx=0).stop 2024-09-16T01:54:26.687 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.685+0000 7f42f7a75640 1 -- 172.21.15.190:0/975693951 shutdown_connections 2024-09-16T01:54:26.687 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.685+0000 7f42f7a75640 1 --2- 172.21.15.190:0/975693951 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f42d4039730 0x7f42d403bbf0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:26.687 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.685+0000 7f42f7a75640 1 --2- 172.21.15.190:0/975693951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42f019c160 0x7f42f019c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:26.687 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.685+0000 7f42f7a75640 1 --2- 172.21.15.190:0/975693951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42f0197890 0x7f42f019bc20 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:26.687 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.685+0000 7f42f7a75640 1 --2- 172.21.15.190:0/975693951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42dc003530 0x7f42f0197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:26.687 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.685+0000 7f42f7a75640 1 -- 172.21.15.190:0/975693951 >> 172.21.15.190:0/975693951 conn(0x7f42f00fd330 msgr2=0x7f42f0106a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:26.687 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.685+0000 7f42f7a75640 1 -- 172.21.15.190:0/975693951 shutdown_connections 2024-09-16T01:54:26.687 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:26.685+0000 7f42f7a75640 1 -- 172.21.15.190:0/975693951 wait complete. 2024-09-16T01:54:27.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:26 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:54:27.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:26 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:54:27.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:26 smithi105 ceph-mon[31197]: from='client.? 172.21.15.190:0/975693951' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T01:54:27.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:26 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:54:27.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:26 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:54:27.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:26 smithi190 ceph-mon[34762]: from='client.? 172.21.15.190:0/975693951' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T01:54:27.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:26 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:54:27.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:26 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:54:27.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:26 smithi137 ceph-mon[34099]: from='client.? 172.21.15.190:0/975693951' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T01:54:27.477 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-16T01:54:27.478 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-16T01:54:27.728 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:54:28.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:27 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:27 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:27 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:27 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:27 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:28.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:27 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:54:28.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:27 smithi190 ceph-mon[34762]: Updating smithi105:/etc/ceph/ceph.conf 2024-09-16T01:54:28.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:27 smithi190 ceph-mon[34762]: Updating smithi137:/etc/ceph/ceph.conf 2024-09-16T01:54:28.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:27 smithi190 ceph-mon[34762]: Updating smithi190:/etc/ceph/ceph.conf 2024-09-16T01:54:28.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:27 smithi190 ceph-mon[34762]: Updating smithi137:/var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:54:28.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:27 smithi190 ceph-mon[34762]: Updating smithi190:/var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:54:28.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:27 smithi190 ceph-mon[34762]: Updating smithi105:/var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:54:28.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:27 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:27 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:27 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:27 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:27 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:27 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:27 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:27 smithi190 ceph-mon[34762]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:28.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:27 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:27 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:27 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:27 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:27 smithi190 ceph-mon[34762]: Reconfiguring mon.a (unknown last config time)... 2024-09-16T01:54:28.095 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:27 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:54:28.095 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:27 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T01:54:28.095 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:27 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:28.095 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:27 smithi190 ceph-mon[34762]: Reconfiguring daemon mon.a on smithi105 2024-09-16T01:54:28.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:27 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:27 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:27 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.146 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:27 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.146 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:27 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:28.146 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:27 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:54:28.146 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:27 smithi105 ceph-mon[31197]: Updating smithi105:/etc/ceph/ceph.conf 2024-09-16T01:54:28.146 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:27 smithi105 ceph-mon[31197]: Updating smithi137:/etc/ceph/ceph.conf 2024-09-16T01:54:28.146 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:27 smithi105 ceph-mon[31197]: Updating smithi190:/etc/ceph/ceph.conf 2024-09-16T01:54:28.147 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:27 smithi105 ceph-mon[31197]: Updating smithi137:/var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:54:28.147 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:27 smithi105 ceph-mon[31197]: Updating smithi190:/var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:54:28.147 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:27 smithi105 ceph-mon[31197]: Updating smithi105:/var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:54:28.147 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:27 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.147 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:27 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.147 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:27 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.148 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:27 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.148 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:27 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.148 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:27 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.148 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:27 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.149 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:27 smithi105 ceph-mon[31197]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:28.149 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:27 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.149 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:27 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.149 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:27 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.149 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:27 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.149 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:27 smithi105 ceph-mon[31197]: Reconfiguring mon.a (unknown last config time)... 2024-09-16T01:54:28.149 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:27 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:54:28.150 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:27 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T01:54:28.150 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:27 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:28.150 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:27 smithi105 ceph-mon[31197]: Reconfiguring daemon mon.a on smithi105 2024-09-16T01:54:28.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:27 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:27 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:27 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:27 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:27 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:28.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:27 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:54:28.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:27 smithi137 ceph-mon[34099]: Updating smithi105:/etc/ceph/ceph.conf 2024-09-16T01:54:28.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:27 smithi137 ceph-mon[34099]: Updating smithi137:/etc/ceph/ceph.conf 2024-09-16T01:54:28.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:27 smithi137 ceph-mon[34099]: Updating smithi190:/etc/ceph/ceph.conf 2024-09-16T01:54:28.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:27 smithi137 ceph-mon[34099]: Updating smithi137:/var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:54:28.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:27 smithi137 ceph-mon[34099]: Updating smithi190:/var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:54:28.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:27 smithi137 ceph-mon[34099]: Updating smithi105:/var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:54:28.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:27 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:27 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:27 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:27 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:27 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:27 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:27 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:27 smithi137 ceph-mon[34099]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:28.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:27 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:27 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.173 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:27 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.173 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:27 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:28.173 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:27 smithi137 ceph-mon[34099]: Reconfiguring mon.a (unknown last config time)... 2024-09-16T01:54:28.173 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:27 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:54:28.173 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:27 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T01:54:28.173 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:27 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:28.173 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:27 smithi137 ceph-mon[34099]: Reconfiguring daemon mon.a on smithi105 2024-09-16T01:54:28.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.764+0000 7f1ecd2fe640 1 -- 172.21.15.105:0/335688498 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ec8106590 msgr2=0x7f1ec8106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:28.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.764+0000 7f1ecd2fe640 1 --2- 172.21.15.105:0/335688498 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ec8106590 0x7f1ec8106970 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8009880 tx=0x7f1eb802a5d0 comp rx=0 tx=0).stop 2024-09-16T01:54:28.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.764+0000 7f1ecd2fe640 1 -- 172.21.15.105:0/335688498 shutdown_connections 2024-09-16T01:54:28.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.764+0000 7f1ecd2fe640 1 --2- 172.21.15.105:0/335688498 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ec8106590 0x7f1ec8106970 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:28.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.764+0000 7f1ecd2fe640 1 -- 172.21.15.105:0/335688498 >> 172.21.15.105:0/335688498 conn(0x7f1ec80fc1a0 msgr2=0x7f1ec80fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:28.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.764+0000 7f1ecd2fe640 1 -- 172.21.15.105:0/335688498 shutdown_connections 2024-09-16T01:54:28.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.764+0000 7f1ecd2fe640 1 -- 172.21.15.105:0/335688498 wait complete. 2024-09-16T01:54:28.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.765+0000 7f1ecd2fe640 1 Processor -- start 2024-09-16T01:54:28.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.765+0000 7f1ecd2fe640 1 -- start start 2024-09-16T01:54:28.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.765+0000 7f1ecd2fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ec8106590 0x7f1ec8198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:28.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.765+0000 7f1ecd2fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ec81985b0 0x7f1ec8198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:28.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.765+0000 7f1ecd2fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ec81a10e0 0x7f1ec81a34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:28.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.765+0000 7f1ecd2fe640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec810aa70 con 0x7f1ec8106590 2024-09-16T01:54:28.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.765+0000 7f1ecd2fe640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec810a8f0 con 0x7f1ec81a10e0 2024-09-16T01:54:28.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.765+0000 7f1ecd2fe640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec810abf0 con 0x7f1ec81985b0 2024-09-16T01:54:28.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.765+0000 7f1ec6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ec8106590 0x7f1ec8198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:28.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.765+0000 7f1ec6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ec8106590 0x7f1ec8198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44788/0 (socket says 172.21.15.105:44788) 2024-09-16T01:54:28.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.765+0000 7f1ec6ffd640 1 -- 172.21.15.105:0/1103585551 learned_addr learned my addr 172.21.15.105:0/1103585551 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:54:28.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.765+0000 7f1ec67fc640 1 --2- 172.21.15.105:0/1103585551 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ec81985b0 0x7f1ec8198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:28.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.766+0000 7f1ec6ffd640 1 -- 172.21.15.105:0/1103585551 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ec81985b0 msgr2=0x7f1ec8198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:28.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.766+0000 7f1ec6ffd640 1 --2- 172.21.15.105:0/1103585551 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ec81985b0 0x7f1ec8198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:28.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.766+0000 7f1ec6ffd640 1 -- 172.21.15.105:0/1103585551 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ec81a10e0 msgr2=0x7f1ec81a34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:28.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.766+0000 7f1ec77fe640 1 --2- 172.21.15.105:0/1103585551 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ec81a10e0 0x7f1ec81a34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:28.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.766+0000 7f1ec6ffd640 1 --2- 172.21.15.105:0/1103585551 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ec81a10e0 0x7f1ec81a34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:28.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.766+0000 7f1ec6ffd640 1 -- 172.21.15.105:0/1103585551 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec81a3d20 con 0x7f1ec8106590 2024-09-16T01:54:28.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.766+0000 7f1ec77fe640 1 --2- 172.21.15.105:0/1103585551 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ec81a10e0 0x7f1ec81a34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:54:28.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.766+0000 7f1ec6ffd640 1 --2- 172.21.15.105:0/1103585551 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ec8106590 0x7f1ec8198070 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8009230 tx=0x7f1eb8003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:28.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.766+0000 7f1eaffff640 1 -- 172.21.15.105:0/1103585551 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb803f070 con 0x7f1ec8106590 2024-09-16T01:54:28.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.766+0000 7f1eaffff640 1 -- 172.21.15.105:0/1103585551 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1eb8031d60 con 0x7f1ec8106590 2024-09-16T01:54:28.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.766+0000 7f1ecd2fe640 1 -- 172.21.15.105:0/1103585551 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ec81a3f50 con 0x7f1ec8106590 2024-09-16T01:54:28.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.766+0000 7f1eaffff640 1 -- 172.21.15.105:0/1103585551 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8035030 con 0x7f1ec8106590 2024-09-16T01:54:28.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.766+0000 7f1ecd2fe640 1 -- 172.21.15.105:0/1103585551 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ec81a43a0 con 0x7f1ec8106590 2024-09-16T01:54:28.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.767+0000 7f1eaffff640 1 -- 172.21.15.105:0/1103585551 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f1eb8042070 con 0x7f1ec8106590 2024-09-16T01:54:28.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.767+0000 7f1ecd2fe640 1 -- 172.21.15.105:0/1103585551 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ec81069f0 con 0x7f1ec8106590 2024-09-16T01:54:28.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.767+0000 7f1eaffff640 1 --2- 172.21.15.105:0/1103585551 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f1eb0039730 0x7f1eb003bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:28.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.767+0000 7f1eaffff640 1 -- 172.21.15.105:0/1103585551 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1eb806d050 con 0x7f1ec8106590 2024-09-16T01:54:28.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.767+0000 7f1ec67fc640 1 --2- 172.21.15.105:0/1103585551 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f1eb0039730 0x7f1eb003bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:28.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.768+0000 7f1ec67fc640 1 --2- 172.21.15.105:0/1103585551 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f1eb0039730 0x7f1eb003bbf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc009950 tx=0x7f1ebc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:28.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.772+0000 7f1eaffff640 1 -- 172.21.15.105:0/1103585551 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1eb8042bc0 con 0x7f1ec8106590 2024-09-16T01:54:28.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.947+0000 7f1ecd2fe640 1 -- 172.21.15.105:0/1103585551 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f1ec81a47b0 con 0x7f1ec8106590 2024-09-16T01:54:28.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.948+0000 7f1eaffff640 1 -- 172.21.15.105:0/1103585551 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7f1eb803a890 con 0x7f1ec8106590 2024-09-16T01:54:28.949 INFO:teuthology.orchestra.run.smithi105.stdout:# minimal ceph.conf for 423b7a2a-73ce-11ef-bceb-c7b262605968 2024-09-16T01:54:28.949 INFO:teuthology.orchestra.run.smithi105.stdout:[global] 2024-09-16T01:54:28.949 INFO:teuthology.orchestra.run.smithi105.stdout: fsid = 423b7a2a-73ce-11ef-bceb-c7b262605968 2024-09-16T01:54:28.949 INFO:teuthology.orchestra.run.smithi105.stdout: mon_host = [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-09-16T01:54:28.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.950+0000 7f1ecd2fe640 1 -- 172.21.15.105:0/1103585551 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f1eb0039730 msgr2=0x7f1eb003bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:28.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.950+0000 7f1ecd2fe640 1 --2- 172.21.15.105:0/1103585551 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f1eb0039730 0x7f1eb003bbf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc009950 tx=0x7f1ebc008040 comp rx=0 tx=0).stop 2024-09-16T01:54:28.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.950+0000 7f1ecd2fe640 1 -- 172.21.15.105:0/1103585551 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ec8106590 msgr2=0x7f1ec8198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:28.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.950+0000 7f1ecd2fe640 1 --2- 172.21.15.105:0/1103585551 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ec8106590 0x7f1ec8198070 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8009230 tx=0x7f1eb8003d50 comp rx=0 tx=0).stop 2024-09-16T01:54:28.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.950+0000 7f1ecd2fe640 1 -- 172.21.15.105:0/1103585551 shutdown_connections 2024-09-16T01:54:28.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.950+0000 7f1ecd2fe640 1 --2- 172.21.15.105:0/1103585551 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f1eb0039730 0x7f1eb003bbf0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:28.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.950+0000 7f1ecd2fe640 1 --2- 172.21.15.105:0/1103585551 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ec81985b0 0x7f1ec8198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:28.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.950+0000 7f1ecd2fe640 1 --2- 172.21.15.105:0/1103585551 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ec81a10e0 0x7f1ec81a34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:28.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.950+0000 7f1ecd2fe640 1 --2- 172.21.15.105:0/1103585551 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ec8106590 0x7f1ec8198070 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:28.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.950+0000 7f1ecd2fe640 1 -- 172.21.15.105:0/1103585551 >> 172.21.15.105:0/1103585551 conn(0x7f1ec80fc1a0 msgr2=0x7f1ec80fc580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:28.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.950+0000 7f1ecd2fe640 1 -- 172.21.15.105:0/1103585551 shutdown_connections 2024-09-16T01:54:28.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:28.951+0000 7f1ecd2fe640 1 -- 172.21.15.105:0/1103585551 wait complete. 2024-09-16T01:54:29.616 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:29 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/1103585551' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:29.616 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:29 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:29.617 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:29 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:29.617 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:29 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:54:29.617 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:29 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T01:54:29.617 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:29 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:29.791 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:29 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/1103585551' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:29.791 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:29 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:29.791 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:29 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:29.791 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:29 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:54:29.791 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:29 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T01:54:29.791 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:29 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:29.817 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-16T01:54:29.817 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-16T01:54:29.817 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/ceph/ceph.conf 2024-09-16T01:54:29.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:29 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/1103585551' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:29.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:29 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:29.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:29 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:29.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:29 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:54:29.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:29 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T01:54:29.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:29 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:29.844 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-16T01:54:29.845 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T01:54:29.910 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-16T01:54:29.910 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/ceph.conf 2024-09-16T01:54:29.939 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-16T01:54:29.939 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T01:54:30.004 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-16T01:54:30.004 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/ceph.conf 2024-09-16T01:54:30.036 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-16T01:54:30.036 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T01:54:30.105 INFO:tasks.cephadm:Adding mgr.x on smithi105 2024-09-16T01:54:30.105 INFO:tasks.cephadm:Adding mgr.y on smithi137 2024-09-16T01:54:30.105 INFO:tasks.cephadm:Adding mgr.z on smithi190 2024-09-16T01:54:30.106 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph orch apply mgr '3;smithi105=x;smithi137=y;smithi190=z' 2024-09-16T01:54:30.391 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T01:54:30.644 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:30 smithi190 ceph-mon[34762]: Reconfiguring mon.b (monmap changed)... 2024-09-16T01:54:30.644 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:30 smithi190 ceph-mon[34762]: Reconfiguring daemon mon.b on smithi137 2024-09-16T01:54:30.644 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:30 smithi190 ceph-mon[34762]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:30.644 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:30 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:30.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:30 smithi137 ceph-mon[34099]: Reconfiguring mon.b (monmap changed)... 2024-09-16T01:54:30.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:30 smithi137 ceph-mon[34099]: Reconfiguring daemon mon.b on smithi137 2024-09-16T01:54:30.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:30 smithi137 ceph-mon[34099]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:30.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:30 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:30.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:30 smithi105 ceph-mon[31197]: Reconfiguring mon.b (monmap changed)... 2024-09-16T01:54:30.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:30 smithi105 ceph-mon[31197]: Reconfiguring daemon mon.b on smithi137 2024-09-16T01:54:30.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:30 smithi105 ceph-mon[31197]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:30.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:30 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:30.851 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:30.849+0000 7f087964d640 1 -- 172.21.15.190:0/1011356974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0874100610 msgr2=0x7f0874100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:30.851 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:30.849+0000 7f087964d640 1 --2- 172.21.15.190:0/1011356974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0874100610 0x7f0874100a10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0864009c20 tx=0x7f086402aa90 comp rx=0 tx=0).stop 2024-09-16T01:54:30.851 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:30.849+0000 7f087964d640 1 -- 172.21.15.190:0/1011356974 shutdown_connections 2024-09-16T01:54:30.851 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:30.849+0000 7f087964d640 1 --2- 172.21.15.190:0/1011356974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0874100610 0x7f0874100a10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:30.851 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:30.849+0000 7f087964d640 1 -- 172.21.15.190:0/1011356974 >> 172.21.15.190:0/1011356974 conn(0x7f08740fbda0 msgr2=0x7f08740fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:30.851 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:30.849+0000 7f087964d640 1 -- 172.21.15.190:0/1011356974 shutdown_connections 2024-09-16T01:54:30.851 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:30.849+0000 7f087964d640 1 -- 172.21.15.190:0/1011356974 wait complete. 2024-09-16T01:54:30.851 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:30.850+0000 7f087964d640 1 Processor -- start 2024-09-16T01:54:30.851 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:30.850+0000 7f087964d640 1 -- start start 2024-09-16T01:54:30.854 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:30.850+0000 7f087964d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0874100610 0x7f0874195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:30.854 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:30.850+0000 7f087964d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0874196330 0x7f087419b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:30.854 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:30.850+0000 7f087964d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08741967b0 0x7f087419d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:30.854 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:30.850+0000 7f087964d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f08741065d0 con 0x7f0874100610 2024-09-16T01:54:30.854 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:30.850+0000 7f087964d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0874106450 con 0x7f0874196330 2024-09-16T01:54:30.854 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:30.850+0000 7f087964d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0874106750 con 0x7f08741967b0 2024-09-16T01:54:30.854 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:30.850+0000 7f08737fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08741967b0 0x7f087419d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:30.854 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:30.850+0000 7f08737fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08741967b0 0x7f087419d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:58830/0 (socket says 172.21.15.190:58830) 2024-09-16T01:54:30.854 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:30.850+0000 7f08737fe640 1 -- 172.21.15.190:0/2327877637 learned_addr learned my addr 172.21.15.190:0/2327877637 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-09-16T01:54:30.854 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:30.850+0000 7f0872ffd640 1 --2- 172.21.15.190:0/2327877637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0874100610 0x7f0874195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:30.855 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:30.850+0000 7f08727fc640 1 --2- 172.21.15.190:0/2327877637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0874196330 0x7f087419b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:30.855 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:30.850+0000 7f08737fe640 1 -- 172.21.15.190:0/2327877637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0874196330 msgr2=0x7f087419b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:30.855 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:30.850+0000 7f08737fe640 1 --2- 172.21.15.190:0/2327877637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0874196330 0x7f087419b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:30.855 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:30.850+0000 7f08737fe640 1 -- 172.21.15.190:0/2327877637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0874100610 msgr2=0x7f0874195df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:30.855 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:30.850+0000 7f08737fe640 1 --2- 172.21.15.190:0/2327877637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0874100610 0x7f0874195df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:30.855 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:30.850+0000 7f08737fe640 1 -- 172.21.15.190:0/2327877637 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f087419e1f0 con 0x7f08741967b0 2024-09-16T01:54:30.855 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:30.850+0000 7f08727fc640 1 --2- 172.21.15.190:0/2327877637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0874196330 0x7f087419b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:54:30.855 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:30.851+0000 7f08737fe640 1 --2- 172.21.15.190:0/2327877637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08741967b0 0x7f087419d930 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f086c00b450 tx=0x7f086c00b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:30.855 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:30.851+0000 7f086bfff640 1 -- 172.21.15.190:0/2327877637 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086c015020 con 0x7f08741967b0 2024-09-16T01:54:30.855 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:30.851+0000 7f086bfff640 1 -- 172.21.15.190:0/2327877637 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f086c011d40 con 0x7f08741967b0 2024-09-16T01:54:30.855 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:30.851+0000 7f087964d640 1 -- 172.21.15.190:0/2327877637 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f087419e4e0 con 0x7f08741967b0 2024-09-16T01:54:30.855 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:30.851+0000 7f086bfff640 1 -- 172.21.15.190:0/2327877637 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086c00f340 con 0x7f08741967b0 2024-09-16T01:54:30.855 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:30.851+0000 7f087964d640 1 -- 172.21.15.190:0/2327877637 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f087406a4b0 con 0x7f08741967b0 2024-09-16T01:54:30.855 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:30.852+0000 7f086bfff640 1 -- 172.21.15.190:0/2327877637 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f086c018020 con 0x7f08741967b0 2024-09-16T01:54:30.856 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:30.852+0000 7f087964d640 1 -- 172.21.15.190:0/2327877637 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f087406af30 con 0x7f08741967b0 2024-09-16T01:54:30.856 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:30.852+0000 7f086bfff640 1 --2- 172.21.15.190:0/2327877637 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f086003b940 0x7f086003de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:30.856 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:30.852+0000 7f086bfff640 1 -- 172.21.15.190:0/2327877637 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f086c04c1b0 con 0x7f08741967b0 2024-09-16T01:54:30.856 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:30.852+0000 7f0872ffd640 1 --2- 172.21.15.190:0/2327877637 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f086003b940 0x7f086003de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:30.856 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:30.853+0000 7f0872ffd640 1 --2- 172.21.15.190:0/2327877637 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f086003b940 0x7f086003de00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0864000f80 tx=0x7f086403a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:30.858 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:30.857+0000 7f086bfff640 1 -- 172.21.15.190:0/2327877637 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f086c01f020 con 0x7f08741967b0 2024-09-16T01:54:31.049 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:31.047+0000 7f087964d640 1 -- 172.21.15.190:0/2327877637 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi105=x;smithi137=y;smithi190=z", "target": ["mon-mgr", ""]}) -- 0x7f087418ef40 con 0x7f086003b940 2024-09-16T01:54:31.055 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:31.054+0000 7f086bfff640 1 -- 172.21.15.190:0/2327877637 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f087418ef40 con 0x7f086003b940 2024-09-16T01:54:31.055 INFO:teuthology.orchestra.run.smithi190.stdout:Scheduled mgr update... 2024-09-16T01:54:31.057 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:31.056+0000 7f087964d640 1 -- 172.21.15.190:0/2327877637 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f086003b940 msgr2=0x7f086003de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:31.057 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:31.056+0000 7f087964d640 1 --2- 172.21.15.190:0/2327877637 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f086003b940 0x7f086003de00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0864000f80 tx=0x7f086403a040 comp rx=0 tx=0).stop 2024-09-16T01:54:31.057 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:31.056+0000 7f087964d640 1 -- 172.21.15.190:0/2327877637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08741967b0 msgr2=0x7f087419d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:31.057 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:31.056+0000 7f087964d640 1 --2- 172.21.15.190:0/2327877637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08741967b0 0x7f087419d930 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f086c00b450 tx=0x7f086c00b910 comp rx=0 tx=0).stop 2024-09-16T01:54:31.057 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:31.056+0000 7f087964d640 1 -- 172.21.15.190:0/2327877637 shutdown_connections 2024-09-16T01:54:31.058 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:31.056+0000 7f087964d640 1 --2- 172.21.15.190:0/2327877637 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f086003b940 0x7f086003de00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:31.058 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:31.056+0000 7f087964d640 1 --2- 172.21.15.190:0/2327877637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08741967b0 0x7f087419d930 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:31.058 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:31.056+0000 7f087964d640 1 --2- 172.21.15.190:0/2327877637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0874196330 0x7f087419b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:31.058 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:31.056+0000 7f087964d640 1 --2- 172.21.15.190:0/2327877637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0874100610 0x7f0874195df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:31.058 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:31.056+0000 7f087964d640 1 -- 172.21.15.190:0/2327877637 >> 172.21.15.190:0/2327877637 conn(0x7f08740fbda0 msgr2=0x7f08740fc930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:31.058 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:31.056+0000 7f087964d640 1 -- 172.21.15.190:0/2327877637 shutdown_connections 2024-09-16T01:54:31.058 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:54:31.056+0000 7f087964d640 1 -- 172.21.15.190:0/2327877637 wait complete. 2024-09-16T01:54:31.735 DEBUG:teuthology.orchestra.run.smithi137:mgr.y> sudo journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mgr.y.service 2024-09-16T01:54:31.738 DEBUG:teuthology.orchestra.run.smithi190:mgr.z> sudo journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mgr.z.service 2024-09-16T01:54:31.743 INFO:tasks.cephadm:Deploying OSDs... 2024-09-16T01:54:31.743 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-16T01:54:31.743 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T01:54:31.763 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T01:54:31.764 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_1 2024-09-16T01:54:31.764 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:31 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:31.765 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:31 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:31.765 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:31 smithi190 ceph-mon[34762]: Reconfiguring mon.c (monmap changed)... 2024-09-16T01:54:31.765 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:31 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:54:31.765 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:31 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T01:54:31.765 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:31 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:31.765 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:31 smithi190 ceph-mon[34762]: Reconfiguring daemon mon.c on smithi190 2024-09-16T01:54:31.765 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:31 smithi190 ceph-mon[34762]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi105=x;smithi137=y;smithi190=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:54:31.765 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:31 smithi190 ceph-mon[34762]: Saving service mgr spec with placement smithi105=x;smithi137=y;smithi190=z;count:3 2024-09-16T01:54:31.766 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:31 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:31.766 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:31 smithi190 ceph-mon[34762]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:31.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:31 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:31.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:31 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:31.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:31 smithi105 ceph-mon[31197]: Reconfiguring mon.c (monmap changed)... 2024-09-16T01:54:31.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:31 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:54:31.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:31 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T01:54:31.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:31 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:31.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:31 smithi105 ceph-mon[31197]: Reconfiguring daemon mon.c on smithi190 2024-09-16T01:54:31.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:31 smithi105 ceph-mon[31197]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi105=x;smithi137=y;smithi190=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:54:31.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:31 smithi105 ceph-mon[31197]: Saving service mgr spec with placement smithi105=x;smithi137=y;smithi190=z;count:3 2024-09-16T01:54:31.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:31 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:31.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:31 smithi105 ceph-mon[31197]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:31.813 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T01:54:31.813 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T01:54:31.813 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-16T01:54:31.814 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T01:54:31.814 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T01:54:31.814 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-09-16 01:53:50.266911608 +0000 2024-09-16T01:54:31.814 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-09-16 01:49:46.338456283 +0000 2024-09-16T01:54:31.814 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-09-16 01:49:46.338456283 +0000 2024-09-16T01:54:31.814 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-09-16 01:49:46.338456283 +0000 2024-09-16T01:54:31.814 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T01:54:31.880 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-09-16T01:54:31.880 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-09-16T01:54:31.880 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000289143 s, 1.8 MB/s 2024-09-16T01:54:31.882 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T01:54:31.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:31 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:31.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:31 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:31.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:31 smithi137 ceph-mon[34099]: Reconfiguring mon.c (monmap changed)... 2024-09-16T01:54:31.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:31 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:54:31.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:31 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T01:54:31.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:31 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:31.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:31 smithi137 ceph-mon[34099]: Reconfiguring daemon mon.c on smithi190 2024-09-16T01:54:31.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:31 smithi137 ceph-mon[34099]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi105=x;smithi137=y;smithi190=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:54:31.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:31 smithi137 ceph-mon[34099]: Saving service mgr spec with placement smithi105=x;smithi137=y;smithi190=z;count:3 2024-09-16T01:54:31.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:31 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:31.923 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:31 smithi137 ceph-mon[34099]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:31.941 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_2 2024-09-16T01:54:32.000 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T01:54:32.000 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T01:54:32.000 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-16T01:54:32.001 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T01:54:32.001 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T01:54:32.001 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-09-16 01:53:50.291911114 +0000 2024-09-16T01:54:32.001 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-09-16 01:49:46.664449831 +0000 2024-09-16T01:54:32.001 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-09-16 01:49:46.664449831 +0000 2024-09-16T01:54:32.001 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-09-16 01:49:46.664449831 +0000 2024-09-16T01:54:32.001 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T01:54:32.068 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-09-16T01:54:32.069 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-09-16T01:54:32.069 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000193637 s, 2.6 MB/s 2024-09-16T01:54:32.070 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T01:54:32.130 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_3 2024-09-16T01:54:32.188 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T01:54:32.189 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T01:54:32.189 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-16T01:54:32.189 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T01:54:32.189 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T01:54:32.189 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-09-16 01:53:50.316910620 +0000 2024-09-16T01:54:32.189 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-09-16 01:49:46.996443260 +0000 2024-09-16T01:54:32.189 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-09-16 01:49:46.996443260 +0000 2024-09-16T01:54:32.189 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-09-16 01:49:46.996443260 +0000 2024-09-16T01:54:32.190 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T01:54:32.256 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-09-16T01:54:32.256 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-09-16T01:54:32.256 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.00020833 s, 2.5 MB/s 2024-09-16T01:54:32.257 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T01:54:32.317 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_4 2024-09-16T01:54:32.375 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T01:54:32.375 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T01:54:32.375 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-16T01:54:32.376 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T01:54:32.376 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T01:54:32.376 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-09-16 01:53:50.342910107 +0000 2024-09-16T01:54:32.376 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-09-16 01:49:47.335436551 +0000 2024-09-16T01:54:32.376 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-09-16 01:49:47.335436551 +0000 2024-09-16T01:54:32.376 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-09-16 01:49:47.335436551 +0000 2024-09-16T01:54:32.376 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T01:54:32.443 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-09-16T01:54:32.443 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-09-16T01:54:32.443 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000228288 s, 2.2 MB/s 2024-09-16T01:54:32.445 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T01:54:32.503 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-16T01:54:32.503 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T01:54:32.522 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T01:54:32.523 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_1 2024-09-16T01:54:32.581 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T01:54:32.581 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T01:54:32.581 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-16T01:54:32.581 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T01:54:32.581 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T01:54:32.581 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-09-16 01:54:20.148382113 +0000 2024-09-16T01:54:32.581 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-09-16 01:49:50.626587665 +0000 2024-09-16T01:54:32.581 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-09-16 01:49:50.626587665 +0000 2024-09-16T01:54:32.582 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-09-16 01:49:50.626587665 +0000 2024-09-16T01:54:32.582 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T01:54:32.648 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-09-16T01:54:32.648 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-09-16T01:54:32.648 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000259934 s, 2.0 MB/s 2024-09-16T01:54:32.649 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T01:54:32.707 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_2 2024-09-16T01:54:32.764 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T01:54:32.764 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T01:54:32.764 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-16T01:54:32.764 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T01:54:32.764 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T01:54:32.764 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-09-16 01:54:20.148382113 +0000 2024-09-16T01:54:32.765 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-09-16 01:49:50.937582360 +0000 2024-09-16T01:54:32.765 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-09-16 01:49:50.937582360 +0000 2024-09-16T01:54:32.765 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-09-16 01:49:50.937582360 +0000 2024-09-16T01:54:32.765 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T01:54:32.829 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-09-16T01:54:32.829 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-09-16T01:54:32.829 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000195907 s, 2.6 MB/s 2024-09-16T01:54:32.830 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T01:54:32.887 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_3 2024-09-16T01:54:32.943 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T01:54:32.943 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T01:54:32.944 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-16T01:54:32.944 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T01:54:32.944 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T01:54:32.944 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-09-16 01:54:20.148382113 +0000 2024-09-16T01:54:32.944 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-09-16 01:49:51.255576938 +0000 2024-09-16T01:54:32.944 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-09-16 01:49:51.255576938 +0000 2024-09-16T01:54:32.944 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-09-16 01:49:51.255576938 +0000 2024-09-16T01:54:32.944 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T01:54:33.008 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-09-16T01:54:33.008 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-09-16T01:54:33.008 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000223787 s, 2.3 MB/s 2024-09-16T01:54:33.010 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T01:54:33.067 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_4 2024-09-16T01:54:33.123 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T01:54:33.123 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T01:54:33.123 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-16T01:54:33.123 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T01:54:33.123 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T01:54:33.123 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-09-16 01:54:20.149382096 +0000 2024-09-16T01:54:33.123 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-09-16 01:49:51.584571328 +0000 2024-09-16T01:54:33.123 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-09-16 01:49:51.584571328 +0000 2024-09-16T01:54:33.123 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-09-16 01:49:51.584571328 +0000 2024-09-16T01:54:33.124 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T01:54:33.187 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-09-16T01:54:33.187 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-09-16T01:54:33.187 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000184211 s, 2.8 MB/s 2024-09-16T01:54:33.188 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T01:54:33.244 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-16T01:54:33.244 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T01:54:33.260 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T01:54:33.260 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_1 2024-09-16T01:54:33.316 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T01:54:33.316 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T01:54:33.316 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-16T01:54:33.316 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T01:54:33.316 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T01:54:33.316 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-09-16 01:54:11.329769040 +0000 2024-09-16T01:54:33.316 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-09-16 01:49:40.516011477 +0000 2024-09-16T01:54:33.316 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-09-16 01:49:40.516011477 +0000 2024-09-16T01:54:33.317 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-09-16 01:49:40.516011477 +0000 2024-09-16T01:54:33.317 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T01:54:33.380 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-09-16T01:54:33.380 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-09-16T01:54:33.381 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000212414 s, 2.4 MB/s 2024-09-16T01:54:33.385 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T01:54:33.441 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_2 2024-09-16T01:54:33.497 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T01:54:33.497 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T01:54:33.497 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-16T01:54:33.497 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T01:54:33.497 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T01:54:33.497 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-09-16 01:54:11.329769040 +0000 2024-09-16T01:54:33.497 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-09-16 01:49:40.867001392 +0000 2024-09-16T01:54:33.497 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-09-16 01:49:40.867001392 +0000 2024-09-16T01:54:33.497 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-09-16 01:49:40.867001392 +0000 2024-09-16T01:54:33.498 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T01:54:33.560 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:33 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:33.560 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:33 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:33.560 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:33 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:54:33.560 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:33 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:33.560 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:33 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:54:33.560 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:33 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:33.560 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:33 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T01:54:33.560 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:33 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T01:54:33.560 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:33 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T01:54:33.560 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:33 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:33.560 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:33 smithi190 ceph-mon[34762]: Deploying daemon mgr.z on smithi190 2024-09-16T01:54:33.561 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-09-16T01:54:33.562 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-09-16T01:54:33.562 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.00019451 s, 2.6 MB/s 2024-09-16T01:54:33.563 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T01:54:33.619 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_3 2024-09-16T01:54:33.675 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T01:54:33.675 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T01:54:33.675 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-16T01:54:33.675 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T01:54:33.675 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T01:54:33.675 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-09-16 01:54:11.329769040 +0000 2024-09-16T01:54:33.675 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-09-16 01:49:41.190992084 +0000 2024-09-16T01:54:33.675 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-09-16 01:49:41.190992084 +0000 2024-09-16T01:54:33.676 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-09-16 01:49:41.190992084 +0000 2024-09-16T01:54:33.676 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T01:54:33.738 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-09-16T01:54:33.738 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-09-16T01:54:33.738 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000189654 s, 2.7 MB/s 2024-09-16T01:54:33.740 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T01:54:33.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:33 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:33.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:33 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:33.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:33 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:54:33.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:33 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:33.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:33 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:54:33.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:33 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:33.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:33 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T01:54:33.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:33 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T01:54:33.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:33 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T01:54:33.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:33 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:33.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:33 smithi105 ceph-mon[31197]: Deploying daemon mgr.z on smithi190 2024-09-16T01:54:33.796 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_4 2024-09-16T01:54:33.851 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T01:54:33.851 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T01:54:33.851 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-16T01:54:33.851 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T01:54:33.851 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T01:54:33.852 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-09-16 01:54:11.329769040 +0000 2024-09-16T01:54:33.852 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-09-16 01:49:41.505983035 +0000 2024-09-16T01:54:33.852 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-09-16 01:49:41.505983035 +0000 2024-09-16T01:54:33.852 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-09-16 01:49:41.505983035 +0000 2024-09-16T01:54:33.852 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T01:54:33.914 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-09-16T01:54:33.915 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-09-16T01:54:33.915 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000190346 s, 2.7 MB/s 2024-09-16T01:54:33.916 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T01:54:33.920 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:33 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:33.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:33 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:33.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:33 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:54:33.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:33 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:33.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:33 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:54:33.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:33 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:33.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:33 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T01:54:33.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:33 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T01:54:33.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:33 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T01:54:33.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:33 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:33.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:33 smithi137 ceph-mon[34099]: Deploying daemon mgr.z on smithi190 2024-09-16T01:54:33.973 INFO:tasks.cephadm:Deploying osd.0 on smithi105 with /dev/vg_nvme/lv_4... 2024-09-16T01:54:33.973 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-16T01:54:34.233 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:54:34.498 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 01:54:34 smithi190 systemd[1]: Starting Ceph mgr.z for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T01:54:34.636 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:34 smithi105 ceph-mon[31197]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:34.750 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:34 smithi190 ceph-mon[34762]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:34.750 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 01:54:34 smithi190 podman[36534]: 2024-09-16 01:54:34.420426861 +0000 UTC m=+0.015995387 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T01:54:34.750 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 01:54:34 smithi190 podman[36534]: 2024-09-16 01:54:34.535832286 +0000 UTC m=+0.131400846 container create ccc24b8c20c82dd74f477c5c0d212485719db2244c0b44208fbc39cccb1f354f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-16T01:54:34.751 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 01:54:34 smithi190 podman[36534]: 2024-09-16 01:54:34.746736843 +0000 UTC m=+0.342305364 container init ccc24b8c20c82dd74f477c5c0d212485719db2244c0b44208fbc39cccb1f354f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, io.buildah.version=1.37.2) 2024-09-16T01:54:34.751 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 01:54:34 smithi190 podman[36534]: 2024-09-16 01:54:34.749557676 +0000 UTC m=+0.345126191 container start ccc24b8c20c82dd74f477c5c0d212485719db2244c0b44208fbc39cccb1f354f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-z, GIT_CLEAN=True, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-16T01:54:34.920 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:34 smithi137 ceph-mon[34099]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:35.078 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 01:54:34 smithi190 ceph-mgr[36553]: -- 172.21.15.190:0/436483169 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f5de5810e0 con 0x55f5de55e800 2024-09-16T01:54:35.078 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 01:54:34 smithi190 bash[36534]: ccc24b8c20c82dd74f477c5c0d212485719db2244c0b44208fbc39cccb1f354f 2024-09-16T01:54:35.078 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 01:54:34 smithi190 systemd[1]: Started Ceph mgr.z for 423b7a2a-73ce-11ef-bceb-c7b262605968. 2024-09-16T01:54:35.078 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 01:54:34 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-z[36549]: 2024-09-16T01:54:34.870+0000 7f9073757040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T01:54:35.078 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 01:54:34 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-z[36549]: 2024-09-16T01:54:34.930+0000 7f9073757040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T01:54:35.078 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 01:54:35 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-z[36549]: 2024-09-16T01:54:35.076+0000 7f9073757040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T01:54:35.342 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 01:54:35 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-z[36549]: 2024-09-16T01:54:35.143+0000 7f9073757040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T01:54:35.342 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 01:54:35 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-z[36549]: 2024-09-16T01:54:35.290+0000 7f9073757040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T01:54:35.739 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 01:54:35 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-z[36549]: 2024-09-16T01:54:35.428+0000 7f9073757040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T01:54:35.740 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 01:54:35 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-z[36549]: 2024-09-16T01:54:35.738+0000 7f9073757040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T01:54:36.092 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 01:54:35 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-z[36549]: 2024-09-16T01:54:35.798+0000 7f9073757040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T01:54:36.093 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 01:54:35 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-z[36549]: 2024-09-16T01:54:35.981+0000 7f9073757040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T01:54:36.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:35 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:36.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:35 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:36.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:35 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:36.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:35 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T01:54:36.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:35 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T01:54:36.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:35 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T01:54:36.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:35 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:36.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:35 smithi190 ceph-mon[34762]: Deploying daemon mgr.y on smithi137 2024-09-16T01:54:36.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:35 smithi190 ceph-mon[34762]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:36.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:35 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:36.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:35 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:36.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:35 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:36.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:35 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T01:54:36.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:35 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T01:54:36.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:35 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T01:54:36.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:35 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:36.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:35 smithi137 ceph-mon[34099]: Deploying daemon mgr.y on smithi137 2024-09-16T01:54:36.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:35 smithi137 ceph-mon[34099]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:36.172 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:35 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:36.172 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:35 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:36.172 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:35 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:36.172 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:35 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T01:54:36.172 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:35 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T01:54:36.172 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:35 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T01:54:36.173 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:35 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:36.173 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:35 smithi105 ceph-mon[31197]: Deploying daemon mgr.y on smithi137 2024-09-16T01:54:36.173 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:35 smithi105 ceph-mon[31197]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:36.707 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:54:36.733 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi105:vg_nvme/lv_4 2024-09-16T01:54:36.897 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 16 01:54:36 smithi137 systemd[1]: Starting Ceph mgr.y for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T01:54:36.973 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:54:37.171 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 16 01:54:36 smithi137 podman[35442]: 2024-09-16 01:54:36.896460497 +0000 UTC m=+0.099936884 container create 19c45ee05332cec61ec7f1d7926fcff04e954f8d922238a5ea0daa3345cb20b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T01:54:37.171 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 16 01:54:36 smithi137 podman[35442]: 2024-09-16 01:54:36.812459791 +0000 UTC m=+0.015936190 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T01:54:37.171 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 16 01:54:37 smithi137 podman[35442]: 2024-09-16 01:54:37.098727465 +0000 UTC m=+0.302203857 container init 19c45ee05332cec61ec7f1d7926fcff04e954f8d922238a5ea0daa3345cb20b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-y, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-16T01:54:37.171 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 16 01:54:37 smithi137 podman[35442]: 2024-09-16 01:54:37.101424225 +0000 UTC m=+0.304900621 container start 19c45ee05332cec61ec7f1d7926fcff04e954f8d922238a5ea0daa3345cb20b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-y, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-16T01:54:37.171 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 16 01:54:37 smithi137 ceph-mgr[35461]: -- 172.21.15.137:0/3615102590 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564b75b910e0 con 0x564b75b6f400 2024-09-16T01:54:37.342 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 01:54:36 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-z[36549]: 2024-09-16T01:54:36.851+0000 7f9073757040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T01:54:37.342 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 01:54:36 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-z[36549]: 2024-09-16T01:54:36.917+0000 7f9073757040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T01:54:37.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.406+0000 7f2e29f8f640 1 -- 172.21.15.105:0/724526988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e24102690 msgr2=0x7f2e24108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:37.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.406+0000 7f2e29f8f640 1 --2- 172.21.15.105:0/724526988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e24102690 0x7f2e24108f20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c009ca0 tx=0x7f2e1c02a8c0 comp rx=0 tx=0).stop 2024-09-16T01:54:37.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.406+0000 7f2e29f8f640 1 -- 172.21.15.105:0/724526988 shutdown_connections 2024-09-16T01:54:37.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.406+0000 7f2e29f8f640 1 --2- 172.21.15.105:0/724526988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e24102690 0x7f2e24108f20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:37.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.406+0000 7f2e29f8f640 1 --2- 172.21.15.105:0/724526988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e24101cd0 0x7f2e24102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:37.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.406+0000 7f2e29f8f640 1 --2- 172.21.15.105:0/724526988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e24100ad0 0x7f2e24100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:37.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.406+0000 7f2e29f8f640 1 -- 172.21.15.105:0/724526988 >> 172.21.15.105:0/724526988 conn(0x7f2e240fc280 msgr2=0x7f2e240fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:37.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.406+0000 7f2e29f8f640 1 -- 172.21.15.105:0/724526988 shutdown_connections 2024-09-16T01:54:37.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.406+0000 7f2e29f8f640 1 -- 172.21.15.105:0/724526988 wait complete. 2024-09-16T01:54:37.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.407+0000 7f2e29f8f640 1 Processor -- start 2024-09-16T01:54:37.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.407+0000 7f2e29f8f640 1 -- start start 2024-09-16T01:54:37.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.407+0000 7f2e29f8f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e24100ad0 0x7f2e24197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:37.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.407+0000 7f2e29f8f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e24101cd0 0x7f2e24198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:37.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.407+0000 7f2e29f8f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e24102690 0x7f2e2419f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:37.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.407+0000 7f2e29f8f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2410d730 con 0x7f2e24102690 2024-09-16T01:54:37.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.407+0000 7f2e29f8f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2410d5b0 con 0x7f2e24101cd0 2024-09-16T01:54:37.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.407+0000 7f2e29f8f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2410d8b0 con 0x7f2e24100ad0 2024-09-16T01:54:37.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.407+0000 7f2e23fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e24102690 0x7f2e2419f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:37.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.407+0000 7f2e23fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e24102690 0x7f2e2419f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44800/0 (socket says 172.21.15.105:44800) 2024-09-16T01:54:37.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.407+0000 7f2e237fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e24100ad0 0x7f2e24197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:37.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.407+0000 7f2e23fff640 1 -- 172.21.15.105:0/3544816369 learned_addr learned my addr 172.21.15.105:0/3544816369 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:54:37.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.407+0000 7f2e1bfff640 1 --2- 172.21.15.105:0/3544816369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e24101cd0 0x7f2e24198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:37.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.407+0000 7f2e23fff640 1 -- 172.21.15.105:0/3544816369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e24100ad0 msgr2=0x7f2e24197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:37.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.407+0000 7f2e23fff640 1 --2- 172.21.15.105:0/3544816369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e24100ad0 0x7f2e24197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:37.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.407+0000 7f2e23fff640 1 -- 172.21.15.105:0/3544816369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e24101cd0 msgr2=0x7f2e24198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:37.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.407+0000 7f2e23fff640 1 --2- 172.21.15.105:0/3544816369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e24101cd0 0x7f2e24198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:37.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.407+0000 7f2e23fff640 1 -- 172.21.15.105:0/3544816369 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e2419fa00 con 0x7f2e24102690 2024-09-16T01:54:37.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.408+0000 7f2e23fff640 1 --2- 172.21.15.105:0/3544816369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e24102690 0x7f2e2419f390 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c009dc0 tx=0x7f2e1c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:37.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.408+0000 7f2e217fa640 1 -- 172.21.15.105:0/3544816369 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c032ec0 con 0x7f2e24102690 2024-09-16T01:54:37.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.408+0000 7f2e217fa640 1 -- 172.21.15.105:0/3544816369 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e1c0044f0 con 0x7f2e24102690 2024-09-16T01:54:37.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.408+0000 7f2e29f8f640 1 -- 172.21.15.105:0/3544816369 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e2419fc90 con 0x7f2e24102690 2024-09-16T01:54:37.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.408+0000 7f2e217fa640 1 -- 172.21.15.105:0/3544816369 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c030d40 con 0x7f2e24102690 2024-09-16T01:54:37.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.408+0000 7f2e29f8f640 1 -- 172.21.15.105:0/3544816369 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e241a01c0 con 0x7f2e24102690 2024-09-16T01:54:37.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.409+0000 7f2e217fa640 1 -- 172.21.15.105:0/3544816369 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f2e1c02f4c0 con 0x7f2e24102690 2024-09-16T01:54:37.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.409+0000 7f2e217fa640 1 --2- 172.21.15.105:0/3544816369 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f2e100396a0 0x7f2e1003bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:37.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.409+0000 7f2e217fa640 1 -- 172.21.15.105:0/3544816369 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2e1c0305c0 con 0x7f2e24102690 2024-09-16T01:54:37.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.409+0000 7f2e237fe640 1 --2- 172.21.15.105:0/3544816369 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f2e100396a0 0x7f2e1003bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:37.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.409+0000 7f2e29f8f640 1 -- 172.21.15.105:0/3544816369 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e24107f40 con 0x7f2e24102690 2024-09-16T01:54:37.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.409+0000 7f2e237fe640 1 --2- 172.21.15.105:0/3544816369 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f2e100396a0 0x7f2e1003bb60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2e14005fd0 tx=0x7f2e1400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:37.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.414+0000 7f2e217fa640 1 -- 172.21.15.105:0/3544816369 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2e1c0443d0 con 0x7f2e24102690 2024-09-16T01:54:37.454 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 16 01:54:37 smithi137 bash[35442]: 19c45ee05332cec61ec7f1d7926fcff04e954f8d922238a5ea0daa3345cb20b5 2024-09-16T01:54:37.454 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 16 01:54:37 smithi137 systemd[1]: Started Ceph mgr.y for 423b7a2a-73ce-11ef-bceb-c7b262605968. 2024-09-16T01:54:37.454 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 16 01:54:37 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-y[35457]: 2024-09-16T01:54:37.226+0000 7f5bd8c9c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T01:54:37.454 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 16 01:54:37 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-y[35457]: 2024-09-16T01:54:37.295+0000 7f5bd8c9c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T01:54:37.454 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 16 01:54:37 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-y[35457]: 2024-09-16T01:54:37.453+0000 7f5bd8c9c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T01:54:37.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:37.591+0000 7f2e29f8f640 1 -- 172.21.15.105:0/3544816369 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f2e24062630 con 0x7f2e100396a0 2024-09-16T01:54:37.833 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 16 01:54:37 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-y[35457]: 2024-09-16T01:54:37.521+0000 7f5bd8c9c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T01:54:37.833 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 16 01:54:37 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-y[35457]: 2024-09-16T01:54:37.678+0000 7f5bd8c9c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T01:54:37.833 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 16 01:54:37 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-y[35457]: 2024-09-16T01:54:37.831+0000 7f5bd8c9c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T01:54:37.940 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 01:54:37 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-z[36549]: 2024-09-16T01:54:37.662+0000 7f9073757040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T01:54:37.940 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 01:54:37 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-z[36549]: 2024-09-16T01:54:37.797+0000 7f9073757040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T01:54:37.941 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 01:54:37 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-z[36549]: 2024-09-16T01:54:37.939+0000 7f9073757040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T01:54:38.228 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 01:54:38 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-z[36549]: 2024-09-16T01:54:38.000+0000 7f9073757040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T01:54:38.228 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 01:54:38 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-z[36549]: 2024-09-16T01:54:38.158+0000 7f9073757040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T01:54:38.228 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 01:54:38 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-z[36549]: 2024-09-16T01:54:38.226+0000 7f9073757040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T01:54:38.408 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 16 01:54:38 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-y[35457]: 2024-09-16T01:54:38.156+0000 7f5bd8c9c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T01:54:38.409 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 16 01:54:38 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-y[35457]: 2024-09-16T01:54:38.219+0000 7f5bd8c9c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T01:54:38.409 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:38 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:38.409 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:38 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:38.409 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:38 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:38.409 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:38 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:38.409 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:38 smithi137 ceph-mon[34099]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:38.409 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:38 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:54:38.409 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:38 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:54:38.409 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:38 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:54:38.409 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:38 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:38.513 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:38 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:38.513 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:38 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:38.513 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:38 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:38.513 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:38 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:38.513 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:38 smithi105 ceph-mon[31197]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:38.513 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:38 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:54:38.514 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:38 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:54:38.514 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:38 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:54:38.514 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:38 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:38.523 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:38 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:38.523 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:38 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:38.523 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:38 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:38.523 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:38 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:38.523 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:38 smithi190 ceph-mon[34762]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:38.524 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:38 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:54:38.524 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:38 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:54:38.524 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:38 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:54:38.524 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:38 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:38.524 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 01:54:38 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-z[36549]: 2024-09-16T01:54:38.288+0000 7f9073757040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T01:54:38.524 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 01:54:38 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-z[36549]: 2024-09-16T01:54:38.448+0000 7f9073757040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T01:54:38.524 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 01:54:38 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-z[36549]: 2024-09-16T01:54:38.522+0000 7f9073757040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T01:54:38.671 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 16 01:54:38 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-y[35457]: 2024-09-16T01:54:38.407+0000 7f5bd8c9c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T01:54:38.793 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 01:54:38 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-z[36549]: 2024-09-16T01:54:38.588+0000 7f9073757040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T01:54:38.794 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 01:54:38 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-z[36549]: 2024-09-16T01:54:38.672+0000 7f9073757040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T01:54:38.794 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 01:54:38 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-z[36549]: 2024-09-16T01:54:38.732+0000 7f9073757040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T01:54:38.794 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 01:54:38 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-z[36549]: 2024-09-16T01:54:38.791+0000 7f9073757040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T01:54:39.671 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 16 01:54:39 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-y[35457]: 2024-09-16T01:54:39.296+0000 7f5bd8c9c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T01:54:39.671 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 16 01:54:39 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-y[35457]: 2024-09-16T01:54:39.363+0000 7f5bd8c9c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T01:54:39.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:39 smithi137 ceph-mon[34099]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:54:39.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:39 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:39.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:39 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:39.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:39 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:39.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:39 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:39.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:39 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:39.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:39 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:54:39.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:39 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:39.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:39 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T01:54:39.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:39 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T01:54:39.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:39 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:39.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:39 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:39.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:39 smithi105 ceph-mon[31197]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:54:39.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:39 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:39.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:39 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:39.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:39 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:39.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:39 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:39.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:39 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:39.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:39 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:54:39.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:39 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:39.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:39 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T01:54:39.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:39 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T01:54:39.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:39 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:39.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:39 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:39.842 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 01:54:39 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-z[36549]: 2024-09-16T01:54:39.574+0000 7f9073757040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T01:54:39.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:39 smithi190 ceph-mon[34762]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:54:39.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:39 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:39.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:39 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:39.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:39 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:39.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:39 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:39.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:39 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:39.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:39 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:54:39.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:39 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:39.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:39 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T01:54:39.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:39 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T01:54:39.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:39 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:39.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:39 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:40.338 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 16 01:54:40 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-y[35457]: 2024-09-16T01:54:40.079+0000 7f5bd8c9c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T01:54:40.339 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 16 01:54:40 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-y[35457]: 2024-09-16T01:54:40.194+0000 7f5bd8c9c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T01:54:40.339 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 16 01:54:40 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-y[35457]: 2024-09-16T01:54:40.337+0000 7f5bd8c9c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T01:54:40.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:40.511+0000 7f2e217fa640 1 -- 172.21.15.105:0/3544816369 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mgrmap(e 14) ==== 92409+0+0 (secure 0 0 0) 0x7f2e1c039cd0 con 0x7f2e24102690 2024-09-16T01:54:40.614 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 16 01:54:40 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-y[35457]: 2024-09-16T01:54:40.397+0000 7f5bd8c9c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T01:54:40.614 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 16 01:54:40 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-y[35457]: 2024-09-16T01:54:40.545+0000 7f5bd8c9c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T01:54:40.614 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 16 01:54:40 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-y[35457]: 2024-09-16T01:54:40.612+0000 7f5bd8c9c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T01:54:40.614 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:40 smithi137 ceph-mon[34099]: Reconfiguring mgr.x (unknown last config time)... 2024-09-16T01:54:40.614 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:40 smithi137 ceph-mon[34099]: Reconfiguring daemon mgr.x on smithi105 2024-09-16T01:54:40.614 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:40 smithi137 ceph-mon[34099]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:40.615 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:40 smithi137 ceph-mon[34099]: Standby manager daemon z started 2024-09-16T01:54:40.615 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:40 smithi137 ceph-mon[34099]: from='mgr.? 172.21.15.190:0/549147832' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-16T01:54:40.615 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:40 smithi137 ceph-mon[34099]: from='mgr.? 172.21.15.190:0/549147832' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T01:54:40.615 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:40 smithi137 ceph-mon[34099]: from='mgr.? 172.21.15.190:0/549147832' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-16T01:54:40.615 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:40 smithi137 ceph-mon[34099]: from='mgr.? 172.21.15.190:0/549147832' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T01:54:40.615 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:40 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/653906680' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a60d6c72-da8b-4b02-bb33-672be795e7f7"}]: dispatch 2024-09-16T01:54:40.615 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:40 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/653906680' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a60d6c72-da8b-4b02-bb33-672be795e7f7"}]': finished 2024-09-16T01:54:40.615 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:40 smithi137 ceph-mon[34099]: osdmap e5: 1 total, 0 up, 1 in 2024-09-16T01:54:40.615 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:40 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:54:40.615 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:40 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/3936438203' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:54:40.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:40 smithi105 ceph-mon[31197]: Reconfiguring mgr.x (unknown last config time)... 2024-09-16T01:54:40.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:40 smithi105 ceph-mon[31197]: Reconfiguring daemon mgr.x on smithi105 2024-09-16T01:54:40.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:40 smithi105 ceph-mon[31197]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:40.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:40 smithi105 ceph-mon[31197]: Standby manager daemon z started 2024-09-16T01:54:40.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:40 smithi105 ceph-mon[31197]: from='mgr.? 172.21.15.190:0/549147832' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-16T01:54:40.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:40 smithi105 ceph-mon[31197]: from='mgr.? 172.21.15.190:0/549147832' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T01:54:40.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:40 smithi105 ceph-mon[31197]: from='mgr.? 172.21.15.190:0/549147832' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-16T01:54:40.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:40 smithi105 ceph-mon[31197]: from='mgr.? 172.21.15.190:0/549147832' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T01:54:40.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:40 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/653906680' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a60d6c72-da8b-4b02-bb33-672be795e7f7"}]: dispatch 2024-09-16T01:54:40.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:40 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/653906680' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a60d6c72-da8b-4b02-bb33-672be795e7f7"}]': finished 2024-09-16T01:54:40.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:40 smithi105 ceph-mon[31197]: osdmap e5: 1 total, 0 up, 1 in 2024-09-16T01:54:40.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:40 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:54:40.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:40 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/3936438203' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:54:40.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:40 smithi190 ceph-mon[34762]: Reconfiguring mgr.x (unknown last config time)... 2024-09-16T01:54:40.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:40 smithi190 ceph-mon[34762]: Reconfiguring daemon mgr.x on smithi105 2024-09-16T01:54:40.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:40 smithi190 ceph-mon[34762]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:40.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:40 smithi190 ceph-mon[34762]: Standby manager daemon z started 2024-09-16T01:54:40.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:40 smithi190 ceph-mon[34762]: from='mgr.? 172.21.15.190:0/549147832' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-16T01:54:40.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:40 smithi190 ceph-mon[34762]: from='mgr.? 172.21.15.190:0/549147832' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T01:54:40.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:40 smithi190 ceph-mon[34762]: from='mgr.? 172.21.15.190:0/549147832' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-16T01:54:40.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:40 smithi190 ceph-mon[34762]: from='mgr.? 172.21.15.190:0/549147832' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T01:54:40.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:40 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/653906680' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a60d6c72-da8b-4b02-bb33-672be795e7f7"}]: dispatch 2024-09-16T01:54:40.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:40 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/653906680' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a60d6c72-da8b-4b02-bb33-672be795e7f7"}]': finished 2024-09-16T01:54:40.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:40 smithi190 ceph-mon[34762]: osdmap e5: 1 total, 0 up, 1 in 2024-09-16T01:54:40.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:40 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:54:40.844 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:40 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/3936438203' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:54:40.895 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 16 01:54:40 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-y[35457]: 2024-09-16T01:54:40.673+0000 7f5bd8c9c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T01:54:40.896 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 16 01:54:40 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-y[35457]: 2024-09-16T01:54:40.830+0000 7f5bd8c9c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T01:54:40.896 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 16 01:54:40 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-y[35457]: 2024-09-16T01:54:40.893+0000 7f5bd8c9c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T01:54:41.158 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 16 01:54:40 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-y[35457]: 2024-09-16T01:54:40.959+0000 7f5bd8c9c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T01:54:41.158 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 16 01:54:41 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-y[35457]: 2024-09-16T01:54:41.039+0000 7f5bd8c9c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T01:54:41.158 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 16 01:54:41 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-y[35457]: 2024-09-16T01:54:41.097+0000 7f5bd8c9c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T01:54:41.158 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 16 01:54:41 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-y[35457]: 2024-09-16T01:54:41.156+0000 7f5bd8c9c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T01:54:41.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:41 smithi105 ceph-mon[31197]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-16T01:54:41.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:41 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-16T01:54:41.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:41 smithi190 ceph-mon[34762]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-16T01:54:41.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:41 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-16T01:54:41.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:41 smithi137 ceph-mon[34099]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-16T01:54:41.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:41 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-16T01:54:42.421 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 16 01:54:41 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-y[35457]: 2024-09-16T01:54:41.943+0000 7f5bd8c9c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T01:54:42.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:42.523+0000 7f2e217fa640 1 -- 172.21.15.105:0/3544816369 <== mon.0 v2:172.21.15.105:3300/0 8 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7f2e1c02fcd0 con 0x7f2e24102690 2024-09-16T01:54:42.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:42 smithi105 ceph-mon[31197]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:42.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:42 smithi105 ceph-mon[31197]: Standby manager daemon y started 2024-09-16T01:54:42.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:42 smithi105 ceph-mon[31197]: from='mgr.? 172.21.15.137:0/563988949' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-16T01:54:42.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:42 smithi105 ceph-mon[31197]: from='mgr.? 172.21.15.137:0/563988949' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T01:54:42.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:42 smithi105 ceph-mon[31197]: from='mgr.? 172.21.15.137:0/563988949' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-16T01:54:42.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:42 smithi105 ceph-mon[31197]: from='mgr.? 172.21.15.137:0/563988949' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T01:54:42.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:42 smithi190 ceph-mon[34762]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:42.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:42 smithi190 ceph-mon[34762]: Standby manager daemon y started 2024-09-16T01:54:42.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:42 smithi190 ceph-mon[34762]: from='mgr.? 172.21.15.137:0/563988949' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-16T01:54:42.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:42 smithi190 ceph-mon[34762]: from='mgr.? 172.21.15.137:0/563988949' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T01:54:42.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:42 smithi190 ceph-mon[34762]: from='mgr.? 172.21.15.137:0/563988949' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-16T01:54:42.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:42 smithi190 ceph-mon[34762]: from='mgr.? 172.21.15.137:0/563988949' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T01:54:42.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:42 smithi137 ceph-mon[34099]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:42.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:42 smithi137 ceph-mon[34099]: Standby manager daemon y started 2024-09-16T01:54:42.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:42 smithi137 ceph-mon[34099]: from='mgr.? 172.21.15.137:0/563988949' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-16T01:54:42.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:42 smithi137 ceph-mon[34099]: from='mgr.? 172.21.15.137:0/563988949' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T01:54:42.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:42 smithi137 ceph-mon[34099]: from='mgr.? 172.21.15.137:0/563988949' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-16T01:54:42.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:42 smithi137 ceph-mon[34099]: from='mgr.? 172.21.15.137:0/563988949' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T01:54:43.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:43 smithi105 ceph-mon[31197]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-09-16T01:54:43.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:43 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T01:54:43.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:43 smithi190 ceph-mon[34762]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-09-16T01:54:43.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:43 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T01:54:43.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:43 smithi137 ceph-mon[34099]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-09-16T01:54:43.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:43 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T01:54:44.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:44 smithi105 ceph-mon[31197]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:44.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:44 smithi190 ceph-mon[34762]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:44.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:44 smithi137 ceph-mon[34099]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:45.544 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:45 smithi105 ceph-mon[31197]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:45.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:45 smithi190 ceph-mon[34762]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:45.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:45 smithi137 ceph-mon[34099]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:48.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:47 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:48.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:47 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:48.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:47 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:54:48.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:47 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:48.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:47 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:54:48.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:47 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:48.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:47 smithi190 ceph-mon[34762]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:48.101 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:47 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:48.101 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:47 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:48.102 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:47 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:54:48.102 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:47 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:48.102 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:47 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:54:48.102 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:47 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:48.102 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:47 smithi105 ceph-mon[31197]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:48.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:47 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:48.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:47 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:48.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:47 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:54:48.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:47 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:48.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:47 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:54:48.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:47 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:48.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:47 smithi137 ceph-mon[34099]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:49.544 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:49 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T01:54:49.544 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:49 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:49.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:49 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T01:54:49.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:49 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:49.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:49 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T01:54:49.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:49 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:50.544 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:50 smithi105 ceph-mon[31197]: Deploying daemon osd.0 on smithi105 2024-09-16T01:54:50.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:50 smithi105 ceph-mon[31197]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:50.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:50 smithi190 ceph-mon[34762]: Deploying daemon osd.0 on smithi105 2024-09-16T01:54:50.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:50 smithi190 ceph-mon[34762]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:50.670 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:50 smithi137 ceph-mon[34099]: Deploying daemon osd.0 on smithi105 2024-09-16T01:54:50.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:50 smithi137 ceph-mon[34099]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:52.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:52 smithi137 ceph-mon[34099]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:52.719 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:52 smithi105 ceph-mon[31197]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:52.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:52 smithi190 ceph-mon[34762]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:54.663 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:54 smithi105 ceph-mon[31197]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:54.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:54 smithi190 ceph-mon[34762]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:54.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:54 smithi137 ceph-mon[34099]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:56.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:56 smithi105 ceph-mon[31197]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:56.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:56 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:54:56.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:56 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:56.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:56 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:56.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:56 smithi190 ceph-mon[34762]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:56.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:56 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:54:56.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:56 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:56.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:56 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:56.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:56 smithi137 ceph-mon[34099]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:56.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:56 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:54:56.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:56 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:56.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:56 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:58.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:58 smithi105 ceph-mon[31197]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:58.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:58 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:58.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:58 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:58.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:58 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:58.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:58 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:54:58.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:54:58 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:58.821 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:58 smithi190 ceph-mon[34762]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:58.822 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:58 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:58.822 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:58 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:58.822 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:58 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:58.822 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:58 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:54:58.822 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:54:58 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:58.909 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:58 smithi137 ceph-mon[34099]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:54:58.909 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:58 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:58.909 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:58 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:58.909 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:58 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:58.909 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:58 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:54:58.909 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:54:58 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:54:59.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:59.652+0000 7f2e217fa640 1 -- 172.21.15.105:0/3544816369 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2e24062630 con 0x7f2e100396a0 2024-09-16T01:54:59.654 INFO:teuthology.orchestra.run.smithi105.stdout:Created osd(s) 0 on host 'smithi105' 2024-09-16T01:54:59.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:59.656+0000 7f2e29f8f640 1 -- 172.21.15.105:0/3544816369 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f2e100396a0 msgr2=0x7f2e1003bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:59.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:59.656+0000 7f2e29f8f640 1 --2- 172.21.15.105:0/3544816369 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f2e100396a0 0x7f2e1003bb60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2e14005fd0 tx=0x7f2e1400b390 comp rx=0 tx=0).stop 2024-09-16T01:54:59.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:59.656+0000 7f2e29f8f640 1 -- 172.21.15.105:0/3544816369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e24102690 msgr2=0x7f2e2419f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:59.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:59.656+0000 7f2e29f8f640 1 --2- 172.21.15.105:0/3544816369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e24102690 0x7f2e2419f390 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c009dc0 tx=0x7f2e1c003ba0 comp rx=0 tx=0).stop 2024-09-16T01:54:59.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:59.656+0000 7f2e29f8f640 1 -- 172.21.15.105:0/3544816369 shutdown_connections 2024-09-16T01:54:59.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:59.656+0000 7f2e29f8f640 1 --2- 172.21.15.105:0/3544816369 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f2e100396a0 0x7f2e1003bb60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:59.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:59.656+0000 7f2e29f8f640 1 --2- 172.21.15.105:0/3544816369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e24102690 0x7f2e2419f390 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:59.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:59.656+0000 7f2e29f8f640 1 --2- 172.21.15.105:0/3544816369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e24101cd0 0x7f2e24198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:59.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:59.656+0000 7f2e29f8f640 1 --2- 172.21.15.105:0/3544816369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e24100ad0 0x7f2e24197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:59.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:59.656+0000 7f2e29f8f640 1 -- 172.21.15.105:0/3544816369 >> 172.21.15.105:0/3544816369 conn(0x7f2e240fc280 msgr2=0x7f2e24105110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:59.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:59.656+0000 7f2e29f8f640 1 -- 172.21.15.105:0/3544816369 shutdown_connections 2024-09-16T01:54:59.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:54:59.656+0000 7f2e29f8f640 1 -- 172.21.15.105:0/3544816369 wait complete. 2024-09-16T01:55:00.290 DEBUG:teuthology.orchestra.run.smithi105:osd.0> sudo journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@osd.0.service 2024-09-16T01:55:00.293 INFO:tasks.cephadm:Deploying osd.1 on smithi137 with /dev/vg_nvme/lv_4... 2024-09-16T01:55:00.293 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-16T01:55:00.522 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:00 smithi105 ceph-mon[31197]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:55:00.522 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:00 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:55:00.522 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:00 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:00.522 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:00 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:00.533 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T01:55:00.652 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:00 smithi137 ceph-mon[34099]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:55:00.652 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:00 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:55:00.652 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:00 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:00.652 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:00 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:00.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:00 smithi190 ceph-mon[34762]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:55:00.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:00 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:55:00.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:00 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:00.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:00 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:02.045 INFO:journalctl@ceph.osd.0.smithi105.stdout:Sep 16 01:55:01 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-0[44633]: 2024-09-16T01:55:01.801+0000 7f53ac76b640 -1 osd.0 0 log_to_monitors true 2024-09-16T01:55:02.792 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:02 smithi105 ceph-mon[31197]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:55:02.793 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:02 smithi105 ceph-mon[31197]: from='osd.0 [v2:172.21.15.105:6802/1226660290,v1:172.21.15.105:6803/1226660290]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T01:55:02.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:02 smithi190 ceph-mon[34762]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:55:02.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:02 smithi190 ceph-mon[34762]: from='osd.0 [v2:172.21.15.105:6802/1226660290,v1:172.21.15.105:6803/1226660290]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T01:55:02.847 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:02 smithi137 ceph-mon[34099]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:55:02.847 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:02 smithi137 ceph-mon[34099]: from='osd.0 [v2:172.21.15.105:6802/1226660290,v1:172.21.15.105:6803/1226660290]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T01:55:02.849 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-16T01:55:02.876 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi137:vg_nvme/lv_4 2024-09-16T01:55:03.123 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T01:55:03.501 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:03 smithi137 ceph-mon[34099]: from='osd.0 [v2:172.21.15.105:6802/1226660290,v1:172.21.15.105:6803/1226660290]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T01:55:03.501 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:03 smithi137 ceph-mon[34099]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T01:55:03.501 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:03 smithi137 ceph-mon[34099]: from='osd.0 [v2:172.21.15.105:6802/1226660290,v1:172.21.15.105:6803/1226660290]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-09-16T01:55:03.501 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:03 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:55:03.502 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:03 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:03.502 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:03 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:03.502 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:03 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:55:03.502 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:03 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:03.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:03 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:55:03.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:03 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:55:03.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:03 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:03.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:03 smithi137 ceph-mon[34099]: from='osd.0 [v2:172.21.15.105:6802/1226660290,v1:172.21.15.105:6803/1226660290]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-09-16T01:55:03.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:03 smithi137 ceph-mon[34099]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T01:55:03.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.582+0000 7f62364fc640 1 -- 172.21.15.137:0/2884471533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6230100de0 msgr2=0x7f6230101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:03.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.582+0000 7f62364fc640 1 --2- 172.21.15.137:0/2884471533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6230100de0 0x7f6230101260 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6218009900 tx=0x7f621802a410 comp rx=0 tx=0).stop 2024-09-16T01:55:03.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.582+0000 7f62364fc640 1 -- 172.21.15.137:0/2884471533 shutdown_connections 2024-09-16T01:55:03.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.582+0000 7f62364fc640 1 --2- 172.21.15.137:0/2884471533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62301017a0 0x7f6230109040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:03.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.582+0000 7f62364fc640 1 --2- 172.21.15.137:0/2884471533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6230100de0 0x7f6230101260 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:03.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.582+0000 7f62364fc640 1 --2- 172.21.15.137:0/2884471533 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62301020a0 0x7f62301024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:03.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.582+0000 7f62364fc640 1 -- 172.21.15.137:0/2884471533 >> 172.21.15.137:0/2884471533 conn(0x7f62300fba40 msgr2=0x7f62300fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:03.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.582+0000 7f62364fc640 1 -- 172.21.15.137:0/2884471533 shutdown_connections 2024-09-16T01:55:03.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.582+0000 7f62364fc640 1 -- 172.21.15.137:0/2884471533 wait complete. 2024-09-16T01:55:03.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.582+0000 7f62364fc640 1 Processor -- start 2024-09-16T01:55:03.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.583+0000 7f62364fc640 1 -- start start 2024-09-16T01:55:03.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.583+0000 7f62364fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6230100de0 0x7f6230197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:03.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.583+0000 7f62364fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62301017a0 0x7f62301983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:03.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.583+0000 7f62364fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62301020a0 0x7f623019f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:03.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.583+0000 7f62364fc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f623010c680 con 0x7f6230100de0 2024-09-16T01:55:03.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.583+0000 7f62364fc640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f623010c500 con 0x7f62301017a0 2024-09-16T01:55:03.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.583+0000 7f62364fc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f623010c800 con 0x7f62301020a0 2024-09-16T01:55:03.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.583+0000 7f6227fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62301017a0 0x7f62301983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:03.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.583+0000 7f6227fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62301017a0 0x7f62301983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56856/0 (socket says 172.21.15.137:56856) 2024-09-16T01:55:03.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.583+0000 7f6227fff640 1 -- 172.21.15.137:0/3791659760 learned_addr learned my addr 172.21.15.137:0/3791659760 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-16T01:55:03.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.583+0000 7f6234a99640 1 --2- 172.21.15.137:0/3791659760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62301020a0 0x7f623019f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:03.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.583+0000 7f622ffff640 1 --2- 172.21.15.137:0/3791659760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6230100de0 0x7f6230197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:03.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.583+0000 7f6227fff640 1 -- 172.21.15.137:0/3791659760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62301020a0 msgr2=0x7f623019f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:03.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.583+0000 7f6227fff640 1 --2- 172.21.15.137:0/3791659760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62301020a0 0x7f623019f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:03.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.583+0000 7f6227fff640 1 -- 172.21.15.137:0/3791659760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6230100de0 msgr2=0x7f6230197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:03.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.583+0000 7f6227fff640 1 --2- 172.21.15.137:0/3791659760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6230100de0 0x7f6230197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:03.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.583+0000 7f6227fff640 1 -- 172.21.15.137:0/3791659760 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f623019fac0 con 0x7f62301017a0 2024-09-16T01:55:03.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.583+0000 7f6227fff640 1 --2- 172.21.15.137:0/3791659760 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62301017a0 0x7f62301983d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6218005ab0 tx=0x7f6218003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:03.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.583+0000 7f622dffb640 1 -- 172.21.15.137:0/3791659760 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621803f070 con 0x7f62301017a0 2024-09-16T01:55:03.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.584+0000 7f62364fc640 1 -- 172.21.15.137:0/3791659760 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f623019fd50 con 0x7f62301017a0 2024-09-16T01:55:03.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.584+0000 7f622dffb640 1 -- 172.21.15.137:0/3791659760 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6218031e40 con 0x7f62301017a0 2024-09-16T01:55:03.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.584+0000 7f622dffb640 1 -- 172.21.15.137:0/3791659760 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6218035030 con 0x7f62301017a0 2024-09-16T01:55:03.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.584+0000 7f62364fc640 1 -- 172.21.15.137:0/3791659760 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62301a01d0 con 0x7f62301017a0 2024-09-16T01:55:03.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.585+0000 7f6226ffd640 1 -- 172.21.15.137:0/3791659760 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6214005290 con 0x7f62301017a0 2024-09-16T01:55:03.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.586+0000 7f622dffb640 1 -- 172.21.15.137:0/3791659760 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7f6218042070 con 0x7f62301017a0 2024-09-16T01:55:03.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.586+0000 7f622dffb640 1 --2- 172.21.15.137:0/3791659760 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f621c0a31e0 0x7f621c0a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:03.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.587+0000 7f622dffb640 1 -- 172.21.15.137:0/3791659760 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7f62180ee5c0 con 0x7f62301017a0 2024-09-16T01:55:03.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.587+0000 7f622ffff640 1 --2- 172.21.15.137:0/3791659760 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f621c0a31e0 0x7f621c0a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:03.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.587+0000 7f622ffff640 1 --2- 172.21.15.137:0/3791659760 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f621c0a31e0 0x7f621c0a56a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f623006a510 tx=0x7f622000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:03.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.591+0000 7f622dffb640 1 -- 172.21.15.137:0/3791659760 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f62180bc820 con 0x7f62301017a0 2024-09-16T01:55:03.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:03.767+0000 7f6226ffd640 1 -- 172.21.15.137:0/3791659760 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f6214000bc0 con 0x7f621c0a31e0 2024-09-16T01:55:03.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:03 smithi105 ceph-mon[31197]: from='osd.0 [v2:172.21.15.105:6802/1226660290,v1:172.21.15.105:6803/1226660290]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T01:55:03.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:03 smithi105 ceph-mon[31197]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T01:55:03.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:03 smithi105 ceph-mon[31197]: from='osd.0 [v2:172.21.15.105:6802/1226660290,v1:172.21.15.105:6803/1226660290]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-09-16T01:55:03.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:03 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:55:03.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:03 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:03.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:03 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:03.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:03 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:55:03.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:03 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:03.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:03 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:55:03.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:03 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:55:03.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:03 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:03.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:03 smithi105 ceph-mon[31197]: from='osd.0 [v2:172.21.15.105:6802/1226660290,v1:172.21.15.105:6803/1226660290]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-09-16T01:55:03.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:03 smithi105 ceph-mon[31197]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T01:55:03.796 INFO:journalctl@ceph.osd.0.smithi105.stdout:Sep 16 01:55:03 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-0[44633]: 2024-09-16T01:55:03.439+0000 7f53a86ee640 -1 osd.0 0 waiting for initial osdmap 2024-09-16T01:55:03.796 INFO:journalctl@ceph.osd.0.smithi105.stdout:Sep 16 01:55:03 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-0[44633]: 2024-09-16T01:55:03.441+0000 7f53a4516640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T01:55:03.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:03 smithi190 ceph-mon[34762]: from='osd.0 [v2:172.21.15.105:6802/1226660290,v1:172.21.15.105:6803/1226660290]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T01:55:03.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:03 smithi190 ceph-mon[34762]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T01:55:03.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:03 smithi190 ceph-mon[34762]: from='osd.0 [v2:172.21.15.105:6802/1226660290,v1:172.21.15.105:6803/1226660290]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-09-16T01:55:03.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:03 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:55:03.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:03 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:03.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:03 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:03.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:03 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:55:03.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:03 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:03.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:03 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:55:03.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:03 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:55:03.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:03 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:03.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:03 smithi190 ceph-mon[34762]: from='osd.0 [v2:172.21.15.105:6802/1226660290,v1:172.21.15.105:6803/1226660290]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-09-16T01:55:03.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:03 smithi190 ceph-mon[34762]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T01:55:04.696 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:04 smithi105 ceph-mon[31197]: Detected new or changed devices on smithi105 2024-09-16T01:55:04.696 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:04 smithi105 ceph-mon[31197]: Adjusting osd_memory_target on smithi105 to 17088M 2024-09-16T01:55:04.696 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:04 smithi105 ceph-mon[31197]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:55:04.696 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:04 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:55:04.696 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:04 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:55:04.696 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:04 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:55:04.696 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:04 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:55:04.696 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:04 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:55:04.697 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:04 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:55:04.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:04 smithi190 ceph-mon[34762]: Detected new or changed devices on smithi105 2024-09-16T01:55:04.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:04 smithi190 ceph-mon[34762]: Adjusting osd_memory_target on smithi105 to 17088M 2024-09-16T01:55:04.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:04 smithi190 ceph-mon[34762]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:55:04.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:04 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:55:04.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:04 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:55:04.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:04 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:55:04.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:04 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:55:04.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:04 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:55:04.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:04 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:55:04.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:04 smithi137 ceph-mon[34099]: Detected new or changed devices on smithi105 2024-09-16T01:55:04.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:04 smithi137 ceph-mon[34099]: Adjusting osd_memory_target on smithi105 to 17088M 2024-09-16T01:55:04.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:04 smithi137 ceph-mon[34099]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:55:04.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:04 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:55:04.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:04 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:55:04.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:04 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:55:04.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:04 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:55:04.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:04 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:55:04.923 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:04 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:55:05.707 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:05 smithi105 ceph-mon[31197]: purged_snaps scrub starts 2024-09-16T01:55:05.708 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:05 smithi105 ceph-mon[31197]: purged_snaps scrub ok 2024-09-16T01:55:05.708 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:05 smithi105 ceph-mon[31197]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:55:05.708 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:05 smithi105 ceph-mon[31197]: osd.0 [v2:172.21.15.105:6802/1226660290,v1:172.21.15.105:6803/1226660290] boot 2024-09-16T01:55:05.708 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:05 smithi105 ceph-mon[31197]: osdmap e8: 1 total, 1 up, 1 in 2024-09-16T01:55:05.708 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:05 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:55:05.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:05 smithi190 ceph-mon[34762]: purged_snaps scrub starts 2024-09-16T01:55:05.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:05 smithi190 ceph-mon[34762]: purged_snaps scrub ok 2024-09-16T01:55:05.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:05 smithi190 ceph-mon[34762]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:55:05.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:05 smithi190 ceph-mon[34762]: osd.0 [v2:172.21.15.105:6802/1226660290,v1:172.21.15.105:6803/1226660290] boot 2024-09-16T01:55:05.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:05 smithi190 ceph-mon[34762]: osdmap e8: 1 total, 1 up, 1 in 2024-09-16T01:55:05.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:05 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:55:05.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:05 smithi137 ceph-mon[34099]: purged_snaps scrub starts 2024-09-16T01:55:05.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:05 smithi137 ceph-mon[34099]: purged_snaps scrub ok 2024-09-16T01:55:05.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:05 smithi137 ceph-mon[34099]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:55:05.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:05 smithi137 ceph-mon[34099]: osd.0 [v2:172.21.15.105:6802/1226660290,v1:172.21.15.105:6803/1226660290] boot 2024-09-16T01:55:05.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:05 smithi137 ceph-mon[34099]: osdmap e8: 1 total, 1 up, 1 in 2024-09-16T01:55:05.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:05 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:55:06.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:06 smithi137 ceph-mon[34099]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:55:06.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:06 smithi137 ceph-mon[34099]: osdmap e9: 1 total, 1 up, 1 in 2024-09-16T01:55:06.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:06 smithi137 ceph-mon[34099]: from='client.? 172.21.15.137:0/2072048274' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b1d9ae84-b9a0-4a7e-b0fe-c402d46809fc"}]: dispatch 2024-09-16T01:55:06.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:06 smithi137 ceph-mon[34099]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b1d9ae84-b9a0-4a7e-b0fe-c402d46809fc"}]: dispatch 2024-09-16T01:55:06.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:06 smithi137 ceph-mon[34099]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b1d9ae84-b9a0-4a7e-b0fe-c402d46809fc"}]': finished 2024-09-16T01:55:06.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:06 smithi137 ceph-mon[34099]: osdmap e10: 2 total, 1 up, 2 in 2024-09-16T01:55:06.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:06 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:55:06.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:06 smithi137 ceph-mon[34099]: from='client.? 172.21.15.137:0/1215515241' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:55:06.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:06 smithi105 ceph-mon[31197]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:55:06.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:06 smithi105 ceph-mon[31197]: osdmap e9: 1 total, 1 up, 1 in 2024-09-16T01:55:06.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:06 smithi105 ceph-mon[31197]: from='client.? 172.21.15.137:0/2072048274' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b1d9ae84-b9a0-4a7e-b0fe-c402d46809fc"}]: dispatch 2024-09-16T01:55:06.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:06 smithi105 ceph-mon[31197]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b1d9ae84-b9a0-4a7e-b0fe-c402d46809fc"}]: dispatch 2024-09-16T01:55:06.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:06 smithi105 ceph-mon[31197]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b1d9ae84-b9a0-4a7e-b0fe-c402d46809fc"}]': finished 2024-09-16T01:55:06.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:06 smithi105 ceph-mon[31197]: osdmap e10: 2 total, 1 up, 2 in 2024-09-16T01:55:06.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:06 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:55:06.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:06 smithi105 ceph-mon[31197]: from='client.? 172.21.15.137:0/1215515241' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:55:06.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:06 smithi190 ceph-mon[34762]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:55:06.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:06 smithi190 ceph-mon[34762]: osdmap e9: 1 total, 1 up, 1 in 2024-09-16T01:55:06.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:06 smithi190 ceph-mon[34762]: from='client.? 172.21.15.137:0/2072048274' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b1d9ae84-b9a0-4a7e-b0fe-c402d46809fc"}]: dispatch 2024-09-16T01:55:06.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:06 smithi190 ceph-mon[34762]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b1d9ae84-b9a0-4a7e-b0fe-c402d46809fc"}]: dispatch 2024-09-16T01:55:06.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:06 smithi190 ceph-mon[34762]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b1d9ae84-b9a0-4a7e-b0fe-c402d46809fc"}]': finished 2024-09-16T01:55:06.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:06 smithi190 ceph-mon[34762]: osdmap e10: 2 total, 1 up, 2 in 2024-09-16T01:55:06.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:06 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:55:06.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:06 smithi190 ceph-mon[34762]: from='client.? 172.21.15.137:0/1215515241' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:55:08.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:08 smithi105 ceph-mon[31197]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:55:08.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:08 smithi190 ceph-mon[34762]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:55:08.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:08 smithi137 ceph-mon[34099]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:55:10.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:10 smithi137 ceph-mon[34099]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:55:10.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:10 smithi105 ceph-mon[31197]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:55:10.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:10 smithi190 ceph-mon[34762]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:55:12.728 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:12 smithi137 ceph-mon[34099]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:55:12.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:12 smithi105 ceph-mon[31197]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:55:12.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:12 smithi190 ceph-mon[34762]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:55:14.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:14 smithi137 ceph-mon[34099]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:55:14.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:14 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T01:55:14.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:14 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:55:14.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:14 smithi105 ceph-mon[31197]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:55:14.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:14 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T01:55:14.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:14 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:55:14.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:14 smithi190 ceph-mon[34762]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:55:14.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:14 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T01:55:14.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:14 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:55:15.576 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:15 smithi137 ceph-mon[34099]: Deploying daemon osd.1 on smithi137 2024-09-16T01:55:15.745 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:15 smithi105 ceph-mon[31197]: Deploying daemon osd.1 on smithi137 2024-09-16T01:55:15.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:15 smithi190 ceph-mon[34762]: Deploying daemon osd.1 on smithi137 2024-09-16T01:55:16.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:16 smithi137 ceph-mon[34099]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:55:16.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:16 smithi105 ceph-mon[31197]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:55:16.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:16 smithi190 ceph-mon[34762]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:55:18.668 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:18 smithi137 ceph-mon[34099]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:55:18.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:18 smithi105 ceph-mon[31197]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:55:18.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:18 smithi190 ceph-mon[34762]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:55:20.535 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:20 smithi137 ceph-mon[34099]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:55:20.535 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:20 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:55:20.535 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:20 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:20.535 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:20 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:20.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:20 smithi105 ceph-mon[31197]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:55:20.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:20 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:55:20.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:20 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:20.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:20 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:20.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:20 smithi190 ceph-mon[34762]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:55:20.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:20 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:55:20.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:20 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:20.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:20 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:22.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:22 smithi105 ceph-mon[31197]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:55:22.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:22 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:22.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:22 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:22.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:22 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:55:22.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:22 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:55:22.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:22 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:22.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:22 smithi190 ceph-mon[34762]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:55:22.844 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:22 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:22.844 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:22 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:22.844 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:22 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:55:22.844 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:22 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:55:22.844 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:22 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:22.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:22 smithi137 ceph-mon[34099]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:55:22.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:22 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:22.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:22 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:22.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:22 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:55:22.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:22 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:55:22.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:22 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:23.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:23.234+0000 7f622dffb640 1 -- 172.21.15.137:0/3791659760 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6214000bc0 con 0x7f621c0a31e0 2024-09-16T01:55:23.236 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 1 on host 'smithi137' 2024-09-16T01:55:23.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:23.237+0000 7f6226ffd640 1 -- 172.21.15.137:0/3791659760 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f621c0a31e0 msgr2=0x7f621c0a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:23.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:23.237+0000 7f6226ffd640 1 --2- 172.21.15.137:0/3791659760 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f621c0a31e0 0x7f621c0a56a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f623006a510 tx=0x7f622000b390 comp rx=0 tx=0).stop 2024-09-16T01:55:23.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:23.237+0000 7f6226ffd640 1 -- 172.21.15.137:0/3791659760 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62301017a0 msgr2=0x7f62301983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:23.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:23.237+0000 7f6226ffd640 1 --2- 172.21.15.137:0/3791659760 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62301017a0 0x7f62301983d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6218005ab0 tx=0x7f6218003cd0 comp rx=0 tx=0).stop 2024-09-16T01:55:23.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:23.237+0000 7f6226ffd640 1 -- 172.21.15.137:0/3791659760 shutdown_connections 2024-09-16T01:55:23.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:23.237+0000 7f6226ffd640 1 --2- 172.21.15.137:0/3791659760 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f621c0a31e0 0x7f621c0a56a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:23.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:23.237+0000 7f6226ffd640 1 --2- 172.21.15.137:0/3791659760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62301020a0 0x7f623019f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:23.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:23.237+0000 7f6226ffd640 1 --2- 172.21.15.137:0/3791659760 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62301017a0 0x7f62301983d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:23.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:23.237+0000 7f6226ffd640 1 --2- 172.21.15.137:0/3791659760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6230100de0 0x7f6230197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:23.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:23.237+0000 7f6226ffd640 1 -- 172.21.15.137:0/3791659760 >> 172.21.15.137:0/3791659760 conn(0x7f62300fba40 msgr2=0x7f62300ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:23.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:23.237+0000 7f6226ffd640 1 -- 172.21.15.137:0/3791659760 shutdown_connections 2024-09-16T01:55:23.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:55:23.237+0000 7f6226ffd640 1 -- 172.21.15.137:0/3791659760 wait complete. 2024-09-16T01:55:23.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:23 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:55:23.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:23 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:23.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:23 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:23.817 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:23 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:55:23.818 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:23 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:23.818 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:23 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:23.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:23 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:55:23.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:23 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:23.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:23 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:23.844 DEBUG:teuthology.orchestra.run.smithi137:osd.1> sudo journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@osd.1.service 2024-09-16T01:55:23.846 INFO:tasks.cephadm:Deploying osd.2 on smithi190 with /dev/vg_nvme/lv_4... 2024-09-16T01:55:23.847 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-16T01:55:24.098 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T01:55:24.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:24 smithi137 ceph-mon[34099]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:55:24.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:24 smithi105 ceph-mon[31197]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:55:24.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:24 smithi190 ceph-mon[34762]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:55:26.171 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 16 01:55:25 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-1[38549]: 2024-09-16T01:55:25.872+0000 7f13658de640 -1 osd.1 0 log_to_monitors true 2024-09-16T01:55:26.579 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T01:55:26.597 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:26 smithi137 ceph-mon[34099]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:55:26.597 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:26 smithi137 ceph-mon[34099]: from='osd.1 [v2:172.21.15.137:6800/34753956,v1:172.21.15.137:6801/34753956]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T01:55:26.597 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:26 smithi137 ceph-mon[34099]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T01:55:26.604 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi190:vg_nvme/lv_4 2024-09-16T01:55:26.615 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:26 smithi190 ceph-mon[34762]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:55:26.615 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:26 smithi190 ceph-mon[34762]: from='osd.1 [v2:172.21.15.137:6800/34753956,v1:172.21.15.137:6801/34753956]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T01:55:26.615 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:26 smithi190 ceph-mon[34762]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T01:55:26.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:26 smithi105 ceph-mon[31197]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:55:26.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:26 smithi105 ceph-mon[31197]: from='osd.1 [v2:172.21.15.137:6800/34753956,v1:172.21.15.137:6801/34753956]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T01:55:26.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:26 smithi105 ceph-mon[31197]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T01:55:26.867 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T01:55:27.320 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.304+0000 7fc95bf25640 1 -- 172.21.15.190:0/2522458379 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc954100ad0 msgr2=0x7fc954100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:27.320 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.304+0000 7fc95bf25640 1 --2- 172.21.15.190:0/2522458379 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc954100ad0 0x7fc954100ed0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc944009900 tx=0x7fc94402a410 comp rx=0 tx=0).stop 2024-09-16T01:55:27.320 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.304+0000 7fc95bf25640 1 -- 172.21.15.190:0/2522458379 shutdown_connections 2024-09-16T01:55:27.321 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.304+0000 7fc95bf25640 1 --2- 172.21.15.190:0/2522458379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc954102690 0x7fc954108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:27.321 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.304+0000 7fc95bf25640 1 --2- 172.21.15.190:0/2522458379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc954101cd0 0x7fc954102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:27.321 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.304+0000 7fc95bf25640 1 --2- 172.21.15.190:0/2522458379 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc954100ad0 0x7fc954100ed0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:27.321 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.304+0000 7fc95bf25640 1 -- 172.21.15.190:0/2522458379 >> 172.21.15.190:0/2522458379 conn(0x7fc9540fc280 msgr2=0x7fc9540fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:27.321 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.304+0000 7fc95bf25640 1 -- 172.21.15.190:0/2522458379 shutdown_connections 2024-09-16T01:55:27.321 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.304+0000 7fc95bf25640 1 -- 172.21.15.190:0/2522458379 wait complete. 2024-09-16T01:55:27.321 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.304+0000 7fc95bf25640 1 Processor -- start 2024-09-16T01:55:27.321 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.304+0000 7fc95bf25640 1 -- start start 2024-09-16T01:55:27.321 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.304+0000 7fc95bf25640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc954100ad0 0x7fc954197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:27.321 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.304+0000 7fc95bf25640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc954101cd0 0x7fc954198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:27.321 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.304+0000 7fc95bf25640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc954102690 0x7fc95419f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:27.321 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.304+0000 7fc95bf25640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc95410c610 con 0x7fc954102690 2024-09-16T01:55:27.321 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.304+0000 7fc95bf25640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc95410c490 con 0x7fc954101cd0 2024-09-16T01:55:27.322 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.304+0000 7fc95bf25640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc95410c790 con 0x7fc954100ad0 2024-09-16T01:55:27.322 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.305+0000 7fc959cc1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc954100ad0 0x7fc954197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:27.322 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.305+0000 7fc959cc1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc954100ad0 0x7fc954197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:58324/0 (socket says 172.21.15.190:58324) 2024-09-16T01:55:27.322 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.305+0000 7fc959cc1640 1 -- 172.21.15.190:0/1977918930 learned_addr learned my addr 172.21.15.190:0/1977918930 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-09-16T01:55:27.322 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.305+0000 7fc9594c0640 1 --2- 172.21.15.190:0/1977918930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc954101cd0 0x7fc954198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:27.322 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.305+0000 7fc95a4c2640 1 --2- 172.21.15.190:0/1977918930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc954102690 0x7fc95419f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:27.322 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.305+0000 7fc959cc1640 1 -- 172.21.15.190:0/1977918930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc954101cd0 msgr2=0x7fc954198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:27.322 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.305+0000 7fc959cc1640 1 --2- 172.21.15.190:0/1977918930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc954101cd0 0x7fc954198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:27.322 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.305+0000 7fc959cc1640 1 -- 172.21.15.190:0/1977918930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc954102690 msgr2=0x7fc95419f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:27.322 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.305+0000 7fc959cc1640 1 --2- 172.21.15.190:0/1977918930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc954102690 0x7fc95419f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:27.322 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.305+0000 7fc959cc1640 1 -- 172.21.15.190:0/1977918930 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc95419fa60 con 0x7fc954100ad0 2024-09-16T01:55:27.322 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.305+0000 7fc95a4c2640 1 --2- 172.21.15.190:0/1977918930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc954102690 0x7fc95419f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:55:27.323 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.305+0000 7fc959cc1640 1 --2- 172.21.15.190:0/1977918930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc954100ad0 0x7fc954197e30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc9440314e0 tx=0x7fc9440042e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:27.323 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.305+0000 7fc94affd640 1 -- 172.21.15.190:0/1977918930 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94403f030 con 0x7fc954100ad0 2024-09-16T01:55:27.323 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.305+0000 7fc94affd640 1 -- 172.21.15.190:0/1977918930 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc944003cb0 con 0x7fc954100ad0 2024-09-16T01:55:27.323 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.305+0000 7fc95bf25640 1 -- 172.21.15.190:0/1977918930 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc95419fcf0 con 0x7fc954100ad0 2024-09-16T01:55:27.323 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.305+0000 7fc94affd640 1 -- 172.21.15.190:0/1977918930 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc944035070 con 0x7fc954100ad0 2024-09-16T01:55:27.323 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.305+0000 7fc95bf25640 1 -- 172.21.15.190:0/1977918930 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9541a0180 con 0x7fc954100ad0 2024-09-16T01:55:27.323 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.306+0000 7fc95bf25640 1 -- 172.21.15.190:0/1977918930 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9541a0350 con 0x7fc954100ad0 2024-09-16T01:55:27.323 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.307+0000 7fc94affd640 1 -- 172.21.15.190:0/1977918930 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7fc944042030 con 0x7fc954100ad0 2024-09-16T01:55:27.323 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.308+0000 7fc94affd640 1 --2- 172.21.15.190:0/1977918930 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc9400a31e0 0x7fc9400a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:27.323 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.308+0000 7fc94affd640 1 -- 172.21.15.190:0/1977918930 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 1903+0+0 (secure 0 0 0) 0x7fc9440ee4b0 con 0x7fc954100ad0 2024-09-16T01:55:27.323 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.309+0000 7fc9594c0640 1 --2- 172.21.15.190:0/1977918930 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc9400a31e0 0x7fc9400a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:27.323 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.309+0000 7fc9594c0640 1 --2- 172.21.15.190:0/1977918930 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc9400a31e0 0x7fc9400a56a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc954199350 tx=0x7fc95000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:27.323 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.313+0000 7fc94affd640 1 -- 172.21.15.190:0/1977918930 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc9440bcbc0 con 0x7fc954100ad0 2024-09-16T01:55:27.490 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:27.489+0000 7fc95bf25640 1 -- 172.21.15.190:0/1977918930 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fc95406a770 con 0x7fc9400a31e0 2024-09-16T01:55:27.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:27 smithi105 ceph-mon[31197]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T01:55:27.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:27 smithi105 ceph-mon[31197]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T01:55:27.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:27 smithi105 ceph-mon[31197]: from='osd.1 [v2:172.21.15.137:6800/34753956,v1:172.21.15.137:6801/34753956]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-09-16T01:55:27.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:27 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:55:27.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:27 smithi105 ceph-mon[31197]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-09-16T01:55:27.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:27 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:27.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:27 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:27.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:27 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:55:27.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:27 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:27.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:27 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:55:27.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:27 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:55:27.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:27 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:27.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:27 smithi105 ceph-mon[31197]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-09-16T01:55:27.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:27 smithi190 ceph-mon[34762]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T01:55:27.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:27 smithi190 ceph-mon[34762]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T01:55:27.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:27 smithi190 ceph-mon[34762]: from='osd.1 [v2:172.21.15.137:6800/34753956,v1:172.21.15.137:6801/34753956]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-09-16T01:55:27.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:27 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:55:27.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:27 smithi190 ceph-mon[34762]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-09-16T01:55:27.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:27 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:27.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:27 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:27.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:27 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:55:27.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:27 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:27.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:27 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:55:27.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:27 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:55:27.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:27 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:27.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:27 smithi190 ceph-mon[34762]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-09-16T01:55:27.879 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:27 smithi137 ceph-mon[34099]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T01:55:27.879 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:27 smithi137 ceph-mon[34099]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T01:55:27.879 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:27 smithi137 ceph-mon[34099]: from='osd.1 [v2:172.21.15.137:6800/34753956,v1:172.21.15.137:6801/34753956]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-09-16T01:55:27.879 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:27 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:55:27.879 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:27 smithi137 ceph-mon[34099]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-09-16T01:55:27.879 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:27 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:27.880 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:27 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:27.880 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:27 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:55:27.880 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:27 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:27.880 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:27 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:55:27.880 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:27 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:55:27.880 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:27 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:27.880 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:27 smithi137 ceph-mon[34099]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-09-16T01:55:28.171 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 16 01:55:27 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-1[38549]: 2024-09-16T01:55:27.877+0000 7f1361861640 -1 osd.1 0 waiting for initial osdmap 2024-09-16T01:55:28.171 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 16 01:55:27 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-1[38549]: 2024-09-16T01:55:27.880+0000 7f1359681640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T01:55:28.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:28 smithi190 ceph-mon[34762]: Detected new or changed devices on smithi137 2024-09-16T01:55:28.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:28 smithi190 ceph-mon[34762]: Adjusting osd_memory_target on smithi137 to 17088M 2024-09-16T01:55:28.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:28 smithi190 ceph-mon[34762]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:55:28.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:28 smithi190 ceph-mon[34762]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:55:28.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:28 smithi190 ceph-mon[34762]: osdmap e12: 2 total, 1 up, 2 in 2024-09-16T01:55:28.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:28 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:55:28.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:28 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:55:28.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:28 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:55:28.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:28 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:55:28.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:28 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:55:28.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:28 smithi190 ceph-mon[34762]: from='osd.1 ' entity='osd.1' 2024-09-16T01:55:28.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:28 smithi105 ceph-mon[31197]: Detected new or changed devices on smithi137 2024-09-16T01:55:28.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:28 smithi105 ceph-mon[31197]: Adjusting osd_memory_target on smithi137 to 17088M 2024-09-16T01:55:28.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:28 smithi105 ceph-mon[31197]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:55:28.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:28 smithi105 ceph-mon[31197]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:55:28.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:28 smithi105 ceph-mon[31197]: osdmap e12: 2 total, 1 up, 2 in 2024-09-16T01:55:28.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:28 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:55:28.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:28 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:55:28.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:28 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:55:28.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:28 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:55:28.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:28 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:55:28.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:28 smithi105 ceph-mon[31197]: from='osd.1 ' entity='osd.1' 2024-09-16T01:55:28.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:28 smithi137 ceph-mon[34099]: Detected new or changed devices on smithi137 2024-09-16T01:55:28.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:28 smithi137 ceph-mon[34099]: Adjusting osd_memory_target on smithi137 to 17088M 2024-09-16T01:55:28.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:28 smithi137 ceph-mon[34099]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:55:28.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:28 smithi137 ceph-mon[34099]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:55:28.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:28 smithi137 ceph-mon[34099]: osdmap e12: 2 total, 1 up, 2 in 2024-09-16T01:55:28.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:28 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:55:28.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:28 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:55:28.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:28 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:55:28.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:28 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:55:28.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:28 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:55:28.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:28 smithi137 ceph-mon[34099]: from='osd.1 ' entity='osd.1' 2024-09-16T01:55:29.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:29 smithi105 ceph-mon[31197]: purged_snaps scrub starts 2024-09-16T01:55:29.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:29 smithi105 ceph-mon[31197]: purged_snaps scrub ok 2024-09-16T01:55:29.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:29 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:55:29.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:29 smithi105 ceph-mon[31197]: osd.1 [v2:172.21.15.137:6800/34753956,v1:172.21.15.137:6801/34753956] boot 2024-09-16T01:55:29.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:29 smithi105 ceph-mon[31197]: osdmap e13: 2 total, 2 up, 2 in 2024-09-16T01:55:29.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:29 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:55:29.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:29 smithi190 ceph-mon[34762]: purged_snaps scrub starts 2024-09-16T01:55:29.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:29 smithi190 ceph-mon[34762]: purged_snaps scrub ok 2024-09-16T01:55:29.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:29 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:55:29.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:29 smithi190 ceph-mon[34762]: osd.1 [v2:172.21.15.137:6800/34753956,v1:172.21.15.137:6801/34753956] boot 2024-09-16T01:55:29.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:29 smithi190 ceph-mon[34762]: osdmap e13: 2 total, 2 up, 2 in 2024-09-16T01:55:29.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:29 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:55:29.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:29 smithi137 ceph-mon[34099]: purged_snaps scrub starts 2024-09-16T01:55:29.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:29 smithi137 ceph-mon[34099]: purged_snaps scrub ok 2024-09-16T01:55:29.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:29 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:55:29.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:29 smithi137 ceph-mon[34099]: osd.1 [v2:172.21.15.137:6800/34753956,v1:172.21.15.137:6801/34753956] boot 2024-09-16T01:55:29.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:29 smithi137 ceph-mon[34099]: osdmap e13: 2 total, 2 up, 2 in 2024-09-16T01:55:29.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:29 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:55:30.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:30 smithi105 ceph-mon[31197]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:55:30.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:30 smithi105 ceph-mon[31197]: from='client.? 172.21.15.190:0/2335408198' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "137faae6-2e8b-44b5-8976-3d95a6eef1aa"}]: dispatch 2024-09-16T01:55:30.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:30 smithi105 ceph-mon[31197]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "137faae6-2e8b-44b5-8976-3d95a6eef1aa"}]: dispatch 2024-09-16T01:55:30.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:30 smithi105 ceph-mon[31197]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "137faae6-2e8b-44b5-8976-3d95a6eef1aa"}]': finished 2024-09-16T01:55:30.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:30 smithi105 ceph-mon[31197]: osdmap e14: 3 total, 2 up, 3 in 2024-09-16T01:55:30.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:30 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:55:30.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:30 smithi105 ceph-mon[31197]: from='client.? 172.21.15.190:0/4062888121' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:55:30.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:30 smithi190 ceph-mon[34762]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:55:30.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:30 smithi190 ceph-mon[34762]: from='client.? 172.21.15.190:0/2335408198' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "137faae6-2e8b-44b5-8976-3d95a6eef1aa"}]: dispatch 2024-09-16T01:55:30.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:30 smithi190 ceph-mon[34762]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "137faae6-2e8b-44b5-8976-3d95a6eef1aa"}]: dispatch 2024-09-16T01:55:30.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:30 smithi190 ceph-mon[34762]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "137faae6-2e8b-44b5-8976-3d95a6eef1aa"}]': finished 2024-09-16T01:55:30.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:30 smithi190 ceph-mon[34762]: osdmap e14: 3 total, 2 up, 3 in 2024-09-16T01:55:30.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:30 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:55:30.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:30 smithi190 ceph-mon[34762]: from='client.? 172.21.15.190:0/4062888121' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:55:30.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:30 smithi137 ceph-mon[34099]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:55:30.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:30 smithi137 ceph-mon[34099]: from='client.? 172.21.15.190:0/2335408198' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "137faae6-2e8b-44b5-8976-3d95a6eef1aa"}]: dispatch 2024-09-16T01:55:30.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:30 smithi137 ceph-mon[34099]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "137faae6-2e8b-44b5-8976-3d95a6eef1aa"}]: dispatch 2024-09-16T01:55:30.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:30 smithi137 ceph-mon[34099]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "137faae6-2e8b-44b5-8976-3d95a6eef1aa"}]': finished 2024-09-16T01:55:30.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:30 smithi137 ceph-mon[34099]: osdmap e14: 3 total, 2 up, 3 in 2024-09-16T01:55:30.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:30 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:55:30.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:30 smithi137 ceph-mon[34099]: from='client.? 172.21.15.190:0/4062888121' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:55:32.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:32 smithi105 ceph-mon[31197]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:55:32.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:32 smithi190 ceph-mon[34762]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:55:32.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:32 smithi137 ceph-mon[34099]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:55:34.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:34 smithi105 ceph-mon[31197]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:55:34.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:34 smithi190 ceph-mon[34762]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:55:34.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:34 smithi137 ceph-mon[34099]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:55:36.564 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:36 smithi190 ceph-mon[34762]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:55:36.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:36 smithi105 ceph-mon[31197]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:55:36.920 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:36 smithi137 ceph-mon[34099]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:55:38.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:38 smithi105 ceph-mon[31197]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:55:38.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:38 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T01:55:38.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:38 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:55:38.826 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:38 smithi190 ceph-mon[34762]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:55:38.826 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:38 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T01:55:38.826 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:38 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:55:38.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:38 smithi137 ceph-mon[34099]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:55:38.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:38 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T01:55:38.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:38 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:55:39.730 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:39 smithi190 ceph-mon[34762]: Deploying daemon osd.2 on smithi190 2024-09-16T01:55:39.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:39 smithi105 ceph-mon[31197]: Deploying daemon osd.2 on smithi190 2024-09-16T01:55:39.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:39 smithi137 ceph-mon[34099]: Deploying daemon osd.2 on smithi190 2024-09-16T01:55:40.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:40 smithi105 ceph-mon[31197]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:55:40.818 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:40 smithi190 ceph-mon[34762]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:55:40.920 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:40 smithi137 ceph-mon[34099]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:55:42.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:42 smithi105 ceph-mon[31197]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:55:42.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:42 smithi190 ceph-mon[34762]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:55:42.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:42 smithi137 ceph-mon[34099]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:55:44.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:44 smithi105 ceph-mon[31197]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:55:44.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:44 smithi190 ceph-mon[34762]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:55:44.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:44 smithi137 ceph-mon[34099]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:55:45.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:45 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:55:45.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:45 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:45.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:45 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:45.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:45 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:55:45.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:45 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:45.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:45 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:45.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:45 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:55:45.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:45 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:45.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:45 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:46.688 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:46 smithi190 ceph-mon[34762]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:55:46.688 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:46 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:46.688 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:46 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:46.688 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:46 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:55:46.689 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:46 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:55:46.689 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:46 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:46.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:46 smithi105 ceph-mon[31197]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:55:46.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:46 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:46.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:46 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:46.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:46 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:55:46.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:46 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:55:46.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:46 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:46.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:46 smithi137 ceph-mon[34099]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:55:46.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:46 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:46.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:46 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:46.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:46 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:55:46.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:46 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:55:46.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:46 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:48.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:48 smithi105 ceph-mon[31197]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:55:48.837 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:48 smithi190 ceph-mon[34762]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:55:48.876 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 2 on host 'smithi190' 2024-09-16T01:55:48.877 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:48.875+0000 7fc94affd640 1 -- 172.21.15.190:0/1977918930 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc95406a770 con 0x7fc9400a31e0 2024-09-16T01:55:48.878 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:48.877+0000 7fc95bf25640 1 -- 172.21.15.190:0/1977918930 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc9400a31e0 msgr2=0x7fc9400a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:48.879 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:48.877+0000 7fc95bf25640 1 --2- 172.21.15.190:0/1977918930 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc9400a31e0 0x7fc9400a56a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc954199350 tx=0x7fc95000b390 comp rx=0 tx=0).stop 2024-09-16T01:55:48.879 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:48.877+0000 7fc95bf25640 1 -- 172.21.15.190:0/1977918930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc954100ad0 msgr2=0x7fc954197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:48.879 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:48.877+0000 7fc95bf25640 1 --2- 172.21.15.190:0/1977918930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc954100ad0 0x7fc954197e30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc9440314e0 tx=0x7fc9440042e0 comp rx=0 tx=0).stop 2024-09-16T01:55:48.879 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:48.877+0000 7fc95bf25640 1 -- 172.21.15.190:0/1977918930 shutdown_connections 2024-09-16T01:55:48.879 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:48.877+0000 7fc95bf25640 1 --2- 172.21.15.190:0/1977918930 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc9400a31e0 0x7fc9400a56a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:48.879 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:48.877+0000 7fc95bf25640 1 --2- 172.21.15.190:0/1977918930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc954102690 0x7fc95419f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:48.879 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:48.877+0000 7fc95bf25640 1 --2- 172.21.15.190:0/1977918930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc954101cd0 0x7fc954198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:48.879 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:48.877+0000 7fc95bf25640 1 --2- 172.21.15.190:0/1977918930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc954100ad0 0x7fc954197e30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:48.879 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:48.877+0000 7fc95bf25640 1 -- 172.21.15.190:0/1977918930 >> 172.21.15.190:0/1977918930 conn(0x7fc9540fc280 msgr2=0x7fc95406a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:48.879 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:48.877+0000 7fc95bf25640 1 -- 172.21.15.190:0/1977918930 shutdown_connections 2024-09-16T01:55:48.879 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:55:48.877+0000 7fc95bf25640 1 -- 172.21.15.190:0/1977918930 wait complete. 2024-09-16T01:55:48.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:48 smithi137 ceph-mon[34099]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:55:49.483 DEBUG:teuthology.orchestra.run.smithi190:osd.2> sudo journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@osd.2.service 2024-09-16T01:55:49.527 INFO:tasks.cephadm:Deploying osd.3 on smithi105 with /dev/vg_nvme/lv_3... 2024-09-16T01:55:49.527 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-16T01:55:49.719 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:49 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:55:49.719 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:49 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:49.719 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:49 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:49.772 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:55:49.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:49 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:55:49.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:49 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:49.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:49 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:49.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:49 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:55:49.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:49 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:49.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:49 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:50.342 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 01:55:50 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-2[39695]: 2024-09-16T01:55:50.137+0000 7fba1ce86640 -1 osd.2 0 log_to_monitors true 2024-09-16T01:55:50.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:50 smithi105 ceph-mon[31197]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:55:50.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:50 smithi105 ceph-mon[31197]: from='osd.2 [v2:172.21.15.190:6800/1159531918,v1:172.21.15.190:6801/1159531918]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T01:55:50.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:50 smithi105 ceph-mon[31197]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T01:55:50.822 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:50 smithi190 ceph-mon[34762]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:55:50.823 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:50 smithi190 ceph-mon[34762]: from='osd.2 [v2:172.21.15.190:6800/1159531918,v1:172.21.15.190:6801/1159531918]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T01:55:50.823 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:50 smithi190 ceph-mon[34762]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T01:55:50.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:50 smithi137 ceph-mon[34099]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:55:50.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:50 smithi137 ceph-mon[34099]: from='osd.2 [v2:172.21.15.190:6800/1159531918,v1:172.21.15.190:6801/1159531918]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T01:55:50.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:50 smithi137 ceph-mon[34099]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T01:55:52.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:51 smithi137 ceph-mon[34099]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T01:55:52.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:51 smithi137 ceph-mon[34099]: osdmap e15: 3 total, 2 up, 3 in 2024-09-16T01:55:52.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:51 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:55:52.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:51 smithi137 ceph-mon[34099]: from='osd.2 [v2:172.21.15.190:6800/1159531918,v1:172.21.15.190:6801/1159531918]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T01:55:52.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:51 smithi137 ceph-mon[34099]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T01:55:52.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:51 smithi137 ceph-mon[34099]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:55:52.222 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:51 smithi105 ceph-mon[31197]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T01:55:52.222 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:51 smithi105 ceph-mon[31197]: osdmap e15: 3 total, 2 up, 3 in 2024-09-16T01:55:52.223 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:51 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:55:52.223 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:51 smithi105 ceph-mon[31197]: from='osd.2 [v2:172.21.15.190:6800/1159531918,v1:172.21.15.190:6801/1159531918]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T01:55:52.223 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:51 smithi105 ceph-mon[31197]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T01:55:52.223 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:51 smithi105 ceph-mon[31197]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:55:52.225 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:55:52.251 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi105:vg_nvme/lv_3 2024-09-16T01:55:52.293 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:51 smithi190 ceph-mon[34762]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T01:55:52.293 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:51 smithi190 ceph-mon[34762]: osdmap e15: 3 total, 2 up, 3 in 2024-09-16T01:55:52.293 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:51 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:55:52.293 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:51 smithi190 ceph-mon[34762]: from='osd.2 [v2:172.21.15.190:6800/1159531918,v1:172.21.15.190:6801/1159531918]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T01:55:52.293 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:51 smithi190 ceph-mon[34762]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T01:55:52.293 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:51 smithi190 ceph-mon[34762]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:55:52.515 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:55:52.592 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 01:55:52 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-2[39695]: 2024-09-16T01:55:52.307+0000 7fba18e09640 -1 osd.2 0 waiting for initial osdmap 2024-09-16T01:55:52.592 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 01:55:52 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-2[39695]: 2024-09-16T01:55:52.310+0000 7fba10c29640 -1 osd.2 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T01:55:53.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.018+0000 7fc2214c6640 1 -- 172.21.15.105:0/1430708737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc21c102690 msgr2=0x7fc21c108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:53.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.018+0000 7fc2214c6640 1 --2- 172.21.15.105:0/1430708737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc21c102690 0x7fc21c108f20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc20c009900 tx=0x7fc20c02a410 comp rx=0 tx=0).stop 2024-09-16T01:55:53.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.018+0000 7fc2214c6640 1 -- 172.21.15.105:0/1430708737 shutdown_connections 2024-09-16T01:55:53.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.018+0000 7fc2214c6640 1 --2- 172.21.15.105:0/1430708737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc21c102690 0x7fc21c108f20 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:53.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.018+0000 7fc2214c6640 1 --2- 172.21.15.105:0/1430708737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc21c101cd0 0x7fc21c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:53.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.018+0000 7fc2214c6640 1 --2- 172.21.15.105:0/1430708737 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc21c100ad0 0x7fc21c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:53.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.018+0000 7fc2214c6640 1 -- 172.21.15.105:0/1430708737 >> 172.21.15.105:0/1430708737 conn(0x7fc21c0fc280 msgr2=0x7fc21c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:53.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.018+0000 7fc2214c6640 1 -- 172.21.15.105:0/1430708737 shutdown_connections 2024-09-16T01:55:53.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.018+0000 7fc2214c6640 1 -- 172.21.15.105:0/1430708737 wait complete. 2024-09-16T01:55:53.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.018+0000 7fc2214c6640 1 Processor -- start 2024-09-16T01:55:53.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.019+0000 7fc2214c6640 1 -- start start 2024-09-16T01:55:53.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.019+0000 7fc2214c6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc21c100ad0 0x7fc21c06aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:53.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.019+0000 7fc2214c6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc21c101cd0 0x7fc21c19cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:53.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.019+0000 7fc2214c6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc21c102690 0x7fc21c19f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:53.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.019+0000 7fc2214c6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc21c10c600 con 0x7fc21c100ad0 2024-09-16T01:55:53.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.019+0000 7fc2214c6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc21c10c480 con 0x7fc21c102690 2024-09-16T01:55:53.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.019+0000 7fc2214c6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc21c10c780 con 0x7fc21c101cd0 2024-09-16T01:55:53.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.019+0000 7fc21affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc21c100ad0 0x7fc21c06aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:53.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.019+0000 7fc21affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc21c100ad0 0x7fc21c06aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53308/0 (socket says 172.21.15.105:53308) 2024-09-16T01:55:53.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.019+0000 7fc21affd640 1 -- 172.21.15.105:0/3456717553 learned_addr learned my addr 172.21.15.105:0/3456717553 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:55:53.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.019+0000 7fc21a7fc640 1 --2- 172.21.15.105:0/3456717553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc21c101cd0 0x7fc21c19cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:53.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.019+0000 7fc21b7fe640 1 --2- 172.21.15.105:0/3456717553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc21c102690 0x7fc21c19f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:53.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.019+0000 7fc21affd640 1 -- 172.21.15.105:0/3456717553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc21c101cd0 msgr2=0x7fc21c19cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:53.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.019+0000 7fc21affd640 1 --2- 172.21.15.105:0/3456717553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc21c101cd0 0x7fc21c19cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:53.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.019+0000 7fc21affd640 1 -- 172.21.15.105:0/3456717553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc21c102690 msgr2=0x7fc21c19f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:53.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.019+0000 7fc21affd640 1 --2- 172.21.15.105:0/3456717553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc21c102690 0x7fc21c19f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:53.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.019+0000 7fc21affd640 1 -- 172.21.15.105:0/3456717553 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc21c19fa10 con 0x7fc21c100ad0 2024-09-16T01:55:53.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.019+0000 7fc21affd640 1 --2- 172.21.15.105:0/3456717553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc21c100ad0 0x7fc21c06aad0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc21400eb70 tx=0x7fc21400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:53.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.020+0000 7fc213fff640 1 -- 172.21.15.105:0/3456717553 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc214003b70 con 0x7fc21c100ad0 2024-09-16T01:55:53.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.020+0000 7fc213fff640 1 -- 172.21.15.105:0/3456717553 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc21400ee70 con 0x7fc21c100ad0 2024-09-16T01:55:53.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.020+0000 7fc2214c6640 1 -- 172.21.15.105:0/3456717553 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc21c19fca0 con 0x7fc21c100ad0 2024-09-16T01:55:53.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.020+0000 7fc213fff640 1 -- 172.21.15.105:0/3456717553 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc214010650 con 0x7fc21c100ad0 2024-09-16T01:55:53.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.020+0000 7fc2214c6640 1 -- 172.21.15.105:0/3456717553 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc21c1a0190 con 0x7fc21c100ad0 2024-09-16T01:55:53.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.021+0000 7fc2214c6640 1 -- 172.21.15.105:0/3456717553 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc21c107f20 con 0x7fc21c100ad0 2024-09-16T01:55:53.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.022+0000 7fc213fff640 1 -- 172.21.15.105:0/3456717553 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7fc214004020 con 0x7fc21c100ad0 2024-09-16T01:55:53.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.023+0000 7fc213fff640 1 --2- 172.21.15.105:0/3456717553 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc2080a3230 0x7fc2080a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:53.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.023+0000 7fc21a7fc640 1 --2- 172.21.15.105:0/3456717553 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc2080a3230 0x7fc2080a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:53.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.023+0000 7fc213fff640 1 -- 172.21.15.105:0/3456717553 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 2571+0+0 (secure 0 0 0) 0x7fc2140ce740 con 0x7fc21c100ad0 2024-09-16T01:55:53.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.023+0000 7fc21a7fc640 1 --2- 172.21.15.105:0/3456717553 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc2080a3230 0x7fc2080a56f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc21c06b050 tx=0x7fc20400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:53.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.027+0000 7fc213fff640 1 -- 172.21.15.105:0/3456717553 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc21409cbb0 con 0x7fc21c100ad0 2024-09-16T01:55:53.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:52 smithi137 ceph-mon[34099]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-09-16T01:55:53.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:52 smithi137 ceph-mon[34099]: osdmap e16: 3 total, 2 up, 3 in 2024-09-16T01:55:53.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:52 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:55:53.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:52 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:55:53.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:52 smithi137 ceph-mon[34099]: from='osd.2 ' entity='osd.2' 2024-09-16T01:55:53.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:52 smithi137 ceph-mon[34099]: Detected new or changed devices on smithi190 2024-09-16T01:55:53.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:52 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:53.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:52 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:53.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:52 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:55:53.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:52 smithi137 ceph-mon[34099]: Adjusting osd_memory_target on smithi190 to 17088M 2024-09-16T01:55:53.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:52 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:53.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:52 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:55:53.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:52 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:55:53.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:52 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:53.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:53.207+0000 7fc2214c6640 1 -- 172.21.15.105:0/3456717553 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc21c062630 con 0x7fc2080a3230 2024-09-16T01:55:53.210 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:52 smithi105 ceph-mon[31197]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-09-16T01:55:53.210 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:52 smithi105 ceph-mon[31197]: osdmap e16: 3 total, 2 up, 3 in 2024-09-16T01:55:53.210 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:52 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:55:53.210 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:52 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:55:53.210 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:52 smithi105 ceph-mon[31197]: from='osd.2 ' entity='osd.2' 2024-09-16T01:55:53.210 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:52 smithi105 ceph-mon[31197]: Detected new or changed devices on smithi190 2024-09-16T01:55:53.210 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:52 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:53.210 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:52 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:53.210 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:52 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:55:53.210 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:52 smithi105 ceph-mon[31197]: Adjusting osd_memory_target on smithi190 to 17088M 2024-09-16T01:55:53.210 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:52 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:53.211 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:52 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:55:53.211 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:52 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:55:53.211 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:52 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:53.342 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:52 smithi190 ceph-mon[34762]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-09-16T01:55:53.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:52 smithi190 ceph-mon[34762]: osdmap e16: 3 total, 2 up, 3 in 2024-09-16T01:55:53.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:52 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:55:53.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:52 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:55:53.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:52 smithi190 ceph-mon[34762]: from='osd.2 ' entity='osd.2' 2024-09-16T01:55:53.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:52 smithi190 ceph-mon[34762]: Detected new or changed devices on smithi190 2024-09-16T01:55:53.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:52 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:53.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:52 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:53.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:52 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:55:53.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:52 smithi190 ceph-mon[34762]: Adjusting osd_memory_target on smithi190 to 17088M 2024-09-16T01:55:53.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:52 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:53.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:52 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:55:53.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:52 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:55:53.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:52 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:54.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:53 smithi137 ceph-mon[34099]: purged_snaps scrub starts 2024-09-16T01:55:54.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:53 smithi137 ceph-mon[34099]: purged_snaps scrub ok 2024-09-16T01:55:54.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:53 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:55:54.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:53 smithi137 ceph-mon[34099]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:55:54.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:53 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:55:54.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:53 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:55:54.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:53 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:55:54.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:53 smithi137 ceph-mon[34099]: osd.2 [v2:172.21.15.190:6800/1159531918,v1:172.21.15.190:6801/1159531918] boot 2024-09-16T01:55:54.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:53 smithi137 ceph-mon[34099]: osdmap e17: 3 total, 3 up, 3 in 2024-09-16T01:55:54.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:53 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:55:54.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:53 smithi137 ceph-mon[34099]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:55:54.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:53 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-16T01:55:54.173 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:53 smithi105 ceph-mon[31197]: purged_snaps scrub starts 2024-09-16T01:55:54.173 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:53 smithi105 ceph-mon[31197]: purged_snaps scrub ok 2024-09-16T01:55:54.173 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:53 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:55:54.174 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:53 smithi105 ceph-mon[31197]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:55:54.174 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:53 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:55:54.174 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:53 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:55:54.174 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:53 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:55:54.174 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:53 smithi105 ceph-mon[31197]: osd.2 [v2:172.21.15.190:6800/1159531918,v1:172.21.15.190:6801/1159531918] boot 2024-09-16T01:55:54.174 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:53 smithi105 ceph-mon[31197]: osdmap e17: 3 total, 3 up, 3 in 2024-09-16T01:55:54.174 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:53 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:55:54.174 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:53 smithi105 ceph-mon[31197]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:55:54.174 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:53 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-16T01:55:54.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:53 smithi190 ceph-mon[34762]: purged_snaps scrub starts 2024-09-16T01:55:54.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:53 smithi190 ceph-mon[34762]: purged_snaps scrub ok 2024-09-16T01:55:54.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:53 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:55:54.345 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:53 smithi190 ceph-mon[34762]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:55:54.345 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:53 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:55:54.345 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:53 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:55:54.345 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:53 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:55:54.345 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:53 smithi190 ceph-mon[34762]: osd.2 [v2:172.21.15.190:6800/1159531918,v1:172.21.15.190:6801/1159531918] boot 2024-09-16T01:55:54.345 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:53 smithi190 ceph-mon[34762]: osdmap e17: 3 total, 3 up, 3 in 2024-09-16T01:55:54.345 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:53 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:55:54.345 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:53 smithi190 ceph-mon[34762]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:55:54.345 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:53 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-16T01:55:55.544 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:55 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T01:55:55.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:55 smithi105 ceph-mon[31197]: osdmap e18: 3 total, 3 up, 3 in 2024-09-16T01:55:55.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:55 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T01:55:55.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:55 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T01:55:55.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:55 smithi190 ceph-mon[34762]: osdmap e18: 3 total, 3 up, 3 in 2024-09-16T01:55:55.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:55 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T01:55:55.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:55 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T01:55:55.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:55 smithi137 ceph-mon[34099]: osdmap e18: 3 total, 3 up, 3 in 2024-09-16T01:55:55.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:55 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T01:55:55.933 INFO:journalctl@ceph.osd.0.smithi105.stdout:Sep 16 01:55:55 smithi105 sudo[49451]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T01:55:55.934 INFO:journalctl@ceph.osd.0.smithi105.stdout:Sep 16 01:55:55 smithi105 sudo[49451]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T01:55:55.934 INFO:journalctl@ceph.osd.0.smithi105.stdout:Sep 16 01:55:55 smithi105 sudo[49451]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T01:55:55.934 INFO:journalctl@ceph.osd.0.smithi105.stdout:Sep 16 01:55:55 smithi105 sudo[49451]: pam_unix(sudo:session): session closed for user root 2024-09-16T01:55:56.294 INFO:journalctl@ceph.osd.0.smithi105.stdout:Sep 16 01:55:55 smithi105 sudo[49479]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T01:55:56.295 INFO:journalctl@ceph.osd.0.smithi105.stdout:Sep 16 01:55:55 smithi105 sudo[49479]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T01:55:56.295 INFO:journalctl@ceph.osd.0.smithi105.stdout:Sep 16 01:55:55 smithi105 sudo[49479]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T01:55:56.295 INFO:journalctl@ceph.osd.0.smithi105.stdout:Sep 16 01:55:55 smithi105 sudo[49479]: pam_unix(sudo:session): session closed for user root 2024-09-16T01:55:56.467 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 16 01:55:56 smithi137 sudo[42649]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T01:55:56.467 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 16 01:55:56 smithi137 sudo[42649]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T01:55:56.467 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 16 01:55:56 smithi137 sudo[42649]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T01:55:56.467 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 16 01:55:56 smithi137 sudo[42649]: pam_unix(sudo:session): session closed for user root 2024-09-16T01:55:56.467 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:56 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T01:55:56.467 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:56 smithi137 ceph-mon[34099]: osdmap e19: 3 total, 3 up, 3 in 2024-09-16T01:55:56.467 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:56 smithi137 ceph-mon[34099]: pgmap v77: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:55:56.467 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:56 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/151618558' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5743c37b-5cd8-42b6-81c8-5cf5afc71e9a"}]: dispatch 2024-09-16T01:55:56.468 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:56 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/151618558' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5743c37b-5cd8-42b6-81c8-5cf5afc71e9a"}]': finished 2024-09-16T01:55:56.468 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:56 smithi137 ceph-mon[34099]: osdmap e20: 4 total, 3 up, 4 in 2024-09-16T01:55:56.468 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:56 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:55:56.468 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:56 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:56.468 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:56 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/2115425828' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:55:56.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:56 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T01:55:56.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:56 smithi190 ceph-mon[34762]: osdmap e19: 3 total, 3 up, 3 in 2024-09-16T01:55:56.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:56 smithi190 ceph-mon[34762]: pgmap v77: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:55:56.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:56 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/151618558' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5743c37b-5cd8-42b6-81c8-5cf5afc71e9a"}]: dispatch 2024-09-16T01:55:56.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:56 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/151618558' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5743c37b-5cd8-42b6-81c8-5cf5afc71e9a"}]': finished 2024-09-16T01:55:56.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:56 smithi190 ceph-mon[34762]: osdmap e20: 4 total, 3 up, 4 in 2024-09-16T01:55:56.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:56 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:55:56.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:56 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:56.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:56 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/2115425828' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:55:56.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:56 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T01:55:56.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:56 smithi105 ceph-mon[31197]: osdmap e19: 3 total, 3 up, 3 in 2024-09-16T01:55:56.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:56 smithi105 ceph-mon[31197]: pgmap v77: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:55:56.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:56 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/151618558' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5743c37b-5cd8-42b6-81c8-5cf5afc71e9a"}]: dispatch 2024-09-16T01:55:56.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:56 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/151618558' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5743c37b-5cd8-42b6-81c8-5cf5afc71e9a"}]': finished 2024-09-16T01:55:56.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:56 smithi105 ceph-mon[31197]: osdmap e20: 4 total, 3 up, 4 in 2024-09-16T01:55:56.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:56 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:55:56.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:56 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:56.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:56 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2115425828' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:55:56.921 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 16 01:55:56 smithi137 sudo[42652]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T01:55:56.921 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 16 01:55:56 smithi137 sudo[42652]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T01:55:56.921 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 16 01:55:56 smithi137 sudo[42652]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T01:55:56.921 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 16 01:55:56 smithi137 sudo[42652]: pam_unix(sudo:session): session closed for user root 2024-09-16T01:55:56.990 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 01:55:56 smithi190 sudo[43289]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T01:55:56.991 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 01:55:56 smithi190 sudo[43289]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T01:55:56.991 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 01:55:56 smithi190 sudo[43289]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T01:55:56.991 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 01:55:56 smithi190 sudo[43289]: pam_unix(sudo:session): session closed for user root 2024-09-16T01:55:57.342 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 01:55:56 smithi190 sudo[43292]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T01:55:57.342 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 01:55:56 smithi190 sudo[43292]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T01:55:57.343 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 01:55:56 smithi190 sudo[43292]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T01:55:57.343 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 01:55:56 smithi190 sudo[43292]: pam_unix(sudo:session): session closed for user root 2024-09-16T01:55:57.517 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:57 smithi105 sudo[49714]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T01:55:57.517 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:57 smithi105 sudo[49714]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T01:55:57.517 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:57 smithi105 sudo[49714]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T01:55:57.517 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:57 smithi105 sudo[49714]: pam_unix(sudo:session): session closed for user root 2024-09-16T01:55:57.781 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:57 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:57.781 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:57 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:57.781 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:57 smithi190 ceph-mon[34762]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T01:55:57.781 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:57 smithi190 sudo[43295]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T01:55:57.781 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:57 smithi190 sudo[43295]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T01:55:57.781 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:57 smithi190 sudo[43295]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T01:55:57.781 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:57 smithi190 sudo[43295]: pam_unix(sudo:session): session closed for user root 2024-09-16T01:55:57.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:57 smithi105 sudo[49900]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T01:55:57.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:57 smithi105 sudo[49900]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T01:55:57.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:57 smithi105 sudo[49900]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T01:55:57.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:57 smithi105 sudo[49900]: pam_unix(sudo:session): session closed for user root 2024-09-16T01:55:57.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:57 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:57.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:57 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:57.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:57 smithi105 ceph-mon[31197]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T01:55:57.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:57 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:57.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:57 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:55:57.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:57 smithi137 ceph-mon[34099]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T01:55:58.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:58 smithi190 sudo[43298]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T01:55:58.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:58 smithi190 sudo[43298]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T01:55:58.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:58 smithi190 sudo[43298]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T01:55:58.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:58 smithi190 sudo[43298]: pam_unix(sudo:session): session closed for user root 2024-09-16T01:55:58.517 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:58 smithi137 sudo[42655]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T01:55:58.517 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:58 smithi137 sudo[42655]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T01:55:58.517 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:58 smithi137 sudo[42655]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T01:55:58.517 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:58 smithi137 sudo[42655]: pam_unix(sudo:session): session closed for user root 2024-09-16T01:55:58.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:55:58.529+0000 7fc213fff640 1 -- 172.21.15.105:0/3456717553 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fc214004310 con 0x7fc21c100ad0 2024-09-16T01:55:58.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:58 smithi105 ceph-mon[31197]: pgmap v79: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:55:58.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:58 smithi105 ceph-mon[31197]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T01:55:58.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:58 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:55:58.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:58 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:55:58.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:58 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:55:58.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:58 smithi105 ceph-mon[31197]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T01:55:58.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:58 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:55:58.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:58 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:55:58.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:58 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:55:58.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:58 smithi105 ceph-mon[31197]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T01:55:58.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:58 smithi105 ceph-mon[31197]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T01:55:58.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:58 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:55:58.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:58 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:55:58.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:58 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:55:58.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:58 smithi190 ceph-mon[34762]: pgmap v79: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:55:58.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:58 smithi190 ceph-mon[34762]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T01:55:58.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:58 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:55:58.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:58 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:55:58.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:58 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:55:58.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:58 smithi190 ceph-mon[34762]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T01:55:58.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:58 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:55:58.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:58 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:55:58.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:58 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:55:58.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:58 smithi190 ceph-mon[34762]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T01:55:58.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:58 smithi190 ceph-mon[34762]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T01:55:58.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:58 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:55:58.844 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:58 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:55:58.844 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:58 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:55:58.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:58 smithi137 sudo[42658]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T01:55:58.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:58 smithi137 sudo[42658]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T01:55:58.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:58 smithi137 sudo[42658]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T01:55:58.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:58 smithi137 sudo[42658]: pam_unix(sudo:session): session closed for user root 2024-09-16T01:55:58.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:58 smithi137 ceph-mon[34099]: pgmap v79: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:55:58.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:58 smithi137 ceph-mon[34099]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T01:55:58.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:58 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:55:58.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:58 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:55:58.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:58 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:55:58.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:58 smithi137 ceph-mon[34099]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T01:55:58.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:58 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:55:58.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:58 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:55:58.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:58 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:55:58.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:58 smithi137 ceph-mon[34099]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T01:55:58.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:58 smithi137 ceph-mon[34099]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T01:55:58.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:58 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:55:58.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:58 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:55:58.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:58 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:55:59.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:59 smithi105 ceph-mon[31197]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T01:55:59.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:55:59 smithi105 ceph-mon[31197]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-16T01:55:59.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:59 smithi190 ceph-mon[34762]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T01:55:59.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:55:59 smithi190 ceph-mon[34762]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-16T01:55:59.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:59 smithi137 ceph-mon[34099]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T01:55:59.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:55:59 smithi137 ceph-mon[34099]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-16T01:56:00.665 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:00 smithi105 ceph-mon[31197]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:56:00.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:00 smithi190 ceph-mon[34762]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:56:00.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:00 smithi137 ceph-mon[34099]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:56:01.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:01 smithi105 ceph-mon[31197]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:56:01.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:01 smithi190 ceph-mon[34762]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:56:01.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:01 smithi137 ceph-mon[34099]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:56:04.544 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:04 smithi105 ceph-mon[31197]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:56:04.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:04 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T01:56:04.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:04 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:56:04.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:04 smithi190 ceph-mon[34762]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:56:04.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:04 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T01:56:04.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:04 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:56:04.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:04 smithi137 ceph-mon[34099]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:56:04.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:04 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T01:56:04.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:04 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:56:05.704 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:05 smithi105 ceph-mon[31197]: Deploying daemon osd.3 on smithi105 2024-09-16T01:56:05.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:05 smithi190 ceph-mon[34762]: Deploying daemon osd.3 on smithi105 2024-09-16T01:56:05.920 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:05 smithi137 ceph-mon[34099]: Deploying daemon osd.3 on smithi105 2024-09-16T01:56:06.544 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:06 smithi105 ceph-mon[31197]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:56:06.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:06 smithi190 ceph-mon[34762]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:56:06.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:06 smithi137 ceph-mon[34099]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:56:08.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:08 smithi105 ceph-mon[31197]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:56:08.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:08 smithi190 ceph-mon[34762]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:56:08.920 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:08 smithi137 ceph-mon[34099]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:56:10.724 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:10 smithi105 ceph-mon[31197]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:56:10.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:10 smithi190 ceph-mon[34762]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:56:10.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:10 smithi137 ceph-mon[34099]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:56:11.536 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:11 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:56:11.536 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:11 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:11.536 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:11 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:11.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:11 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:56:11.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:11 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:11.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:11 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:11.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:11 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:56:11.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:11 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:11.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:11 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:12.718 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:12 smithi105 ceph-mon[31197]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:56:12.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:12 smithi190 ceph-mon[34762]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:56:12.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:12 smithi137 ceph-mon[34099]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:56:14.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:56:14.266+0000 7fc213fff640 1 -- 172.21.15.105:0/3456717553 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc21c062630 con 0x7fc2080a3230 2024-09-16T01:56:14.268 INFO:teuthology.orchestra.run.smithi105.stdout:Created osd(s) 3 on host 'smithi105' 2024-09-16T01:56:14.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:56:14.268+0000 7fc2214c6640 1 -- 172.21.15.105:0/3456717553 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc2080a3230 msgr2=0x7fc2080a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:14.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:56:14.268+0000 7fc2214c6640 1 --2- 172.21.15.105:0/3456717553 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc2080a3230 0x7fc2080a56f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc21c06b050 tx=0x7fc20400a2d0 comp rx=0 tx=0).stop 2024-09-16T01:56:14.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:56:14.268+0000 7fc2214c6640 1 -- 172.21.15.105:0/3456717553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc21c100ad0 msgr2=0x7fc21c06aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:14.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:56:14.268+0000 7fc2214c6640 1 --2- 172.21.15.105:0/3456717553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc21c100ad0 0x7fc21c06aad0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc21400eb70 tx=0x7fc21400c6a0 comp rx=0 tx=0).stop 2024-09-16T01:56:14.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:56:14.269+0000 7fc2214c6640 1 -- 172.21.15.105:0/3456717553 shutdown_connections 2024-09-16T01:56:14.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:56:14.269+0000 7fc2214c6640 1 --2- 172.21.15.105:0/3456717553 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc2080a3230 0x7fc2080a56f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:14.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:56:14.269+0000 7fc2214c6640 1 --2- 172.21.15.105:0/3456717553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc21c101cd0 0x7fc21c19cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:14.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:56:14.269+0000 7fc2214c6640 1 --2- 172.21.15.105:0/3456717553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc21c102690 0x7fc21c19f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:14.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:56:14.269+0000 7fc2214c6640 1 --2- 172.21.15.105:0/3456717553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc21c100ad0 0x7fc21c06aad0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:14.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:56:14.269+0000 7fc2214c6640 1 -- 172.21.15.105:0/3456717553 >> 172.21.15.105:0/3456717553 conn(0x7fc21c0fc280 msgr2=0x7fc21c1050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:14.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:56:14.269+0000 7fc2214c6640 1 -- 172.21.15.105:0/3456717553 shutdown_connections 2024-09-16T01:56:14.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:56:14.269+0000 7fc2214c6640 1 -- 172.21.15.105:0/3456717553 wait complete. 2024-09-16T01:56:14.544 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:14 smithi105 ceph-mon[31197]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:56:14.544 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:14 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:14.544 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:14 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:14.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:14 smithi190 ceph-mon[34762]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:56:14.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:14 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:14.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:14 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:14.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:14 smithi137 ceph-mon[34099]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:56:14.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:14 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:14.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:14 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:15.017 DEBUG:teuthology.orchestra.run.smithi105:osd.3> sudo journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@osd.3.service 2024-09-16T01:56:15.020 INFO:tasks.cephadm:Deploying osd.4 on smithi137 with /dev/vg_nvme/lv_3... 2024-09-16T01:56:15.020 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-16T01:56:15.274 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T01:56:16.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:16 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:16.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:16 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:16.297 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:16 smithi105 ceph-mon[31197]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:56:16.349 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:16 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:16.350 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:16 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:16.350 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:16 smithi137 ceph-mon[34099]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:56:16.559 INFO:journalctl@ceph.osd.3.smithi105.stdout:Sep 16 01:56:16 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-3[51503]: 2024-09-16T01:56:16.294+0000 7f3161d89640 -1 osd.3 0 log_to_monitors true 2024-09-16T01:56:16.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:16 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:16.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:16 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:16.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:16 smithi190 ceph-mon[34762]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:56:17.385 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:17 smithi137 ceph-mon[34099]: from='osd.3 [v2:172.21.15.105:6810/604905679,v1:172.21.15.105:6811/604905679]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T01:56:17.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:17 smithi105 ceph-mon[31197]: from='osd.3 [v2:172.21.15.105:6810/604905679,v1:172.21.15.105:6811/604905679]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T01:56:17.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:17 smithi190 ceph-mon[34762]: from='osd.3 [v2:172.21.15.105:6810/604905679,v1:172.21.15.105:6811/604905679]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T01:56:17.643 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-16T01:56:17.670 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi137:vg_nvme/lv_3 2024-09-16T01:56:17.917 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T01:56:18.351 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:18 smithi137 ceph-mon[34099]: from='osd.3 [v2:172.21.15.105:6810/604905679,v1:172.21.15.105:6811/604905679]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T01:56:18.351 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:18 smithi137 ceph-mon[34099]: osdmap e21: 4 total, 3 up, 4 in 2024-09-16T01:56:18.351 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:18 smithi137 ceph-mon[34099]: from='osd.3 [v2:172.21.15.105:6810/604905679,v1:172.21.15.105:6811/604905679]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-09-16T01:56:18.351 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:18 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:56:18.352 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:18 smithi137 ceph-mon[34099]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:56:18.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.361+0000 7f8fcc884640 1 -- 172.21.15.137:0/821497425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fc8101cd0 msgr2=0x7f8fc8102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:18.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.361+0000 7f8fcc884640 1 --2- 172.21.15.137:0/821497425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fc8101cd0 0x7f8fc8102150 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc009900 tx=0x7f8fbc02a410 comp rx=0 tx=0).stop 2024-09-16T01:56:18.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.362+0000 7f8fcc884640 1 -- 172.21.15.137:0/821497425 shutdown_connections 2024-09-16T01:56:18.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.362+0000 7f8fcc884640 1 --2- 172.21.15.137:0/821497425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fc8102690 0x7f8fc8108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:18.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.362+0000 7f8fcc884640 1 --2- 172.21.15.137:0/821497425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fc8101cd0 0x7f8fc8102150 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:18.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.362+0000 7f8fcc884640 1 --2- 172.21.15.137:0/821497425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fc8100ad0 0x7f8fc8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:18.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.362+0000 7f8fcc884640 1 -- 172.21.15.137:0/821497425 >> 172.21.15.137:0/821497425 conn(0x7f8fc80fc280 msgr2=0x7f8fc80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:18.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.362+0000 7f8fcc884640 1 -- 172.21.15.137:0/821497425 shutdown_connections 2024-09-16T01:56:18.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.362+0000 7f8fcc884640 1 -- 172.21.15.137:0/821497425 wait complete. 2024-09-16T01:56:18.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.362+0000 7f8fcc884640 1 Processor -- start 2024-09-16T01:56:18.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.363+0000 7f8fcc884640 1 -- start start 2024-09-16T01:56:18.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.363+0000 7f8fcc884640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fc8100ad0 0x7f8fc8197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:18.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.363+0000 7f8fcc884640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fc8101cd0 0x7f8fc8198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:18.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.363+0000 7f8fcc884640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fc8102690 0x7f8fc819f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:18.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.363+0000 7f8fcc884640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc810c5e0 con 0x7f8fc8102690 2024-09-16T01:56:18.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.363+0000 7f8fcc884640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc810c460 con 0x7f8fc8101cd0 2024-09-16T01:56:18.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.363+0000 7f8fcc884640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc810c760 con 0x7f8fc8100ad0 2024-09-16T01:56:18.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.363+0000 7f8fc5d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fc8101cd0 0x7f8fc8198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:18.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.363+0000 7f8fc5d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fc8101cd0 0x7f8fc8198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37508/0 (socket says 172.21.15.137:37508) 2024-09-16T01:56:18.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.363+0000 7f8fc5d9b640 1 -- 172.21.15.137:0/3600569775 learned_addr learned my addr 172.21.15.137:0/3600569775 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-16T01:56:18.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.363+0000 7f8fc6d9d640 1 --2- 172.21.15.137:0/3600569775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fc8102690 0x7f8fc819f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:18.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.363+0000 7f8fc659c640 1 --2- 172.21.15.137:0/3600569775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fc8100ad0 0x7f8fc8197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:18.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.363+0000 7f8fc5d9b640 1 -- 172.21.15.137:0/3600569775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fc8100ad0 msgr2=0x7f8fc8197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:18.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.363+0000 7f8fc5d9b640 1 --2- 172.21.15.137:0/3600569775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fc8100ad0 0x7f8fc8197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:18.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.363+0000 7f8fc5d9b640 1 -- 172.21.15.137:0/3600569775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fc8102690 msgr2=0x7f8fc819f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:18.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.363+0000 7f8fc5d9b640 1 --2- 172.21.15.137:0/3600569775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fc8102690 0x7f8fc819f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:18.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.363+0000 7f8fc5d9b640 1 -- 172.21.15.137:0/3600569775 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc8102ef0 con 0x7f8fc8101cd0 2024-09-16T01:56:18.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.363+0000 7f8fc6d9d640 1 --2- 172.21.15.137:0/3600569775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fc8102690 0x7f8fc819f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:56:18.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.363+0000 7f8fc5d9b640 1 --2- 172.21.15.137:0/3600569775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fc8101cd0 0x7f8fc8198330 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc005940 tx=0x7f8fbc0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:18.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.364+0000 7f8fb77fe640 1 -- 172.21.15.137:0/3600569775 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc03f030 con 0x7f8fc8101cd0 2024-09-16T01:56:18.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.364+0000 7f8fb77fe640 1 -- 172.21.15.137:0/3600569775 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fbc003cb0 con 0x7f8fc8101cd0 2024-09-16T01:56:18.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.364+0000 7f8fcc884640 1 -- 172.21.15.137:0/3600569775 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fc8103180 con 0x7f8fc8101cd0 2024-09-16T01:56:18.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.364+0000 7f8fb77fe640 1 -- 172.21.15.137:0/3600569775 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc035070 con 0x7f8fc8101cd0 2024-09-16T01:56:18.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.364+0000 7f8fcc884640 1 -- 172.21.15.137:0/3600569775 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fc81036b0 con 0x7f8fc8101cd0 2024-09-16T01:56:18.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.365+0000 7f8fcc884640 1 -- 172.21.15.137:0/3600569775 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fa8005290 con 0x7f8fc8101cd0 2024-09-16T01:56:18.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.366+0000 7f8fb77fe640 1 -- 172.21.15.137:0/3600569775 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f8fbc042030 con 0x7f8fc8101cd0 2024-09-16T01:56:18.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.367+0000 7f8fb77fe640 1 --2- 172.21.15.137:0/3600569775 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f8fb00a32f0 0x7f8fb00a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:18.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.367+0000 7f8fb77fe640 1 -- 172.21.15.137:0/3600569775 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 3389+0+0 (secure 0 0 0) 0x7f8fbc0eefa0 con 0x7f8fc8101cd0 2024-09-16T01:56:18.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.367+0000 7f8fc659c640 1 --2- 172.21.15.137:0/3600569775 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f8fb00a32f0 0x7f8fb00a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:18.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.367+0000 7f8fc659c640 1 --2- 172.21.15.137:0/3600569775 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f8fb00a32f0 0x7f8fb00a57b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8101b30 tx=0x7f8fb8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:18.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.370+0000 7f8fb77fe640 1 -- 172.21.15.137:0/3600569775 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8fbc0bc9b0 con 0x7f8fc8101cd0 2024-09-16T01:56:18.405 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:18 smithi105 ceph-mon[31197]: from='osd.3 [v2:172.21.15.105:6810/604905679,v1:172.21.15.105:6811/604905679]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T01:56:18.405 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:18 smithi105 ceph-mon[31197]: osdmap e21: 4 total, 3 up, 4 in 2024-09-16T01:56:18.405 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:18 smithi105 ceph-mon[31197]: from='osd.3 [v2:172.21.15.105:6810/604905679,v1:172.21.15.105:6811/604905679]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-09-16T01:56:18.405 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:18 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:56:18.405 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:18 smithi105 ceph-mon[31197]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:56:18.493 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:18 smithi190 ceph-mon[34762]: from='osd.3 [v2:172.21.15.105:6810/604905679,v1:172.21.15.105:6811/604905679]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T01:56:18.494 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:18 smithi190 ceph-mon[34762]: osdmap e21: 4 total, 3 up, 4 in 2024-09-16T01:56:18.494 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:18 smithi190 ceph-mon[34762]: from='osd.3 [v2:172.21.15.105:6810/604905679,v1:172.21.15.105:6811/604905679]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-09-16T01:56:18.494 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:18 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:56:18.494 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:18 smithi190 ceph-mon[34762]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:56:18.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:18.551+0000 7f8fcc884640 1 -- 172.21.15.137:0/3600569775 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f8fa8000bc0 con 0x7f8fb00a32f0 2024-09-16T01:56:18.794 INFO:journalctl@ceph.osd.3.smithi105.stdout:Sep 16 01:56:18 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-3[51503]: 2024-09-16T01:56:18.520+0000 7f315dd0c640 -1 osd.3 0 waiting for initial osdmap 2024-09-16T01:56:18.794 INFO:journalctl@ceph.osd.3.smithi105.stdout:Sep 16 01:56:18 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-3[51503]: 2024-09-16T01:56:18.524+0000 7f3155b2c640 -1 osd.3 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T01:56:19.221 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:19 smithi137 ceph-mon[34099]: from='osd.3 [v2:172.21.15.105:6810/604905679,v1:172.21.15.105:6811/604905679]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-09-16T01:56:19.221 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:19 smithi137 ceph-mon[34099]: osdmap e22: 4 total, 3 up, 4 in 2024-09-16T01:56:19.221 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:19 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:56:19.222 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:19 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:56:19.222 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:19 smithi137 ceph-mon[34099]: Detected new or changed devices on smithi105 2024-09-16T01:56:19.222 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:19 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:19.222 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:19 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:19.222 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:19 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:56:19.222 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:19 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:56:19.222 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:19 smithi137 ceph-mon[34099]: Adjusting osd_memory_target on smithi105 to 8544M 2024-09-16T01:56:19.222 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:19 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:19.222 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:19 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:56:19.222 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:19 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:56:19.222 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:19 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:19.222 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:19 smithi137 ceph-mon[34099]: from='osd.3 [v2:172.21.15.105:6810/604905679,v1:172.21.15.105:6811/604905679]' entity='osd.3' 2024-09-16T01:56:19.222 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:19 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:56:19.223 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:19 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:56:19.223 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:19 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:56:19.223 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:19 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:56:19.223 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:19 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:56:19.223 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:19 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:56:19.223 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:19 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:19.544 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:19 smithi105 ceph-mon[31197]: from='osd.3 [v2:172.21.15.105:6810/604905679,v1:172.21.15.105:6811/604905679]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-09-16T01:56:19.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:19 smithi105 ceph-mon[31197]: osdmap e22: 4 total, 3 up, 4 in 2024-09-16T01:56:19.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:19 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:56:19.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:19 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:56:19.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:19 smithi105 ceph-mon[31197]: Detected new or changed devices on smithi105 2024-09-16T01:56:19.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:19 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:19.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:19 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:19.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:19 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:56:19.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:19 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:56:19.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:19 smithi105 ceph-mon[31197]: Adjusting osd_memory_target on smithi105 to 8544M 2024-09-16T01:56:19.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:19 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:19.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:19 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:56:19.546 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:19 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:56:19.546 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:19 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:19.546 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:19 smithi105 ceph-mon[31197]: from='osd.3 [v2:172.21.15.105:6810/604905679,v1:172.21.15.105:6811/604905679]' entity='osd.3' 2024-09-16T01:56:19.546 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:19 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:56:19.546 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:19 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:56:19.546 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:19 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:56:19.546 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:19 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:56:19.546 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:19 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:56:19.546 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:19 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:56:19.546 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:19 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:19.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:19 smithi190 ceph-mon[34762]: from='osd.3 [v2:172.21.15.105:6810/604905679,v1:172.21.15.105:6811/604905679]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-09-16T01:56:19.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:19 smithi190 ceph-mon[34762]: osdmap e22: 4 total, 3 up, 4 in 2024-09-16T01:56:19.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:19 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:56:19.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:19 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:56:19.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:19 smithi190 ceph-mon[34762]: Detected new or changed devices on smithi105 2024-09-16T01:56:19.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:19 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:19.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:19 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:19.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:19 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:56:19.594 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:19 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:56:19.594 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:19 smithi190 ceph-mon[34762]: Adjusting osd_memory_target on smithi105 to 8544M 2024-09-16T01:56:19.594 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:19 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:19.594 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:19 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:56:19.594 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:19 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:56:19.594 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:19 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:19.594 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:19 smithi190 ceph-mon[34762]: from='osd.3 [v2:172.21.15.105:6810/604905679,v1:172.21.15.105:6811/604905679]' entity='osd.3' 2024-09-16T01:56:19.594 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:19 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:56:19.594 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:19 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:56:19.594 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:19 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:56:19.594 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:19 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:56:19.594 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:19 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:56:19.594 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:19 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:56:19.594 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:19 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:20.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:20 smithi137 ceph-mon[34099]: purged_snaps scrub starts 2024-09-16T01:56:20.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:20 smithi137 ceph-mon[34099]: purged_snaps scrub ok 2024-09-16T01:56:20.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:20 smithi137 ceph-mon[34099]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:56:20.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:20 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:56:20.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:20 smithi137 ceph-mon[34099]: osd.3 [v2:172.21.15.105:6810/604905679,v1:172.21.15.105:6811/604905679] boot 2024-09-16T01:56:20.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:20 smithi137 ceph-mon[34099]: osdmap e23: 4 total, 4 up, 4 in 2024-09-16T01:56:20.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:20 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:56:20.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:20 smithi137 ceph-mon[34099]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:56:20.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:20 smithi137 ceph-mon[34099]: osdmap e24: 4 total, 4 up, 4 in 2024-09-16T01:56:20.544 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:20 smithi105 ceph-mon[31197]: purged_snaps scrub starts 2024-09-16T01:56:20.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:20 smithi105 ceph-mon[31197]: purged_snaps scrub ok 2024-09-16T01:56:20.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:20 smithi105 ceph-mon[31197]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:56:20.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:20 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:56:20.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:20 smithi105 ceph-mon[31197]: osd.3 [v2:172.21.15.105:6810/604905679,v1:172.21.15.105:6811/604905679] boot 2024-09-16T01:56:20.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:20 smithi105 ceph-mon[31197]: osdmap e23: 4 total, 4 up, 4 in 2024-09-16T01:56:20.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:20 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:56:20.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:20 smithi105 ceph-mon[31197]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:56:20.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:20 smithi105 ceph-mon[31197]: osdmap e24: 4 total, 4 up, 4 in 2024-09-16T01:56:20.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:20 smithi190 ceph-mon[34762]: purged_snaps scrub starts 2024-09-16T01:56:20.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:20 smithi190 ceph-mon[34762]: purged_snaps scrub ok 2024-09-16T01:56:20.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:20 smithi190 ceph-mon[34762]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:56:20.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:20 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:56:20.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:20 smithi190 ceph-mon[34762]: osd.3 [v2:172.21.15.105:6810/604905679,v1:172.21.15.105:6811/604905679] boot 2024-09-16T01:56:20.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:20 smithi190 ceph-mon[34762]: osdmap e23: 4 total, 4 up, 4 in 2024-09-16T01:56:20.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:20 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:56:20.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:20 smithi190 ceph-mon[34762]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:56:20.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:20 smithi190 ceph-mon[34762]: osdmap e24: 4 total, 4 up, 4 in 2024-09-16T01:56:21.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:21 smithi105 ceph-mon[31197]: osdmap e25: 4 total, 4 up, 4 in 2024-09-16T01:56:21.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:21 smithi105 ceph-mon[31197]: from='client.? 172.21.15.137:0/167781869' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e439d4a2-b54c-4de9-97f6-3510afe57f36"}]: dispatch 2024-09-16T01:56:21.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:21 smithi105 ceph-mon[31197]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e439d4a2-b54c-4de9-97f6-3510afe57f36"}]: dispatch 2024-09-16T01:56:21.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:21 smithi105 ceph-mon[31197]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e439d4a2-b54c-4de9-97f6-3510afe57f36"}]': finished 2024-09-16T01:56:21.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:21 smithi105 ceph-mon[31197]: osdmap e26: 5 total, 4 up, 5 in 2024-09-16T01:56:21.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:21 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:56:21.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:21 smithi190 ceph-mon[34762]: osdmap e25: 4 total, 4 up, 4 in 2024-09-16T01:56:21.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:21 smithi190 ceph-mon[34762]: from='client.? 172.21.15.137:0/167781869' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e439d4a2-b54c-4de9-97f6-3510afe57f36"}]: dispatch 2024-09-16T01:56:21.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:21 smithi190 ceph-mon[34762]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e439d4a2-b54c-4de9-97f6-3510afe57f36"}]: dispatch 2024-09-16T01:56:21.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:21 smithi190 ceph-mon[34762]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e439d4a2-b54c-4de9-97f6-3510afe57f36"}]': finished 2024-09-16T01:56:21.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:21 smithi190 ceph-mon[34762]: osdmap e26: 5 total, 4 up, 5 in 2024-09-16T01:56:21.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:21 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:56:21.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:21 smithi137 ceph-mon[34099]: osdmap e25: 4 total, 4 up, 4 in 2024-09-16T01:56:21.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:21 smithi137 ceph-mon[34099]: from='client.? 172.21.15.137:0/167781869' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e439d4a2-b54c-4de9-97f6-3510afe57f36"}]: dispatch 2024-09-16T01:56:21.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:21 smithi137 ceph-mon[34099]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e439d4a2-b54c-4de9-97f6-3510afe57f36"}]: dispatch 2024-09-16T01:56:21.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:21 smithi137 ceph-mon[34099]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e439d4a2-b54c-4de9-97f6-3510afe57f36"}]': finished 2024-09-16T01:56:21.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:21 smithi137 ceph-mon[34099]: osdmap e26: 5 total, 4 up, 5 in 2024-09-16T01:56:21.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:21 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:56:22.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:22 smithi105 ceph-mon[31197]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:56:22.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:22 smithi105 ceph-mon[31197]: from='client.? 172.21.15.137:0/3069045403' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:56:22.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:22 smithi105 ceph-mon[31197]: osdmap e27: 5 total, 4 up, 5 in 2024-09-16T01:56:22.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:22 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:56:22.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:22 smithi190 ceph-mon[34762]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:56:22.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:22 smithi190 ceph-mon[34762]: from='client.? 172.21.15.137:0/3069045403' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:56:22.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:22 smithi190 ceph-mon[34762]: osdmap e27: 5 total, 4 up, 5 in 2024-09-16T01:56:22.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:22 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:56:22.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:22 smithi137 ceph-mon[34099]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:56:22.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:22 smithi137 ceph-mon[34099]: from='client.? 172.21.15.137:0/3069045403' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:56:22.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:22 smithi137 ceph-mon[34099]: osdmap e27: 5 total, 4 up, 5 in 2024-09-16T01:56:22.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:22 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:56:24.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:24 smithi105 ceph-mon[31197]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:56:24.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:24 smithi190 ceph-mon[34762]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:56:24.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:24 smithi137 ceph-mon[34099]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:56:26.626 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:26 smithi137 ceph-mon[34099]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-16T01:56:26.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:26 smithi105 ceph-mon[31197]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-16T01:56:26.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:26 smithi190 ceph-mon[34762]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-16T01:56:28.631 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:28 smithi137 ceph-mon[34099]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-16T01:56:28.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:28 smithi105 ceph-mon[31197]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-16T01:56:28.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:28 smithi190 ceph-mon[34762]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-16T01:56:29.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:29 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T01:56:29.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:29 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:56:29.771 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:29 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T01:56:29.771 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:29 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:56:29.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:29 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T01:56:29.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:29 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:56:30.637 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:30 smithi137 ceph-mon[34099]: Deploying daemon osd.4 on smithi137 2024-09-16T01:56:30.638 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:30 smithi137 ceph-mon[34099]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-16T01:56:30.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:30 smithi105 ceph-mon[31197]: Deploying daemon osd.4 on smithi137 2024-09-16T01:56:30.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:30 smithi105 ceph-mon[31197]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-16T01:56:30.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:30 smithi190 ceph-mon[34762]: Deploying daemon osd.4 on smithi137 2024-09-16T01:56:30.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:30 smithi190 ceph-mon[34762]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-16T01:56:32.717 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:32 smithi137 ceph-mon[34099]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T01:56:32.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:32 smithi105 ceph-mon[31197]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T01:56:32.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:32 smithi190 ceph-mon[34762]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T01:56:34.569 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:34 smithi137 ceph-mon[34099]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-16T01:56:34.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:34 smithi105 ceph-mon[31197]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-16T01:56:34.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:34 smithi190 ceph-mon[34762]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-16T01:56:35.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:35 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:56:35.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:35 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:35.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:35 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:35.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:35 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:56:35.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:35 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:35.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:35 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:35.857 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:35 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:56:35.857 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:35 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:35.857 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:35 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:36.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:36 smithi105 ceph-mon[31197]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T01:56:36.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:36 smithi190 ceph-mon[34762]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T01:56:36.854 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:36 smithi137 ceph-mon[34099]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T01:56:38.663 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:38 smithi137 ceph-mon[34099]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:56:38.663 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:38 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:38.663 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:38 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:38.663 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:38 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:56:38.664 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:38 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:56:38.664 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:38 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:38.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:38 smithi105 ceph-mon[31197]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:56:38.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:38 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:38.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:38 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:38.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:38 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:56:38.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:38 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:56:38.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:38 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:38.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:38 smithi190 ceph-mon[34762]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:56:38.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:38 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:38.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:38 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:38.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:38 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:56:38.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:38 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:56:38.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:38 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:38.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:38.953+0000 7f8fb77fe640 1 -- 172.21.15.137:0/3600569775 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8fa8000bc0 con 0x7f8fb00a32f0 2024-09-16T01:56:38.956 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 4 on host 'smithi137' 2024-09-16T01:56:38.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:38.956+0000 7f8fcc884640 1 -- 172.21.15.137:0/3600569775 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f8fb00a32f0 msgr2=0x7f8fb00a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:38.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:38.956+0000 7f8fcc884640 1 --2- 172.21.15.137:0/3600569775 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f8fb00a32f0 0x7f8fb00a57b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8101b30 tx=0x7f8fb8005f90 comp rx=0 tx=0).stop 2024-09-16T01:56:38.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:38.956+0000 7f8fcc884640 1 -- 172.21.15.137:0/3600569775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fc8101cd0 msgr2=0x7f8fc8198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:38.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:38.956+0000 7f8fcc884640 1 --2- 172.21.15.137:0/3600569775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fc8101cd0 0x7f8fc8198330 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc005940 tx=0x7f8fbc0042f0 comp rx=0 tx=0).stop 2024-09-16T01:56:38.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:38.956+0000 7f8fcc884640 1 -- 172.21.15.137:0/3600569775 shutdown_connections 2024-09-16T01:56:38.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:38.956+0000 7f8fcc884640 1 --2- 172.21.15.137:0/3600569775 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f8fb00a32f0 0x7f8fb00a57b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:38.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:38.956+0000 7f8fcc884640 1 --2- 172.21.15.137:0/3600569775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fc8102690 0x7f8fc819f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:38.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:38.956+0000 7f8fcc884640 1 --2- 172.21.15.137:0/3600569775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fc8101cd0 0x7f8fc8198330 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:38.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:38.956+0000 7f8fcc884640 1 --2- 172.21.15.137:0/3600569775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fc8100ad0 0x7f8fc8197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:38.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:38.956+0000 7f8fcc884640 1 -- 172.21.15.137:0/3600569775 >> 172.21.15.137:0/3600569775 conn(0x7f8fc80fc280 msgr2=0x7f8fc8069fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:38.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:38.956+0000 7f8fcc884640 1 -- 172.21.15.137:0/3600569775 shutdown_connections 2024-09-16T01:56:38.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:56:38.956+0000 7f8fcc884640 1 -- 172.21.15.137:0/3600569775 wait complete. 2024-09-16T01:56:39.689 DEBUG:teuthology.orchestra.run.smithi137:osd.4> sudo journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@osd.4.service 2024-09-16T01:56:39.692 INFO:tasks.cephadm:Deploying osd.5 on smithi190 with /dev/vg_nvme/lv_3... 2024-09-16T01:56:39.692 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-16T01:56:39.721 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:39 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:56:39.721 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:39 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:39.721 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:39 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:39.772 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:39 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:56:39.772 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:39 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:39.772 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:39 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:39.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:39 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:56:39.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:39 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:39.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:39 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:39.946 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T01:56:40.773 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:40 smithi190 ceph-mon[34762]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:56:40.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:40 smithi105 ceph-mon[31197]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:56:40.827 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:40 smithi137 ceph-mon[34099]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:56:41.421 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 16 01:56:41 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-4[45404]: 2024-09-16T01:56:41.291+0000 7f7a4e7b6640 -1 osd.4 0 log_to_monitors true 2024-09-16T01:56:41.756 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:41 smithi190 ceph-mon[34762]: from='osd.4 [v2:172.21.15.137:6808/3758109169,v1:172.21.15.137:6809/3758109169]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T01:56:41.756 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:41 smithi190 ceph-mon[34762]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T01:56:41.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:41 smithi105 ceph-mon[31197]: from='osd.4 [v2:172.21.15.137:6808/3758109169,v1:172.21.15.137:6809/3758109169]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T01:56:41.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:41 smithi105 ceph-mon[31197]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T01:56:41.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:41 smithi137 ceph-mon[34099]: from='osd.4 [v2:172.21.15.137:6808/3758109169,v1:172.21.15.137:6809/3758109169]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T01:56:41.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:41 smithi137 ceph-mon[34099]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T01:56:42.479 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T01:56:42.508 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi190:vg_nvme/lv_3 2024-09-16T01:56:42.571 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:42 smithi190 ceph-mon[34762]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:56:42.571 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:42 smithi190 ceph-mon[34762]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T01:56:42.571 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:42 smithi190 ceph-mon[34762]: from='osd.4 [v2:172.21.15.137:6808/3758109169,v1:172.21.15.137:6809/3758109169]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-09-16T01:56:42.571 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:42 smithi190 ceph-mon[34762]: osdmap e28: 5 total, 4 up, 5 in 2024-09-16T01:56:42.572 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:42 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:56:42.572 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:42 smithi190 ceph-mon[34762]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-09-16T01:56:42.572 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:42 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:42.572 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:42 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:42.572 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:42 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:56:42.572 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:42 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:56:42.572 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:42 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:42.572 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:42 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:56:42.572 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:42 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:56:42.572 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:42 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:42.671 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 16 01:56:42 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-4[45404]: 2024-09-16T01:56:42.484+0000 7f7a4a739640 -1 osd.4 0 waiting for initial osdmap 2024-09-16T01:56:42.671 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 16 01:56:42 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-4[45404]: 2024-09-16T01:56:42.488+0000 7f7a46561640 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T01:56:42.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:42 smithi137 ceph-mon[34099]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:56:42.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:42 smithi137 ceph-mon[34099]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T01:56:42.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:42 smithi137 ceph-mon[34099]: from='osd.4 [v2:172.21.15.137:6808/3758109169,v1:172.21.15.137:6809/3758109169]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-09-16T01:56:42.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:42 smithi137 ceph-mon[34099]: osdmap e28: 5 total, 4 up, 5 in 2024-09-16T01:56:42.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:42 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:56:42.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:42 smithi137 ceph-mon[34099]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-09-16T01:56:42.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:42 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:42.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:42 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:42.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:42 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:56:42.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:42 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:56:42.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:42 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:42.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:42 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:56:42.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:42 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:56:42.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:42 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:42.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:42 smithi105 ceph-mon[31197]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:56:42.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:42 smithi105 ceph-mon[31197]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T01:56:42.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:42 smithi105 ceph-mon[31197]: from='osd.4 [v2:172.21.15.137:6808/3758109169,v1:172.21.15.137:6809/3758109169]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-09-16T01:56:42.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:42 smithi105 ceph-mon[31197]: osdmap e28: 5 total, 4 up, 5 in 2024-09-16T01:56:42.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:42 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:56:42.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:42 smithi105 ceph-mon[31197]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-09-16T01:56:42.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:42 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:42.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:42 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:42.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:42 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:56:42.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:42 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:56:42.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:42 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:42.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:42 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:56:42.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:42 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:56:42.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:42 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:56:42.796 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T01:56:43.267 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.242+0000 7f2ff1b6c640 1 -- 172.21.15.190:0/3194997992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fec100ad0 msgr2=0x7f2fec100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:43.267 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.242+0000 7f2ff1b6c640 1 --2- 172.21.15.190:0/3194997992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fec100ad0 0x7f2fec100ed0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009900 tx=0x7f2fe002a410 comp rx=0 tx=0).stop 2024-09-16T01:56:43.268 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.242+0000 7f2ff1b6c640 1 -- 172.21.15.190:0/3194997992 shutdown_connections 2024-09-16T01:56:43.268 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.242+0000 7f2ff1b6c640 1 --2- 172.21.15.190:0/3194997992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fec102690 0x7f2fec108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:43.268 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.242+0000 7f2ff1b6c640 1 --2- 172.21.15.190:0/3194997992 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fec101cd0 0x7f2fec102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:43.268 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.242+0000 7f2ff1b6c640 1 --2- 172.21.15.190:0/3194997992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fec100ad0 0x7f2fec100ed0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:43.268 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.242+0000 7f2ff1b6c640 1 -- 172.21.15.190:0/3194997992 >> 172.21.15.190:0/3194997992 conn(0x7f2fec0fc280 msgr2=0x7f2fec0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:43.268 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.242+0000 7f2ff1b6c640 1 -- 172.21.15.190:0/3194997992 shutdown_connections 2024-09-16T01:56:43.268 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.242+0000 7f2ff1b6c640 1 -- 172.21.15.190:0/3194997992 wait complete. 2024-09-16T01:56:43.269 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.242+0000 7f2ff1b6c640 1 Processor -- start 2024-09-16T01:56:43.269 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.243+0000 7f2ff1b6c640 1 -- start start 2024-09-16T01:56:43.269 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.243+0000 7f2ff1b6c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fec100ad0 0x7f2fec10eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:43.269 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.243+0000 7f2ff1b6c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fec101cd0 0x7f2fec10d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:43.269 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.243+0000 7f2ff1b6c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fec102690 0x7f2fec10d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:43.269 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.243+0000 7f2ff1b6c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2fec10c610 con 0x7f2fec102690 2024-09-16T01:56:43.269 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.243+0000 7f2ff1b6c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2fec10c490 con 0x7f2fec100ad0 2024-09-16T01:56:43.269 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.243+0000 7f2ff1b6c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2fec10c790 con 0x7f2fec101cd0 2024-09-16T01:56:43.269 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.243+0000 7f2feaffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fec101cd0 0x7f2fec10d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:43.270 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.243+0000 7f2feaffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fec101cd0 0x7f2fec10d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:46322/0 (socket says 172.21.15.190:46322) 2024-09-16T01:56:43.270 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.243+0000 7f2feaffd640 1 -- 172.21.15.190:0/2293424581 learned_addr learned my addr 172.21.15.190:0/2293424581 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-09-16T01:56:43.270 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.243+0000 7f2febfff640 1 --2- 172.21.15.190:0/2293424581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fec102690 0x7f2fec10d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:43.270 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.243+0000 7f2feb7fe640 1 --2- 172.21.15.190:0/2293424581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fec100ad0 0x7f2fec10eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:43.270 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.243+0000 7f2feaffd640 1 -- 172.21.15.190:0/2293424581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fec100ad0 msgr2=0x7f2fec10eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:43.270 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.243+0000 7f2feaffd640 1 --2- 172.21.15.190:0/2293424581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fec100ad0 0x7f2fec10eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:43.270 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.243+0000 7f2feaffd640 1 -- 172.21.15.190:0/2293424581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fec102690 msgr2=0x7f2fec10d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:43.270 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.243+0000 7f2feaffd640 1 --2- 172.21.15.190:0/2293424581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fec102690 0x7f2fec10d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:43.270 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.243+0000 7f2feaffd640 1 -- 172.21.15.190:0/2293424581 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fec10dfa0 con 0x7f2fec101cd0 2024-09-16T01:56:43.270 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.243+0000 7f2febfff640 1 --2- 172.21.15.190:0/2293424581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fec102690 0x7f2fec10d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:56:43.270 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.244+0000 7f2feaffd640 1 --2- 172.21.15.190:0/2293424581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fec101cd0 0x7f2fec10d1a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc00d950 tx=0x7f2fdc00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:43.271 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.244+0000 7f2fe8ff9640 1 -- 172.21.15.190:0/2293424581 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc017070 con 0x7f2fec101cd0 2024-09-16T01:56:43.271 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.244+0000 7f2ff1b6c640 1 -- 172.21.15.190:0/2293424581 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fec1a1f80 con 0x7f2fec101cd0 2024-09-16T01:56:43.271 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.244+0000 7f2fe8ff9640 1 -- 172.21.15.190:0/2293424581 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fdc009c30 con 0x7f2fec101cd0 2024-09-16T01:56:43.271 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.244+0000 7f2fe8ff9640 1 -- 172.21.15.190:0/2293424581 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc00f620 con 0x7f2fec101cd0 2024-09-16T01:56:43.271 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.244+0000 7f2ff1b6c640 1 -- 172.21.15.190:0/2293424581 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fec1a2430 con 0x7f2fec101cd0 2024-09-16T01:56:43.271 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.245+0000 7f2ff1b6c640 1 -- 172.21.15.190:0/2293424581 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fec1a2720 con 0x7f2fec101cd0 2024-09-16T01:56:43.271 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.246+0000 7f2fe8ff9640 1 -- 172.21.15.190:0/2293424581 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f2fdc0109a0 con 0x7f2fec101cd0 2024-09-16T01:56:43.271 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.247+0000 7f2fe8ff9640 1 --2- 172.21.15.190:0/2293424581 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f2fd40a32c0 0x7f2fd40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:43.271 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.247+0000 7f2fe8ff9640 1 -- 172.21.15.190:0/2293424581 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3821+0+0 (secure 0 0 0) 0x7f2fdc0ce3f0 con 0x7f2fec101cd0 2024-09-16T01:56:43.271 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.247+0000 7f2feb7fe640 1 --2- 172.21.15.190:0/2293424581 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f2fd40a32c0 0x7f2fd40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:43.271 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.248+0000 7f2feb7fe640 1 --2- 172.21.15.190:0/2293424581 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f2fd40a32c0 0x7f2fd40a5780 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2fe002a930 tx=0x7f2fe0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:43.271 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.251+0000 7f2fe8ff9640 1 -- 172.21.15.190:0/2293424581 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2fdc009de0 con 0x7f2fec101cd0 2024-09-16T01:56:43.433 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:56:43.432+0000 7f2ff1b6c640 1 -- 172.21.15.190:0/2293424581 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f2fec0fd0c0 con 0x7f2fd40a32c0 2024-09-16T01:56:43.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:43 smithi105 ceph-mon[31197]: Detected new or changed devices on smithi137 2024-09-16T01:56:43.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:43 smithi105 ceph-mon[31197]: Adjusting osd_memory_target on smithi137 to 8544M 2024-09-16T01:56:43.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:43 smithi105 ceph-mon[31197]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-09-16T01:56:43.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:43 smithi105 ceph-mon[31197]: osdmap e29: 5 total, 4 up, 5 in 2024-09-16T01:56:43.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:43 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:56:43.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:43 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:56:43.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:43 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:56:43.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:43 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:56:43.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:43 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:56:43.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:43 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:56:43.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:43 smithi190 ceph-mon[34762]: Detected new or changed devices on smithi137 2024-09-16T01:56:43.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:43 smithi190 ceph-mon[34762]: Adjusting osd_memory_target on smithi137 to 8544M 2024-09-16T01:56:43.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:43 smithi190 ceph-mon[34762]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-09-16T01:56:43.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:43 smithi190 ceph-mon[34762]: osdmap e29: 5 total, 4 up, 5 in 2024-09-16T01:56:43.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:43 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:56:43.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:43 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:56:43.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:43 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:56:43.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:43 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:56:43.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:43 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:56:43.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:43 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:56:43.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:43 smithi137 ceph-mon[34099]: Detected new or changed devices on smithi137 2024-09-16T01:56:43.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:43 smithi137 ceph-mon[34099]: Adjusting osd_memory_target on smithi137 to 8544M 2024-09-16T01:56:43.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:43 smithi137 ceph-mon[34099]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-09-16T01:56:43.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:43 smithi137 ceph-mon[34099]: osdmap e29: 5 total, 4 up, 5 in 2024-09-16T01:56:43.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:43 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:56:43.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:43 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:56:43.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:43 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:56:43.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:43 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:56:43.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:43 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:56:43.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:43 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:56:44.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:44 smithi105 ceph-mon[31197]: purged_snaps scrub starts 2024-09-16T01:56:44.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:44 smithi105 ceph-mon[31197]: purged_snaps scrub ok 2024-09-16T01:56:44.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:44 smithi105 ceph-mon[31197]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:56:44.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:44 smithi105 ceph-mon[31197]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:56:44.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:44 smithi105 ceph-mon[31197]: osd.4 [v2:172.21.15.137:6808/3758109169,v1:172.21.15.137:6809/3758109169] boot 2024-09-16T01:56:44.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:44 smithi105 ceph-mon[31197]: osdmap e30: 5 total, 5 up, 5 in 2024-09-16T01:56:44.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:44 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:56:44.815 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:44 smithi190 ceph-mon[34762]: purged_snaps scrub starts 2024-09-16T01:56:44.815 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:44 smithi190 ceph-mon[34762]: purged_snaps scrub ok 2024-09-16T01:56:44.815 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:44 smithi190 ceph-mon[34762]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:56:44.815 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:44 smithi190 ceph-mon[34762]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:56:44.816 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:44 smithi190 ceph-mon[34762]: osd.4 [v2:172.21.15.137:6808/3758109169,v1:172.21.15.137:6809/3758109169] boot 2024-09-16T01:56:44.816 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:44 smithi190 ceph-mon[34762]: osdmap e30: 5 total, 5 up, 5 in 2024-09-16T01:56:44.816 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:44 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:56:44.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:44 smithi137 ceph-mon[34099]: purged_snaps scrub starts 2024-09-16T01:56:44.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:44 smithi137 ceph-mon[34099]: purged_snaps scrub ok 2024-09-16T01:56:44.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:44 smithi137 ceph-mon[34099]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:56:44.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:44 smithi137 ceph-mon[34099]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:56:44.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:44 smithi137 ceph-mon[34099]: osd.4 [v2:172.21.15.137:6808/3758109169,v1:172.21.15.137:6809/3758109169] boot 2024-09-16T01:56:44.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:44 smithi137 ceph-mon[34099]: osdmap e30: 5 total, 5 up, 5 in 2024-09-16T01:56:44.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:44 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:56:45.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:45 smithi105 ceph-mon[31197]: osdmap e31: 5 total, 5 up, 5 in 2024-09-16T01:56:45.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:45 smithi190 ceph-mon[34762]: osdmap e31: 5 total, 5 up, 5 in 2024-09-16T01:56:45.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:45 smithi137 ceph-mon[34099]: osdmap e31: 5 total, 5 up, 5 in 2024-09-16T01:56:46.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:46 smithi105 ceph-mon[31197]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:56:46.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:46 smithi105 ceph-mon[31197]: from='client.? 172.21.15.190:0/1070994836' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1e2a7d5-99f8-40fa-af6e-0305cc0af677"}]: dispatch 2024-09-16T01:56:46.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:46 smithi105 ceph-mon[31197]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1e2a7d5-99f8-40fa-af6e-0305cc0af677"}]: dispatch 2024-09-16T01:56:46.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:46 smithi105 ceph-mon[31197]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d1e2a7d5-99f8-40fa-af6e-0305cc0af677"}]': finished 2024-09-16T01:56:46.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:46 smithi105 ceph-mon[31197]: osdmap e32: 6 total, 5 up, 6 in 2024-09-16T01:56:46.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:46 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T01:56:46.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:46 smithi105 ceph-mon[31197]: from='client.? 172.21.15.190:0/3883368349' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:56:46.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:46 smithi190 ceph-mon[34762]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:56:46.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:46 smithi190 ceph-mon[34762]: from='client.? 172.21.15.190:0/1070994836' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1e2a7d5-99f8-40fa-af6e-0305cc0af677"}]: dispatch 2024-09-16T01:56:46.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:46 smithi190 ceph-mon[34762]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1e2a7d5-99f8-40fa-af6e-0305cc0af677"}]: dispatch 2024-09-16T01:56:46.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:46 smithi190 ceph-mon[34762]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d1e2a7d5-99f8-40fa-af6e-0305cc0af677"}]': finished 2024-09-16T01:56:46.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:46 smithi190 ceph-mon[34762]: osdmap e32: 6 total, 5 up, 6 in 2024-09-16T01:56:46.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:46 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T01:56:46.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:46 smithi190 ceph-mon[34762]: from='client.? 172.21.15.190:0/3883368349' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:56:46.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:46 smithi137 ceph-mon[34099]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:56:46.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:46 smithi137 ceph-mon[34099]: from='client.? 172.21.15.190:0/1070994836' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1e2a7d5-99f8-40fa-af6e-0305cc0af677"}]: dispatch 2024-09-16T01:56:46.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:46 smithi137 ceph-mon[34099]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1e2a7d5-99f8-40fa-af6e-0305cc0af677"}]: dispatch 2024-09-16T01:56:46.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:46 smithi137 ceph-mon[34099]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d1e2a7d5-99f8-40fa-af6e-0305cc0af677"}]': finished 2024-09-16T01:56:46.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:46 smithi137 ceph-mon[34099]: osdmap e32: 6 total, 5 up, 6 in 2024-09-16T01:56:46.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:46 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T01:56:46.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:46 smithi137 ceph-mon[34099]: from='client.? 172.21.15.190:0/3883368349' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:56:48.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:48 smithi105 ceph-mon[31197]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:56:48.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:48 smithi190 ceph-mon[34762]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:56:48.920 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:48 smithi137 ceph-mon[34099]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:56:50.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:50 smithi105 ceph-mon[31197]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:56:50.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:50 smithi190 ceph-mon[34762]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:56:50.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:50 smithi137 ceph-mon[34099]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:56:52.559 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:52 smithi190 ceph-mon[34762]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:56:52.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:52 smithi105 ceph-mon[31197]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:56:52.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:52 smithi137 ceph-mon[34099]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:56:54.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:54 smithi190 ceph-mon[34762]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:56:54.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:54 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T01:56:54.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:54 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:56:54.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:54 smithi105 ceph-mon[31197]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:56:54.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:54 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T01:56:54.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:54 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:56:54.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:54 smithi137 ceph-mon[34099]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:56:54.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:54 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T01:56:54.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:54 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:56:55.588 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:55 smithi190 ceph-mon[34762]: Deploying daemon osd.5 on smithi190 2024-09-16T01:56:55.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:55 smithi105 ceph-mon[31197]: Deploying daemon osd.5 on smithi190 2024-09-16T01:56:55.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:55 smithi137 ceph-mon[34099]: Deploying daemon osd.5 on smithi190 2024-09-16T01:56:56.725 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:56 smithi190 ceph-mon[34762]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:56:56.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:56 smithi105 ceph-mon[31197]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:56:56.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:56 smithi137 ceph-mon[34099]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:56:58.628 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:56:58 smithi190 ceph-mon[34762]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:56:58.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:56:58 smithi105 ceph-mon[31197]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:56:58.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:56:58 smithi137 ceph-mon[34099]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:57:00.707 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:00 smithi190 ceph-mon[34762]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:57:00.707 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:00 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:00.707 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:00 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:00.707 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:00 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:00.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:00 smithi105 ceph-mon[31197]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:57:00.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:00 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:00.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:00 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:00.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:00 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:00.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:00 smithi137 ceph-mon[34099]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:57:00.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:00 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:00.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:00 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:00.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:00 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:02.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:02 smithi105 ceph-mon[31197]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:57:02.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:02 smithi190 ceph-mon[34762]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:57:02.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:02 smithi137 ceph-mon[34099]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:57:04.013 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:03 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:04.013 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:03 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:04.013 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:03 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:04.013 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:03 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:04.013 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:03 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:04.013 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:03 smithi190 ceph-mon[34762]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:57:04.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:03 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:04.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:03 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:04.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:03 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:04.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:03 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:04.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:03 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:04.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:03 smithi105 ceph-mon[31197]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:57:04.053 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:04.050+0000 7f2fe8ff9640 1 -- 172.21.15.190:0/2293424581 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2fec0fd0c0 con 0x7f2fd40a32c0 2024-09-16T01:57:04.053 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 5 on host 'smithi190' 2024-09-16T01:57:04.055 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:04.052+0000 7f2ff1b6c640 1 -- 172.21.15.190:0/2293424581 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f2fd40a32c0 msgr2=0x7f2fd40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:04.055 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:04.052+0000 7f2ff1b6c640 1 --2- 172.21.15.190:0/2293424581 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f2fd40a32c0 0x7f2fd40a5780 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2fe002a930 tx=0x7f2fe0000bc0 comp rx=0 tx=0).stop 2024-09-16T01:57:04.055 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:04.052+0000 7f2ff1b6c640 1 -- 172.21.15.190:0/2293424581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fec101cd0 msgr2=0x7f2fec10d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:04.055 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:04.052+0000 7f2ff1b6c640 1 --2- 172.21.15.190:0/2293424581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fec101cd0 0x7f2fec10d1a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc00d950 tx=0x7f2fdc00de10 comp rx=0 tx=0).stop 2024-09-16T01:57:04.055 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:04.052+0000 7f2ff1b6c640 1 -- 172.21.15.190:0/2293424581 shutdown_connections 2024-09-16T01:57:04.055 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:04.052+0000 7f2ff1b6c640 1 --2- 172.21.15.190:0/2293424581 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f2fd40a32c0 0x7f2fd40a5780 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:04.055 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:04.052+0000 7f2ff1b6c640 1 --2- 172.21.15.190:0/2293424581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fec101cd0 0x7f2fec10d1a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:04.056 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:04.052+0000 7f2ff1b6c640 1 --2- 172.21.15.190:0/2293424581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fec102690 0x7f2fec10d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:04.056 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:04.052+0000 7f2ff1b6c640 1 --2- 172.21.15.190:0/2293424581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fec100ad0 0x7f2fec10eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:04.056 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:04.052+0000 7f2ff1b6c640 1 -- 172.21.15.190:0/2293424581 >> 172.21.15.190:0/2293424581 conn(0x7f2fec0fc280 msgr2=0x7f2fec0fc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:04.056 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:04.053+0000 7f2ff1b6c640 1 -- 172.21.15.190:0/2293424581 shutdown_connections 2024-09-16T01:57:04.056 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:04.053+0000 7f2ff1b6c640 1 -- 172.21.15.190:0/2293424581 wait complete. 2024-09-16T01:57:04.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:03 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:04.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:03 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:04.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:03 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:04.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:03 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:04.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:03 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:04.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:03 smithi137 ceph-mon[34099]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:57:04.765 DEBUG:teuthology.orchestra.run.smithi190:osd.5> sudo journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@osd.5.service 2024-09-16T01:57:04.808 INFO:tasks.cephadm:Deploying osd.6 on smithi105 with /dev/vg_nvme/lv_2... 2024-09-16T01:57:04.808 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-16T01:57:04.992 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:04 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:04.992 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:04 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:04.992 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:04 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:05.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:04 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:05.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:04 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:05.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:04 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:05.068 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:57:05.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:04 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:05.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:04 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:05.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:04 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:06.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:05 smithi105 ceph-mon[31197]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:57:06.054 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:05 smithi190 ceph-mon[34762]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:57:06.170 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:05 smithi137 ceph-mon[34099]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:57:06.343 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 01:57:06 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5[46111]: 2024-09-16T01:57:06.051+0000 7f07542de640 -1 osd.5 0 log_to_monitors true 2024-09-16T01:57:07.009 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:06 smithi105 ceph-mon[31197]: from='osd.5 [v2:172.21.15.190:6808/2086773754,v1:172.21.15.190:6809/2086773754]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T01:57:07.010 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:06 smithi105 ceph-mon[31197]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T01:57:07.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:06 smithi190 ceph-mon[34762]: from='osd.5 [v2:172.21.15.190:6808/2086773754,v1:172.21.15.190:6809/2086773754]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T01:57:07.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:06 smithi190 ceph-mon[34762]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T01:57:07.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:06 smithi137 ceph-mon[34099]: from='osd.5 [v2:172.21.15.190:6808/2086773754,v1:172.21.15.190:6809/2086773754]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T01:57:07.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:06 smithi137 ceph-mon[34099]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T01:57:07.713 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:57:07.737 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi105:vg_nvme/lv_2 2024-09-16T01:57:07.983 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:57:08.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:07 smithi105 ceph-mon[31197]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T01:57:08.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:07 smithi105 ceph-mon[31197]: osdmap e33: 6 total, 5 up, 6 in 2024-09-16T01:57:08.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:07 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T01:57:08.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:07 smithi105 ceph-mon[31197]: from='osd.5 [v2:172.21.15.190:6808/2086773754,v1:172.21.15.190:6809/2086773754]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T01:57:08.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:07 smithi105 ceph-mon[31197]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T01:57:08.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:07 smithi105 ceph-mon[31197]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:57:08.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:07 smithi105 ceph-mon[31197]: Detected new or changed devices on smithi190 2024-09-16T01:57:08.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:07 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:08.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:07 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:08.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:07 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:57:08.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:07 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:57:08.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:07 smithi105 ceph-mon[31197]: Adjusting osd_memory_target on smithi190 to 8544M 2024-09-16T01:57:08.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:07 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:08.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:07 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:08.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:07 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:08.046 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:07 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:08.093 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 01:57:07 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5[46111]: 2024-09-16T01:57:07.771+0000 7f0750261640 -1 osd.5 0 waiting for initial osdmap 2024-09-16T01:57:08.093 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 01:57:07 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5[46111]: 2024-09-16T01:57:07.778+0000 7f074c089640 -1 osd.5 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T01:57:08.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:07 smithi190 ceph-mon[34762]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T01:57:08.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:07 smithi190 ceph-mon[34762]: osdmap e33: 6 total, 5 up, 6 in 2024-09-16T01:57:08.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:07 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T01:57:08.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:07 smithi190 ceph-mon[34762]: from='osd.5 [v2:172.21.15.190:6808/2086773754,v1:172.21.15.190:6809/2086773754]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T01:57:08.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:07 smithi190 ceph-mon[34762]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T01:57:08.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:07 smithi190 ceph-mon[34762]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:57:08.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:07 smithi190 ceph-mon[34762]: Detected new or changed devices on smithi190 2024-09-16T01:57:08.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:07 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:08.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:07 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:08.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:07 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:57:08.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:07 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:57:08.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:07 smithi190 ceph-mon[34762]: Adjusting osd_memory_target on smithi190 to 8544M 2024-09-16T01:57:08.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:07 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:08.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:07 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:08.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:07 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:08.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:07 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:08.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:07 smithi137 ceph-mon[34099]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T01:57:08.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:07 smithi137 ceph-mon[34099]: osdmap e33: 6 total, 5 up, 6 in 2024-09-16T01:57:08.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:07 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T01:57:08.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:07 smithi137 ceph-mon[34099]: from='osd.5 [v2:172.21.15.190:6808/2086773754,v1:172.21.15.190:6809/2086773754]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T01:57:08.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:07 smithi137 ceph-mon[34099]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T01:57:08.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:07 smithi137 ceph-mon[34099]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:57:08.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:07 smithi137 ceph-mon[34099]: Detected new or changed devices on smithi190 2024-09-16T01:57:08.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:07 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:08.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:07 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:08.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:07 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:57:08.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:07 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:57:08.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:07 smithi137 ceph-mon[34099]: Adjusting osd_memory_target on smithi190 to 8544M 2024-09-16T01:57:08.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:07 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:08.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:07 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:08.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:07 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:08.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:07 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:08.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.470+0000 7f8ad8fa8640 1 -- 172.21.15.105:0/2136798736 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ad4103870 msgr2=0x7f8ad410a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:08.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.470+0000 7f8ad8fa8640 1 --2- 172.21.15.105:0/2136798736 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ad4103870 0x7f8ad410a100 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8acc009cd0 tx=0x7f8acc02a8e0 comp rx=0 tx=0).stop 2024-09-16T01:57:08.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.470+0000 7f8ad8fa8640 1 -- 172.21.15.105:0/2136798736 shutdown_connections 2024-09-16T01:57:08.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.470+0000 7f8ad8fa8640 1 --2- 172.21.15.105:0/2136798736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ad4102eb0 0x7f8ad4103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:08.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.470+0000 7f8ad8fa8640 1 --2- 172.21.15.105:0/2136798736 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ad4103870 0x7f8ad410a100 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:08.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.470+0000 7f8ad8fa8640 1 --2- 172.21.15.105:0/2136798736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ad40ffed0 0x7f8ad41002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:08.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.470+0000 7f8ad8fa8640 1 -- 172.21.15.105:0/2136798736 >> 172.21.15.105:0/2136798736 conn(0x7f8ad40fba40 msgr2=0x7f8ad40fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:08.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.470+0000 7f8ad8fa8640 1 -- 172.21.15.105:0/2136798736 shutdown_connections 2024-09-16T01:57:08.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.470+0000 7f8ad8fa8640 1 -- 172.21.15.105:0/2136798736 wait complete. 2024-09-16T01:57:08.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.471+0000 7f8ad8fa8640 1 Processor -- start 2024-09-16T01:57:08.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.471+0000 7f8ad8fa8640 1 -- start start 2024-09-16T01:57:08.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.471+0000 7f8ad8fa8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ad40ffed0 0x7f8ad4102810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:08.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.471+0000 7f8ad8fa8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ad4102eb0 0x7f8ad4100e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:08.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.471+0000 7f8ad8fa8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ad4103870 0x7f8ad41013a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:08.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.471+0000 7f8ad8fa8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad410d7c0 con 0x7f8ad4103870 2024-09-16T01:57:08.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.471+0000 7f8ad8fa8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad410d640 con 0x7f8ad40ffed0 2024-09-16T01:57:08.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.471+0000 7f8ad8fa8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad410d940 con 0x7f8ad4102eb0 2024-09-16T01:57:08.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.471+0000 7f8ad2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ad4103870 0x7f8ad41013a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:08.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.471+0000 7f8ad2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ad4103870 0x7f8ad41013a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33976/0 (socket says 172.21.15.105:33976) 2024-09-16T01:57:08.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.471+0000 7f8ad2d9d640 1 -- 172.21.15.105:0/3967664620 learned_addr learned my addr 172.21.15.105:0/3967664620 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:57:08.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.471+0000 7f8ad259c640 1 --2- 172.21.15.105:0/3967664620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ad40ffed0 0x7f8ad4102810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:08.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.471+0000 7f8ad1d9b640 1 --2- 172.21.15.105:0/3967664620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ad4102eb0 0x7f8ad4100e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:08.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.471+0000 7f8ad2d9d640 1 -- 172.21.15.105:0/3967664620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ad4102eb0 msgr2=0x7f8ad4100e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:08.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.471+0000 7f8ad2d9d640 1 --2- 172.21.15.105:0/3967664620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ad4102eb0 0x7f8ad4100e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:08.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.471+0000 7f8ad2d9d640 1 -- 172.21.15.105:0/3967664620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ad40ffed0 msgr2=0x7f8ad4102810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:08.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.471+0000 7f8ad2d9d640 1 --2- 172.21.15.105:0/3967664620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ad40ffed0 0x7f8ad4102810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:08.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.471+0000 7f8ad2d9d640 1 -- 172.21.15.105:0/3967664620 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad4101c60 con 0x7f8ad4103870 2024-09-16T01:57:08.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.472+0000 7f8ad2d9d640 1 --2- 172.21.15.105:0/3967664620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ad4103870 0x7f8ad41013a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8acc009df0 tx=0x7f8acc003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:08.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.472+0000 7f8ac37fe640 1 -- 172.21.15.105:0/3967664620 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8acc004450 con 0x7f8ad4103870 2024-09-16T01:57:08.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.472+0000 7f8ac37fe640 1 -- 172.21.15.105:0/3967664620 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8acc0045f0 con 0x7f8ad4103870 2024-09-16T01:57:08.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.472+0000 7f8ad8fa8640 1 -- 172.21.15.105:0/3967664620 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ad41a1f50 con 0x7f8ad4103870 2024-09-16T01:57:08.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.472+0000 7f8ac37fe640 1 -- 172.21.15.105:0/3967664620 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8acc0466e0 con 0x7f8ad4103870 2024-09-16T01:57:08.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.472+0000 7f8ad8fa8640 1 -- 172.21.15.105:0/3967664620 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ad41a2480 con 0x7f8ad4103870 2024-09-16T01:57:08.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.473+0000 7f8ad8fa8640 1 -- 172.21.15.105:0/3967664620 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ad4109100 con 0x7f8ad4103870 2024-09-16T01:57:08.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.474+0000 7f8ac37fe640 1 -- 172.21.15.105:0/3967664620 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f8acc0305a0 con 0x7f8ad4103870 2024-09-16T01:57:08.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.475+0000 7f8ac37fe640 1 --2- 172.21.15.105:0/3967664620 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f8abc0a3240 0x7f8abc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:08.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.475+0000 7f8ac37fe640 1 -- 172.21.15.105:0/3967664620 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4253+0+0 (secure 0 0 0) 0x7f8acc02e030 con 0x7f8ad4103870 2024-09-16T01:57:08.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.475+0000 7f8ad259c640 1 --2- 172.21.15.105:0/3967664620 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f8abc0a3240 0x7f8abc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:08.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.475+0000 7f8ad259c640 1 --2- 172.21.15.105:0/3967664620 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f8abc0a3240 0x7f8abc0a5700 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8ad406a590 tx=0x7f8ac4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:08.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.478+0000 7f8ac37fe640 1 -- 172.21.15.105:0/3967664620 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8acc04b060 con 0x7f8ad4103870 2024-09-16T01:57:08.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:08.652+0000 7f8ad8fa8640 1 -- 172.21.15.105:0/3967664620 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8ad4062630 con 0x7f8abc0a3240 2024-09-16T01:57:09.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:08 smithi105 ceph-mon[31197]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-09-16T01:57:09.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:08 smithi105 ceph-mon[31197]: osdmap e34: 6 total, 5 up, 6 in 2024-09-16T01:57:09.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:08 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T01:57:09.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:08 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T01:57:09.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:08 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:57:09.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:08 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:57:09.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:08 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:09.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:08 smithi105 ceph-mon[31197]: osd.5 [v2:172.21.15.190:6808/2086773754,v1:172.21.15.190:6809/2086773754] boot 2024-09-16T01:57:09.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:08 smithi105 ceph-mon[31197]: osdmap e35: 6 total, 6 up, 6 in 2024-09-16T01:57:09.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:08 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T01:57:09.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:08 smithi190 ceph-mon[34762]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-09-16T01:57:09.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:08 smithi190 ceph-mon[34762]: osdmap e34: 6 total, 5 up, 6 in 2024-09-16T01:57:09.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:08 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T01:57:09.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:08 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T01:57:09.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:08 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:57:09.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:08 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:57:09.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:08 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:09.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:08 smithi190 ceph-mon[34762]: osd.5 [v2:172.21.15.190:6808/2086773754,v1:172.21.15.190:6809/2086773754] boot 2024-09-16T01:57:09.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:08 smithi190 ceph-mon[34762]: osdmap e35: 6 total, 6 up, 6 in 2024-09-16T01:57:09.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:08 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T01:57:09.173 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:08 smithi137 ceph-mon[34099]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-09-16T01:57:09.173 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:08 smithi137 ceph-mon[34099]: osdmap e34: 6 total, 5 up, 6 in 2024-09-16T01:57:09.173 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:08 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T01:57:09.173 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:08 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T01:57:09.173 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:08 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:57:09.173 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:08 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:57:09.173 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:08 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:09.173 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:08 smithi137 ceph-mon[34099]: osd.5 [v2:172.21.15.190:6808/2086773754,v1:172.21.15.190:6809/2086773754] boot 2024-09-16T01:57:09.174 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:08 smithi137 ceph-mon[34099]: osdmap e35: 6 total, 6 up, 6 in 2024-09-16T01:57:09.174 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:08 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T01:57:10.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:09 smithi105 ceph-mon[31197]: purged_snaps scrub starts 2024-09-16T01:57:10.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:09 smithi105 ceph-mon[31197]: purged_snaps scrub ok 2024-09-16T01:57:10.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:09 smithi105 ceph-mon[31197]: from='client.14376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:57:10.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:09 smithi105 ceph-mon[31197]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:57:10.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:09 smithi105 ceph-mon[31197]: osdmap e36: 6 total, 6 up, 6 in 2024-09-16T01:57:10.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:09 smithi190 ceph-mon[34762]: purged_snaps scrub starts 2024-09-16T01:57:10.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:09 smithi190 ceph-mon[34762]: purged_snaps scrub ok 2024-09-16T01:57:10.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:09 smithi190 ceph-mon[34762]: from='client.14376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:57:10.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:09 smithi190 ceph-mon[34762]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:57:10.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:09 smithi190 ceph-mon[34762]: osdmap e36: 6 total, 6 up, 6 in 2024-09-16T01:57:10.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:09 smithi137 ceph-mon[34099]: purged_snaps scrub starts 2024-09-16T01:57:10.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:09 smithi137 ceph-mon[34099]: purged_snaps scrub ok 2024-09-16T01:57:10.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:09 smithi137 ceph-mon[34099]: from='client.14376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:57:10.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:09 smithi137 ceph-mon[34099]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:57:10.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:09 smithi137 ceph-mon[34099]: osdmap e36: 6 total, 6 up, 6 in 2024-09-16T01:57:11.719 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:11 smithi105 ceph-mon[31197]: osdmap e37: 6 total, 6 up, 6 in 2024-09-16T01:57:11.719 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:11 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/1358509063' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd418159-a0f6-44c7-9213-c010b132fb56"}]: dispatch 2024-09-16T01:57:11.719 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:11 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/1358509063' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cd418159-a0f6-44c7-9213-c010b132fb56"}]': finished 2024-09-16T01:57:11.719 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:11 smithi105 ceph-mon[31197]: osdmap e38: 7 total, 6 up, 7 in 2024-09-16T01:57:11.720 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:11 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:57:11.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:11 smithi190 ceph-mon[34762]: osdmap e37: 6 total, 6 up, 6 in 2024-09-16T01:57:11.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:11 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/1358509063' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd418159-a0f6-44c7-9213-c010b132fb56"}]: dispatch 2024-09-16T01:57:11.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:11 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/1358509063' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cd418159-a0f6-44c7-9213-c010b132fb56"}]': finished 2024-09-16T01:57:11.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:11 smithi190 ceph-mon[34762]: osdmap e38: 7 total, 6 up, 7 in 2024-09-16T01:57:11.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:11 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:57:11.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:11 smithi137 ceph-mon[34099]: osdmap e37: 6 total, 6 up, 6 in 2024-09-16T01:57:11.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:11 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/1358509063' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd418159-a0f6-44c7-9213-c010b132fb56"}]: dispatch 2024-09-16T01:57:11.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:11 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/1358509063' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cd418159-a0f6-44c7-9213-c010b132fb56"}]': finished 2024-09-16T01:57:11.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:11 smithi137 ceph-mon[34099]: osdmap e38: 7 total, 6 up, 7 in 2024-09-16T01:57:11.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:11 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:57:12.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:12 smithi105 ceph-mon[31197]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:57:12.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:12 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/988698467' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:57:12.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:12 smithi105 ceph-mon[31197]: osdmap e39: 7 total, 6 up, 7 in 2024-09-16T01:57:12.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:12 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:57:12.845 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:12 smithi190 ceph-mon[34762]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:57:12.846 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:12 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/988698467' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:57:12.846 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:12 smithi190 ceph-mon[34762]: osdmap e39: 7 total, 6 up, 7 in 2024-09-16T01:57:12.846 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:12 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:57:12.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:12 smithi137 ceph-mon[34099]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:57:12.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:12 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/988698467' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:57:12.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:12 smithi137 ceph-mon[34099]: osdmap e39: 7 total, 6 up, 7 in 2024-09-16T01:57:12.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:12 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:57:14.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:14 smithi105 ceph-mon[31197]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:57:14.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:14 smithi190 ceph-mon[34762]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:57:14.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:14 smithi137 ceph-mon[34099]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:57:16.696 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:16 smithi105 ceph-mon[31197]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-16T01:57:16.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:16 smithi190 ceph-mon[34762]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-16T01:57:16.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:16 smithi137 ceph-mon[34099]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-16T01:57:18.522 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:18 smithi105 ceph-mon[31197]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-16T01:57:18.723 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:18 smithi137 ceph-mon[34099]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-16T01:57:18.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:18 smithi190 ceph-mon[34762]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-16T01:57:20.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:20 smithi105 ceph-mon[31197]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-16T01:57:20.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:20 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-16T01:57:20.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:20 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:20.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:20 smithi190 ceph-mon[34762]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-16T01:57:20.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:20 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-16T01:57:20.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:20 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:20.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:20 smithi137 ceph-mon[34099]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-16T01:57:20.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:20 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-16T01:57:20.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:20 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:21.749 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:21 smithi105 ceph-mon[31197]: Deploying daemon osd.6 on smithi105 2024-09-16T01:57:21.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:21 smithi190 ceph-mon[34762]: Deploying daemon osd.6 on smithi105 2024-09-16T01:57:21.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:21 smithi137 ceph-mon[34099]: Deploying daemon osd.6 on smithi105 2024-09-16T01:57:22.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:22 smithi105 ceph-mon[31197]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T01:57:22.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:22 smithi190 ceph-mon[34762]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T01:57:22.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:22 smithi137 ceph-mon[34099]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T01:57:24.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:24 smithi105 ceph-mon[31197]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-16T01:57:24.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:24 smithi190 ceph-mon[34762]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-16T01:57:24.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:24 smithi137 ceph-mon[34099]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-16T01:57:26.623 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:26 smithi105 ceph-mon[31197]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T01:57:26.623 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:26 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:26.623 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:26 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:26.623 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:26 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:26.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:26 smithi190 ceph-mon[34762]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T01:57:26.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:26 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:26.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:26 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:26.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:26 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:26.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:26 smithi137 ceph-mon[34099]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T01:57:26.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:26 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:26.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:26 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:26.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:26 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:28.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:28 smithi105 ceph-mon[31197]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:57:28.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:28 smithi190 ceph-mon[34762]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:57:28.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:28 smithi137 ceph-mon[34099]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:57:29.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:29 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:29.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:29 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:29.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:29 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:29.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:29 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:29.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:29 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:29.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:29 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:29.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:29 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:29.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:29 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:29.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:29 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:29.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:29 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:29.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:29.877+0000 7f8ac37fe640 1 -- 172.21.15.105:0/3967664620 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8ad4062630 con 0x7f8abc0a3240 2024-09-16T01:57:29.880 INFO:teuthology.orchestra.run.smithi105.stdout:Created osd(s) 6 on host 'smithi105' 2024-09-16T01:57:29.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:29.879+0000 7f8ad8fa8640 1 -- 172.21.15.105:0/3967664620 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f8abc0a3240 msgr2=0x7f8abc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:29.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:29.879+0000 7f8ad8fa8640 1 --2- 172.21.15.105:0/3967664620 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f8abc0a3240 0x7f8abc0a5700 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8ad406a590 tx=0x7f8ac4005f90 comp rx=0 tx=0).stop 2024-09-16T01:57:29.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:29.879+0000 7f8ad8fa8640 1 -- 172.21.15.105:0/3967664620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ad4103870 msgr2=0x7f8ad41013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:29.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:29.879+0000 7f8ad8fa8640 1 --2- 172.21.15.105:0/3967664620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ad4103870 0x7f8ad41013a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8acc009df0 tx=0x7f8acc003d60 comp rx=0 tx=0).stop 2024-09-16T01:57:29.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:29.879+0000 7f8ad8fa8640 1 -- 172.21.15.105:0/3967664620 shutdown_connections 2024-09-16T01:57:29.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:29.879+0000 7f8ad8fa8640 1 --2- 172.21.15.105:0/3967664620 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f8abc0a3240 0x7f8abc0a5700 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:29.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:29.879+0000 7f8ad8fa8640 1 --2- 172.21.15.105:0/3967664620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ad4102eb0 0x7f8ad4100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:29.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:29.879+0000 7f8ad8fa8640 1 --2- 172.21.15.105:0/3967664620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ad4103870 0x7f8ad41013a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:29.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:29.879+0000 7f8ad8fa8640 1 --2- 172.21.15.105:0/3967664620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ad40ffed0 0x7f8ad4102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:29.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:29.879+0000 7f8ad8fa8640 1 -- 172.21.15.105:0/3967664620 >> 172.21.15.105:0/3967664620 conn(0x7f8ad40fba40 msgr2=0x7f8ad41062d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:29.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:29.880+0000 7f8ad8fa8640 1 -- 172.21.15.105:0/3967664620 shutdown_connections 2024-09-16T01:57:29.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:57:29.880+0000 7f8ad8fa8640 1 -- 172.21.15.105:0/3967664620 wait complete. 2024-09-16T01:57:29.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:29 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:29.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:29 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:29.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:29 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:29.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:29 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:29.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:29 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:30.601 DEBUG:teuthology.orchestra.run.smithi105:osd.6> sudo journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@osd.6.service 2024-09-16T01:57:30.604 INFO:tasks.cephadm:Deploying osd.7 on smithi137 with /dev/vg_nvme/lv_2... 2024-09-16T01:57:30.604 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-16T01:57:30.787 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:30 smithi105 ceph-mon[31197]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:57:30.788 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:30 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:30.788 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:30 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:30.788 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:30 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:30.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:30 smithi190 ceph-mon[34762]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:57:30.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:30 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:30.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:30 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:30.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:30 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:30.855 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T01:57:30.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:30 smithi137 ceph-mon[34099]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:57:30.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:30 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:30.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:30 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:30.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:30 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:31.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:31 smithi137 ceph-mon[34099]: from='osd.6 [v2:172.21.15.105:6818/4012018725,v1:172.21.15.105:6819/4012018725]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T01:57:31.790 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:31 smithi105 ceph-mon[31197]: from='osd.6 [v2:172.21.15.105:6818/4012018725,v1:172.21.15.105:6819/4012018725]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T01:57:31.791 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 01:57:31 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T01:57:31.474+0000 7f249256b640 -1 osd.6 0 log_to_monitors true 2024-09-16T01:57:31.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:31 smithi190 ceph-mon[34762]: from='osd.6 [v2:172.21.15.105:6818/4012018725,v1:172.21.15.105:6819/4012018725]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T01:57:32.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:32 smithi137 ceph-mon[34099]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:57:32.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:32 smithi137 ceph-mon[34099]: from='osd.6 [v2:172.21.15.105:6818/4012018725,v1:172.21.15.105:6819/4012018725]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-16T01:57:32.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:32 smithi137 ceph-mon[34099]: osdmap e40: 7 total, 6 up, 7 in 2024-09-16T01:57:32.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:32 smithi137 ceph-mon[34099]: from='osd.6 [v2:172.21.15.105:6818/4012018725,v1:172.21.15.105:6819/4012018725]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-09-16T01:57:32.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:32 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:57:32.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:32 smithi105 ceph-mon[31197]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:57:32.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:32 smithi105 ceph-mon[31197]: from='osd.6 [v2:172.21.15.105:6818/4012018725,v1:172.21.15.105:6819/4012018725]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-16T01:57:32.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:32 smithi105 ceph-mon[31197]: osdmap e40: 7 total, 6 up, 7 in 2024-09-16T01:57:32.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:32 smithi105 ceph-mon[31197]: from='osd.6 [v2:172.21.15.105:6818/4012018725,v1:172.21.15.105:6819/4012018725]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-09-16T01:57:32.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:32 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:57:32.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:32 smithi190 ceph-mon[34762]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:57:32.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:32 smithi190 ceph-mon[34762]: from='osd.6 [v2:172.21.15.105:6818/4012018725,v1:172.21.15.105:6819/4012018725]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-16T01:57:32.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:32 smithi190 ceph-mon[34762]: osdmap e40: 7 total, 6 up, 7 in 2024-09-16T01:57:32.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:32 smithi190 ceph-mon[34762]: from='osd.6 [v2:172.21.15.105:6818/4012018725,v1:172.21.15.105:6819/4012018725]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-09-16T01:57:32.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:32 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:57:33.294 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 01:57:32 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T01:57:32.940+0000 7f248e4ee640 -1 osd.6 0 waiting for initial osdmap 2024-09-16T01:57:33.295 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 01:57:32 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T01:57:32.951+0000 7f248630e640 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T01:57:33.473 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-16T01:57:33.500 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi137:vg_nvme/lv_2 2024-09-16T01:57:33.737 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:33 smithi105 ceph-mon[31197]: from='osd.6 [v2:172.21.15.105:6818/4012018725,v1:172.21.15.105:6819/4012018725]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-09-16T01:57:33.737 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:33 smithi105 ceph-mon[31197]: osdmap e41: 7 total, 6 up, 7 in 2024-09-16T01:57:33.737 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:33 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:57:33.737 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:33 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:57:33.737 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:33 smithi105 ceph-mon[31197]: from='osd.6 [v2:172.21.15.105:6818/4012018725,v1:172.21.15.105:6819/4012018725]' entity='osd.6' 2024-09-16T01:57:33.744 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T01:57:33.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:33 smithi190 ceph-mon[34762]: from='osd.6 [v2:172.21.15.105:6818/4012018725,v1:172.21.15.105:6819/4012018725]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-09-16T01:57:33.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:33 smithi190 ceph-mon[34762]: osdmap e41: 7 total, 6 up, 7 in 2024-09-16T01:57:33.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:33 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:57:33.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:33 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:57:33.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:33 smithi190 ceph-mon[34762]: from='osd.6 [v2:172.21.15.105:6818/4012018725,v1:172.21.15.105:6819/4012018725]' entity='osd.6' 2024-09-16T01:57:33.865 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:33 smithi137 ceph-mon[34099]: from='osd.6 [v2:172.21.15.105:6818/4012018725,v1:172.21.15.105:6819/4012018725]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-09-16T01:57:33.865 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:33 smithi137 ceph-mon[34099]: osdmap e41: 7 total, 6 up, 7 in 2024-09-16T01:57:33.865 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:33 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:57:33.865 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:33 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:57:33.866 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:33 smithi137 ceph-mon[34099]: from='osd.6 [v2:172.21.15.105:6818/4012018725,v1:172.21.15.105:6819/4012018725]' entity='osd.6' 2024-09-16T01:57:34.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.213+0000 7f786e6e0640 1 -- 172.21.15.137:0/4099384766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7868107900 msgr2=0x7f7868109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:34.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.213+0000 7f786e6e0640 1 --2- 172.21.15.137:0/4099384766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7868107900 0x7f7868109d50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7860009ca0 tx=0x7f786002a8c0 comp rx=0 tx=0).stop 2024-09-16T01:57:34.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.213+0000 7f786e6e0640 1 -- 172.21.15.137:0/4099384766 shutdown_connections 2024-09-16T01:57:34.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.213+0000 7f786e6e0640 1 --2- 172.21.15.137:0/4099384766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7868104fd0 0x7f78681073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:34.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.213+0000 7f786e6e0640 1 --2- 172.21.15.137:0/4099384766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7868107900 0x7f7868109d50 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:34.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.213+0000 7f786e6e0640 1 --2- 172.21.15.137:0/4099384766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78681026a0 0x7f7868104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:34.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.213+0000 7f786e6e0640 1 -- 172.21.15.137:0/4099384766 >> 172.21.15.137:0/4099384766 conn(0x7f78680fc2a0 msgr2=0x7f78680fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:34.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.213+0000 7f786e6e0640 1 -- 172.21.15.137:0/4099384766 shutdown_connections 2024-09-16T01:57:34.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.213+0000 7f786e6e0640 1 -- 172.21.15.137:0/4099384766 wait complete. 2024-09-16T01:57:34.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.213+0000 7f786e6e0640 1 Processor -- start 2024-09-16T01:57:34.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.213+0000 7f786e6e0640 1 -- start start 2024-09-16T01:57:34.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.214+0000 7f786e6e0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78681026a0 0x7f7868102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:34.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.214+0000 7f786e6e0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7868104fd0 0x7f7868100650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:34.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.214+0000 7f786e6e0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7868107900 0x7f7868100b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:34.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.214+0000 7f786e6e0640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f786810d7f0 con 0x7f78681026a0 2024-09-16T01:57:34.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.214+0000 7f786e6e0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f786810d670 con 0x7f7868104fd0 2024-09-16T01:57:34.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.214+0000 7f786e6e0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f786810d970 con 0x7f7868107900 2024-09-16T01:57:34.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.214+0000 7f78677fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7868104fd0 0x7f7868100650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:34.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.214+0000 7f78677fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7868104fd0 0x7f7868100650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51470/0 (socket says 172.21.15.137:51470) 2024-09-16T01:57:34.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.214+0000 7f78677fe640 1 -- 172.21.15.137:0/1391622937 learned_addr learned my addr 172.21.15.137:0/1391622937 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-16T01:57:34.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.214+0000 7f7867fff640 1 --2- 172.21.15.137:0/1391622937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78681026a0 0x7f7868102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:34.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.214+0000 7f786cc7d640 1 --2- 172.21.15.137:0/1391622937 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7868107900 0x7f7868100b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:34.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.214+0000 7f78677fe640 1 -- 172.21.15.137:0/1391622937 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7868107900 msgr2=0x7f7868100b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:34.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.214+0000 7f78677fe640 1 --2- 172.21.15.137:0/1391622937 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7868107900 0x7f7868100b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:34.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.214+0000 7f78677fe640 1 -- 172.21.15.137:0/1391622937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78681026a0 msgr2=0x7f7868102000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:34.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.214+0000 7f78677fe640 1 --2- 172.21.15.137:0/1391622937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78681026a0 0x7f7868102000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:34.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.214+0000 7f78677fe640 1 -- 172.21.15.137:0/1391622937 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7868101450 con 0x7f7868104fd0 2024-09-16T01:57:34.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.214+0000 7f7867fff640 1 --2- 172.21.15.137:0/1391622937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78681026a0 0x7f7868102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:57:34.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.214+0000 7f78677fe640 1 --2- 172.21.15.137:0/1391622937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7868104fd0 0x7f7868100650 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f785c00e9f0 tx=0x7f785c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:34.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.214+0000 7f78657fa640 1 -- 172.21.15.137:0/1391622937 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785c00ccc0 con 0x7f7868104fd0 2024-09-16T01:57:34.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.214+0000 7f786e6e0640 1 -- 172.21.15.137:0/1391622937 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78681a1fe0 con 0x7f7868104fd0 2024-09-16T01:57:34.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.214+0000 7f78657fa640 1 -- 172.21.15.137:0/1391622937 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f785c0054d0 con 0x7f7868104fd0 2024-09-16T01:57:34.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.214+0000 7f78657fa640 1 -- 172.21.15.137:0/1391622937 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785c010430 con 0x7f7868104fd0 2024-09-16T01:57:34.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.215+0000 7f786e6e0640 1 -- 172.21.15.137:0/1391622937 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78681a2440 con 0x7f7868104fd0 2024-09-16T01:57:34.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.215+0000 7f786e6e0640 1 -- 172.21.15.137:0/1391622937 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78681053b0 con 0x7f7868104fd0 2024-09-16T01:57:34.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.217+0000 7f78657fa640 1 -- 172.21.15.137:0/1391622937 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f785c005000 con 0x7f7868104fd0 2024-09-16T01:57:34.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.217+0000 7f78657fa640 1 --2- 172.21.15.137:0/1391622937 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f78500a3240 0x7f78500a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:34.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.217+0000 7f78657fa640 1 -- 172.21.15.137:0/1391622937 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4965+0+0 (secure 0 0 0) 0x7f785c0cf220 con 0x7f7868104fd0 2024-09-16T01:57:34.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.218+0000 7f7867fff640 1 --2- 172.21.15.137:0/1391622937 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f78500a3240 0x7f78500a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:34.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.218+0000 7f7867fff640 1 --2- 172.21.15.137:0/1391622937 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f78500a3240 0x7f78500a5700 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7858009d60 tx=0x7f7858009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:34.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.220+0000 7f78657fa640 1 -- 172.21.15.137:0/1391622937 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f785c0541b0 con 0x7f7868104fd0 2024-09-16T01:57:34.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:34.394+0000 7f786e6e0640 1 -- 172.21.15.137:0/1391622937 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f7868062630 con 0x7f78500a3240 2024-09-16T01:57:34.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:34 smithi137 ceph-mon[34099]: purged_snaps scrub starts 2024-09-16T01:57:34.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:34 smithi137 ceph-mon[34099]: purged_snaps scrub ok 2024-09-16T01:57:34.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:34 smithi137 ceph-mon[34099]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:57:34.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:34 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:57:34.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:34 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:34.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:34 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:34.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:34 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:57:34.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:34 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:57:34.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:34 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:57:34.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:34 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:34.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:34 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:34.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:34 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:34.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:34 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:34.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:34 smithi137 ceph-mon[34099]: osd.6 [v2:172.21.15.105:6818/4012018725,v1:172.21.15.105:6819/4012018725] boot 2024-09-16T01:57:34.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:34 smithi137 ceph-mon[34099]: osdmap e42: 7 total, 7 up, 7 in 2024-09-16T01:57:34.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:34 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:57:34.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:34 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:57:34.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:34 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:57:34.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:34 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:34.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:34 smithi105 ceph-mon[31197]: purged_snaps scrub starts 2024-09-16T01:57:34.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:34 smithi105 ceph-mon[31197]: purged_snaps scrub ok 2024-09-16T01:57:34.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:34 smithi105 ceph-mon[31197]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:57:34.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:34 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:57:34.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:34 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:34.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:34 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:34.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:34 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:57:34.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:34 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:57:34.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:34 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:57:34.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:34 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:34.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:34 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:34.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:34 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:34.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:34 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:34.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:34 smithi105 ceph-mon[31197]: osd.6 [v2:172.21.15.105:6818/4012018725,v1:172.21.15.105:6819/4012018725] boot 2024-09-16T01:57:34.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:34 smithi105 ceph-mon[31197]: osdmap e42: 7 total, 7 up, 7 in 2024-09-16T01:57:34.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:34 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:57:34.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:34 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:57:34.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:34 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:57:34.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:34 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:34.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:34 smithi190 ceph-mon[34762]: purged_snaps scrub starts 2024-09-16T01:57:34.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:34 smithi190 ceph-mon[34762]: purged_snaps scrub ok 2024-09-16T01:57:34.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:34 smithi190 ceph-mon[34762]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:57:34.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:34 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:57:34.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:34 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:34.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:34 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:34.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:34 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:57:34.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:34 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:57:34.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:34 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:57:34.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:34 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:34.844 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:34 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:34.844 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:34 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:34.844 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:34 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:34.844 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:34 smithi190 ceph-mon[34762]: osd.6 [v2:172.21.15.105:6818/4012018725,v1:172.21.15.105:6819/4012018725] boot 2024-09-16T01:57:34.844 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:34 smithi190 ceph-mon[34762]: osdmap e42: 7 total, 7 up, 7 in 2024-09-16T01:57:34.844 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:34 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:57:34.844 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:34 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:57:34.844 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:34 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:57:34.844 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:34 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:35.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:35 smithi105 ceph-mon[31197]: Detected new or changed devices on smithi105 2024-09-16T01:57:35.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:35 smithi105 ceph-mon[31197]: Adjusting osd_memory_target on smithi105 to 5696M 2024-09-16T01:57:35.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:35 smithi105 ceph-mon[31197]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:57:35.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:35 smithi105 ceph-mon[31197]: osdmap e43: 7 total, 7 up, 7 in 2024-09-16T01:57:35.812 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:35 smithi137 ceph-mon[34099]: Detected new or changed devices on smithi105 2024-09-16T01:57:35.813 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:35 smithi137 ceph-mon[34099]: Adjusting osd_memory_target on smithi105 to 5696M 2024-09-16T01:57:35.813 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:35 smithi137 ceph-mon[34099]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:57:35.813 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:35 smithi137 ceph-mon[34099]: osdmap e43: 7 total, 7 up, 7 in 2024-09-16T01:57:35.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:35 smithi190 ceph-mon[34762]: Detected new or changed devices on smithi105 2024-09-16T01:57:35.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:35 smithi190 ceph-mon[34762]: Adjusting osd_memory_target on smithi105 to 5696M 2024-09-16T01:57:35.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:35 smithi190 ceph-mon[34762]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:57:35.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:35 smithi190 ceph-mon[34762]: osdmap e43: 7 total, 7 up, 7 in 2024-09-16T01:57:36.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:36 smithi105 ceph-mon[31197]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:57:36.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:36 smithi190 ceph-mon[34762]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:57:36.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:36 smithi137 ceph-mon[34099]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:57:37.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:37 smithi137 ceph-mon[34099]: from='client.? 172.21.15.137:0/2390829156' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76c15863-801e-4a72-be73-ba9df46c15b3"}]: dispatch 2024-09-16T01:57:37.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:37 smithi137 ceph-mon[34099]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76c15863-801e-4a72-be73-ba9df46c15b3"}]: dispatch 2024-09-16T01:57:37.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:37 smithi137 ceph-mon[34099]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "76c15863-801e-4a72-be73-ba9df46c15b3"}]': finished 2024-09-16T01:57:37.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:37 smithi137 ceph-mon[34099]: osdmap e44: 8 total, 7 up, 8 in 2024-09-16T01:57:37.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:37 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T01:57:37.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:37 smithi137 ceph-mon[34099]: from='client.? 172.21.15.137:0/2814450041' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:57:37.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:37 smithi105 ceph-mon[31197]: from='client.? 172.21.15.137:0/2390829156' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76c15863-801e-4a72-be73-ba9df46c15b3"}]: dispatch 2024-09-16T01:57:37.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:37 smithi105 ceph-mon[31197]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76c15863-801e-4a72-be73-ba9df46c15b3"}]: dispatch 2024-09-16T01:57:37.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:37 smithi105 ceph-mon[31197]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "76c15863-801e-4a72-be73-ba9df46c15b3"}]': finished 2024-09-16T01:57:37.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:37 smithi105 ceph-mon[31197]: osdmap e44: 8 total, 7 up, 8 in 2024-09-16T01:57:37.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:37 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T01:57:37.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:37 smithi105 ceph-mon[31197]: from='client.? 172.21.15.137:0/2814450041' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:57:37.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:37 smithi190 ceph-mon[34762]: from='client.? 172.21.15.137:0/2390829156' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76c15863-801e-4a72-be73-ba9df46c15b3"}]: dispatch 2024-09-16T01:57:37.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:37 smithi190 ceph-mon[34762]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76c15863-801e-4a72-be73-ba9df46c15b3"}]: dispatch 2024-09-16T01:57:37.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:37 smithi190 ceph-mon[34762]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "76c15863-801e-4a72-be73-ba9df46c15b3"}]': finished 2024-09-16T01:57:37.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:37 smithi190 ceph-mon[34762]: osdmap e44: 8 total, 7 up, 8 in 2024-09-16T01:57:37.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:37 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T01:57:37.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:37 smithi190 ceph-mon[34762]: from='client.? 172.21.15.137:0/2814450041' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:57:38.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:38 smithi105 ceph-mon[31197]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:57:38.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:38 smithi190 ceph-mon[34762]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:57:38.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:38 smithi137 ceph-mon[34099]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:57:40.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:40 smithi105 ceph-mon[31197]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:57:40.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:40 smithi190 ceph-mon[34762]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:57:40.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:40 smithi137 ceph-mon[34099]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:57:42.613 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:42 smithi137 ceph-mon[34099]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:57:42.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:42 smithi190 ceph-mon[34762]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:57:43.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:42 smithi105 ceph-mon[31197]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:57:44.783 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:44 smithi137 ceph-mon[34099]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:57:44.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:44 smithi190 ceph-mon[34762]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:57:45.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:44 smithi105 ceph-mon[31197]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:57:45.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:45 smithi105 ceph-mon[31197]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:57:45.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:45 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-16T01:57:45.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:45 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:45.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:45 smithi105 ceph-mon[31197]: Deploying daemon osd.7 on smithi137 2024-09-16T01:57:45.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:45 smithi190 ceph-mon[34762]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:57:45.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:45 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-16T01:57:45.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:45 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:45.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:45 smithi190 ceph-mon[34762]: Deploying daemon osd.7 on smithi137 2024-09-16T01:57:45.883 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:45 smithi137 ceph-mon[34099]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:57:45.883 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:45 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-16T01:57:45.883 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:45 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:45.883 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:45 smithi137 ceph-mon[34099]: Deploying daemon osd.7 on smithi137 2024-09-16T01:57:48.727 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:48 smithi137 ceph-mon[34099]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:57:48.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:48 smithi105 ceph-mon[31197]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:57:48.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:48 smithi190 ceph-mon[34762]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:57:50.658 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:50 smithi137 ceph-mon[34099]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:57:50.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:50 smithi105 ceph-mon[31197]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:57:50.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:50 smithi190 ceph-mon[34762]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:57:52.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:52 smithi137 ceph-mon[34099]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:57:52.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:52 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:52.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:52 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:52.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:52 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:52.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:52 smithi105 ceph-mon[31197]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:57:52.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:52 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:52.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:52 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:52.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:52 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:52.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:52 smithi190 ceph-mon[34762]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:57:52.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:52 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:52.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:52 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:52.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:52 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:54.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:54 smithi105 ceph-mon[31197]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:57:54.806 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:54 smithi137 ceph-mon[34099]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:57:54.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:54 smithi190 ceph-mon[34762]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:57:54.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:54.845+0000 7f78657fa640 1 -- 172.21.15.137:0/1391622937 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7868062630 con 0x7f78500a3240 2024-09-16T01:57:54.846 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 7 on host 'smithi137' 2024-09-16T01:57:54.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:54.847+0000 7f786e6e0640 1 -- 172.21.15.137:0/1391622937 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f78500a3240 msgr2=0x7f78500a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:54.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:54.847+0000 7f786e6e0640 1 --2- 172.21.15.137:0/1391622937 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f78500a3240 0x7f78500a5700 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7858009d60 tx=0x7f7858009110 comp rx=0 tx=0).stop 2024-09-16T01:57:54.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:54.847+0000 7f786e6e0640 1 -- 172.21.15.137:0/1391622937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7868104fd0 msgr2=0x7f7868100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:54.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:54.847+0000 7f786e6e0640 1 --2- 172.21.15.137:0/1391622937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7868104fd0 0x7f7868100650 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f785c00e9f0 tx=0x7f785c00edf0 comp rx=0 tx=0).stop 2024-09-16T01:57:54.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:54.847+0000 7f786e6e0640 1 -- 172.21.15.137:0/1391622937 shutdown_connections 2024-09-16T01:57:54.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:54.847+0000 7f786e6e0640 1 --2- 172.21.15.137:0/1391622937 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f78500a3240 0x7f78500a5700 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:54.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:54.847+0000 7f786e6e0640 1 --2- 172.21.15.137:0/1391622937 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7868107900 0x7f7868100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:54.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:54.847+0000 7f786e6e0640 1 --2- 172.21.15.137:0/1391622937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7868104fd0 0x7f7868100650 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:54.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:54.847+0000 7f786e6e0640 1 --2- 172.21.15.137:0/1391622937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78681026a0 0x7f7868102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:54.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:54.847+0000 7f786e6e0640 1 -- 172.21.15.137:0/1391622937 >> 172.21.15.137:0/1391622937 conn(0x7f78680fc2a0 msgr2=0x7f7868103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:54.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:54.847+0000 7f786e6e0640 1 -- 172.21.15.137:0/1391622937 shutdown_connections 2024-09-16T01:57:54.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:57:54.848+0000 7f786e6e0640 1 -- 172.21.15.137:0/1391622937 wait complete. 2024-09-16T01:57:55.555 DEBUG:teuthology.orchestra.run.smithi137:osd.7> sudo journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@osd.7.service 2024-09-16T01:57:55.557 INFO:tasks.cephadm:Deploying osd.8 on smithi190 with /dev/vg_nvme/lv_2... 2024-09-16T01:57:55.557 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-16T01:57:55.814 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T01:57:56.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:55 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:56.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:55 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:56.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:55 smithi190 ceph-mon[34762]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:57:56.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:55 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:56.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:55 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:56.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:55 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:56.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:55 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:56.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:55 smithi137 ceph-mon[34099]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:57:56.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:55 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:56.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:55 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:56.294 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:55 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:56.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:55 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:56.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:55 smithi105 ceph-mon[31197]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:57:56.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:55 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:56.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:55 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:57.921 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 01:57:57 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T01:57:57.609+0000 7fcd44dfd640 -1 osd.7 0 log_to_monitors true 2024-09-16T01:57:58.505 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T01:57:58.531 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi190:vg_nvme/lv_2 2024-09-16T01:57:58.609 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:58 smithi137 ceph-mon[34099]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:57:58.610 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:58 smithi137 ceph-mon[34099]: from='osd.7 [v2:172.21.15.137:6816/888678288,v1:172.21.15.137:6817/888678288]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T01:57:58.610 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:58 smithi137 ceph-mon[34099]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T01:57:58.788 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T01:57:58.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:58 smithi105 ceph-mon[31197]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:57:58.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:58 smithi105 ceph-mon[31197]: from='osd.7 [v2:172.21.15.137:6816/888678288,v1:172.21.15.137:6817/888678288]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T01:57:58.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:58 smithi105 ceph-mon[31197]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T01:57:58.814 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:58 smithi190 ceph-mon[34762]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:57:58.814 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:58 smithi190 ceph-mon[34762]: from='osd.7 [v2:172.21.15.137:6816/888678288,v1:172.21.15.137:6817/888678288]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T01:57:58.814 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:58 smithi190 ceph-mon[34762]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T01:57:59.265 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.249+0000 7f7ebcf71640 1 -- 172.21.15.190:0/3400092191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7eb81078f0 msgr2=0x7f7eb8109d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:59.265 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.249+0000 7f7ebcf71640 1 --2- 172.21.15.190:0/3400092191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7eb81078f0 0x7f7eb8109d40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0009ca0 tx=0x7f7eb002a8c0 comp rx=0 tx=0).stop 2024-09-16T01:57:59.265 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.249+0000 7f7ebcf71640 1 -- 172.21.15.190:0/3400092191 shutdown_connections 2024-09-16T01:57:59.265 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.249+0000 7f7ebcf71640 1 --2- 172.21.15.190:0/3400092191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7eb81078f0 0x7f7eb8109d40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:59.265 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.249+0000 7f7ebcf71640 1 --2- 172.21.15.190:0/3400092191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7eb8104fc0 0x7f7eb81073b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:59.265 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.249+0000 7f7ebcf71640 1 --2- 172.21.15.190:0/3400092191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7eb8102690 0x7f7eb8104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:59.266 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.249+0000 7f7ebcf71640 1 -- 172.21.15.190:0/3400092191 >> 172.21.15.190:0/3400092191 conn(0x7f7eb80fc270 msgr2=0x7f7eb80fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:59.266 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.249+0000 7f7ebcf71640 1 -- 172.21.15.190:0/3400092191 shutdown_connections 2024-09-16T01:57:59.266 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.249+0000 7f7ebcf71640 1 -- 172.21.15.190:0/3400092191 wait complete. 2024-09-16T01:57:59.266 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.250+0000 7f7ebcf71640 1 Processor -- start 2024-09-16T01:57:59.266 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.250+0000 7f7ebcf71640 1 -- start start 2024-09-16T01:57:59.266 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.250+0000 7f7ebcf71640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7eb8102690 0x7f7eb8101ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:59.266 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.250+0000 7f7ebcf71640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7eb8104fc0 0x7f7eb8100640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:59.266 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.250+0000 7f7ebcf71640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7eb81078f0 0x7f7eb8100b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:59.266 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.250+0000 7f7ebcf71640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb810d760 con 0x7f7eb8104fc0 2024-09-16T01:57:59.266 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.250+0000 7f7ebcf71640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb810d5e0 con 0x7f7eb8102690 2024-09-16T01:57:59.266 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.250+0000 7f7ebcf71640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb810d8e0 con 0x7f7eb81078f0 2024-09-16T01:57:59.266 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.250+0000 7f7eb6d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7eb81078f0 0x7f7eb8100b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:59.267 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.250+0000 7f7eb659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7eb8102690 0x7f7eb8101ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:59.267 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.250+0000 7f7eb659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7eb8102690 0x7f7eb8101ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.190:45584/0 (socket says 172.21.15.190:45584) 2024-09-16T01:57:59.267 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.250+0000 7f7eb659c640 1 -- 172.21.15.190:0/2446610129 learned_addr learned my addr 172.21.15.190:0/2446610129 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-09-16T01:57:59.267 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.251+0000 7f7eaffff640 1 --2- 172.21.15.190:0/2446610129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7eb8104fc0 0x7f7eb8100640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:59.267 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.251+0000 7f7eb659c640 1 -- 172.21.15.190:0/2446610129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7eb81078f0 msgr2=0x7f7eb8100b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:59.267 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.251+0000 7f7eb659c640 1 --2- 172.21.15.190:0/2446610129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7eb81078f0 0x7f7eb8100b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:59.267 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.251+0000 7f7eb659c640 1 -- 172.21.15.190:0/2446610129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7eb8104fc0 msgr2=0x7f7eb8100640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:59.267 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.251+0000 7f7eb659c640 1 --2- 172.21.15.190:0/2446610129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7eb8104fc0 0x7f7eb8100640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:59.267 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.251+0000 7f7eb659c640 1 -- 172.21.15.190:0/2446610129 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb8101440 con 0x7f7eb8102690 2024-09-16T01:57:59.268 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.251+0000 7f7eb6d9d640 1 --2- 172.21.15.190:0/2446610129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7eb81078f0 0x7f7eb8100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:57:59.268 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.251+0000 7f7eaffff640 1 --2- 172.21.15.190:0/2446610129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7eb8104fc0 0x7f7eb8100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:57:59.268 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.251+0000 7f7eb659c640 1 --2- 172.21.15.190:0/2446610129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7eb8102690 0x7f7eb8101ff0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7ea800d950 tx=0x7f7ea800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:59.268 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.251+0000 7f7eaf7fe640 1 -- 172.21.15.190:0/2446610129 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea8017030 con 0x7f7eb8102690 2024-09-16T01:57:59.268 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.251+0000 7f7eaf7fe640 1 -- 172.21.15.190:0/2446610129 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ea8004020 con 0x7f7eb8102690 2024-09-16T01:57:59.268 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.251+0000 7f7ebcf71640 1 -- 172.21.15.190:0/2446610129 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eb81a1fd0 con 0x7f7eb8102690 2024-09-16T01:57:59.268 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.251+0000 7f7eaf7fe640 1 -- 172.21.15.190:0/2446610129 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea80047a0 con 0x7f7eb8102690 2024-09-16T01:57:59.268 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.251+0000 7f7ebcf71640 1 -- 172.21.15.190:0/2446610129 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eb81a2430 con 0x7f7eb8102690 2024-09-16T01:57:59.268 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.252+0000 7f7ebcf71640 1 -- 172.21.15.190:0/2446610129 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eb80fdd60 con 0x7f7eb8102690 2024-09-16T01:57:59.269 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.253+0000 7f7eaf7fe640 1 -- 172.21.15.190:0/2446610129 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f7ea80041d0 con 0x7f7eb8102690 2024-09-16T01:57:59.269 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.254+0000 7f7eaf7fe640 1 --2- 172.21.15.190:0/2446610129 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7ea40a3240 0x7f7ea40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:59.269 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.254+0000 7f7eaf7fe640 1 -- 172.21.15.190:0/2446610129 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5101+0+0 (secure 0 0 0) 0x7f7ea80ceea0 con 0x7f7eb8102690 2024-09-16T01:57:59.269 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.254+0000 7f7eaffff640 1 --2- 172.21.15.190:0/2446610129 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7ea40a3240 0x7f7ea40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:59.269 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.255+0000 7f7eaffff640 1 --2- 172.21.15.190:0/2446610129 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7ea40a3240 0x7f7ea40a5700 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0005fd0 tx=0x7f7ea000b5d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:59.269 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.257+0000 7f7eaf7fe640 1 -- 172.21.15.190:0/2446610129 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7ea809c7b0 con 0x7f7eb8102690 2024-09-16T01:57:59.438 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:57:59.436+0000 7f7ebcf71640 1 -- 172.21.15.190:0/2446610129 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f7eb8062630 con 0x7f7ea40a3240 2024-09-16T01:57:59.792 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:59 smithi137 ceph-mon[34099]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-16T01:57:59.793 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:59 smithi137 ceph-mon[34099]: from='osd.7 [v2:172.21.15.137:6816/888678288,v1:172.21.15.137:6817/888678288]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-09-16T01:57:59.793 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:59 smithi137 ceph-mon[34099]: osdmap e45: 8 total, 7 up, 8 in 2024-09-16T01:57:59.793 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:59 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T01:57:59.793 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:59 smithi137 ceph-mon[34099]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-09-16T01:57:59.793 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:59 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:59.793 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:59 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:59.793 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:59 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:57:59.793 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:59 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:57:59.793 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:59 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:57:59.793 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:59 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:59.793 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:59 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:59.793 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:59 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:59.794 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:59 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:59.794 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:59 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:59.794 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:59 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:59.794 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:59 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:59.794 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:59 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:59.794 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:59 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:57:59.794 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:59 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:57:59.794 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:59 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:59.794 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:59 smithi137 ceph-mon[34099]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-09-16T01:57:59.794 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:59 smithi137 ceph-mon[34099]: osdmap e46: 8 total, 7 up, 8 in 2024-09-16T01:57:59.794 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:57:59 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T01:57:59.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:59 smithi105 ceph-mon[31197]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-16T01:57:59.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:59 smithi105 ceph-mon[31197]: from='osd.7 [v2:172.21.15.137:6816/888678288,v1:172.21.15.137:6817/888678288]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-09-16T01:57:59.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:59 smithi105 ceph-mon[31197]: osdmap e45: 8 total, 7 up, 8 in 2024-09-16T01:57:59.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:59 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T01:57:59.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:59 smithi105 ceph-mon[31197]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-09-16T01:57:59.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:59 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:59.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:59 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:59.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:59 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:57:59.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:59 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:57:59.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:59 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:57:59.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:59 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:59.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:59 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:59.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:59 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:59.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:59 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:59.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:59 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:59.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:59 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:59.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:59 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:59.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:59 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:59.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:59 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:57:59.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:59 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:57:59.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:59 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:59.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:59 smithi105 ceph-mon[31197]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-09-16T01:57:59.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:59 smithi105 ceph-mon[31197]: osdmap e46: 8 total, 7 up, 8 in 2024-09-16T01:57:59.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:57:59 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T01:57:59.816 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:59 smithi190 ceph-mon[34762]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-16T01:57:59.816 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:59 smithi190 ceph-mon[34762]: from='osd.7 [v2:172.21.15.137:6816/888678288,v1:172.21.15.137:6817/888678288]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-09-16T01:57:59.816 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:59 smithi190 ceph-mon[34762]: osdmap e45: 8 total, 7 up, 8 in 2024-09-16T01:57:59.816 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:59 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T01:57:59.816 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:59 smithi190 ceph-mon[34762]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-09-16T01:57:59.816 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:59 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:59.817 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:59 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:59.817 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:59 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:57:59.817 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:59 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:57:59.817 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:59 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:57:59.817 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:59 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:59.817 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:59 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:59.817 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:59 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:59.817 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:59 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:59.817 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:59 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:59.817 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:59 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:59.817 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:59 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:59.817 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:59 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:57:59.817 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:59 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:57:59.817 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:59 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:57:59.818 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:59 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:59.818 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:59 smithi190 ceph-mon[34762]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-09-16T01:57:59.818 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:59 smithi190 ceph-mon[34762]: osdmap e46: 8 total, 7 up, 8 in 2024-09-16T01:57:59.818 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:57:59 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T01:58:00.171 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 01:57:59 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T01:57:59.863+0000 7fcd40d80640 -1 osd.7 0 waiting for initial osdmap 2024-09-16T01:58:00.171 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 01:57:59 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T01:57:59.872+0000 7fcd38ba0640 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T01:58:00.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:00 smithi137 ceph-mon[34099]: Detected new or changed devices on smithi137 2024-09-16T01:58:00.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:00 smithi137 ceph-mon[34099]: Adjusting osd_memory_target on smithi137 to 5696M 2024-09-16T01:58:00.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:00 smithi137 ceph-mon[34099]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:58:00.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:00 smithi137 ceph-mon[34099]: from='client.24311 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:58:00.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:00 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T01:58:00.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:00 smithi137 ceph-mon[34099]: from='osd.7 ' entity='osd.7' 2024-09-16T01:58:00.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:00 smithi137 ceph-mon[34099]: osd.7 [v2:172.21.15.137:6816/888678288,v1:172.21.15.137:6817/888678288] boot 2024-09-16T01:58:00.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:00 smithi137 ceph-mon[34099]: osdmap e47: 8 total, 8 up, 8 in 2024-09-16T01:58:00.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:00 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T01:58:00.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:00 smithi105 ceph-mon[31197]: Detected new or changed devices on smithi137 2024-09-16T01:58:00.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:00 smithi105 ceph-mon[31197]: Adjusting osd_memory_target on smithi137 to 5696M 2024-09-16T01:58:00.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:00 smithi105 ceph-mon[31197]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:58:00.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:00 smithi105 ceph-mon[31197]: from='client.24311 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:58:00.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:00 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T01:58:00.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:00 smithi105 ceph-mon[31197]: from='osd.7 ' entity='osd.7' 2024-09-16T01:58:00.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:00 smithi105 ceph-mon[31197]: osd.7 [v2:172.21.15.137:6816/888678288,v1:172.21.15.137:6817/888678288] boot 2024-09-16T01:58:00.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:00 smithi105 ceph-mon[31197]: osdmap e47: 8 total, 8 up, 8 in 2024-09-16T01:58:00.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:00 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T01:58:00.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:00 smithi190 ceph-mon[34762]: Detected new or changed devices on smithi137 2024-09-16T01:58:00.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:00 smithi190 ceph-mon[34762]: Adjusting osd_memory_target on smithi137 to 5696M 2024-09-16T01:58:00.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:00 smithi190 ceph-mon[34762]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:58:00.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:00 smithi190 ceph-mon[34762]: from='client.24311 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:58:00.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:00 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T01:58:00.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:00 smithi190 ceph-mon[34762]: from='osd.7 ' entity='osd.7' 2024-09-16T01:58:00.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:00 smithi190 ceph-mon[34762]: osd.7 [v2:172.21.15.137:6816/888678288,v1:172.21.15.137:6817/888678288] boot 2024-09-16T01:58:00.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:00 smithi190 ceph-mon[34762]: osdmap e47: 8 total, 8 up, 8 in 2024-09-16T01:58:00.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:00 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T01:58:01.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:01 smithi105 ceph-mon[31197]: purged_snaps scrub starts 2024-09-16T01:58:01.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:01 smithi105 ceph-mon[31197]: purged_snaps scrub ok 2024-09-16T01:58:01.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:01 smithi105 ceph-mon[31197]: osdmap e48: 8 total, 8 up, 8 in 2024-09-16T01:58:01.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:01 smithi190 ceph-mon[34762]: purged_snaps scrub starts 2024-09-16T01:58:01.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:01 smithi190 ceph-mon[34762]: purged_snaps scrub ok 2024-09-16T01:58:01.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:01 smithi190 ceph-mon[34762]: osdmap e48: 8 total, 8 up, 8 in 2024-09-16T01:58:01.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:01 smithi137 ceph-mon[34099]: purged_snaps scrub starts 2024-09-16T01:58:01.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:01 smithi137 ceph-mon[34099]: purged_snaps scrub ok 2024-09-16T01:58:01.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:01 smithi137 ceph-mon[34099]: osdmap e48: 8 total, 8 up, 8 in 2024-09-16T01:58:02.548 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:02 smithi190 ceph-mon[34762]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:58:02.548 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:02 smithi190 ceph-mon[34762]: from='client.? 172.21.15.190:0/2639346016' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6718597c-6f42-48ee-9493-edb2396fe707"}]: dispatch 2024-09-16T01:58:02.548 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:02 smithi190 ceph-mon[34762]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6718597c-6f42-48ee-9493-edb2396fe707"}]: dispatch 2024-09-16T01:58:02.549 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:02 smithi190 ceph-mon[34762]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6718597c-6f42-48ee-9493-edb2396fe707"}]': finished 2024-09-16T01:58:02.549 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:02 smithi190 ceph-mon[34762]: osdmap e49: 9 total, 8 up, 9 in 2024-09-16T01:58:02.549 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:02 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:58:02.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:02 smithi105 ceph-mon[31197]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:58:02.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:02 smithi105 ceph-mon[31197]: from='client.? 172.21.15.190:0/2639346016' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6718597c-6f42-48ee-9493-edb2396fe707"}]: dispatch 2024-09-16T01:58:02.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:02 smithi105 ceph-mon[31197]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6718597c-6f42-48ee-9493-edb2396fe707"}]: dispatch 2024-09-16T01:58:02.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:02 smithi105 ceph-mon[31197]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6718597c-6f42-48ee-9493-edb2396fe707"}]': finished 2024-09-16T01:58:02.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:02 smithi105 ceph-mon[31197]: osdmap e49: 9 total, 8 up, 9 in 2024-09-16T01:58:02.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:02 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:58:02.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:02 smithi137 ceph-mon[34099]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:58:02.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:02 smithi137 ceph-mon[34099]: from='client.? 172.21.15.190:0/2639346016' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6718597c-6f42-48ee-9493-edb2396fe707"}]: dispatch 2024-09-16T01:58:02.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:02 smithi137 ceph-mon[34099]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6718597c-6f42-48ee-9493-edb2396fe707"}]: dispatch 2024-09-16T01:58:02.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:02 smithi137 ceph-mon[34099]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6718597c-6f42-48ee-9493-edb2396fe707"}]': finished 2024-09-16T01:58:02.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:02 smithi137 ceph-mon[34099]: osdmap e49: 9 total, 8 up, 9 in 2024-09-16T01:58:02.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:02 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:58:03.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:03 smithi105 ceph-mon[31197]: from='client.? 172.21.15.190:0/2295678604' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:58:03.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:03 smithi190 ceph-mon[34762]: from='client.? 172.21.15.190:0/2295678604' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:58:03.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:03 smithi137 ceph-mon[34099]: from='client.? 172.21.15.190:0/2295678604' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:58:04.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:04 smithi105 ceph-mon[31197]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:58:04.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:04 smithi105 ceph-mon[31197]: osdmap e50: 9 total, 8 up, 9 in 2024-09-16T01:58:04.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:04 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:58:04.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:04 smithi190 ceph-mon[34762]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:58:04.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:04 smithi190 ceph-mon[34762]: osdmap e50: 9 total, 8 up, 9 in 2024-09-16T01:58:04.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:04 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:58:04.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:04 smithi137 ceph-mon[34099]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:58:04.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:04 smithi137 ceph-mon[34099]: osdmap e50: 9 total, 8 up, 9 in 2024-09-16T01:58:04.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:04 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:58:05.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:05 smithi105 ceph-mon[31197]: osdmap e51: 9 total, 8 up, 9 in 2024-09-16T01:58:05.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:05 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:58:05.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:05 smithi190 ceph-mon[34762]: osdmap e51: 9 total, 8 up, 9 in 2024-09-16T01:58:05.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:05 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:58:05.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:05 smithi137 ceph-mon[34099]: osdmap e51: 9 total, 8 up, 9 in 2024-09-16T01:58:05.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:05 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:58:06.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:06 smithi190 ceph-mon[34762]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:58:06.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:06 smithi105 ceph-mon[31197]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:58:06.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:06 smithi137 ceph-mon[34099]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:58:08.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:08 smithi105 ceph-mon[31197]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:58:08.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:08 smithi190 ceph-mon[34762]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:58:08.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:08 smithi137 ceph-mon[34099]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:58:10.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:10 smithi105 ceph-mon[31197]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:58:10.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:10 smithi190 ceph-mon[34762]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:58:10.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:10 smithi137 ceph-mon[34099]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:58:11.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:11 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-16T01:58:11.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:11 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:11.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:11 smithi190 ceph-mon[34762]: Deploying daemon osd.8 on smithi190 2024-09-16T01:58:11.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:11 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-16T01:58:11.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:11 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:11.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:11 smithi105 ceph-mon[31197]: Deploying daemon osd.8 on smithi190 2024-09-16T01:58:11.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:11 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-16T01:58:11.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:11 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:11.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:11 smithi137 ceph-mon[34099]: Deploying daemon osd.8 on smithi190 2024-09-16T01:58:12.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:12 smithi190 ceph-mon[34762]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:58:12.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:12 smithi105 ceph-mon[31197]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:58:12.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:12 smithi137 ceph-mon[34099]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:58:14.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:14 smithi190 ceph-mon[34762]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:58:14.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:14 smithi105 ceph-mon[31197]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:58:14.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:14 smithi137 ceph-mon[34099]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:58:16.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:16 smithi105 ceph-mon[31197]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:58:16.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:16 smithi190 ceph-mon[34762]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:58:16.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:16 smithi137 ceph-mon[34099]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:58:17.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:17 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:58:17.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:17 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:17.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:17 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:17.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:17 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:58:17.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:17 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:17.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:17 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:17.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:17 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:58:17.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:17 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:17.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:17 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:18.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:18 smithi105 ceph-mon[31197]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:58:18.822 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:18 smithi137 ceph-mon[34099]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:58:18.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:18 smithi190 ceph-mon[34762]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:58:20.136 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 8 on host 'smithi190' 2024-09-16T01:58:20.136 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:58:20.134+0000 7f7eaf7fe640 1 -- 172.21.15.190:0/2446610129 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7eb8062630 con 0x7f7ea40a3240 2024-09-16T01:58:20.139 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:58:20.137+0000 7f7ebcf71640 1 -- 172.21.15.190:0/2446610129 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7ea40a3240 msgr2=0x7f7ea40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:20.139 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:58:20.137+0000 7f7ebcf71640 1 --2- 172.21.15.190:0/2446610129 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7ea40a3240 0x7f7ea40a5700 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0005fd0 tx=0x7f7ea000b5d0 comp rx=0 tx=0).stop 2024-09-16T01:58:20.139 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:58:20.137+0000 7f7ebcf71640 1 -- 172.21.15.190:0/2446610129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7eb8102690 msgr2=0x7f7eb8101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:20.139 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:58:20.137+0000 7f7ebcf71640 1 --2- 172.21.15.190:0/2446610129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7eb8102690 0x7f7eb8101ff0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7ea800d950 tx=0x7f7ea800de10 comp rx=0 tx=0).stop 2024-09-16T01:58:20.139 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:58:20.138+0000 7f7ebcf71640 1 -- 172.21.15.190:0/2446610129 shutdown_connections 2024-09-16T01:58:20.139 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:58:20.138+0000 7f7ebcf71640 1 --2- 172.21.15.190:0/2446610129 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7ea40a3240 0x7f7ea40a5700 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:20.139 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:58:20.138+0000 7f7ebcf71640 1 --2- 172.21.15.190:0/2446610129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7eb81078f0 0x7f7eb8100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:20.139 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:58:20.138+0000 7f7ebcf71640 1 --2- 172.21.15.190:0/2446610129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7eb8104fc0 0x7f7eb8100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:20.139 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:58:20.138+0000 7f7ebcf71640 1 --2- 172.21.15.190:0/2446610129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7eb8102690 0x7f7eb8101ff0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:20.140 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:58:20.138+0000 7f7ebcf71640 1 -- 172.21.15.190:0/2446610129 >> 172.21.15.190:0/2446610129 conn(0x7f7eb80fc270 msgr2=0x7f7eb8103070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:20.140 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:58:20.138+0000 7f7ebcf71640 1 -- 172.21.15.190:0/2446610129 shutdown_connections 2024-09-16T01:58:20.140 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:58:20.138+0000 7f7ebcf71640 1 -- 172.21.15.190:0/2446610129 wait complete. 2024-09-16T01:58:20.790 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:20 smithi190 ceph-mon[34762]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:58:20.790 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:20 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:20.790 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:20 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:20.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:20 smithi105 ceph-mon[31197]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:58:20.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:20 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:20.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:20 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:20.819 DEBUG:teuthology.orchestra.run.smithi190:osd.8> sudo journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@osd.8.service 2024-09-16T01:58:20.821 INFO:tasks.cephadm:Deploying osd.9 on smithi105 with /dev/vg_nvme/lv_1... 2024-09-16T01:58:20.821 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-16T01:58:20.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:20 smithi137 ceph-mon[34099]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:58:20.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:20 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:20.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:20 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:21.076 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:58:22.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:21 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:22.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:21 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:22.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:21 smithi137 ceph-mon[34099]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:58:22.222 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:21 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:22.222 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:21 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:22.222 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:21 smithi190 ceph-mon[34762]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:58:22.227 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:21 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:22.227 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:21 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:22.227 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:21 smithi105 ceph-mon[31197]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:58:22.543 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 01:58:22 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T01:58:22.459+0000 7fadab161640 -1 osd.8 0 log_to_monitors true 2024-09-16T01:58:23.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:22 smithi105 ceph-mon[31197]: from='osd.8 [v2:172.21.15.190:6816/2132638341,v1:172.21.15.190:6817/2132638341]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T01:58:23.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:22 smithi105 ceph-mon[31197]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T01:58:23.176 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:22 smithi137 ceph-mon[34099]: from='osd.8 [v2:172.21.15.190:6816/2132638341,v1:172.21.15.190:6817/2132638341]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T01:58:23.177 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:22 smithi137 ceph-mon[34099]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T01:58:23.240 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:22 smithi190 ceph-mon[34762]: from='osd.8 [v2:172.21.15.190:6816/2132638341,v1:172.21.15.190:6817/2132638341]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T01:58:23.240 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:22 smithi190 ceph-mon[34762]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T01:58:23.765 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:58:23.791 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi105:vg_nvme/lv_1 2024-09-16T01:58:24.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:23 smithi105 ceph-mon[31197]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-16T01:58:24.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:23 smithi105 ceph-mon[31197]: from='osd.8 [v2:172.21.15.190:6816/2132638341,v1:172.21.15.190:6817/2132638341]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T01:58:24.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:23 smithi105 ceph-mon[31197]: osdmap e52: 9 total, 8 up, 9 in 2024-09-16T01:58:24.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:23 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:58:24.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:23 smithi105 ceph-mon[31197]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T01:58:24.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:23 smithi105 ceph-mon[31197]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:58:24.045 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:58:24.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:23 smithi137 ceph-mon[34099]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-16T01:58:24.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:23 smithi137 ceph-mon[34099]: from='osd.8 [v2:172.21.15.190:6816/2132638341,v1:172.21.15.190:6817/2132638341]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T01:58:24.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:23 smithi137 ceph-mon[34099]: osdmap e52: 9 total, 8 up, 9 in 2024-09-16T01:58:24.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:23 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:58:24.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:23 smithi137 ceph-mon[34099]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T01:58:24.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:23 smithi137 ceph-mon[34099]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:58:24.342 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:23 smithi190 ceph-mon[34762]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-16T01:58:24.342 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:23 smithi190 ceph-mon[34762]: from='osd.8 [v2:172.21.15.190:6816/2132638341,v1:172.21.15.190:6817/2132638341]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T01:58:24.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:23 smithi190 ceph-mon[34762]: osdmap e52: 9 total, 8 up, 9 in 2024-09-16T01:58:24.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:23 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:58:24.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:23 smithi190 ceph-mon[34762]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T01:58:24.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:23 smithi190 ceph-mon[34762]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:58:24.343 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 01:58:23 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T01:58:23.875+0000 7fada70e4640 -1 osd.8 0 waiting for initial osdmap 2024-09-16T01:58:24.343 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 01:58:23 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T01:58:23.884+0000 7fada2f0c640 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T01:58:24.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.507+0000 7f5ab41a3640 1 -- 172.21.15.105:0/3745765133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5aac109a70 msgr2=0x7f5aac10bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:24.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.507+0000 7f5ab41a3640 1 --2- 172.21.15.105:0/3745765133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5aac109a70 0x7f5aac10bf30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4009900 tx=0x7f5aa402a410 comp rx=0 tx=0).stop 2024-09-16T01:58:24.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.507+0000 7f5ab41a3640 1 -- 172.21.15.105:0/3745765133 shutdown_connections 2024-09-16T01:58:24.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.507+0000 7f5ab41a3640 1 --2- 172.21.15.105:0/3745765133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5aac074b40 0x7f5aac074fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:24.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.507+0000 7f5ab41a3640 1 --2- 172.21.15.105:0/3745765133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5aac109a70 0x7f5aac10bf30 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:24.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.507+0000 7f5ab41a3640 1 --2- 172.21.15.105:0/3745765133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5aac0761a0 0x7f5aac074600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:24.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.507+0000 7f5ab41a3640 1 -- 172.21.15.105:0/3745765133 >> 172.21.15.105:0/3745765133 conn(0x7f5aac0fd390 msgr2=0x7f5aac0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:24.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.507+0000 7f5ab41a3640 1 -- 172.21.15.105:0/3745765133 shutdown_connections 2024-09-16T01:58:24.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.507+0000 7f5ab41a3640 1 -- 172.21.15.105:0/3745765133 wait complete. 2024-09-16T01:58:24.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.508+0000 7f5ab41a3640 1 Processor -- start 2024-09-16T01:58:24.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.508+0000 7f5ab41a3640 1 -- start start 2024-09-16T01:58:24.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.508+0000 7f5ab41a3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5aac074b40 0x7f5aac199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:24.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.508+0000 7f5ab41a3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5aac0761a0 0x7f5aac19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:24.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.508+0000 7f5ab41a3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5aac109a70 0x7f5aac1a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:24.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.508+0000 7f5ab41a3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac10f900 con 0x7f5aac074b40 2024-09-16T01:58:24.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.508+0000 7f5ab41a3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac10f780 con 0x7f5aac109a70 2024-09-16T01:58:24.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.508+0000 7f5ab41a3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac10fa80 con 0x7f5aac0761a0 2024-09-16T01:58:24.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.508+0000 7f5ab1f3f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5aac074b40 0x7f5aac199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:24.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.508+0000 7f5ab1f3f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5aac074b40 0x7f5aac199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50908/0 (socket says 172.21.15.105:50908) 2024-09-16T01:58:24.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.508+0000 7f5ab1f3f640 1 -- 172.21.15.105:0/1355777871 learned_addr learned my addr 172.21.15.105:0/1355777871 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:58:24.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.508+0000 7f5ab173e640 1 --2- 172.21.15.105:0/1355777871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5aac0761a0 0x7f5aac19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:24.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.508+0000 7f5ab2740640 1 --2- 172.21.15.105:0/1355777871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5aac109a70 0x7f5aac1a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:24.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.508+0000 7f5ab1f3f640 1 -- 172.21.15.105:0/1355777871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5aac0761a0 msgr2=0x7f5aac19a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:24.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.508+0000 7f5ab1f3f640 1 --2- 172.21.15.105:0/1355777871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5aac0761a0 0x7f5aac19a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:24.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.508+0000 7f5ab1f3f640 1 -- 172.21.15.105:0/1355777871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5aac109a70 msgr2=0x7f5aac1a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:24.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.508+0000 7f5ab1f3f640 1 --2- 172.21.15.105:0/1355777871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5aac109a70 0x7f5aac1a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:24.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.508+0000 7f5ab1f3f640 1 -- 172.21.15.105:0/1355777871 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aac1a1bd0 con 0x7f5aac074b40 2024-09-16T01:58:24.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.508+0000 7f5ab2740640 1 --2- 172.21.15.105:0/1355777871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5aac109a70 0x7f5aac1a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:58:24.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.508+0000 7f5ab1f3f640 1 --2- 172.21.15.105:0/1355777871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5aac074b40 0x7f5aac199fa0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c00d950 tx=0x7f5a9c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:24.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.509+0000 7f5aaaffd640 1 -- 172.21.15.105:0/1355777871 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c017030 con 0x7f5aac074b40 2024-09-16T01:58:24.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.509+0000 7f5aaaffd640 1 -- 172.21.15.105:0/1355777871 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a9c004020 con 0x7f5aac074b40 2024-09-16T01:58:24.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.509+0000 7f5ab41a3640 1 -- 172.21.15.105:0/1355777871 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aac1a1ec0 con 0x7f5aac074b40 2024-09-16T01:58:24.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.509+0000 7f5aaaffd640 1 -- 172.21.15.105:0/1355777871 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c0047a0 con 0x7f5aac074b40 2024-09-16T01:58:24.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.509+0000 7f5ab41a3640 1 -- 172.21.15.105:0/1355777871 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aac1a2310 con 0x7f5aac074b40 2024-09-16T01:58:24.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.510+0000 7f5ab41a3640 1 -- 172.21.15.105:0/1355777871 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5aac108880 con 0x7f5aac074b40 2024-09-16T01:58:24.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.511+0000 7f5aaaffd640 1 -- 172.21.15.105:0/1355777871 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f5a9c0041d0 con 0x7f5aac074b40 2024-09-16T01:58:24.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.511+0000 7f5aaaffd640 1 --2- 172.21.15.105:0/1355777871 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5a940a3270 0x7f5a940a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:24.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.511+0000 7f5ab173e640 1 --2- 172.21.15.105:0/1355777871 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5a940a3270 0x7f5a940a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:24.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.511+0000 7f5aaaffd640 1 -- 172.21.15.105:0/1355777871 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5862+0+0 (secure 0 0 0) 0x7f5a9c0ce9e0 con 0x7f5aac074b40 2024-09-16T01:58:24.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.512+0000 7f5ab173e640 1 --2- 172.21.15.105:0/1355777871 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5a940a3270 0x7f5a940a5730 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5aac19b4c0 tx=0x7f5aa0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:24.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.516+0000 7f5aaaffd640 1 -- 172.21.15.105:0/1355777871 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5a9c09c090 con 0x7f5aac074b40 2024-09-16T01:58:24.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:24.692+0000 7f5ab41a3640 1 -- 172.21.15.105:0/1355777871 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f5aac062630 con 0x7f5a940a3270 2024-09-16T01:58:25.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:24 smithi105 ceph-mon[31197]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-09-16T01:58:25.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:24 smithi105 ceph-mon[31197]: osdmap e53: 9 total, 8 up, 9 in 2024-09-16T01:58:25.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:24 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:58:25.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:24 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:58:25.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:24 smithi105 ceph-mon[31197]: Detected new or changed devices on smithi190 2024-09-16T01:58:25.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:24 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:25.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:24 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:25.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:24 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:58:25.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:24 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:58:25.046 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:24 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:58:25.046 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:24 smithi105 ceph-mon[31197]: Adjusting osd_memory_target on smithi190 to 5696M 2024-09-16T01:58:25.046 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:24 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:25.046 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:24 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:25.046 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:24 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:58:25.046 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:24 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:25.046 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:24 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:58:25.046 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:24 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:25.046 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:24 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:58:25.046 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:24 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:25.046 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:24 smithi105 ceph-mon[31197]: osd.8 [v2:172.21.15.190:6816/2132638341,v1:172.21.15.190:6817/2132638341] boot 2024-09-16T01:58:25.047 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:24 smithi105 ceph-mon[31197]: osdmap e54: 9 total, 9 up, 9 in 2024-09-16T01:58:25.047 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:24 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:58:25.047 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:24 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:58:25.047 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:24 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:58:25.047 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:24 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:25.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:24 smithi137 ceph-mon[34099]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-09-16T01:58:25.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:24 smithi137 ceph-mon[34099]: osdmap e53: 9 total, 8 up, 9 in 2024-09-16T01:58:25.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:24 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:58:25.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:24 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:58:25.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:24 smithi137 ceph-mon[34099]: Detected new or changed devices on smithi190 2024-09-16T01:58:25.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:24 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:25.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:24 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:25.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:24 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:58:25.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:24 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:58:25.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:24 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:58:25.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:24 smithi137 ceph-mon[34099]: Adjusting osd_memory_target on smithi190 to 5696M 2024-09-16T01:58:25.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:24 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:25.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:24 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:25.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:24 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:58:25.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:24 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:25.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:24 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:58:25.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:24 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:25.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:24 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:58:25.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:24 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:25.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:24 smithi137 ceph-mon[34099]: osd.8 [v2:172.21.15.190:6816/2132638341,v1:172.21.15.190:6817/2132638341] boot 2024-09-16T01:58:25.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:24 smithi137 ceph-mon[34099]: osdmap e54: 9 total, 9 up, 9 in 2024-09-16T01:58:25.173 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:24 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:58:25.173 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:24 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:58:25.173 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:24 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:58:25.173 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:24 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:25.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:24 smithi190 ceph-mon[34762]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-09-16T01:58:25.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:24 smithi190 ceph-mon[34762]: osdmap e53: 9 total, 8 up, 9 in 2024-09-16T01:58:25.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:24 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:58:25.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:24 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:58:25.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:24 smithi190 ceph-mon[34762]: Detected new or changed devices on smithi190 2024-09-16T01:58:25.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:24 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:25.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:24 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:25.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:24 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:58:25.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:24 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:58:25.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:24 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:58:25.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:24 smithi190 ceph-mon[34762]: Adjusting osd_memory_target on smithi190 to 5696M 2024-09-16T01:58:25.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:24 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:25.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:24 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:25.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:24 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:58:25.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:24 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:25.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:24 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:58:25.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:24 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:25.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:24 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:58:25.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:24 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:25.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:24 smithi190 ceph-mon[34762]: osd.8 [v2:172.21.15.190:6816/2132638341,v1:172.21.15.190:6817/2132638341] boot 2024-09-16T01:58:25.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:24 smithi190 ceph-mon[34762]: osdmap e54: 9 total, 9 up, 9 in 2024-09-16T01:58:25.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:24 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:58:25.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:24 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:58:25.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:24 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:58:25.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:24 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:26.533 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:26 smithi105 ceph-mon[31197]: purged_snaps scrub starts 2024-09-16T01:58:26.533 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:26 smithi105 ceph-mon[31197]: purged_snaps scrub ok 2024-09-16T01:58:26.533 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:26 smithi105 ceph-mon[31197]: from='client.14439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:58:26.533 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:26 smithi105 ceph-mon[31197]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:58:26.533 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:26 smithi105 ceph-mon[31197]: osdmap e55: 9 total, 9 up, 9 in 2024-09-16T01:58:26.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:26 smithi190 ceph-mon[34762]: purged_snaps scrub starts 2024-09-16T01:58:26.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:26 smithi190 ceph-mon[34762]: purged_snaps scrub ok 2024-09-16T01:58:26.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:26 smithi190 ceph-mon[34762]: from='client.14439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:58:26.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:26 smithi190 ceph-mon[34762]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:58:26.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:26 smithi190 ceph-mon[34762]: osdmap e55: 9 total, 9 up, 9 in 2024-09-16T01:58:26.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:26 smithi137 ceph-mon[34099]: purged_snaps scrub starts 2024-09-16T01:58:26.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:26 smithi137 ceph-mon[34099]: purged_snaps scrub ok 2024-09-16T01:58:26.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:26 smithi137 ceph-mon[34099]: from='client.14439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:58:26.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:26 smithi137 ceph-mon[34099]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:58:26.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:26 smithi137 ceph-mon[34099]: osdmap e55: 9 total, 9 up, 9 in 2024-09-16T01:58:27.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:27 smithi105 ceph-mon[31197]: osdmap e56: 9 total, 9 up, 9 in 2024-09-16T01:58:27.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:27 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/585073821' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5178edc0-a2e8-491c-821c-6f41ec5aeb2c"}]: dispatch 2024-09-16T01:58:27.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:27 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/585073821' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5178edc0-a2e8-491c-821c-6f41ec5aeb2c"}]': finished 2024-09-16T01:58:27.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:27 smithi105 ceph-mon[31197]: osdmap e57: 10 total, 9 up, 10 in 2024-09-16T01:58:27.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:27 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:58:27.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:27 smithi190 ceph-mon[34762]: osdmap e56: 9 total, 9 up, 9 in 2024-09-16T01:58:27.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:27 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/585073821' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5178edc0-a2e8-491c-821c-6f41ec5aeb2c"}]: dispatch 2024-09-16T01:58:27.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:27 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/585073821' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5178edc0-a2e8-491c-821c-6f41ec5aeb2c"}]': finished 2024-09-16T01:58:27.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:27 smithi190 ceph-mon[34762]: osdmap e57: 10 total, 9 up, 10 in 2024-09-16T01:58:27.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:27 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:58:27.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:27 smithi137 ceph-mon[34099]: osdmap e56: 9 total, 9 up, 9 in 2024-09-16T01:58:27.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:27 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/585073821' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5178edc0-a2e8-491c-821c-6f41ec5aeb2c"}]: dispatch 2024-09-16T01:58:27.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:27 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/585073821' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5178edc0-a2e8-491c-821c-6f41ec5aeb2c"}]': finished 2024-09-16T01:58:27.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:27 smithi137 ceph-mon[34099]: osdmap e57: 10 total, 9 up, 10 in 2024-09-16T01:58:27.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:27 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:58:28.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:28 smithi105 ceph-mon[31197]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:58:28.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:28 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2670176840' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:58:28.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:28 smithi105 ceph-mon[31197]: osdmap e58: 10 total, 9 up, 10 in 2024-09-16T01:58:28.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:28 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:58:28.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:28 smithi190 ceph-mon[34762]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:58:28.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:28 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/2670176840' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:58:28.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:28 smithi190 ceph-mon[34762]: osdmap e58: 10 total, 9 up, 10 in 2024-09-16T01:58:28.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:28 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:58:28.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:28 smithi137 ceph-mon[34099]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:58:28.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:28 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/2670176840' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:58:28.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:28 smithi137 ceph-mon[34099]: osdmap e58: 10 total, 9 up, 10 in 2024-09-16T01:58:28.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:28 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:58:30.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:30 smithi105 ceph-mon[31197]: pgmap v193: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:58:30.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:30 smithi190 ceph-mon[34762]: pgmap v193: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:58:30.920 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:30 smithi137 ceph-mon[34099]: pgmap v193: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:58:32.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:32 smithi105 ceph-mon[31197]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-16T01:58:32.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:32 smithi190 ceph-mon[34762]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-16T01:58:32.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:32 smithi137 ceph-mon[34099]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-16T01:58:34.723 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:34 smithi105 ceph-mon[31197]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-16T01:58:34.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:34 smithi190 ceph-mon[34762]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-16T01:58:34.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:34 smithi137 ceph-mon[34099]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-16T01:58:36.708 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:36 smithi105 ceph-mon[31197]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-16T01:58:36.708 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:36 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-16T01:58:36.708 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:36 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:36.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:36 smithi190 ceph-mon[34762]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-16T01:58:36.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:36 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-16T01:58:36.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:36 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:36.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:36 smithi137 ceph-mon[34099]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-16T01:58:36.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:36 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-16T01:58:36.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:36 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:37.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:37 smithi105 ceph-mon[31197]: Deploying daemon osd.9 on smithi105 2024-09-16T01:58:37.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:37 smithi190 ceph-mon[34762]: Deploying daemon osd.9 on smithi105 2024-09-16T01:58:37.920 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:37 smithi137 ceph-mon[34099]: Deploying daemon osd.9 on smithi105 2024-09-16T01:58:38.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:38 smithi105 ceph-mon[31197]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T01:58:38.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:38 smithi190 ceph-mon[34762]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T01:58:38.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:38 smithi137 ceph-mon[34099]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T01:58:40.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:40 smithi105 ceph-mon[31197]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-16T01:58:40.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:40 smithi190 ceph-mon[34762]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-16T01:58:40.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:40 smithi137 ceph-mon[34099]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-16T01:58:42.763 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:42 smithi105 ceph-mon[31197]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T01:58:42.763 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:42 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:58:42.763 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:42 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:42.763 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:42 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:42.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:42 smithi190 ceph-mon[34762]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T01:58:42.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:42 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:58:42.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:42 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:42.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:42 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:42.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:42 smithi137 ceph-mon[34099]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T01:58:42.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:42 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:58:42.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:42 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:42.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:42 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:44.751 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:44 smithi105 ceph-mon[31197]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:58:44.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:44 smithi190 ceph-mon[34762]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:58:44.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:44 smithi137 ceph-mon[34099]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:58:45.596 INFO:teuthology.orchestra.run.smithi105.stdout:Created osd(s) 9 on host 'smithi105' 2024-09-16T01:58:45.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:45.595+0000 7f5aaaffd640 1 -- 172.21.15.105:0/1355777871 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5aac062630 con 0x7f5a940a3270 2024-09-16T01:58:45.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:45.597+0000 7f5ab41a3640 1 -- 172.21.15.105:0/1355777871 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5a940a3270 msgr2=0x7f5a940a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:45.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:45.597+0000 7f5ab41a3640 1 --2- 172.21.15.105:0/1355777871 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5a940a3270 0x7f5a940a5730 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5aac19b4c0 tx=0x7f5aa0009500 comp rx=0 tx=0).stop 2024-09-16T01:58:45.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:45.597+0000 7f5ab41a3640 1 -- 172.21.15.105:0/1355777871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5aac074b40 msgr2=0x7f5aac199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:45.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:45.597+0000 7f5ab41a3640 1 --2- 172.21.15.105:0/1355777871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5aac074b40 0x7f5aac199fa0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c00d950 tx=0x7f5a9c00de10 comp rx=0 tx=0).stop 2024-09-16T01:58:45.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:45.597+0000 7f5ab41a3640 1 -- 172.21.15.105:0/1355777871 shutdown_connections 2024-09-16T01:58:45.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:45.597+0000 7f5ab41a3640 1 --2- 172.21.15.105:0/1355777871 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5a940a3270 0x7f5a940a5730 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:45.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:45.597+0000 7f5ab41a3640 1 --2- 172.21.15.105:0/1355777871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5aac0761a0 0x7f5aac19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:45.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:45.597+0000 7f5ab41a3640 1 --2- 172.21.15.105:0/1355777871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5aac109a70 0x7f5aac1a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:45.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:45.597+0000 7f5ab41a3640 1 --2- 172.21.15.105:0/1355777871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5aac074b40 0x7f5aac199fa0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:45.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:45.597+0000 7f5ab41a3640 1 -- 172.21.15.105:0/1355777871 >> 172.21.15.105:0/1355777871 conn(0x7f5aac0fd390 msgr2=0x7f5aac105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:45.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:45.598+0000 7f5ab41a3640 1 -- 172.21.15.105:0/1355777871 shutdown_connections 2024-09-16T01:58:45.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:58:45.598+0000 7f5ab41a3640 1 -- 172.21.15.105:0/1355777871 wait complete. 2024-09-16T01:58:46.639 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:46 smithi105 ceph-mon[31197]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:58:46.639 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:46 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:46.639 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:46 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:46.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:46 smithi190 ceph-mon[34762]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:58:46.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:46 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:46.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:46 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:46.860 DEBUG:teuthology.orchestra.run.smithi105:osd.9> sudo journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@osd.9.service 2024-09-16T01:58:46.862 INFO:tasks.cephadm:Deploying osd.10 on smithi137 with /dev/vg_nvme/lv_1... 2024-09-16T01:58:46.862 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-16T01:58:46.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:46 smithi137 ceph-mon[34099]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:58:46.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:46 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:46.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:46 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:47.117 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T01:58:48.107 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 01:58:47 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T01:58:47.866+0000 7f330f346640 -1 osd.9 0 log_to_monitors true 2024-09-16T01:58:48.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:48 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:48.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:48 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:48.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:48 smithi137 ceph-mon[34099]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:58:48.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:48 smithi137 ceph-mon[34099]: from='osd.9 [v2:172.21.15.105:6826/3375034979,v1:172.21.15.105:6827/3375034979]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-16T01:58:48.537 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:48 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:48.537 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:48 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:48.537 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:48 smithi105 ceph-mon[31197]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:58:48.537 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:48 smithi105 ceph-mon[31197]: from='osd.9 [v2:172.21.15.105:6826/3375034979,v1:172.21.15.105:6827/3375034979]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-16T01:58:48.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:48 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:48.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:48 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:48.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:48 smithi190 ceph-mon[34762]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:58:48.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:48 smithi190 ceph-mon[34762]: from='osd.9 [v2:172.21.15.105:6826/3375034979,v1:172.21.15.105:6827/3375034979]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-16T01:58:49.332 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:49 smithi137 ceph-mon[34099]: from='osd.9 [v2:172.21.15.105:6826/3375034979,v1:172.21.15.105:6827/3375034979]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-16T01:58:49.332 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:49 smithi137 ceph-mon[34099]: osdmap e59: 10 total, 9 up, 10 in 2024-09-16T01:58:49.332 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:49 smithi137 ceph-mon[34099]: from='osd.9 [v2:172.21.15.105:6826/3375034979,v1:172.21.15.105:6827/3375034979]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-09-16T01:58:49.332 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:49 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:58:49.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:49 smithi105 ceph-mon[31197]: from='osd.9 [v2:172.21.15.105:6826/3375034979,v1:172.21.15.105:6827/3375034979]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-16T01:58:49.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:49 smithi105 ceph-mon[31197]: osdmap e59: 10 total, 9 up, 10 in 2024-09-16T01:58:49.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:49 smithi105 ceph-mon[31197]: from='osd.9 [v2:172.21.15.105:6826/3375034979,v1:172.21.15.105:6827/3375034979]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-09-16T01:58:49.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:49 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:58:49.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:49 smithi190 ceph-mon[34762]: from='osd.9 [v2:172.21.15.105:6826/3375034979,v1:172.21.15.105:6827/3375034979]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-16T01:58:49.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:49 smithi190 ceph-mon[34762]: osdmap e59: 10 total, 9 up, 10 in 2024-09-16T01:58:49.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:49 smithi190 ceph-mon[34762]: from='osd.9 [v2:172.21.15.105:6826/3375034979,v1:172.21.15.105:6827/3375034979]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-09-16T01:58:49.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:49 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:58:49.659 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-16T01:58:49.694 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi137:vg_nvme/lv_1 2024-09-16T01:58:49.986 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T01:58:50.044 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 01:58:49 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T01:58:49.545+0000 7f330b2c9640 -1 osd.9 0 waiting for initial osdmap 2024-09-16T01:58:50.045 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 01:58:49 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T01:58:49.557+0000 7f33030e9640 -1 osd.9 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T01:58:50.359 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:50 smithi137 ceph-mon[34099]: from='osd.9 [v2:172.21.15.105:6826/3375034979,v1:172.21.15.105:6827/3375034979]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-09-16T01:58:50.359 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:50 smithi137 ceph-mon[34099]: osdmap e60: 10 total, 9 up, 10 in 2024-09-16T01:58:50.360 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:50 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:58:50.360 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:50 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:58:50.360 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:50 smithi137 ceph-mon[34099]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:58:50.360 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:50 smithi137 ceph-mon[34099]: from='osd.9 [v2:172.21.15.105:6826/3375034979,v1:172.21.15.105:6827/3375034979]' entity='osd.9' 2024-09-16T01:58:50.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.471+0000 7f2382279640 1 -- 172.21.15.137:0/2919617086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f237c101cd0 msgr2=0x7f237c102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:50.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.471+0000 7f2382279640 1 --2- 172.21.15.137:0/2919617086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f237c101cd0 0x7f237c102150 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2364009900 tx=0x7f236402a410 comp rx=0 tx=0).stop 2024-09-16T01:58:50.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.471+0000 7f2382279640 1 -- 172.21.15.137:0/2919617086 shutdown_connections 2024-09-16T01:58:50.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.471+0000 7f2382279640 1 --2- 172.21.15.137:0/2919617086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f237c102690 0x7f237c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:50.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.471+0000 7f2382279640 1 --2- 172.21.15.137:0/2919617086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f237c101cd0 0x7f237c102150 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:50.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.471+0000 7f2382279640 1 --2- 172.21.15.137:0/2919617086 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f237c100ad0 0x7f237c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:50.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.471+0000 7f2382279640 1 -- 172.21.15.137:0/2919617086 >> 172.21.15.137:0/2919617086 conn(0x7f237c0fc280 msgr2=0x7f237c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:50.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.471+0000 7f2382279640 1 -- 172.21.15.137:0/2919617086 shutdown_connections 2024-09-16T01:58:50.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.471+0000 7f2382279640 1 -- 172.21.15.137:0/2919617086 wait complete. 2024-09-16T01:58:50.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.471+0000 7f2382279640 1 Processor -- start 2024-09-16T01:58:50.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.471+0000 7f2382279640 1 -- start start 2024-09-16T01:58:50.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.472+0000 7f2382279640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f237c100ad0 0x7f237c197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:50.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.472+0000 7f2382279640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f237c101cd0 0x7f237c198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:50.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.472+0000 7f2382279640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f237c102690 0x7f237c19f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:50.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.472+0000 7f2382279640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f237c10c5e0 con 0x7f237c101cd0 2024-09-16T01:58:50.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.472+0000 7f2382279640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f237c10c460 con 0x7f237c102690 2024-09-16T01:58:50.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.472+0000 7f2382279640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f237c10c760 con 0x7f237c100ad0 2024-09-16T01:58:50.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.472+0000 7f2380816640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f237c102690 0x7f237c19f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:50.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.472+0000 7f2380816640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f237c102690 0x7f237c19f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56838/0 (socket says 172.21.15.137:56838) 2024-09-16T01:58:50.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.472+0000 7f2380816640 1 -- 172.21.15.137:0/1425750432 learned_addr learned my addr 172.21.15.137:0/1425750432 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-16T01:58:50.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.472+0000 7f237bfff640 1 --2- 172.21.15.137:0/1425750432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f237c100ad0 0x7f237c197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:50.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.472+0000 7f237b7fe640 1 --2- 172.21.15.137:0/1425750432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f237c101cd0 0x7f237c198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:50.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.472+0000 7f2380816640 1 -- 172.21.15.137:0/1425750432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f237c100ad0 msgr2=0x7f237c197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:50.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.472+0000 7f2380816640 1 --2- 172.21.15.137:0/1425750432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f237c100ad0 0x7f237c197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:50.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.472+0000 7f2380816640 1 -- 172.21.15.137:0/1425750432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f237c101cd0 msgr2=0x7f237c198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:50.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.472+0000 7f2380816640 1 --2- 172.21.15.137:0/1425750432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f237c101cd0 0x7f237c198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:50.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.472+0000 7f2380816640 1 -- 172.21.15.137:0/1425750432 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f237c19fa20 con 0x7f237c102690 2024-09-16T01:58:50.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.472+0000 7f2380816640 1 --2- 172.21.15.137:0/1425750432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f237c102690 0x7f237c19f3b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f237400eaa0 tx=0x7f237400ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:50.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.473+0000 7f23797fa640 1 -- 172.21.15.137:0/1425750432 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237400cdb0 con 0x7f237c102690 2024-09-16T01:58:50.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.473+0000 7f2382279640 1 -- 172.21.15.137:0/1425750432 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f237c19fd10 con 0x7f237c102690 2024-09-16T01:58:50.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.473+0000 7f23797fa640 1 -- 172.21.15.137:0/1425750432 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2374005020 con 0x7f237c102690 2024-09-16T01:58:50.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.473+0000 7f23797fa640 1 -- 172.21.15.137:0/1425750432 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23740116d0 con 0x7f237c102690 2024-09-16T01:58:50.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.473+0000 7f2382279640 1 -- 172.21.15.137:0/1425750432 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f237c1a0170 con 0x7f237c102690 2024-09-16T01:58:50.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.474+0000 7f2382279640 1 -- 172.21.15.137:0/1425750432 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f237c1a0400 con 0x7f237c102690 2024-09-16T01:58:50.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.475+0000 7f23797fa640 1 -- 172.21.15.137:0/1425750432 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f23740051d0 con 0x7f237c102690 2024-09-16T01:58:50.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.475+0000 7f23797fa640 1 --2- 172.21.15.137:0/1425750432 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f23680a3270 0x7f23680a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:50.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.476+0000 7f23797fa640 1 -- 172.21.15.137:0/1425750432 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5981+0+0 (secure 0 0 0) 0x7f23740cf490 con 0x7f237c102690 2024-09-16T01:58:50.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.476+0000 7f237bfff640 1 --2- 172.21.15.137:0/1425750432 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f23680a3270 0x7f23680a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:50.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.476+0000 7f237bfff640 1 --2- 172.21.15.137:0/1425750432 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f23680a3270 0x7f23680a5730 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f237c101b30 tx=0x7f236c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:50.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.479+0000 7f23797fa640 1 -- 172.21.15.137:0/1425750432 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2374016030 con 0x7f237c102690 2024-09-16T01:58:50.544 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:50 smithi105 ceph-mon[31197]: from='osd.9 [v2:172.21.15.105:6826/3375034979,v1:172.21.15.105:6827/3375034979]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-09-16T01:58:50.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:50 smithi105 ceph-mon[31197]: osdmap e60: 10 total, 9 up, 10 in 2024-09-16T01:58:50.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:50 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:58:50.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:50 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:58:50.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:50 smithi105 ceph-mon[31197]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:58:50.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:50 smithi105 ceph-mon[31197]: from='osd.9 [v2:172.21.15.105:6826/3375034979,v1:172.21.15.105:6827/3375034979]' entity='osd.9' 2024-09-16T01:58:50.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:50 smithi190 ceph-mon[34762]: from='osd.9 [v2:172.21.15.105:6826/3375034979,v1:172.21.15.105:6827/3375034979]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-09-16T01:58:50.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:50 smithi190 ceph-mon[34762]: osdmap e60: 10 total, 9 up, 10 in 2024-09-16T01:58:50.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:50 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:58:50.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:50 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:58:50.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:50 smithi190 ceph-mon[34762]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:58:50.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:50 smithi190 ceph-mon[34762]: from='osd.9 [v2:172.21.15.105:6826/3375034979,v1:172.21.15.105:6827/3375034979]' entity='osd.9' 2024-09-16T01:58:50.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:58:50.656+0000 7f2382279640 1 -- 172.21.15.137:0/1425750432 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f237c1a05d0 con 0x7f23680a3270 2024-09-16T01:58:51.321 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:51 smithi137 ceph-mon[34099]: purged_snaps scrub starts 2024-09-16T01:58:51.321 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:51 smithi137 ceph-mon[34099]: purged_snaps scrub ok 2024-09-16T01:58:51.322 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:51 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:58:51.322 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:51 smithi137 ceph-mon[34099]: osd.9 [v2:172.21.15.105:6826/3375034979,v1:172.21.15.105:6827/3375034979] boot 2024-09-16T01:58:51.322 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:51 smithi137 ceph-mon[34099]: osdmap e61: 10 total, 10 up, 10 in 2024-09-16T01:58:51.322 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:51 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:58:51.322 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:51 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:51.322 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:51 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:51.322 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:51 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:58:51.322 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:51 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:58:51.322 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:51 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:58:51.322 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:51 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:58:51.322 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:51 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:51.322 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:51 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:51.322 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:51 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:58:51.323 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:51 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:51.323 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:51 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:58:51.323 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:51 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:58:51.323 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:51 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:51.323 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:51 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:58:51.323 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:51 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:51.323 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:51 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:58:51.323 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:51 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:51.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:51 smithi105 ceph-mon[31197]: purged_snaps scrub starts 2024-09-16T01:58:51.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:51 smithi105 ceph-mon[31197]: purged_snaps scrub ok 2024-09-16T01:58:51.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:51 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:58:51.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:51 smithi105 ceph-mon[31197]: osd.9 [v2:172.21.15.105:6826/3375034979,v1:172.21.15.105:6827/3375034979] boot 2024-09-16T01:58:51.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:51 smithi105 ceph-mon[31197]: osdmap e61: 10 total, 10 up, 10 in 2024-09-16T01:58:51.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:51 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:58:51.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:51 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:51.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:51 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:51.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:51 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:58:51.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:51 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:58:51.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:51 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:58:51.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:51 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:58:51.546 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:51 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:51.546 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:51 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:51.546 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:51 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:58:51.546 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:51 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:51.546 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:51 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:58:51.546 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:51 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:58:51.546 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:51 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:51.546 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:51 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:58:51.546 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:51 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:51.546 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:51 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:58:51.546 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:51 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:51.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:51 smithi190 ceph-mon[34762]: purged_snaps scrub starts 2024-09-16T01:58:51.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:51 smithi190 ceph-mon[34762]: purged_snaps scrub ok 2024-09-16T01:58:51.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:51 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:58:51.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:51 smithi190 ceph-mon[34762]: osd.9 [v2:172.21.15.105:6826/3375034979,v1:172.21.15.105:6827/3375034979] boot 2024-09-16T01:58:51.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:51 smithi190 ceph-mon[34762]: osdmap e61: 10 total, 10 up, 10 in 2024-09-16T01:58:51.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:51 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:58:51.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:51 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:51.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:51 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:51.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:51 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:58:51.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:51 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:58:51.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:51 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:58:51.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:51 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:58:51.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:51 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:51.594 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:51 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:51.594 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:51 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:58:51.594 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:51 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:51.594 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:51 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:58:51.594 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:51 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:58:51.594 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:51 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:51.594 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:51 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:58:51.594 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:51 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:51.594 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:51 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:58:51.594 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:51 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:58:52.353 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:52 smithi137 ceph-mon[34099]: Detected new or changed devices on smithi105 2024-09-16T01:58:52.353 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:52 smithi137 ceph-mon[34099]: Adjusting osd_memory_target on smithi105 to 4272M 2024-09-16T01:58:52.353 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:52 smithi137 ceph-mon[34099]: from='client.24350 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:58:52.353 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:52 smithi137 ceph-mon[34099]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:58:52.353 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:52 smithi137 ceph-mon[34099]: osdmap e62: 10 total, 10 up, 10 in 2024-09-16T01:58:52.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:52 smithi105 ceph-mon[31197]: Detected new or changed devices on smithi105 2024-09-16T01:58:52.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:52 smithi105 ceph-mon[31197]: Adjusting osd_memory_target on smithi105 to 4272M 2024-09-16T01:58:52.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:52 smithi105 ceph-mon[31197]: from='client.24350 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:58:52.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:52 smithi105 ceph-mon[31197]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:58:52.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:52 smithi105 ceph-mon[31197]: osdmap e62: 10 total, 10 up, 10 in 2024-09-16T01:58:52.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:52 smithi190 ceph-mon[34762]: Detected new or changed devices on smithi105 2024-09-16T01:58:52.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:52 smithi190 ceph-mon[34762]: Adjusting osd_memory_target on smithi105 to 4272M 2024-09-16T01:58:52.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:52 smithi190 ceph-mon[34762]: from='client.24350 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:58:52.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:52 smithi190 ceph-mon[34762]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:58:52.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:52 smithi190 ceph-mon[34762]: osdmap e62: 10 total, 10 up, 10 in 2024-09-16T01:58:53.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:53 smithi137 ceph-mon[34099]: from='client.? 172.21.15.137:0/178941327' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad145d3e-a6e2-4dd7-bea3-bcd01eb3413e"}]: dispatch 2024-09-16T01:58:53.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:53 smithi137 ceph-mon[34099]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad145d3e-a6e2-4dd7-bea3-bcd01eb3413e"}]: dispatch 2024-09-16T01:58:53.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:53 smithi137 ceph-mon[34099]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad145d3e-a6e2-4dd7-bea3-bcd01eb3413e"}]': finished 2024-09-16T01:58:53.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:53 smithi137 ceph-mon[34099]: osdmap e63: 11 total, 10 up, 11 in 2024-09-16T01:58:53.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:53 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T01:58:53.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:53 smithi105 ceph-mon[31197]: from='client.? 172.21.15.137:0/178941327' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad145d3e-a6e2-4dd7-bea3-bcd01eb3413e"}]: dispatch 2024-09-16T01:58:53.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:53 smithi105 ceph-mon[31197]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad145d3e-a6e2-4dd7-bea3-bcd01eb3413e"}]: dispatch 2024-09-16T01:58:53.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:53 smithi105 ceph-mon[31197]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad145d3e-a6e2-4dd7-bea3-bcd01eb3413e"}]': finished 2024-09-16T01:58:53.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:53 smithi105 ceph-mon[31197]: osdmap e63: 11 total, 10 up, 11 in 2024-09-16T01:58:53.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:53 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T01:58:53.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:53 smithi190 ceph-mon[34762]: from='client.? 172.21.15.137:0/178941327' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad145d3e-a6e2-4dd7-bea3-bcd01eb3413e"}]: dispatch 2024-09-16T01:58:53.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:53 smithi190 ceph-mon[34762]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad145d3e-a6e2-4dd7-bea3-bcd01eb3413e"}]: dispatch 2024-09-16T01:58:53.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:53 smithi190 ceph-mon[34762]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad145d3e-a6e2-4dd7-bea3-bcd01eb3413e"}]': finished 2024-09-16T01:58:53.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:53 smithi190 ceph-mon[34762]: osdmap e63: 11 total, 10 up, 11 in 2024-09-16T01:58:53.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:53 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T01:58:54.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:54 smithi137 ceph-mon[34099]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:58:54.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:54 smithi137 ceph-mon[34099]: from='client.? 172.21.15.137:0/1837820220' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:58:54.544 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:54 smithi105 ceph-mon[31197]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:58:54.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:54 smithi105 ceph-mon[31197]: from='client.? 172.21.15.137:0/1837820220' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:58:54.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:54 smithi190 ceph-mon[34762]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:58:54.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:54 smithi190 ceph-mon[34762]: from='client.? 172.21.15.137:0/1837820220' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:58:56.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:56 smithi105 ceph-mon[31197]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:58:56.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:56 smithi190 ceph-mon[34762]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:58:56.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:56 smithi137 ceph-mon[34099]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:58:58.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:58:58 smithi105 ceph-mon[31197]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:58:58.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:58:58 smithi190 ceph-mon[34762]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:58:58.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:58:58 smithi137 ceph-mon[34099]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:59:00.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:00 smithi137 ceph-mon[34099]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:59:00.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:00 smithi105 ceph-mon[31197]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:59:00.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:00 smithi190 ceph-mon[34762]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:59:02.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:02 smithi137 ceph-mon[34099]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:59:02.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:02 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-16T01:59:02.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:02 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:02.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:02 smithi105 ceph-mon[31197]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:59:02.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:02 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-16T01:59:02.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:02 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:02.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:02 smithi190 ceph-mon[34762]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:59:02.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:02 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-16T01:59:02.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:02 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:03.615 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:03 smithi137 ceph-mon[34099]: Deploying daemon osd.10 on smithi137 2024-09-16T01:59:03.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:03 smithi105 ceph-mon[31197]: Deploying daemon osd.10 on smithi137 2024-09-16T01:59:03.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:03 smithi190 ceph-mon[34762]: Deploying daemon osd.10 on smithi137 2024-09-16T01:59:04.595 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:04 smithi137 ceph-mon[34099]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:59:04.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:04 smithi105 ceph-mon[31197]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:59:04.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:04 smithi190 ceph-mon[34762]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:59:06.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:06 smithi105 ceph-mon[31197]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:59:06.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:06 smithi190 ceph-mon[34762]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:59:06.887 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:06 smithi137 ceph-mon[34099]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:59:08.537 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:08 smithi137 ceph-mon[34099]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:59:08.537 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:08 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:59:08.537 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:08 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:08.538 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:08 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:08.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:08 smithi105 ceph-mon[31197]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:59:08.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:08 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:59:08.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:08 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:08.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:08 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:08.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:08 smithi190 ceph-mon[34762]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:59:08.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:08 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:59:08.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:08 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:08.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:08 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:10.648 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:10 smithi137 ceph-mon[34099]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:59:10.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:10 smithi105 ceph-mon[31197]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:59:10.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:10 smithi190 ceph-mon[34762]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:59:11.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:59:11.354+0000 7f23797fa640 1 -- 172.21.15.137:0/1425750432 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f237c1a05d0 con 0x7f23680a3270 2024-09-16T01:59:11.356 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 10 on host 'smithi137' 2024-09-16T01:59:11.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:59:11.357+0000 7f2382279640 1 -- 172.21.15.137:0/1425750432 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f23680a3270 msgr2=0x7f23680a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:11.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:59:11.357+0000 7f2382279640 1 --2- 172.21.15.137:0/1425750432 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f23680a3270 0x7f23680a5730 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f237c101b30 tx=0x7f236c005f90 comp rx=0 tx=0).stop 2024-09-16T01:59:11.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:59:11.357+0000 7f2382279640 1 -- 172.21.15.137:0/1425750432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f237c102690 msgr2=0x7f237c19f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:11.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:59:11.357+0000 7f2382279640 1 --2- 172.21.15.137:0/1425750432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f237c102690 0x7f237c19f3b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f237400eaa0 tx=0x7f237400ef60 comp rx=0 tx=0).stop 2024-09-16T01:59:11.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:59:11.357+0000 7f2382279640 1 -- 172.21.15.137:0/1425750432 shutdown_connections 2024-09-16T01:59:11.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:59:11.357+0000 7f2382279640 1 --2- 172.21.15.137:0/1425750432 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f23680a3270 0x7f23680a5730 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:11.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:59:11.357+0000 7f2382279640 1 --2- 172.21.15.137:0/1425750432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f237c102690 0x7f237c19f3b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:11.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:59:11.357+0000 7f2382279640 1 --2- 172.21.15.137:0/1425750432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f237c101cd0 0x7f237c198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:11.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:59:11.357+0000 7f2382279640 1 --2- 172.21.15.137:0/1425750432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f237c100ad0 0x7f237c197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:11.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:59:11.357+0000 7f2382279640 1 -- 172.21.15.137:0/1425750432 >> 172.21.15.137:0/1425750432 conn(0x7f237c0fc280 msgr2=0x7f237c069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:11.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:59:11.357+0000 7f2382279640 1 -- 172.21.15.137:0/1425750432 shutdown_connections 2024-09-16T01:59:11.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-16T01:59:11.357+0000 7f2382279640 1 -- 172.21.15.137:0/1425750432 wait complete. 2024-09-16T01:59:11.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:11 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:11.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:11 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:11.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:11 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:11.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:11 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:59:11.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:11 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:11.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:11 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:59:11.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:11 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:11.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:11 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:11.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:11 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:11.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:11 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:11.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:11 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:11.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:11 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:59:11.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:11 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:11.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:11 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:59:11.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:11 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:11.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:11 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:12.013 DEBUG:teuthology.orchestra.run.smithi137:osd.10> sudo journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@osd.10.service 2024-09-16T01:59:12.016 INFO:tasks.cephadm:Deploying osd.11 on smithi190 with /dev/vg_nvme/lv_1... 2024-09-16T01:59:12.016 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-16T01:59:12.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:11 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:12.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:11 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:12.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:11 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:12.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:11 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:59:12.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:11 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:12.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:11 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:59:12.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:11 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:12.046 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:11 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:12.268 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T01:59:12.673 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:12 smithi190 ceph-mon[34762]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:59:12.915 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:12 smithi137 ceph-mon[34099]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:59:13.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:12 smithi105 ceph-mon[31197]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:59:14.171 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 01:59:13 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T01:59:13.889+0000 7fa7f12d8640 -1 osd.10 0 log_to_monitors true 2024-09-16T01:59:14.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:14 smithi190 ceph-mon[34762]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:59:14.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:14 smithi190 ceph-mon[34762]: from='osd.10 [v2:172.21.15.137:6824/318562234,v1:172.21.15.137:6825/318562234]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T01:59:14.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:14 smithi190 ceph-mon[34762]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T01:59:14.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:14 smithi137 ceph-mon[34099]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:59:14.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:14 smithi137 ceph-mon[34099]: from='osd.10 [v2:172.21.15.137:6824/318562234,v1:172.21.15.137:6825/318562234]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T01:59:14.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:14 smithi137 ceph-mon[34099]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T01:59:15.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:14 smithi105 ceph-mon[31197]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:59:15.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:14 smithi105 ceph-mon[31197]: from='osd.10 [v2:172.21.15.137:6824/318562234,v1:172.21.15.137:6825/318562234]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T01:59:15.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:14 smithi105 ceph-mon[31197]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T01:59:15.073 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T01:59:15.102 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi190:vg_nvme/lv_1 2024-09-16T01:59:15.352 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T01:59:15.782 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:15 smithi190 ceph-mon[34762]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-16T01:59:15.782 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:15 smithi190 ceph-mon[34762]: osdmap e64: 11 total, 10 up, 11 in 2024-09-16T01:59:15.783 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:15 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T01:59:15.783 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:15 smithi190 ceph-mon[34762]: from='osd.10 [v2:172.21.15.137:6824/318562234,v1:172.21.15.137:6825/318562234]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-09-16T01:59:15.783 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:15 smithi190 ceph-mon[34762]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-09-16T01:59:15.783 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:15 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:15.783 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:15 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:15.783 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:15 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:59:15.783 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:15 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:59:15.783 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:15 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:59:15.783 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:15 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:59:15.783 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:15 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:15.783 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:15 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:15.783 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:15 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:59:15.783 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:15 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:15.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:15 smithi105 ceph-mon[31197]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-16T01:59:15.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:15 smithi105 ceph-mon[31197]: osdmap e64: 11 total, 10 up, 11 in 2024-09-16T01:59:15.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:15 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T01:59:15.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:15 smithi105 ceph-mon[31197]: from='osd.10 [v2:172.21.15.137:6824/318562234,v1:172.21.15.137:6825/318562234]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-09-16T01:59:15.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:15 smithi105 ceph-mon[31197]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-09-16T01:59:15.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:15 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:15.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:15 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:15.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:15 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:59:15.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:15 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:59:15.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:15 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:59:15.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:15 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:59:15.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:15 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:15.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:15 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:15.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:15 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:59:15.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:15 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:15.882 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.863+0000 7f9d0839c640 1 -- 172.21.15.190:0/4113836065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d00101c00 msgr2=0x7f9d00102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:15.882 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.863+0000 7f9d0839c640 1 --2- 172.21.15.190:0/4113836065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d00101c00 0x7f9d00102000 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4009900 tx=0x7f9cf402a410 comp rx=0 tx=0).stop 2024-09-16T01:59:15.882 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.863+0000 7f9d0839c640 1 -- 172.21.15.190:0/4113836065 shutdown_connections 2024-09-16T01:59:15.882 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.863+0000 7f9d0839c640 1 --2- 172.21.15.190:0/4113836065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d001037c0 0x7f9d0010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:15.883 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.863+0000 7f9d0839c640 1 --2- 172.21.15.190:0/4113836065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d00102e00 0x7f9d00103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:15.883 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.863+0000 7f9d0839c640 1 --2- 172.21.15.190:0/4113836065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d00101c00 0x7f9d00102000 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:15.883 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.863+0000 7f9d0839c640 1 -- 172.21.15.190:0/4113836065 >> 172.21.15.190:0/4113836065 conn(0x7f9d000fd390 msgr2=0x7f9d000ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:15.883 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.863+0000 7f9d0839c640 1 -- 172.21.15.190:0/4113836065 shutdown_connections 2024-09-16T01:59:15.883 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.863+0000 7f9d0839c640 1 -- 172.21.15.190:0/4113836065 wait complete. 2024-09-16T01:59:15.883 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.864+0000 7f9d0839c640 1 Processor -- start 2024-09-16T01:59:15.883 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.864+0000 7f9d0839c640 1 -- start start 2024-09-16T01:59:15.883 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.864+0000 7f9d0839c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d00101c00 0x7f9d00197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:15.883 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.864+0000 7f9d0839c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d00102e00 0x7f9d001983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:15.883 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.864+0000 7f9d0839c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d001037c0 0x7f9d0019f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:15.883 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.864+0000 7f9d0839c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0010d790 con 0x7f9d001037c0 2024-09-16T01:59:15.883 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.864+0000 7f9d0839c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0010d610 con 0x7f9d00101c00 2024-09-16T01:59:15.883 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.864+0000 7f9d0839c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0010d910 con 0x7f9d00102e00 2024-09-16T01:59:15.884 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.864+0000 7f9d05937640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d00102e00 0x7f9d001983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:15.884 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.864+0000 7f9d05937640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d00102e00 0x7f9d001983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:44076/0 (socket says 172.21.15.190:44076) 2024-09-16T01:59:15.884 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.864+0000 7f9d05937640 1 -- 172.21.15.190:0/659342069 learned_addr learned my addr 172.21.15.190:0/659342069 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-09-16T01:59:15.884 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.864+0000 7f9d06138640 1 --2- 172.21.15.190:0/659342069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d00101c00 0x7f9d00197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:15.884 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.864+0000 7f9d06939640 1 --2- 172.21.15.190:0/659342069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d001037c0 0x7f9d0019f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:15.884 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.865+0000 7f9d05937640 1 -- 172.21.15.190:0/659342069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d00101c00 msgr2=0x7f9d00197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:15.884 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.865+0000 7f9d05937640 1 --2- 172.21.15.190:0/659342069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d00101c00 0x7f9d00197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:15.884 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.865+0000 7f9d05937640 1 -- 172.21.15.190:0/659342069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d001037c0 msgr2=0x7f9d0019f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:15.884 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.865+0000 7f9d05937640 1 --2- 172.21.15.190:0/659342069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d001037c0 0x7f9d0019f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:15.884 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.865+0000 7f9d05937640 1 -- 172.21.15.190:0/659342069 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d0019fad0 con 0x7f9d00102e00 2024-09-16T01:59:15.884 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.865+0000 7f9d05937640 1 --2- 172.21.15.190:0/659342069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d00102e00 0x7f9d001983e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc00d950 tx=0x7f9cfc00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:15.884 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.865+0000 7f9cf37fe640 1 -- 172.21.15.190:0/659342069 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc017030 con 0x7f9d00102e00 2024-09-16T01:59:15.884 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.865+0000 7f9d0839c640 1 -- 172.21.15.190:0/659342069 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d0019fdc0 con 0x7f9d00102e00 2024-09-16T01:59:15.885 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.865+0000 7f9cf37fe640 1 -- 172.21.15.190:0/659342069 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9cfc004020 con 0x7f9d00102e00 2024-09-16T01:59:15.885 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.865+0000 7f9d0839c640 1 -- 172.21.15.190:0/659342069 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d001a0280 con 0x7f9d00102e00 2024-09-16T01:59:15.885 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.865+0000 7f9cf37fe640 1 -- 172.21.15.190:0/659342069 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc0055c0 con 0x7f9d00102e00 2024-09-16T01:59:15.885 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.866+0000 7f9d0839c640 1 -- 172.21.15.190:0/659342069 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d001090a0 con 0x7f9d00102e00 2024-09-16T01:59:15.885 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.867+0000 7f9cf37fe640 1 -- 172.21.15.190:0/659342069 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f9cfc009a30 con 0x7f9d00102e00 2024-09-16T01:59:15.885 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.867+0000 7f9cf37fe640 1 --2- 172.21.15.190:0/659342069 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f9cec0a3270 0x7f9cec0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:15.885 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.868+0000 7f9cf37fe640 1 -- 172.21.15.190:0/659342069 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6414+0+0 (secure 0 0 0) 0x7f9cfc0ce950 con 0x7f9d00102e00 2024-09-16T01:59:15.885 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.868+0000 7f9d06138640 1 --2- 172.21.15.190:0/659342069 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f9cec0a3270 0x7f9cec0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:15.885 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.868+0000 7f9d06138640 1 --2- 172.21.15.190:0/659342069 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f9cec0a3270 0x7f9cec0a5730 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4004770 tx=0x7f9cf4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:15.885 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:15.872+0000 7f9cf37fe640 1 -- 172.21.15.190:0/659342069 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9cfc022020 con 0x7f9d00102e00 2024-09-16T01:59:15.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:15 smithi137 ceph-mon[34099]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-16T01:59:15.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:15 smithi137 ceph-mon[34099]: osdmap e64: 11 total, 10 up, 11 in 2024-09-16T01:59:15.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:15 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T01:59:15.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:15 smithi137 ceph-mon[34099]: from='osd.10 [v2:172.21.15.137:6824/318562234,v1:172.21.15.137:6825/318562234]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-09-16T01:59:15.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:15 smithi137 ceph-mon[34099]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-09-16T01:59:15.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:15 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:15.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:15 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:15.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:15 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:59:15.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:15 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:59:15.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:15 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:59:15.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:15 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:59:15.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:15 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:15.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:15 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:15.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:15 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:59:15.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:15 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:16.048 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:16.046+0000 7f9d0839c640 1 -- 172.21.15.190:0/659342069 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f9d00062630 con 0x7f9cec0a3270 2024-09-16T01:59:16.421 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 01:59:15 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T01:59:15.978+0000 7fa7ed25b640 -1 osd.10 0 waiting for initial osdmap 2024-09-16T01:59:16.421 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 01:59:15 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T01:59:15.988+0000 7fa7e507b640 -1 osd.10 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T01:59:16.710 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:16 smithi190 ceph-mon[34762]: Detected new or changed devices on smithi137 2024-09-16T01:59:16.710 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:16 smithi190 ceph-mon[34762]: Adjusting osd_memory_target on smithi137 to 4272M 2024-09-16T01:59:16.710 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:16 smithi190 ceph-mon[34762]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:59:16.710 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:16 smithi190 ceph-mon[34762]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-09-16T01:59:16.710 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:16 smithi190 ceph-mon[34762]: osdmap e65: 11 total, 10 up, 11 in 2024-09-16T01:59:16.710 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:16 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T01:59:16.711 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:16 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T01:59:16.711 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:16 smithi190 ceph-mon[34762]: from='osd.10 ' entity='osd.10' 2024-09-16T01:59:16.711 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:16 smithi190 ceph-mon[34762]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:59:16.711 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:16 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:59:16.711 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:16 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:59:16.711 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:16 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:16.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:16 smithi137 ceph-mon[34099]: Detected new or changed devices on smithi137 2024-09-16T01:59:16.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:16 smithi137 ceph-mon[34099]: Adjusting osd_memory_target on smithi137 to 4272M 2024-09-16T01:59:16.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:16 smithi137 ceph-mon[34099]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:59:16.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:16 smithi137 ceph-mon[34099]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-09-16T01:59:16.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:16 smithi137 ceph-mon[34099]: osdmap e65: 11 total, 10 up, 11 in 2024-09-16T01:59:16.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:16 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T01:59:16.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:16 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T01:59:16.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:16 smithi137 ceph-mon[34099]: from='osd.10 ' entity='osd.10' 2024-09-16T01:59:16.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:16 smithi137 ceph-mon[34099]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:59:16.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:16 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:59:16.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:16 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:59:16.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:16 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:17.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:16 smithi105 ceph-mon[31197]: Detected new or changed devices on smithi137 2024-09-16T01:59:17.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:16 smithi105 ceph-mon[31197]: Adjusting osd_memory_target on smithi137 to 4272M 2024-09-16T01:59:17.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:16 smithi105 ceph-mon[31197]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:59:17.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:16 smithi105 ceph-mon[31197]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-09-16T01:59:17.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:16 smithi105 ceph-mon[31197]: osdmap e65: 11 total, 10 up, 11 in 2024-09-16T01:59:17.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:16 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T01:59:17.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:16 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T01:59:17.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:16 smithi105 ceph-mon[31197]: from='osd.10 ' entity='osd.10' 2024-09-16T01:59:17.046 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:16 smithi105 ceph-mon[31197]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:59:17.046 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:16 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:59:17.046 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:16 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:59:17.046 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:16 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:17.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:17 smithi190 ceph-mon[34762]: purged_snaps scrub starts 2024-09-16T01:59:17.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:17 smithi190 ceph-mon[34762]: purged_snaps scrub ok 2024-09-16T01:59:17.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:17 smithi190 ceph-mon[34762]: osd.10 [v2:172.21.15.137:6824/318562234,v1:172.21.15.137:6825/318562234] boot 2024-09-16T01:59:17.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:17 smithi190 ceph-mon[34762]: osdmap e66: 11 total, 11 up, 11 in 2024-09-16T01:59:17.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:17 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T01:59:17.844 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:17 smithi190 ceph-mon[34762]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:59:17.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:17 smithi137 ceph-mon[34099]: purged_snaps scrub starts 2024-09-16T01:59:17.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:17 smithi137 ceph-mon[34099]: purged_snaps scrub ok 2024-09-16T01:59:17.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:17 smithi137 ceph-mon[34099]: osd.10 [v2:172.21.15.137:6824/318562234,v1:172.21.15.137:6825/318562234] boot 2024-09-16T01:59:17.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:17 smithi137 ceph-mon[34099]: osdmap e66: 11 total, 11 up, 11 in 2024-09-16T01:59:17.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:17 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T01:59:17.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:17 smithi137 ceph-mon[34099]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:59:18.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:17 smithi105 ceph-mon[31197]: purged_snaps scrub starts 2024-09-16T01:59:18.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:17 smithi105 ceph-mon[31197]: purged_snaps scrub ok 2024-09-16T01:59:18.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:17 smithi105 ceph-mon[31197]: osd.10 [v2:172.21.15.137:6824/318562234,v1:172.21.15.137:6825/318562234] boot 2024-09-16T01:59:18.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:17 smithi105 ceph-mon[31197]: osdmap e66: 11 total, 11 up, 11 in 2024-09-16T01:59:18.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:17 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T01:59:18.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:17 smithi105 ceph-mon[31197]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:59:18.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:18 smithi190 ceph-mon[34762]: osdmap e67: 11 total, 11 up, 11 in 2024-09-16T01:59:18.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:18 smithi137 ceph-mon[34099]: osdmap e67: 11 total, 11 up, 11 in 2024-09-16T01:59:19.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:18 smithi105 ceph-mon[31197]: osdmap e67: 11 total, 11 up, 11 in 2024-09-16T01:59:19.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:19 smithi190 ceph-mon[34762]: osdmap e68: 11 total, 11 up, 11 in 2024-09-16T01:59:19.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:19 smithi190 ceph-mon[34762]: from='client.? 172.21.15.190:0/3802401961' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d44bced-3a8b-4763-a93e-0ea34acb4cb4"}]: dispatch 2024-09-16T01:59:19.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:19 smithi190 ceph-mon[34762]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d44bced-3a8b-4763-a93e-0ea34acb4cb4"}]: dispatch 2024-09-16T01:59:19.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:19 smithi190 ceph-mon[34762]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d44bced-3a8b-4763-a93e-0ea34acb4cb4"}]': finished 2024-09-16T01:59:19.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:19 smithi190 ceph-mon[34762]: osdmap e69: 12 total, 11 up, 12 in 2024-09-16T01:59:19.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:19 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:59:19.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:19 smithi190 ceph-mon[34762]: from='client.? 172.21.15.190:0/456081086' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:59:19.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:19 smithi190 ceph-mon[34762]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-16T01:59:19.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:19 smithi137 ceph-mon[34099]: osdmap e68: 11 total, 11 up, 11 in 2024-09-16T01:59:19.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:19 smithi137 ceph-mon[34099]: from='client.? 172.21.15.190:0/3802401961' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d44bced-3a8b-4763-a93e-0ea34acb4cb4"}]: dispatch 2024-09-16T01:59:19.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:19 smithi137 ceph-mon[34099]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d44bced-3a8b-4763-a93e-0ea34acb4cb4"}]: dispatch 2024-09-16T01:59:19.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:19 smithi137 ceph-mon[34099]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d44bced-3a8b-4763-a93e-0ea34acb4cb4"}]': finished 2024-09-16T01:59:19.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:19 smithi137 ceph-mon[34099]: osdmap e69: 12 total, 11 up, 12 in 2024-09-16T01:59:19.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:19 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:59:19.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:19 smithi137 ceph-mon[34099]: from='client.? 172.21.15.190:0/456081086' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:59:19.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:19 smithi137 ceph-mon[34099]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-16T01:59:20.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:19 smithi105 ceph-mon[31197]: osdmap e68: 11 total, 11 up, 11 in 2024-09-16T01:59:20.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:19 smithi105 ceph-mon[31197]: from='client.? 172.21.15.190:0/3802401961' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d44bced-3a8b-4763-a93e-0ea34acb4cb4"}]: dispatch 2024-09-16T01:59:20.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:19 smithi105 ceph-mon[31197]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d44bced-3a8b-4763-a93e-0ea34acb4cb4"}]: dispatch 2024-09-16T01:59:20.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:19 smithi105 ceph-mon[31197]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d44bced-3a8b-4763-a93e-0ea34acb4cb4"}]': finished 2024-09-16T01:59:20.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:19 smithi105 ceph-mon[31197]: osdmap e69: 12 total, 11 up, 12 in 2024-09-16T01:59:20.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:19 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:59:20.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:19 smithi105 ceph-mon[31197]: from='client.? 172.21.15.190:0/456081086' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:59:20.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:19 smithi105 ceph-mon[31197]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-16T01:59:21.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:20 smithi105 ceph-mon[31197]: osdmap e70: 12 total, 11 up, 12 in 2024-09-16T01:59:21.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:20 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:59:21.048 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:20 smithi137 ceph-mon[34099]: osdmap e70: 12 total, 11 up, 12 in 2024-09-16T01:59:21.048 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:20 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:59:21.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:20 smithi190 ceph-mon[34762]: osdmap e70: 12 total, 11 up, 12 in 2024-09-16T01:59:21.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:20 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:59:22.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:21 smithi190 ceph-mon[34762]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-16T01:59:22.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:21 smithi137 ceph-mon[34099]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-16T01:59:22.294 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:21 smithi105 ceph-mon[31197]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-16T01:59:24.751 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:24 smithi190 ceph-mon[34762]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-09-16T01:59:24.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:24 smithi105 ceph-mon[31197]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-09-16T01:59:24.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:24 smithi137 ceph-mon[34099]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-09-16T01:59:26.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:26 smithi190 ceph-mon[34762]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-16T01:59:26.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:26 smithi105 ceph-mon[31197]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-16T01:59:26.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:26 smithi137 ceph-mon[34099]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-16T01:59:28.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:28 smithi105 ceph-mon[31197]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-16T01:59:28.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:28 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-16T01:59:28.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:28 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:28.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:28 smithi190 ceph-mon[34762]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-16T01:59:28.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:28 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-16T01:59:28.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:28 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:28.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:28 smithi137 ceph-mon[34099]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-16T01:59:28.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:28 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-16T01:59:28.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:28 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:29.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:29 smithi190 ceph-mon[34762]: Deploying daemon osd.11 on smithi190 2024-09-16T01:59:29.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:29 smithi105 ceph-mon[31197]: Deploying daemon osd.11 on smithi190 2024-09-16T01:59:29.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:29 smithi137 ceph-mon[34099]: Deploying daemon osd.11 on smithi190 2024-09-16T01:59:30.550 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:30 smithi190 ceph-mon[34762]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T01:59:30.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:30 smithi105 ceph-mon[31197]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T01:59:30.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:30 smithi137 ceph-mon[34099]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T01:59:32.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:32 smithi105 ceph-mon[31197]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-16T01:59:32.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:32 smithi190 ceph-mon[34762]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-16T01:59:32.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:32 smithi137 ceph-mon[34099]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-16T01:59:34.564 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:34 smithi190 ceph-mon[34762]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-16T01:59:34.564 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:34 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:59:34.564 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:34 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:34.564 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:34 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:34.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:34 smithi105 ceph-mon[31197]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-16T01:59:34.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:34 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:59:34.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:34 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:34.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:34 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:34.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:34 smithi137 ceph-mon[34099]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-16T01:59:34.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:34 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:59:34.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:34 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:34.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:34 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:36.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:36 smithi105 ceph-mon[31197]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-16T01:59:36.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:36 smithi190 ceph-mon[34762]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-16T01:59:36.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:36 smithi137 ceph-mon[34099]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-16T01:59:37.072 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 11 on host 'smithi190' 2024-09-16T01:59:37.072 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:37.070+0000 7f9cf37fe640 1 -- 172.21.15.190:0/659342069 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f9d00062630 con 0x7f9cec0a3270 2024-09-16T01:59:37.074 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:37.072+0000 7f9d0839c640 1 -- 172.21.15.190:0/659342069 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f9cec0a3270 msgr2=0x7f9cec0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:37.074 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:37.072+0000 7f9d0839c640 1 --2- 172.21.15.190:0/659342069 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f9cec0a3270 0x7f9cec0a5730 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4004770 tx=0x7f9cf4000bc0 comp rx=0 tx=0).stop 2024-09-16T01:59:37.074 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:37.072+0000 7f9d0839c640 1 -- 172.21.15.190:0/659342069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d00102e00 msgr2=0x7f9d001983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:37.074 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:37.072+0000 7f9d0839c640 1 --2- 172.21.15.190:0/659342069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d00102e00 0x7f9d001983e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc00d950 tx=0x7f9cfc00de10 comp rx=0 tx=0).stop 2024-09-16T01:59:37.074 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:37.072+0000 7f9d0839c640 1 -- 172.21.15.190:0/659342069 shutdown_connections 2024-09-16T01:59:37.074 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:37.072+0000 7f9d0839c640 1 --2- 172.21.15.190:0/659342069 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f9cec0a3270 0x7f9cec0a5730 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:37.075 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:37.072+0000 7f9d0839c640 1 --2- 172.21.15.190:0/659342069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d00102e00 0x7f9d001983e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:37.075 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:37.072+0000 7f9d0839c640 1 --2- 172.21.15.190:0/659342069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d001037c0 0x7f9d0019f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:37.075 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:37.072+0000 7f9d0839c640 1 --2- 172.21.15.190:0/659342069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d00101c00 0x7f9d00197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:37.075 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:37.072+0000 7f9d0839c640 1 -- 172.21.15.190:0/659342069 >> 172.21.15.190:0/659342069 conn(0x7f9d000fd390 msgr2=0x7f9d00106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:37.075 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:37.072+0000 7f9d0839c640 1 -- 172.21.15.190:0/659342069 shutdown_connections 2024-09-16T01:59:37.075 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:37.072+0000 7f9d0839c640 1 -- 172.21.15.190:0/659342069 wait complete. 2024-09-16T01:59:38.107 DEBUG:teuthology.orchestra.run.smithi190:osd.11> sudo journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@osd.11.service 2024-09-16T01:59:38.109 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-16T01:59:38.109 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-16T01:59:38.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:38 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:38.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:38 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:38.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:38 smithi190 ceph-mon[34762]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-16T01:59:38.366 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:59:38.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:38 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:38.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:38 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:38.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:38 smithi137 ceph-mon[34099]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-16T01:59:38.486 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:38 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:38.486 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:38 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:38.486 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:38 smithi105 ceph-mon[31197]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-16T01:59:38.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.935+0000 7f5053bde640 1 -- 172.21.15.105:0/2456236508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f504c101cd0 msgr2=0x7f504c102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:38.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.935+0000 7f5053bde640 1 --2- 172.21.15.105:0/2456236508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f504c101cd0 0x7f504c102150 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f503c009900 tx=0x7f503c02a410 comp rx=0 tx=0).stop 2024-09-16T01:59:38.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.935+0000 7f5053bde640 1 -- 172.21.15.105:0/2456236508 shutdown_connections 2024-09-16T01:59:38.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.935+0000 7f5053bde640 1 --2- 172.21.15.105:0/2456236508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f504c102690 0x7f504c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:38.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.935+0000 7f5053bde640 1 --2- 172.21.15.105:0/2456236508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f504c101cd0 0x7f504c102150 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:38.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.935+0000 7f5053bde640 1 --2- 172.21.15.105:0/2456236508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f504c100ad0 0x7f504c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:38.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.935+0000 7f5053bde640 1 -- 172.21.15.105:0/2456236508 >> 172.21.15.105:0/2456236508 conn(0x7f504c0fc280 msgr2=0x7f504c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:38.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.935+0000 7f5053bde640 1 -- 172.21.15.105:0/2456236508 shutdown_connections 2024-09-16T01:59:38.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.935+0000 7f5053bde640 1 -- 172.21.15.105:0/2456236508 wait complete. 2024-09-16T01:59:38.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.936+0000 7f5053bde640 1 Processor -- start 2024-09-16T01:59:38.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.936+0000 7f5053bde640 1 -- start start 2024-09-16T01:59:38.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.936+0000 7f5053bde640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f504c100ad0 0x7f504c06aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:38.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.936+0000 7f5053bde640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f504c101cd0 0x7f504c19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:38.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.936+0000 7f5053bde640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f504c102690 0x7f504c19f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:38.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.936+0000 7f5053bde640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f504c10c700 con 0x7f504c101cd0 2024-09-16T01:59:38.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.936+0000 7f5053bde640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f504c10c580 con 0x7f504c100ad0 2024-09-16T01:59:38.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.936+0000 7f5053bde640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f504c10c880 con 0x7f504c102690 2024-09-16T01:59:38.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.936+0000 7f5051179640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f504c101cd0 0x7f504c19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:38.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.936+0000 7f5051179640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f504c101cd0 0x7f504c19cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59866/0 (socket says 172.21.15.105:59866) 2024-09-16T01:59:38.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.936+0000 7f5051179640 1 -- 172.21.15.105:0/3180232973 learned_addr learned my addr 172.21.15.105:0/3180232973 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:59:38.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.936+0000 7f505217b640 1 --2- 172.21.15.105:0/3180232973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f504c102690 0x7f504c19f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:38.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.936+0000 7f505197a640 1 --2- 172.21.15.105:0/3180232973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f504c100ad0 0x7f504c06aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:38.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.936+0000 7f5051179640 1 -- 172.21.15.105:0/3180232973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f504c102690 msgr2=0x7f504c19f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:38.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.936+0000 7f5051179640 1 --2- 172.21.15.105:0/3180232973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f504c102690 0x7f504c19f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:38.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.936+0000 7f5051179640 1 -- 172.21.15.105:0/3180232973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f504c100ad0 msgr2=0x7f504c06aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:38.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.936+0000 7f5051179640 1 --2- 172.21.15.105:0/3180232973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f504c100ad0 0x7f504c06aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:38.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.936+0000 7f5051179640 1 -- 172.21.15.105:0/3180232973 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f504c19fb40 con 0x7f504c101cd0 2024-09-16T01:59:38.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.936+0000 7f5051179640 1 --2- 172.21.15.105:0/3180232973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f504c101cd0 0x7f504c19cc10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f503c0314e0 tx=0x7f503c003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:38.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.937+0000 7f5042ffd640 1 -- 172.21.15.105:0/3180232973 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503c03f070 con 0x7f504c101cd0 2024-09-16T01:59:38.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.937+0000 7f5053bde640 1 -- 172.21.15.105:0/3180232973 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f504c19fd70 con 0x7f504c101cd0 2024-09-16T01:59:38.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.937+0000 7f5042ffd640 1 -- 172.21.15.105:0/3180232973 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f503c031e40 con 0x7f504c101cd0 2024-09-16T01:59:38.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.937+0000 7f5042ffd640 1 -- 172.21.15.105:0/3180232973 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503c035030 con 0x7f504c101cd0 2024-09-16T01:59:38.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.937+0000 7f5053bde640 1 -- 172.21.15.105:0/3180232973 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f504c1a0270 con 0x7f504c101cd0 2024-09-16T01:59:38.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.938+0000 7f5053bde640 1 -- 172.21.15.105:0/3180232973 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f504c107f20 con 0x7f504c101cd0 2024-09-16T01:59:38.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.939+0000 7f5042ffd640 1 -- 172.21.15.105:0/3180232973 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f503c042070 con 0x7f504c101cd0 2024-09-16T01:59:38.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.940+0000 7f5042ffd640 1 --2- 172.21.15.105:0/3180232973 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f50380a3240 0x7f50380a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:38.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.940+0000 7f505197a640 1 --2- 172.21.15.105:0/3180232973 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f50380a3240 0x7f50380a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:38.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.940+0000 7f5042ffd640 1 -- 172.21.15.105:0/3180232973 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6809+0+0 (secure 0 0 0) 0x7f503c0f01a0 con 0x7f504c101cd0 2024-09-16T01:59:38.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.941+0000 7f505197a640 1 --2- 172.21.15.105:0/3180232973 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f50380a3240 0x7f50380a5700 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f504c101b30 tx=0x7f5048008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:38.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:38.944+0000 7f5042ffd640 1 -- 172.21.15.105:0/3180232973 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f503c0bd4a0 con 0x7f504c101cd0 2024-09-16T01:59:39.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:39.112+0000 7f5053bde640 1 -- 172.21.15.105:0/3180232973 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f504c02da10 con 0x7f504c101cd0 2024-09-16T01:59:39.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:39.112+0000 7f5042ffd640 1 -- 172.21.15.105:0/3180232973 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7f503c0bcda0 con 0x7f504c101cd0 2024-09-16T01:59:39.113 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:59:39.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:39.114+0000 7f5053bde640 1 -- 172.21.15.105:0/3180232973 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f50380a3240 msgr2=0x7f50380a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:39.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:39.114+0000 7f5053bde640 1 --2- 172.21.15.105:0/3180232973 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f50380a3240 0x7f50380a5700 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f504c101b30 tx=0x7f5048008040 comp rx=0 tx=0).stop 2024-09-16T01:59:39.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:39.114+0000 7f5053bde640 1 -- 172.21.15.105:0/3180232973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f504c101cd0 msgr2=0x7f504c19cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:39.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:39.114+0000 7f5053bde640 1 --2- 172.21.15.105:0/3180232973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f504c101cd0 0x7f504c19cc10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f503c0314e0 tx=0x7f503c003c80 comp rx=0 tx=0).stop 2024-09-16T01:59:39.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:39.114+0000 7f5053bde640 1 -- 172.21.15.105:0/3180232973 shutdown_connections 2024-09-16T01:59:39.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:39.114+0000 7f5053bde640 1 --2- 172.21.15.105:0/3180232973 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f50380a3240 0x7f50380a5700 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:39.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:39.114+0000 7f5053bde640 1 --2- 172.21.15.105:0/3180232973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f504c102690 0x7f504c19f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:39.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:39.114+0000 7f5053bde640 1 --2- 172.21.15.105:0/3180232973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f504c101cd0 0x7f504c19cc10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:39.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:39.114+0000 7f5053bde640 1 --2- 172.21.15.105:0/3180232973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f504c100ad0 0x7f504c06aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:39.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:39.114+0000 7f5053bde640 1 -- 172.21.15.105:0/3180232973 >> 172.21.15.105:0/3180232973 conn(0x7f504c0fc280 msgr2=0x7f504c1050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:39.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:39.114+0000 7f5053bde640 1 -- 172.21.15.105:0/3180232973 shutdown_connections 2024-09-16T01:59:39.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:39.115+0000 7f5053bde640 1 -- 172.21.15.105:0/3180232973 wait complete. 2024-09-16T01:59:39.544 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:39 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:39.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:39 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:39.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:39 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/3180232973' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T01:59:39.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:39 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:39.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:39 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:39.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:39 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/3180232973' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T01:59:39.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:39 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:39.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:39 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:39.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:39 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/3180232973' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T01:59:40.039 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":70,"num_osds":12,"num_up_osds":11,"osd_up_since":1726451956,"num_in_osds":12,"osd_in_since":1726451958,"num_remapped_pgs":0} 2024-09-16T01:59:40.079 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 01:59:39 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T01:59:39.883+0000 7fdddab68640 -1 osd.11 0 log_to_monitors true 2024-09-16T01:59:40.294 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:40 smithi105 ceph-mon[31197]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-16T01:59:40.294 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:40 smithi105 ceph-mon[31197]: from='osd.11 [v2:172.21.15.190:6824/3820460689,v1:172.21.15.190:6825/3820460689]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T01:59:40.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:40 smithi105 ceph-mon[31197]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T01:59:40.342 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:40 smithi190 ceph-mon[34762]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-16T01:59:40.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:40 smithi190 ceph-mon[34762]: from='osd.11 [v2:172.21.15.190:6824/3820460689,v1:172.21.15.190:6825/3820460689]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T01:59:40.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:40 smithi190 ceph-mon[34762]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T01:59:40.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:40 smithi137 ceph-mon[34099]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-16T01:59:40.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:40 smithi137 ceph-mon[34099]: from='osd.11 [v2:172.21.15.190:6824/3820460689,v1:172.21.15.190:6825/3820460689]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T01:59:40.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:40 smithi137 ceph-mon[34099]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T01:59:41.040 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-16T01:59:41.290 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:59:41.413 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:41 smithi105 ceph-mon[31197]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-16T01:59:41.413 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:41 smithi105 ceph-mon[31197]: osdmap e71: 12 total, 11 up, 12 in 2024-09-16T01:59:41.413 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:41 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:59:41.413 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:41 smithi105 ceph-mon[31197]: from='osd.11 [v2:172.21.15.190:6824/3820460689,v1:172.21.15.190:6825/3820460689]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T01:59:41.413 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:41 smithi105 ceph-mon[31197]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T01:59:41.514 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 01:59:41 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T01:59:41.220+0000 7fddd6aeb640 -1 osd.11 0 waiting for initial osdmap 2024-09-16T01:59:41.514 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 01:59:41 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T01:59:41.233+0000 7fddd2913640 -1 osd.11 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T01:59:41.515 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:41 smithi190 ceph-mon[34762]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-16T01:59:41.515 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:41 smithi190 ceph-mon[34762]: osdmap e71: 12 total, 11 up, 12 in 2024-09-16T01:59:41.515 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:41 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:59:41.515 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:41 smithi190 ceph-mon[34762]: from='osd.11 [v2:172.21.15.190:6824/3820460689,v1:172.21.15.190:6825/3820460689]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T01:59:41.515 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:41 smithi190 ceph-mon[34762]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T01:59:41.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:41 smithi137 ceph-mon[34099]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-16T01:59:41.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:41 smithi137 ceph-mon[34099]: osdmap e71: 12 total, 11 up, 12 in 2024-09-16T01:59:41.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:41 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:59:41.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:41 smithi137 ceph-mon[34099]: from='osd.11 [v2:172.21.15.190:6824/3820460689,v1:172.21.15.190:6825/3820460689]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T01:59:41.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:41 smithi137 ceph-mon[34099]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T01:59:41.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.836+0000 7fd89da65640 1 -- 172.21.15.105:0/2874454278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd898100ad0 msgr2=0x7fd898100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:41.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.836+0000 7fd89da65640 1 --2- 172.21.15.105:0/2874454278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd898100ad0 0x7fd898100ed0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd888009900 tx=0x7fd88802a410 comp rx=0 tx=0).stop 2024-09-16T01:59:41.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.836+0000 7fd89da65640 1 -- 172.21.15.105:0/2874454278 shutdown_connections 2024-09-16T01:59:41.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.836+0000 7fd89da65640 1 --2- 172.21.15.105:0/2874454278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd898102690 0x7fd898108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:41.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.836+0000 7fd89da65640 1 --2- 172.21.15.105:0/2874454278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd898101cd0 0x7fd898102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:41.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.836+0000 7fd89da65640 1 --2- 172.21.15.105:0/2874454278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd898100ad0 0x7fd898100ed0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:41.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.836+0000 7fd89da65640 1 -- 172.21.15.105:0/2874454278 >> 172.21.15.105:0/2874454278 conn(0x7fd8980fc280 msgr2=0x7fd8980fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:41.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.836+0000 7fd89da65640 1 -- 172.21.15.105:0/2874454278 shutdown_connections 2024-09-16T01:59:41.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.836+0000 7fd89da65640 1 -- 172.21.15.105:0/2874454278 wait complete. 2024-09-16T01:59:41.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.836+0000 7fd89da65640 1 Processor -- start 2024-09-16T01:59:41.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.836+0000 7fd89da65640 1 -- start start 2024-09-16T01:59:41.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.837+0000 7fd89da65640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd898100ad0 0x7fd898197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:41.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.837+0000 7fd89da65640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd898101cd0 0x7fd8981982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:41.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.837+0000 7fd89da65640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd898102690 0x7fd89819f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:41.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.837+0000 7fd89da65640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd89810c610 con 0x7fd898101cd0 2024-09-16T01:59:41.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.837+0000 7fd89da65640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd89810c490 con 0x7fd898100ad0 2024-09-16T01:59:41.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.837+0000 7fd89da65640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd89810c790 con 0x7fd898102690 2024-09-16T01:59:41.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.837+0000 7fd88ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd898101cd0 0x7fd8981982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:41.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.837+0000 7fd88ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd898101cd0 0x7fd8981982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59886/0 (socket says 172.21.15.105:59886) 2024-09-16T01:59:41.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.837+0000 7fd88ffff640 1 -- 172.21.15.105:0/611189101 learned_addr learned my addr 172.21.15.105:0/611189101 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:59:41.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.837+0000 7fd897fff640 1 --2- 172.21.15.105:0/611189101 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd898102690 0x7fd89819f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:41.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.837+0000 7fd8977fe640 1 --2- 172.21.15.105:0/611189101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd898100ad0 0x7fd898197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:41.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.837+0000 7fd88ffff640 1 -- 172.21.15.105:0/611189101 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd898102690 msgr2=0x7fd89819f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:41.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.837+0000 7fd88ffff640 1 --2- 172.21.15.105:0/611189101 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd898102690 0x7fd89819f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:41.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.837+0000 7fd88ffff640 1 -- 172.21.15.105:0/611189101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd898100ad0 msgr2=0x7fd898197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:41.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.837+0000 7fd88ffff640 1 --2- 172.21.15.105:0/611189101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd898100ad0 0x7fd898197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:41.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.837+0000 7fd88ffff640 1 -- 172.21.15.105:0/611189101 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd89819f9d0 con 0x7fd898101cd0 2024-09-16T01:59:41.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.837+0000 7fd897fff640 1 --2- 172.21.15.105:0/611189101 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd898102690 0x7fd89819f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:59:41.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.837+0000 7fd88ffff640 1 --2- 172.21.15.105:0/611189101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd898101cd0 0x7fd8981982f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd88000c920 tx=0x7fd88000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:41.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.837+0000 7fd8957fa640 1 -- 172.21.15.105:0/611189101 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd880007c70 con 0x7fd898101cd0 2024-09-16T01:59:41.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.837+0000 7fd8957fa640 1 -- 172.21.15.105:0/611189101 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd880007e10 con 0x7fd898101cd0 2024-09-16T01:59:41.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.837+0000 7fd89da65640 1 -- 172.21.15.105:0/611189101 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd89819fcc0 con 0x7fd898101cd0 2024-09-16T01:59:41.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.837+0000 7fd8957fa640 1 -- 172.21.15.105:0/611189101 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd880018820 con 0x7fd898101cd0 2024-09-16T01:59:41.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.837+0000 7fd89da65640 1 -- 172.21.15.105:0/611189101 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8981a01b0 con 0x7fd898101cd0 2024-09-16T01:59:41.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.838+0000 7fd89da65640 1 -- 172.21.15.105:0/611189101 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8981a03e0 con 0x7fd898101cd0 2024-09-16T01:59:41.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.840+0000 7fd8957fa640 1 -- 172.21.15.105:0/611189101 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fd8800189c0 con 0x7fd898101cd0 2024-09-16T01:59:41.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.840+0000 7fd8957fa640 1 --2- 172.21.15.105:0/611189101 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fd8840a3240 0x7fd8840a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:41.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.840+0000 7fd8977fe640 1 --2- 172.21.15.105:0/611189101 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fd8840a3240 0x7fd8840a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:41.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.840+0000 7fd8957fa640 1 -- 172.21.15.105:0/611189101 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6847+0+0 (secure 0 0 0) 0x7fd8800cf800 con 0x7fd898101cd0 2024-09-16T01:59:41.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.841+0000 7fd8977fe640 1 --2- 172.21.15.105:0/611189101 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fd8840a3240 0x7fd8840a5700 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd888006010 tx=0x7fd888000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:41.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:41.843+0000 7fd8957fa640 1 -- 172.21.15.105:0/611189101 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd880014030 con 0x7fd898101cd0 2024-09-16T01:59:42.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:42.010+0000 7fd89da65640 1 -- 172.21.15.105:0/611189101 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fd8981a0650 con 0x7fd898101cd0 2024-09-16T01:59:42.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:42.010+0000 7fd8957fa640 1 -- 172.21.15.105:0/611189101 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7fd88009cb70 con 0x7fd898101cd0 2024-09-16T01:59:42.012 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:59:42.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:42.012+0000 7fd89da65640 1 -- 172.21.15.105:0/611189101 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fd8840a3240 msgr2=0x7fd8840a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:42.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:42.012+0000 7fd89da65640 1 --2- 172.21.15.105:0/611189101 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fd8840a3240 0x7fd8840a5700 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd888006010 tx=0x7fd888000bc0 comp rx=0 tx=0).stop 2024-09-16T01:59:42.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:42.012+0000 7fd89da65640 1 -- 172.21.15.105:0/611189101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd898101cd0 msgr2=0x7fd8981982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:42.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:42.012+0000 7fd89da65640 1 --2- 172.21.15.105:0/611189101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd898101cd0 0x7fd8981982f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd88000c920 tx=0x7fd88000cde0 comp rx=0 tx=0).stop 2024-09-16T01:59:42.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:42.012+0000 7fd89da65640 1 -- 172.21.15.105:0/611189101 shutdown_connections 2024-09-16T01:59:42.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:42.012+0000 7fd89da65640 1 --2- 172.21.15.105:0/611189101 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fd8840a3240 0x7fd8840a5700 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:42.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:42.012+0000 7fd89da65640 1 --2- 172.21.15.105:0/611189101 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd898102690 0x7fd89819f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:42.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:42.012+0000 7fd89da65640 1 --2- 172.21.15.105:0/611189101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd898101cd0 0x7fd8981982f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:42.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:42.012+0000 7fd89da65640 1 --2- 172.21.15.105:0/611189101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd898100ad0 0x7fd898197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:42.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:42.012+0000 7fd89da65640 1 -- 172.21.15.105:0/611189101 >> 172.21.15.105:0/611189101 conn(0x7fd8980fc280 msgr2=0x7fd898069fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:42.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:42.012+0000 7fd89da65640 1 -- 172.21.15.105:0/611189101 shutdown_connections 2024-09-16T01:59:42.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:42.012+0000 7fd89da65640 1 -- 172.21.15.105:0/611189101 wait complete. 2024-09-16T01:59:42.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:42 smithi105 ceph-mon[31197]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-09-16T01:59:42.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:42 smithi105 ceph-mon[31197]: osdmap e72: 12 total, 11 up, 12 in 2024-09-16T01:59:42.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:42 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:59:42.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:42 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:59:42.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:42 smithi105 ceph-mon[31197]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-16T01:59:42.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:42 smithi105 ceph-mon[31197]: Detected new or changed devices on smithi190 2024-09-16T01:59:42.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:42 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:42.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:42 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:42.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:42 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:59:42.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:42 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:59:42.297 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:42 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:59:42.297 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:42 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:59:42.297 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:42 smithi105 ceph-mon[31197]: Adjusting osd_memory_target on smithi190 to 4272M 2024-09-16T01:59:42.297 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:42 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:42.297 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:42 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:42.297 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:42 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:59:42.297 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:42 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:42.297 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:42 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:59:42.297 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:42 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:42.297 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:42 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:59:42.298 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:42 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:42.298 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:42 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/611189101' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T01:59:42.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:42 smithi190 ceph-mon[34762]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-09-16T01:59:42.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:42 smithi190 ceph-mon[34762]: osdmap e72: 12 total, 11 up, 12 in 2024-09-16T01:59:42.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:42 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:59:42.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:42 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:59:42.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:42 smithi190 ceph-mon[34762]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-16T01:59:42.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:42 smithi190 ceph-mon[34762]: Detected new or changed devices on smithi190 2024-09-16T01:59:42.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:42 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:42.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:42 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:42.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:42 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:59:42.594 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:42 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:59:42.594 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:42 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:59:42.594 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:42 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:59:42.594 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:42 smithi190 ceph-mon[34762]: Adjusting osd_memory_target on smithi190 to 4272M 2024-09-16T01:59:42.594 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:42 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:42.594 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:42 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:42.594 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:42 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:59:42.594 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:42 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:42.594 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:42 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:59:42.594 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:42 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:42.594 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:42 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:59:42.594 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:42 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:42.594 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:42 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/611189101' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T01:59:42.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:42 smithi137 ceph-mon[34099]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-09-16T01:59:42.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:42 smithi137 ceph-mon[34099]: osdmap e72: 12 total, 11 up, 12 in 2024-09-16T01:59:42.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:42 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:59:42.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:42 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:59:42.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:42 smithi137 ceph-mon[34099]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-16T01:59:42.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:42 smithi137 ceph-mon[34099]: Detected new or changed devices on smithi190 2024-09-16T01:59:42.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:42 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:42.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:42 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:42.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:42 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:59:42.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:42 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:59:42.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:42 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:59:42.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:42 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:59:42.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:42 smithi137 ceph-mon[34099]: Adjusting osd_memory_target on smithi190 to 4272M 2024-09-16T01:59:42.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:42 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:42.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:42 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:42.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:42 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:59:42.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:42 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:42.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:42 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:59:42.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:42 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:42.673 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:42 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:59:42.673 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:42 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:42.673 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:42 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/611189101' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T01:59:42.722 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":72,"num_osds":12,"num_up_osds":11,"osd_up_since":1726451956,"num_in_osds":12,"osd_in_since":1726451958,"num_remapped_pgs":0} 2024-09-16T01:59:43.544 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:43 smithi105 ceph-mon[31197]: purged_snaps scrub starts 2024-09-16T01:59:43.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:43 smithi105 ceph-mon[31197]: purged_snaps scrub ok 2024-09-16T01:59:43.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:43 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:59:43.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:43 smithi105 ceph-mon[31197]: osd.11 [v2:172.21.15.190:6824/3820460689,v1:172.21.15.190:6825/3820460689] boot 2024-09-16T01:59:43.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:43 smithi105 ceph-mon[31197]: osdmap e73: 12 total, 12 up, 12 in 2024-09-16T01:59:43.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:43 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:59:43.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:43 smithi190 ceph-mon[34762]: purged_snaps scrub starts 2024-09-16T01:59:43.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:43 smithi190 ceph-mon[34762]: purged_snaps scrub ok 2024-09-16T01:59:43.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:43 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:59:43.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:43 smithi190 ceph-mon[34762]: osd.11 [v2:172.21.15.190:6824/3820460689,v1:172.21.15.190:6825/3820460689] boot 2024-09-16T01:59:43.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:43 smithi190 ceph-mon[34762]: osdmap e73: 12 total, 12 up, 12 in 2024-09-16T01:59:43.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:43 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:59:43.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:43 smithi137 ceph-mon[34099]: purged_snaps scrub starts 2024-09-16T01:59:43.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:43 smithi137 ceph-mon[34099]: purged_snaps scrub ok 2024-09-16T01:59:43.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:43 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:59:43.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:43 smithi137 ceph-mon[34099]: osd.11 [v2:172.21.15.190:6824/3820460689,v1:172.21.15.190:6825/3820460689] boot 2024-09-16T01:59:43.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:43 smithi137 ceph-mon[34099]: osdmap e73: 12 total, 12 up, 12 in 2024-09-16T01:59:43.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:43 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:59:43.723 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-16T01:59:43.974 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:59:44.391 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:44 smithi105 ceph-mon[31197]: osdmap e74: 12 total, 12 up, 12 in 2024-09-16T01:59:44.391 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:44 smithi105 ceph-mon[31197]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:59:44.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.467+0000 7f09d8713640 1 -- 172.21.15.105:0/2850154000 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09d0103e20 msgr2=0x7f09d0104220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:44.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.467+0000 7f09d8713640 1 --2- 172.21.15.105:0/2850154000 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09d0103e20 0x7f09d0104220 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f09cc009900 tx=0x7f09cc02a410 comp rx=0 tx=0).stop 2024-09-16T01:59:44.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.467+0000 7f09d8713640 1 -- 172.21.15.105:0/2850154000 shutdown_connections 2024-09-16T01:59:44.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.467+0000 7f09d8713640 1 --2- 172.21.15.105:0/2850154000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09d0105940 0x7f09d010c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:44.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.467+0000 7f09d8713640 1 --2- 172.21.15.105:0/2850154000 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09d0104f80 0x7f09d0105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:44.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.467+0000 7f09d8713640 1 --2- 172.21.15.105:0/2850154000 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09d0103e20 0x7f09d0104220 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:44.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.467+0000 7f09d8713640 1 -- 172.21.15.105:0/2850154000 >> 172.21.15.105:0/2850154000 conn(0x7f09d00ff5b0 msgr2=0x7f09d01019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:44.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.467+0000 7f09d8713640 1 -- 172.21.15.105:0/2850154000 shutdown_connections 2024-09-16T01:59:44.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.467+0000 7f09d8713640 1 -- 172.21.15.105:0/2850154000 wait complete. 2024-09-16T01:59:44.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.467+0000 7f09d8713640 1 Processor -- start 2024-09-16T01:59:44.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.467+0000 7f09d8713640 1 -- start start 2024-09-16T01:59:44.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.467+0000 7f09d8713640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09d0103e20 0x7f09d0199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:44.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.467+0000 7f09d8713640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09d0104f80 0x7f09d019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:44.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.467+0000 7f09d8713640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09d0105940 0x7f09d01a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:44.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.467+0000 7f09d8713640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f09d010f9e0 con 0x7f09d0104f80 2024-09-16T01:59:44.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.467+0000 7f09d8713640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f09d010f860 con 0x7f09d0105940 2024-09-16T01:59:44.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.467+0000 7f09d8713640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f09d010fb60 con 0x7f09d0103e20 2024-09-16T01:59:44.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.468+0000 7f09d5cae640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09d0104f80 0x7f09d019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:44.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.468+0000 7f09d5cae640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09d0104f80 0x7f09d019a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59916/0 (socket says 172.21.15.105:59916) 2024-09-16T01:59:44.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.468+0000 7f09d5cae640 1 -- 172.21.15.105:0/448065528 learned_addr learned my addr 172.21.15.105:0/448065528 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:59:44.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.468+0000 7f09d6cb0640 1 --2- 172.21.15.105:0/448065528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09d0105940 0x7f09d01a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:44.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.468+0000 7f09d5cae640 1 -- 172.21.15.105:0/448065528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09d0103e20 msgr2=0x7f09d0199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:44.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.468+0000 7f09d64af640 1 --2- 172.21.15.105:0/448065528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09d0103e20 0x7f09d0199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:44.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.468+0000 7f09d5cae640 1 --2- 172.21.15.105:0/448065528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09d0103e20 0x7f09d0199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:44.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.468+0000 7f09d5cae640 1 -- 172.21.15.105:0/448065528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09d0105940 msgr2=0x7f09d01a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:44.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.468+0000 7f09d5cae640 1 --2- 172.21.15.105:0/448065528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09d0105940 0x7f09d01a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:44.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.468+0000 7f09d5cae640 1 -- 172.21.15.105:0/448065528 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d01a1c20 con 0x7f09d0104f80 2024-09-16T01:59:44.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.468+0000 7f09d6cb0640 1 --2- 172.21.15.105:0/448065528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09d0105940 0x7f09d01a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:59:44.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.468+0000 7f09d5cae640 1 --2- 172.21.15.105:0/448065528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09d0104f80 0x7f09d019a530 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f09c000c970 tx=0x7f09c000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:44.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.468+0000 7f09c77fe640 1 -- 172.21.15.105:0/448065528 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c0007c10 con 0x7f09d0104f80 2024-09-16T01:59:44.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.468+0000 7f09d8713640 1 -- 172.21.15.105:0/448065528 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09d01a1f10 con 0x7f09d0104f80 2024-09-16T01:59:44.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.468+0000 7f09c77fe640 1 -- 172.21.15.105:0/448065528 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09c0007db0 con 0x7f09d0104f80 2024-09-16T01:59:44.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.468+0000 7f09c77fe640 1 -- 172.21.15.105:0/448065528 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c000f7d0 con 0x7f09d0104f80 2024-09-16T01:59:44.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.468+0000 7f09d8713640 1 -- 172.21.15.105:0/448065528 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09d01a2440 con 0x7f09d0104f80 2024-09-16T01:59:44.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.469+0000 7f09d8713640 1 -- 172.21.15.105:0/448065528 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09d010b1b0 con 0x7f09d0104f80 2024-09-16T01:59:44.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.471+0000 7f09c77fe640 1 -- 172.21.15.105:0/448065528 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f09c0018480 con 0x7f09d0104f80 2024-09-16T01:59:44.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.471+0000 7f09c77fe640 1 --2- 172.21.15.105:0/448065528 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f09bc0a3270 0x7f09bc0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:44.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.471+0000 7f09d64af640 1 --2- 172.21.15.105:0/448065528 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f09bc0a3270 0x7f09bc0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:44.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.471+0000 7f09c77fe640 1 -- 172.21.15.105:0/448065528 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f09c00cfb10 con 0x7f09d0104f80 2024-09-16T01:59:44.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.472+0000 7f09d64af640 1 --2- 172.21.15.105:0/448065528 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f09bc0a3270 0x7f09bc0a5730 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f09cc02a930 tx=0x7f09cc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:44.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.474+0000 7f09c77fe640 1 -- 172.21.15.105:0/448065528 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f09c0026080 con 0x7f09d0104f80 2024-09-16T01:59:44.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:44 smithi190 ceph-mon[34762]: osdmap e74: 12 total, 12 up, 12 in 2024-09-16T01:59:44.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:44 smithi190 ceph-mon[34762]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:59:44.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.639+0000 7f09d8713640 1 -- 172.21.15.105:0/448065528 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f09d002da30 con 0x7f09d0104f80 2024-09-16T01:59:44.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.639+0000 7f09c77fe640 1 -- 172.21.15.105:0/448065528 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v74) ==== 74+0+133 (secure 0 0 0) 0x7f09c009cd60 con 0x7f09d0104f80 2024-09-16T01:59:44.641 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:59:44.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.641+0000 7f09d8713640 1 -- 172.21.15.105:0/448065528 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f09bc0a3270 msgr2=0x7f09bc0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:44.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.641+0000 7f09d8713640 1 --2- 172.21.15.105:0/448065528 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f09bc0a3270 0x7f09bc0a5730 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f09cc02a930 tx=0x7f09cc000bc0 comp rx=0 tx=0).stop 2024-09-16T01:59:44.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.641+0000 7f09d8713640 1 -- 172.21.15.105:0/448065528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09d0104f80 msgr2=0x7f09d019a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:44.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.641+0000 7f09d8713640 1 --2- 172.21.15.105:0/448065528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09d0104f80 0x7f09d019a530 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f09c000c970 tx=0x7f09c000ce30 comp rx=0 tx=0).stop 2024-09-16T01:59:44.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.641+0000 7f09d8713640 1 -- 172.21.15.105:0/448065528 shutdown_connections 2024-09-16T01:59:44.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.641+0000 7f09d8713640 1 --2- 172.21.15.105:0/448065528 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f09bc0a3270 0x7f09bc0a5730 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:44.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.641+0000 7f09d8713640 1 --2- 172.21.15.105:0/448065528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09d0105940 0x7f09d01a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:44.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.641+0000 7f09d8713640 1 --2- 172.21.15.105:0/448065528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09d0104f80 0x7f09d019a530 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:44.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.641+0000 7f09d8713640 1 --2- 172.21.15.105:0/448065528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09d0103e20 0x7f09d0199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:44.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.641+0000 7f09d8713640 1 -- 172.21.15.105:0/448065528 >> 172.21.15.105:0/448065528 conn(0x7f09d00ff5b0 msgr2=0x7f09d0108380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:44.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.642+0000 7f09d8713640 1 -- 172.21.15.105:0/448065528 shutdown_connections 2024-09-16T01:59:44.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:44.642+0000 7f09d8713640 1 -- 172.21.15.105:0/448065528 wait complete. 2024-09-16T01:59:44.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:44 smithi137 ceph-mon[34099]: osdmap e74: 12 total, 12 up, 12 in 2024-09-16T01:59:44.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:44 smithi137 ceph-mon[34099]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:59:45.473 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:45 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/448065528' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T01:59:45.503 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":74,"num_osds":12,"num_up_osds":12,"osd_up_since":1726451982,"num_in_osds":12,"osd_in_since":1726451958,"num_remapped_pgs":0} 2024-09-16T01:59:45.503 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T01:59:45.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:45 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/448065528' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T01:59:45.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:45 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/448065528' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T01:59:45.745 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:59:46.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.196+0000 7fa97f787640 1 -- 172.21.15.105:0/2597699730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa978106f10 msgr2=0x7fa97810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:46.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.196+0000 7fa97f787640 1 --2- 172.21.15.105:0/2597699730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa978106f10 0x7fa97810ea90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa970009ca0 tx=0x7fa97002a8c0 comp rx=0 tx=0).stop 2024-09-16T01:59:46.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.196+0000 7fa97f787640 1 -- 172.21.15.105:0/2597699730 shutdown_connections 2024-09-16T01:59:46.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.196+0000 7fa97f787640 1 --2- 172.21.15.105:0/2597699730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa978102790 0x7fa9781067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:46.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.196+0000 7fa97f787640 1 --2- 172.21.15.105:0/2597699730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa978106f10 0x7fa97810ea90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:46.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.196+0000 7fa97f787640 1 --2- 172.21.15.105:0/2597699730 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa978101de0 0x7fa9781021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:46.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.196+0000 7fa97f787640 1 -- 172.21.15.105:0/2597699730 >> 172.21.15.105:0/2597699730 conn(0x7fa9780fd8c0 msgr2=0x7fa9780ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:46.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.196+0000 7fa97f787640 1 -- 172.21.15.105:0/2597699730 shutdown_connections 2024-09-16T01:59:46.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.196+0000 7fa97f787640 1 -- 172.21.15.105:0/2597699730 wait complete. 2024-09-16T01:59:46.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.197+0000 7fa97f787640 1 Processor -- start 2024-09-16T01:59:46.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.197+0000 7fa97f787640 1 -- start start 2024-09-16T01:59:46.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.197+0000 7fa97f787640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa978101de0 0x7fa9781982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:46.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.197+0000 7fa97f787640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa978102790 0x7fa9781987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:46.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.197+0000 7fa97f787640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa978106f10 0x7fa97819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:46.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.197+0000 7fa97f787640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa978111e20 con 0x7fa978102790 2024-09-16T01:59:46.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.197+0000 7fa97f787640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa978111ca0 con 0x7fa978101de0 2024-09-16T01:59:46.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.197+0000 7fa97f787640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa978111fa0 con 0x7fa978106f10 2024-09-16T01:59:46.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.197+0000 7fa97cd22640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa978102790 0x7fa9781987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:46.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.197+0000 7fa97cd22640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa978102790 0x7fa9781987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59940/0 (socket says 172.21.15.105:59940) 2024-09-16T01:59:46.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.197+0000 7fa97cd22640 1 -- 172.21.15.105:0/2402005538 learned_addr learned my addr 172.21.15.105:0/2402005538 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:59:46.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.197+0000 7fa97dd24640 1 --2- 172.21.15.105:0/2402005538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa978106f10 0x7fa97819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:46.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.197+0000 7fa97d523640 1 --2- 172.21.15.105:0/2402005538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa978101de0 0x7fa9781982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:46.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.198+0000 7fa97cd22640 1 -- 172.21.15.105:0/2402005538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa978106f10 msgr2=0x7fa97819cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:46.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.198+0000 7fa97cd22640 1 --2- 172.21.15.105:0/2402005538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa978106f10 0x7fa97819cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:46.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.198+0000 7fa97cd22640 1 -- 172.21.15.105:0/2402005538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa978101de0 msgr2=0x7fa9781982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:46.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.198+0000 7fa97cd22640 1 --2- 172.21.15.105:0/2402005538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa978101de0 0x7fa9781982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:46.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.198+0000 7fa97cd22640 1 -- 172.21.15.105:0/2402005538 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa97819d250 con 0x7fa978102790 2024-09-16T01:59:46.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.198+0000 7fa97d523640 1 --2- 172.21.15.105:0/2402005538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa978101de0 0x7fa9781982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:59:46.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.198+0000 7fa97cd22640 1 --2- 172.21.15.105:0/2402005538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa978102790 0x7fa9781987e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa96800c920 tx=0x7fa96800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:46.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.198+0000 7fa96e7fc640 1 -- 172.21.15.105:0/2402005538 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa968007c70 con 0x7fa978102790 2024-09-16T01:59:46.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.198+0000 7fa96e7fc640 1 -- 172.21.15.105:0/2402005538 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa968007e10 con 0x7fa978102790 2024-09-16T01:59:46.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.198+0000 7fa97f787640 1 -- 172.21.15.105:0/2402005538 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa97819d540 con 0x7fa978102790 2024-09-16T01:59:46.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.198+0000 7fa96e7fc640 1 -- 172.21.15.105:0/2402005538 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa968018820 con 0x7fa978102790 2024-09-16T01:59:46.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.198+0000 7fa97f787640 1 -- 172.21.15.105:0/2402005538 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9781a4e20 con 0x7fa978102790 2024-09-16T01:59:46.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.199+0000 7fa97f787640 1 -- 172.21.15.105:0/2402005538 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9781021c0 con 0x7fa978102790 2024-09-16T01:59:46.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.201+0000 7fa96e7fc640 1 -- 172.21.15.105:0/2402005538 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fa9680189c0 con 0x7fa978102790 2024-09-16T01:59:46.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.201+0000 7fa96e7fc640 1 --2- 172.21.15.105:0/2402005538 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa9640a32c0 0x7fa9640a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:46.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.201+0000 7fa97d523640 1 --2- 172.21.15.105:0/2402005538 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa9640a32c0 0x7fa9640a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:46.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.201+0000 7fa96e7fc640 1 -- 172.21.15.105:0/2402005538 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fa9680cf810 con 0x7fa978102790 2024-09-16T01:59:46.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.202+0000 7fa97d523640 1 --2- 172.21.15.105:0/2402005538 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa9640a32c0 0x7fa9640a5780 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa974009b40 tx=0x7fa974005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:46.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.204+0000 7fa96e7fc640 1 -- 172.21.15.105:0/2402005538 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa968014030 con 0x7fa978102790 2024-09-16T01:59:46.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.372+0000 7fa97f787640 1 -- 172.21.15.105:0/2402005538 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa978199600 con 0x7fa978102790 2024-09-16T01:59:46.374 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:46 smithi105 ceph-mon[31197]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:59:46.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.373+0000 7fa96e7fc640 1 -- 172.21.15.105:0/2402005538 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+18516 (secure 0 0 0) 0x7fa978199600 con 0x7fa978102790 2024-09-16T01:59:46.375 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:59:46.375 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":74,"fsid":"423b7a2a-73ce-11ef-bceb-c7b262605968","created":"2024-09-16T01:52:44.388033+0000","modified":"2024-09-16T01:59:43.222943+0000","last_up_change":"2024-09-16T01:59:42.218034+0000","last_in_change":"2024-09-16T01:59:18.785252+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T01:55:53.680808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a60d6c72-da8b-4b02-bb33-672be795e7f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6803","nonce":1226660290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6805","nonce":1226660290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6809","nonce":1226660290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6807","nonce":1226660290}]},"public_addr":"172.21.15.105:6803/1226660290","cluster_addr":"172.21.15.105:6805/1226660290","heartbeat_back_addr":"172.21.15.105:6809/1226660290","heartbeat_front_addr":"172.21.15.105:6807/1226660290","state":["exists","up"]},{"osd":1,"uuid":"b1d9ae84-b9a0-4a7e-b0fe-c402d46809fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6801","nonce":34753956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6803","nonce":34753956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6807","nonce":34753956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6805","nonce":34753956}]},"public_addr":"172.21.15.137:6801/34753956","cluster_addr":"172.21.15.137:6803/34753956","heartbeat_back_addr":"172.21.15.137:6807/34753956","heartbeat_front_addr":"172.21.15.137:6805/34753956","state":["exists","up"]},{"osd":2,"uuid":"137faae6-2e8b-44b5-8976-3d95a6eef1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6801","nonce":1159531918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6803","nonce":1159531918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6807","nonce":1159531918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6805","nonce":1159531918}]},"public_addr":"172.21.15.190:6801/1159531918","cluster_addr":"172.21.15.190:6803/1159531918","heartbeat_back_addr":"172.21.15.190:6807/1159531918","heartbeat_front_addr":"172.21.15.190:6805/1159531918","state":["exists","up"]},{"osd":3,"uuid":"5743c37b-5cd8-42b6-81c8-5cf5afc71e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6811","nonce":604905679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6813","nonce":604905679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6817","nonce":604905679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6815","nonce":604905679}]},"public_addr":"172.21.15.105:6811/604905679","cluster_addr":"172.21.15.105:6813/604905679","heartbeat_back_addr":"172.21.15.105:6817/604905679","heartbeat_front_addr":"172.21.15.105:6815/604905679","state":["exists","up"]},{"osd":4,"uuid":"e439d4a2-b54c-4de9-97f6-3510afe57f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6809","nonce":3758109169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6811","nonce":3758109169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6815","nonce":3758109169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6813","nonce":3758109169}]},"public_addr":"172.21.15.137:6809/3758109169","cluster_addr":"172.21.15.137:6811/3758109169","heartbeat_back_addr":"172.21.15.137:6815/3758109169","heartbeat_front_addr":"172.21.15.137:6813/3758109169","state":["exists","up"]},{"osd":5,"uuid":"d1e2a7d5-99f8-40fa-af6e-0305cc0af677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6809","nonce":2086773754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6811","nonce":2086773754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6815","nonce":2086773754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6813","nonce":2086773754}]},"public_addr":"172.21.15.190:6809/2086773754","cluster_addr":"172.21.15.190:6811/2086773754","heartbeat_back_addr":"172.21.15.190:6815/2086773754","heartbeat_front_addr":"172.21.15.190:6813/2086773754","state":["exists","up"]},{"osd":6,"uuid":"cd418159-a0f6-44c7-9213-c010b132fb56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6819","nonce":4012018725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6821","nonce":4012018725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6825","nonce":4012018725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6823","nonce":4012018725}]},"public_addr":"172.21.15.105:6819/4012018725","cluster_addr":"172.21.15.105:6821/4012018725","heartbeat_back_addr":"172.21.15.105:6825/4012018725","heartbeat_front_addr":"172.21.15.105:6823/4012018725","state":["exists","up"]},{"osd":7,"uuid":"76c15863-801e-4a72-be73-ba9df46c15b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6817","nonce":888678288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6819","nonce":888678288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6823","nonce":888678288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6821","nonce":888678288}]},"public_addr":"172.21.15.137:6817/888678288","cluster_addr":"172.21.15.137:6819/888678288","heartbeat_back_addr":"172.21.15.137:6823/888678288","heartbeat_front_addr":"172.21.15.137:6821/888678288","state":["exists","up"]},{"osd":8,"uuid":"6718597c-6f42-48ee-9493-edb2396fe707","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6817","nonce":2132638341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6819","nonce":2132638341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6823","nonce":2132638341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6821","nonce":2132638341}]},"public_addr":"172.21.15.190:6817/2132638341","cluster_addr":"172.21.15.190:6819/2132638341","heartbeat_back_addr":"172.21.15.190:6823/2132638341","heartbeat_front_addr":"172.21.15.190:6821/2132638341","state":["exists","up"]},{"osd":9,"uuid":"5178edc0-a2e8-491c-821c-6f41ec5aeb2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6827","nonce":3375034979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6829","nonce":3375034979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6833","nonce":3375034979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6831","nonce":3375034979}]},"public_addr":"172.21.15.105:6827/3375034979","cluster_addr":"172.21.15.105:6829/3375034979","heartbeat_back_addr":"172.21.15.105:6833/3375034979","heartbeat_front_addr":"172.21.15.105:6831/3375034979","state":["exists","up"]},{"osd":10,"uuid":"ad145d3e-a6e2-4dd7-bea3-bcd01eb3413e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6825","nonce":318562234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6827","nonce":318562234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6831","nonce":318562234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6829","nonce":318562234}]},"public_addr":"172.21.15.137:6825/318562234","cluster_addr":"172.21.15.137:6827/318562234","heartbeat_back_addr":"172.21.15.137:6831/318562234","heartbeat_front_addr":"172.21.15.137:6829/318562234","state":["exists","up"]},{"osd":11,"uuid":"6d44bced-3a8b-4763-a93e-0ea34acb4cb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6825","nonce":3820460689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6827","nonce":3820460689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6831","nonce":3820460689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6829","nonce":3820460689}]},"public_addr":"172.21.15.190:6825/3820460689","cluster_addr":"172.21.15.190:6827/3820460689","heartbeat_back_addr":"172.21.15.190:6831/3820460689","heartbeat_front_addr":"172.21.15.190:6829/3820460689","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:55:02.759033+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:55:26.829142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:55:51.120200+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:56:17.277870+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:56:42.313701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:07.013416+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:32.477916+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:58.643764+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:23.454908+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:48.869082+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:14.921573+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:40.908993+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:0/2453739657":"2024-09-17T01:53:39.351314+0000","172.21.15.105:6801/1469189484":"2024-09-17T01:53:39.351314+0000","172.21.15.105:0/3466166879":"2024-09-17T01:53:39.351314+0000","172.21.15.105:6801/1268348742":"2024-09-17T01:53:15.522227+0000","172.21.15.105:0/2897248478":"2024-09-17T01:53:39.351314+0000","172.21.15.105:0/3709363371":"2024-09-17T01:53:15.522227+0000","172.21.15.105:6800/1268348742":"2024-09-17T01:53:15.522227+0000","172.21.15.105:6800/1469189484":"2024-09-17T01:53:39.351314+0000","172.21.15.105:0/1198176299":"2024-09-17T01:53:15.522227+0000","172.21.15.105:0/3031917848":"2024-09-17T01:53:15.522227+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T01:59:46.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.375+0000 7fa97f787640 1 -- 172.21.15.105:0/2402005538 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa9640a32c0 msgr2=0x7fa9640a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:46.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.375+0000 7fa97f787640 1 --2- 172.21.15.105:0/2402005538 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa9640a32c0 0x7fa9640a5780 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa974009b40 tx=0x7fa974005e50 comp rx=0 tx=0).stop 2024-09-16T01:59:46.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.375+0000 7fa97f787640 1 -- 172.21.15.105:0/2402005538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa978102790 msgr2=0x7fa9781987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:46.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.375+0000 7fa97f787640 1 --2- 172.21.15.105:0/2402005538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa978102790 0x7fa9781987e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa96800c920 tx=0x7fa96800cde0 comp rx=0 tx=0).stop 2024-09-16T01:59:46.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.375+0000 7fa97f787640 1 -- 172.21.15.105:0/2402005538 shutdown_connections 2024-09-16T01:59:46.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.375+0000 7fa97f787640 1 --2- 172.21.15.105:0/2402005538 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa9640a32c0 0x7fa9640a5780 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:46.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.375+0000 7fa97f787640 1 --2- 172.21.15.105:0/2402005538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa978106f10 0x7fa97819cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:46.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.375+0000 7fa97f787640 1 --2- 172.21.15.105:0/2402005538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa978102790 0x7fa9781987e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:46.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.375+0000 7fa97f787640 1 --2- 172.21.15.105:0/2402005538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa978101de0 0x7fa9781982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:46.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.375+0000 7fa97f787640 1 -- 172.21.15.105:0/2402005538 >> 172.21.15.105:0/2402005538 conn(0x7fa9780fd8c0 msgr2=0x7fa9780feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:46.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.375+0000 7fa97f787640 1 -- 172.21.15.105:0/2402005538 shutdown_connections 2024-09-16T01:59:46.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:46.375+0000 7fa97f787640 1 -- 172.21.15.105:0/2402005538 wait complete. 2024-09-16T01:59:46.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:46 smithi190 ceph-mon[34762]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:59:46.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:46 smithi137 ceph-mon[34099]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:59:47.129 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T01:55:53.680808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T01:59:47.129 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-16T01:59:47.370 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:59:47.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:47 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2402005538' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:59:47.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:47 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/2402005538' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:59:47.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:47 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/2402005538' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:59:47.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.841+0000 7f0c3d5a9640 1 -- 172.21.15.105:0/1075880009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c3810c170 msgr2=0x7f0c3810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:47.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.841+0000 7f0c3d5a9640 1 --2- 172.21.15.105:0/1075880009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c3810c170 0x7f0c3810e610 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0c30009ca0 tx=0x7f0c3002a950 comp rx=0 tx=0).stop 2024-09-16T01:59:47.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.841+0000 7f0c3d5a9640 1 -- 172.21.15.105:0/1075880009 shutdown_connections 2024-09-16T01:59:47.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.841+0000 7f0c3d5a9640 1 --2- 172.21.15.105:0/1075880009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c3810c170 0x7f0c3810e610 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:47.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.841+0000 7f0c3d5a9640 1 --2- 172.21.15.105:0/1075880009 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c38103660 0x7f0c3810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:47.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.841+0000 7f0c3d5a9640 1 --2- 172.21.15.105:0/1075880009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c38102d40 0x7f0c38103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:47.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.841+0000 7f0c3d5a9640 1 -- 172.21.15.105:0/1075880009 >> 172.21.15.105:0/1075880009 conn(0x7f0c380fc7a0 msgr2=0x7f0c380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:47.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.842+0000 7f0c3d5a9640 1 -- 172.21.15.105:0/1075880009 shutdown_connections 2024-09-16T01:59:47.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.842+0000 7f0c3d5a9640 1 -- 172.21.15.105:0/1075880009 wait complete. 2024-09-16T01:59:47.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.842+0000 7f0c3d5a9640 1 Processor -- start 2024-09-16T01:59:47.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.842+0000 7f0c3d5a9640 1 -- start start 2024-09-16T01:59:47.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.843+0000 7f0c3d5a9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c38102d40 0x7f0c38198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:47.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.843+0000 7f0c3d5a9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c38103660 0x7f0c38198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:47.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.843+0000 7f0c3d5a9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c3810c170 0x7f0c3819caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:47.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.843+0000 7f0c3d5a9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0c38111e20 con 0x7f0c38103660 2024-09-16T01:59:47.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.843+0000 7f0c3d5a9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0c38111ca0 con 0x7f0c3810c170 2024-09-16T01:59:47.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.843+0000 7f0c3d5a9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0c38111fa0 con 0x7f0c38102d40 2024-09-16T01:59:47.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.843+0000 7f0c2e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c38103660 0x7f0c38198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:47.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.843+0000 7f0c2e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c38103660 0x7f0c38198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59960/0 (socket says 172.21.15.105:59960) 2024-09-16T01:59:47.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.843+0000 7f0c2e7fc640 1 -- 172.21.15.105:0/617588438 learned_addr learned my addr 172.21.15.105:0/617588438 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:59:47.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.843+0000 7f0c36ffd640 1 --2- 172.21.15.105:0/617588438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c38102d40 0x7f0c38198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:47.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.843+0000 7f0c377fe640 1 --2- 172.21.15.105:0/617588438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c3810c170 0x7f0c3819caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:47.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.843+0000 7f0c2e7fc640 1 -- 172.21.15.105:0/617588438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c38102d40 msgr2=0x7f0c38198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:47.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.843+0000 7f0c2e7fc640 1 --2- 172.21.15.105:0/617588438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c38102d40 0x7f0c38198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:47.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.843+0000 7f0c2e7fc640 1 -- 172.21.15.105:0/617588438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c3810c170 msgr2=0x7f0c3819caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:47.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.843+0000 7f0c2e7fc640 1 --2- 172.21.15.105:0/617588438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c3810c170 0x7f0c3819caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:47.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.843+0000 7f0c2e7fc640 1 -- 172.21.15.105:0/617588438 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c3819d1d0 con 0x7f0c38103660 2024-09-16T01:59:47.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.843+0000 7f0c2e7fc640 1 --2- 172.21.15.105:0/617588438 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c38103660 0x7f0c38198760 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0c2000c970 tx=0x7f0c2000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:47.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.843+0000 7f0c34ff9640 1 -- 172.21.15.105:0/617588438 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20007bb0 con 0x7f0c38103660 2024-09-16T01:59:47.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.843+0000 7f0c3d5a9640 1 -- 172.21.15.105:0/617588438 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c3819d4c0 con 0x7f0c38103660 2024-09-16T01:59:47.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.843+0000 7f0c34ff9640 1 -- 172.21.15.105:0/617588438 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c20007d50 con 0x7f0c38103660 2024-09-16T01:59:47.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.843+0000 7f0c34ff9640 1 -- 172.21.15.105:0/617588438 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20018780 con 0x7f0c38103660 2024-09-16T01:59:47.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.844+0000 7f0c3d5a9640 1 -- 172.21.15.105:0/617588438 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c381a4da0 con 0x7f0c38103660 2024-09-16T01:59:47.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.845+0000 7f0c3d5a9640 1 -- 172.21.15.105:0/617588438 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c38103120 con 0x7f0c38103660 2024-09-16T01:59:47.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.846+0000 7f0c34ff9640 1 -- 172.21.15.105:0/617588438 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f0c2000f450 con 0x7f0c38103660 2024-09-16T01:59:47.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.846+0000 7f0c34ff9640 1 --2- 172.21.15.105:0/617588438 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f0c240a3240 0x7f0c240a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:47.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.846+0000 7f0c36ffd640 1 --2- 172.21.15.105:0/617588438 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f0c240a3240 0x7f0c240a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:47.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.846+0000 7f0c34ff9640 1 -- 172.21.15.105:0/617588438 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f0c200cf4a0 con 0x7f0c38103660 2024-09-16T01:59:47.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.847+0000 7f0c36ffd640 1 --2- 172.21.15.105:0/617588438 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f0c240a3240 0x7f0c240a5700 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0c2800bbe0 tx=0x7f0c2800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:47.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:47.850+0000 7f0c34ff9640 1 -- 172.21.15.105:0/617588438 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0c20014030 con 0x7f0c38103660 2024-09-16T01:59:48.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:48.018+0000 7f0c3d5a9640 1 -- 172.21.15.105:0/617588438 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0c381a5000 con 0x7f0c38103660 2024-09-16T01:59:48.020 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-09-16T01:59:48.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:48.018+0000 7f0c34ff9640 1 -- 172.21.15.105:0/617588438 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) ==== 93+0+10 (secure 0 0 0) 0x7f0c2009c660 con 0x7f0c38103660 2024-09-16T01:59:48.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:48.020+0000 7f0c3d5a9640 1 -- 172.21.15.105:0/617588438 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f0c240a3240 msgr2=0x7f0c240a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:48.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:48.020+0000 7f0c3d5a9640 1 --2- 172.21.15.105:0/617588438 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f0c240a3240 0x7f0c240a5700 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0c2800bbe0 tx=0x7f0c2800b390 comp rx=0 tx=0).stop 2024-09-16T01:59:48.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:48.020+0000 7f0c3d5a9640 1 -- 172.21.15.105:0/617588438 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c38103660 msgr2=0x7f0c38198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:48.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:48.020+0000 7f0c3d5a9640 1 --2- 172.21.15.105:0/617588438 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c38103660 0x7f0c38198760 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0c2000c970 tx=0x7f0c2000ce30 comp rx=0 tx=0).stop 2024-09-16T01:59:48.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:48.020+0000 7f0c3d5a9640 1 -- 172.21.15.105:0/617588438 shutdown_connections 2024-09-16T01:59:48.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:48.020+0000 7f0c3d5a9640 1 --2- 172.21.15.105:0/617588438 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f0c240a3240 0x7f0c240a5700 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:48.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:48.020+0000 7f0c3d5a9640 1 --2- 172.21.15.105:0/617588438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c3810c170 0x7f0c3819caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:48.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:48.020+0000 7f0c3d5a9640 1 --2- 172.21.15.105:0/617588438 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c38103660 0x7f0c38198760 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:48.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:48.020+0000 7f0c3d5a9640 1 --2- 172.21.15.105:0/617588438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c38102d40 0x7f0c38198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:48.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:48.020+0000 7f0c3d5a9640 1 -- 172.21.15.105:0/617588438 >> 172.21.15.105:0/617588438 conn(0x7f0c380fc7a0 msgr2=0x7f0c380fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:48.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:48.020+0000 7f0c3d5a9640 1 -- 172.21.15.105:0/617588438 shutdown_connections 2024-09-16T01:59:48.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:48.020+0000 7f0c3d5a9640 1 -- 172.21.15.105:0/617588438 wait complete. 2024-09-16T01:59:48.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:48 smithi105 ceph-mon[31197]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:59:48.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:48 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/617588438' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T01:59:48.572 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:48 smithi137 ceph-mon[34099]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:59:48.572 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:48 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/617588438' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T01:59:48.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:48 smithi190 ceph-mon[34762]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:59:48.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:48 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/617588438' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T01:59:48.853 INFO:tasks.cephadm:Adding mds.a on smithi105 2024-09-16T01:59:48.853 INFO:tasks.cephadm:Adding mds.d on smithi105 2024-09-16T01:59:48.853 INFO:tasks.cephadm:Adding mds.g on smithi105 2024-09-16T01:59:48.854 INFO:tasks.cephadm:Adding mds.j on smithi105 2024-09-16T01:59:48.854 INFO:tasks.cephadm:Adding mds.b on smithi137 2024-09-16T01:59:48.854 INFO:tasks.cephadm:Adding mds.e on smithi137 2024-09-16T01:59:48.854 INFO:tasks.cephadm:Adding mds.h on smithi137 2024-09-16T01:59:48.854 INFO:tasks.cephadm:Adding mds.k on smithi137 2024-09-16T01:59:48.854 INFO:tasks.cephadm:Adding mds.c on smithi190 2024-09-16T01:59:48.854 INFO:tasks.cephadm:Adding mds.f on smithi190 2024-09-16T01:59:48.854 INFO:tasks.cephadm:Adding mds.i on smithi190 2024-09-16T01:59:48.854 INFO:tasks.cephadm:Adding mds.l on smithi190 2024-09-16T01:59:48.854 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph orch apply mds all '12;smithi105=a;smithi105=d;smithi105=g;smithi105=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l' 2024-09-16T01:59:49.122 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T01:59:49.635 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.618+0000 7fc762726640 1 -- 172.21.15.190:0/2856358470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc75c101bf0 msgr2=0x7fc75c101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:49.635 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.618+0000 7fc762726640 1 --2- 172.21.15.190:0/2856358470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc75c101bf0 0x7fc75c101ff0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc74c009900 tx=0x7fc74c02a410 comp rx=0 tx=0).stop 2024-09-16T01:59:49.635 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.618+0000 7fc762726640 1 -- 172.21.15.190:0/2856358470 shutdown_connections 2024-09-16T01:59:49.635 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.618+0000 7fc762726640 1 --2- 172.21.15.190:0/2856358470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc75c1037b0 0x7fc75c10a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:49.635 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.618+0000 7fc762726640 1 --2- 172.21.15.190:0/2856358470 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc75c102df0 0x7fc75c103270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:49.636 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.618+0000 7fc762726640 1 --2- 172.21.15.190:0/2856358470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc75c101bf0 0x7fc75c101ff0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:49.636 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.618+0000 7fc762726640 1 -- 172.21.15.190:0/2856358470 >> 172.21.15.190:0/2856358470 conn(0x7fc75c0fd3c0 msgr2=0x7fc75c0ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:49.636 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.618+0000 7fc762726640 1 -- 172.21.15.190:0/2856358470 shutdown_connections 2024-09-16T01:59:49.636 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.618+0000 7fc762726640 1 -- 172.21.15.190:0/2856358470 wait complete. 2024-09-16T01:59:49.636 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.618+0000 7fc762726640 1 Processor -- start 2024-09-16T01:59:49.636 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.618+0000 7fc762726640 1 -- start start 2024-09-16T01:59:49.636 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.619+0000 7fc762726640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc75c101bf0 0x7fc75c075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:49.636 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.619+0000 7fc762726640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc75c102df0 0x7fc75c0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:49.636 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.619+0000 7fc762726640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc75c1037b0 0x7fc75c074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:49.636 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.619+0000 7fc762726640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c10d800 con 0x7fc75c1037b0 2024-09-16T01:59:49.636 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.619+0000 7fc762726640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c10d680 con 0x7fc75c102df0 2024-09-16T01:59:49.637 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.619+0000 7fc762726640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c10d980 con 0x7fc75c101bf0 2024-09-16T01:59:49.637 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.619+0000 7fc75bfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc75c101bf0 0x7fc75c075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:49.637 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.619+0000 7fc75bfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc75c101bf0 0x7fc75c075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:48128/0 (socket says 172.21.15.190:48128) 2024-09-16T01:59:49.637 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.619+0000 7fc75bfff640 1 -- 172.21.15.190:0/2113165943 learned_addr learned my addr 172.21.15.190:0/2113165943 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-09-16T01:59:49.637 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.619+0000 7fc75b7fe640 1 --2- 172.21.15.190:0/2113165943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc75c102df0 0x7fc75c0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:49.637 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.619+0000 7fc760cc3640 1 --2- 172.21.15.190:0/2113165943 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc75c1037b0 0x7fc75c074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:49.637 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.619+0000 7fc75bfff640 1 -- 172.21.15.190:0/2113165943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc75c102df0 msgr2=0x7fc75c0745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:49.637 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.619+0000 7fc75bfff640 1 --2- 172.21.15.190:0/2113165943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc75c102df0 0x7fc75c0745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:49.637 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.619+0000 7fc75bfff640 1 -- 172.21.15.190:0/2113165943 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc75c1037b0 msgr2=0x7fc75c074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:49.637 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.619+0000 7fc75bfff640 1 --2- 172.21.15.190:0/2113165943 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc75c1037b0 0x7fc75c074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:49.637 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.619+0000 7fc75bfff640 1 -- 172.21.15.190:0/2113165943 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc75c0753e0 con 0x7fc75c101bf0 2024-09-16T01:59:49.637 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.619+0000 7fc75bfff640 1 --2- 172.21.15.190:0/2113165943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc75c101bf0 0x7fc75c075f90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc74c009230 tx=0x7fc74c004340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:49.638 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.619+0000 7fc7597fa640 1 -- 172.21.15.190:0/2113165943 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c03f030 con 0x7fc75c101bf0 2024-09-16T01:59:49.638 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.620+0000 7fc762726640 1 -- 172.21.15.190:0/2113165943 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc75c1a20a0 con 0x7fc75c101bf0 2024-09-16T01:59:49.638 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.620+0000 7fc7597fa640 1 -- 172.21.15.190:0/2113165943 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc74c003cb0 con 0x7fc75c101bf0 2024-09-16T01:59:49.638 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.620+0000 7fc7597fa640 1 -- 172.21.15.190:0/2113165943 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c035070 con 0x7fc75c101bf0 2024-09-16T01:59:49.638 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.620+0000 7fc762726640 1 -- 172.21.15.190:0/2113165943 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc75c1a2620 con 0x7fc75c101bf0 2024-09-16T01:59:49.638 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.621+0000 7fc762726640 1 -- 172.21.15.190:0/2113165943 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc75c109130 con 0x7fc75c101bf0 2024-09-16T01:59:49.638 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.622+0000 7fc7597fa640 1 -- 172.21.15.190:0/2113165943 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fc74c042030 con 0x7fc75c101bf0 2024-09-16T01:59:49.638 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.623+0000 7fc7597fa640 1 --2- 172.21.15.190:0/2113165943 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc7440a3240 0x7fc7440a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:49.638 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.623+0000 7fc7597fa640 1 -- 172.21.15.190:0/2113165943 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fc74c0eff30 con 0x7fc75c101bf0 2024-09-16T01:59:49.638 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.623+0000 7fc75b7fe640 1 --2- 172.21.15.190:0/2113165943 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc7440a3240 0x7fc7440a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:49.638 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.623+0000 7fc75b7fe640 1 --2- 172.21.15.190:0/2113165943 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc7440a3240 0x7fc7440a5700 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc75c075d10 tx=0x7fc750009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:49.639 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.626+0000 7fc7597fa640 1 -- 172.21.15.190:0/2113165943 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc74c0bd050 con 0x7fc75c101bf0 2024-09-16T01:59:49.809 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.807+0000 7fc762726640 1 -- 172.21.15.190:0/2113165943 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi105=a;smithi105=d;smithi105=g;smithi105=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l", "target": ["mon-mgr", ""]}) -- 0x7fc75c0627f0 con 0x7fc7440a3240 2024-09-16T01:59:49.816 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.815+0000 7fc7597fa640 1 -- 172.21.15.190:0/2113165943 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fc75c0627f0 con 0x7fc7440a3240 2024-09-16T01:59:49.817 INFO:teuthology.orchestra.run.smithi190.stdout:Scheduled mds.all update... 2024-09-16T01:59:49.819 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.817+0000 7fc762726640 1 -- 172.21.15.190:0/2113165943 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc7440a3240 msgr2=0x7fc7440a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:49.819 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.817+0000 7fc762726640 1 --2- 172.21.15.190:0/2113165943 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc7440a3240 0x7fc7440a5700 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc75c075d10 tx=0x7fc750009110 comp rx=0 tx=0).stop 2024-09-16T01:59:49.819 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.817+0000 7fc762726640 1 -- 172.21.15.190:0/2113165943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc75c101bf0 msgr2=0x7fc75c075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:49.819 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.817+0000 7fc762726640 1 --2- 172.21.15.190:0/2113165943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc75c101bf0 0x7fc75c075f90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc74c009230 tx=0x7fc74c004340 comp rx=0 tx=0).stop 2024-09-16T01:59:49.819 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.817+0000 7fc762726640 1 -- 172.21.15.190:0/2113165943 shutdown_connections 2024-09-16T01:59:49.819 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.817+0000 7fc762726640 1 --2- 172.21.15.190:0/2113165943 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc7440a3240 0x7fc7440a5700 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:49.819 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.817+0000 7fc762726640 1 --2- 172.21.15.190:0/2113165943 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc75c1037b0 0x7fc75c074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:49.819 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.817+0000 7fc762726640 1 --2- 172.21.15.190:0/2113165943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc75c102df0 0x7fc75c0745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:49.819 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.817+0000 7fc762726640 1 --2- 172.21.15.190:0/2113165943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc75c101bf0 0x7fc75c075f90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:49.820 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.817+0000 7fc762726640 1 -- 172.21.15.190:0/2113165943 >> 172.21.15.190:0/2113165943 conn(0x7fc75c0fd3c0 msgr2=0x7fc75c106300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:49.820 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.818+0000 7fc762726640 1 -- 172.21.15.190:0/2113165943 shutdown_connections 2024-09-16T01:59:49.820 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-16T01:59:49.818+0000 7fc762726640 1 -- 172.21.15.190:0/2113165943 wait complete. 2024-09-16T01:59:50.550 DEBUG:teuthology.orchestra.run.smithi105:mds.a> sudo journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mds.a.service 2024-09-16T01:59:50.553 DEBUG:teuthology.orchestra.run.smithi105:mds.d> sudo journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mds.d.service 2024-09-16T01:59:50.596 DEBUG:teuthology.orchestra.run.smithi105:mds.g> sudo journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mds.g.service 2024-09-16T01:59:50.639 DEBUG:teuthology.orchestra.run.smithi105:mds.j> sudo journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mds.j.service 2024-09-16T01:59:50.683 DEBUG:teuthology.orchestra.run.smithi137:mds.b> sudo journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mds.b.service 2024-09-16T01:59:50.685 DEBUG:teuthology.orchestra.run.smithi137:mds.e> sudo journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mds.e.service 2024-09-16T01:59:50.728 DEBUG:teuthology.orchestra.run.smithi137:mds.h> sudo journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mds.h.service 2024-09-16T01:59:50.732 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:50 smithi190 ceph-mon[34762]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:59:50.733 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:50 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:50.733 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:50 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:59:50.733 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:50 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:50.733 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:50 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:59:50.733 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:50 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:50.733 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:50 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:59:50.733 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:50 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:59:50.733 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:50 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:50.747 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:50 smithi137 ceph-mon[34099]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:59:50.747 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:50 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:50.747 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:50 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:59:50.747 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:50 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:50.748 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:50 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:59:50.748 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:50 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:50.748 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:50 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:59:50.748 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:50 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:59:50.748 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:50 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:50.749 DEBUG:teuthology.orchestra.run.smithi137:mds.k> sudo journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mds.k.service 2024-09-16T01:59:50.792 DEBUG:teuthology.orchestra.run.smithi190:mds.c> sudo journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mds.c.service 2024-09-16T01:59:50.794 DEBUG:teuthology.orchestra.run.smithi190:mds.f> sudo journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mds.f.service 2024-09-16T01:59:50.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:50 smithi105 ceph-mon[31197]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:59:50.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:50 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:50.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:50 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:59:50.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:50 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:50.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:50 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:59:50.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:50 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:50.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:50 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:59:50.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:50 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:59:50.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:50 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:50.837 DEBUG:teuthology.orchestra.run.smithi190:mds.i> sudo journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mds.i.service 2024-09-16T01:59:50.880 DEBUG:teuthology.orchestra.run.smithi190:mds.l> sudo journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mds.l.service 2024-09-16T01:59:50.923 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-16T01:59:50.923 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:59:51.168 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:59:51.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:51 smithi105 ceph-mon[31197]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi105=a;smithi105=d;smithi105=g;smithi105=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:59:51.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:51 smithi105 ceph-mon[31197]: Saving service mds.all spec with placement smithi105=a;smithi105=d;smithi105=g;smithi105=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l;count:12 2024-09-16T01:59:51.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:51 smithi105 ceph-mon[31197]: Deploying daemon mds.l on smithi190 2024-09-16T01:59:51.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.647+0000 7f524c495640 1 -- 172.21.15.105:0/2140667141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5244102d40 msgr2=0x7f5244103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:51.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.647+0000 7f524c495640 1 --2- 172.21.15.105:0/2140667141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5244102d40 0x7f5244103120 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5234009900 tx=0x7f523402a480 comp rx=0 tx=0).stop 2024-09-16T01:59:51.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.647+0000 7f524c495640 1 -- 172.21.15.105:0/2140667141 shutdown_connections 2024-09-16T01:59:51.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.647+0000 7f524c495640 1 --2- 172.21.15.105:0/2140667141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5244103660 0x7f524410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:51.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.647+0000 7f524c495640 1 --2- 172.21.15.105:0/2140667141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f524410c170 0x7f524410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:51.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.647+0000 7f524c495640 1 --2- 172.21.15.105:0/2140667141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5244102d40 0x7f5244103120 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:51.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.647+0000 7f524c495640 1 -- 172.21.15.105:0/2140667141 >> 172.21.15.105:0/2140667141 conn(0x7f52440fc7a0 msgr2=0x7f52440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:51.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.647+0000 7f524c495640 1 -- 172.21.15.105:0/2140667141 shutdown_connections 2024-09-16T01:59:51.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.647+0000 7f524c495640 1 -- 172.21.15.105:0/2140667141 wait complete. 2024-09-16T01:59:51.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.648+0000 7f524c495640 1 Processor -- start 2024-09-16T01:59:51.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.648+0000 7f524c495640 1 -- start start 2024-09-16T01:59:51.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.648+0000 7f524c495640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5244102d40 0x7f52441984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:51.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.648+0000 7f524c495640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5244103660 0x7f5244198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:51.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.648+0000 7f524c495640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f524410c170 0x7f524419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:51.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.648+0000 7f524c495640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5244111f30 con 0x7f5244102d40 2024-09-16T01:59:51.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.648+0000 7f524c495640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5244111db0 con 0x7f524410c170 2024-09-16T01:59:51.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.648+0000 7f524c495640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f52441120b0 con 0x7f5244103660 2024-09-16T01:59:51.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.648+0000 7f524a231640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5244102d40 0x7f52441984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:51.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.648+0000 7f524a231640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5244102d40 0x7f52441984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51930/0 (socket says 172.21.15.105:51930) 2024-09-16T01:59:51.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.648+0000 7f524a231640 1 -- 172.21.15.105:0/2741080299 learned_addr learned my addr 172.21.15.105:0/2741080299 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:59:51.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.648+0000 7f5249a30640 1 --2- 172.21.15.105:0/2741080299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5244103660 0x7f5244198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:51.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.648+0000 7f524aa32640 1 --2- 172.21.15.105:0/2741080299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f524410c170 0x7f524419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:51.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.648+0000 7f524a231640 1 -- 172.21.15.105:0/2741080299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5244103660 msgr2=0x7f5244198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:51.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.648+0000 7f524a231640 1 --2- 172.21.15.105:0/2741080299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5244103660 0x7f5244198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:51.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.648+0000 7f524a231640 1 -- 172.21.15.105:0/2741080299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f524410c170 msgr2=0x7f524419cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:51.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.648+0000 7f524a231640 1 --2- 172.21.15.105:0/2741080299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f524410c170 0x7f524419cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:51.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.648+0000 7f524a231640 1 -- 172.21.15.105:0/2741080299 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f524419d480 con 0x7f5244102d40 2024-09-16T01:59:51.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.649+0000 7f524a231640 1 --2- 172.21.15.105:0/2741080299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5244102d40 0x7f52441984d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5234005ab0 tx=0x7f5234003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:51.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.649+0000 7f523b7fe640 1 -- 172.21.15.105:0/2741080299 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523403f070 con 0x7f5244102d40 2024-09-16T01:59:51.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.649+0000 7f523b7fe640 1 -- 172.21.15.105:0/2741080299 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5234031d60 con 0x7f5244102d40 2024-09-16T01:59:51.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.649+0000 7f524c495640 1 -- 172.21.15.105:0/2741080299 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f524419d710 con 0x7f5244102d40 2024-09-16T01:59:51.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.649+0000 7f523b7fe640 1 -- 172.21.15.105:0/2741080299 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5234035030 con 0x7f5244102d40 2024-09-16T01:59:51.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.649+0000 7f524c495640 1 -- 172.21.15.105:0/2741080299 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52441a5050 con 0x7f5244102d40 2024-09-16T01:59:51.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.650+0000 7f524c495640 1 -- 172.21.15.105:0/2741080299 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f524406b070 con 0x7f5244102d40 2024-09-16T01:59:51.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.651+0000 7f523b7fe640 1 -- 172.21.15.105:0/2741080299 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f5234042070 con 0x7f5244102d40 2024-09-16T01:59:51.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.652+0000 7f523b7fe640 1 --2- 172.21.15.105:0/2741080299 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f52300a3240 0x7f52300a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:51.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.652+0000 7f5249a30640 1 --2- 172.21.15.105:0/2741080299 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f52300a3240 0x7f52300a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:51.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.652+0000 7f523b7fe640 1 -- 172.21.15.105:0/2741080299 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f52340efe90 con 0x7f5244102d40 2024-09-16T01:59:51.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.652+0000 7f5249a30640 1 --2- 172.21.15.105:0/2741080299 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f52300a3240 0x7f52300a5700 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f52440fee10 tx=0x7f5240009060 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:51.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.656+0000 7f523b7fe640 1 -- 172.21.15.105:0/2741080299 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f52340bd050 con 0x7f5244102d40 2024-09-16T01:59:51.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:51 smithi190 ceph-mon[34762]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi105=a;smithi105=d;smithi105=g;smithi105=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:59:51.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:51 smithi190 ceph-mon[34762]: Saving service mds.all spec with placement smithi105=a;smithi105=d;smithi105=g;smithi105=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l;count:12 2024-09-16T01:59:51.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:51 smithi190 ceph-mon[34762]: Deploying daemon mds.l on smithi190 2024-09-16T01:59:51.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.855+0000 7f524c495640 1 -- 172.21.15.105:0/2741080299 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52441071e0 con 0x7f5244102d40 2024-09-16T01:59:51.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.855+0000 7f523b7fe640 1 -- 172.21.15.105:0/2741080299 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f52340bc950 con 0x7f5244102d40 2024-09-16T01:59:51.858 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:59:51.858 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":1,"btime":"2024-09-16T01:52:44:387843+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-16T01:59:51.858 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 1 2024-09-16T01:59:51.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.857+0000 7f524c495640 1 -- 172.21.15.105:0/2741080299 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f52300a3240 msgr2=0x7f52300a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:51.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.857+0000 7f524c495640 1 --2- 172.21.15.105:0/2741080299 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f52300a3240 0x7f52300a5700 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f52440fee10 tx=0x7f5240009060 comp rx=0 tx=0).stop 2024-09-16T01:59:51.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.857+0000 7f524c495640 1 -- 172.21.15.105:0/2741080299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5244102d40 msgr2=0x7f52441984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:51.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.857+0000 7f524c495640 1 --2- 172.21.15.105:0/2741080299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5244102d40 0x7f52441984d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5234005ab0 tx=0x7f5234003d50 comp rx=0 tx=0).stop 2024-09-16T01:59:51.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.857+0000 7f524c495640 1 -- 172.21.15.105:0/2741080299 shutdown_connections 2024-09-16T01:59:51.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.857+0000 7f524c495640 1 --2- 172.21.15.105:0/2741080299 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f52300a3240 0x7f52300a5700 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:51.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.857+0000 7f524c495640 1 --2- 172.21.15.105:0/2741080299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5244103660 0x7f5244198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:51.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.857+0000 7f524c495640 1 --2- 172.21.15.105:0/2741080299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f524410c170 0x7f524419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:51.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.858+0000 7f524c495640 1 --2- 172.21.15.105:0/2741080299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5244102d40 0x7f52441984d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:51.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.858+0000 7f524c495640 1 -- 172.21.15.105:0/2741080299 >> 172.21.15.105:0/2741080299 conn(0x7f52440fc7a0 msgr2=0x7f52440fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:51.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.858+0000 7f524c495640 1 -- 172.21.15.105:0/2741080299 shutdown_connections 2024-09-16T01:59:51.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:51.858+0000 7f524c495640 1 -- 172.21.15.105:0/2741080299 wait complete. 2024-09-16T01:59:51.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:51 smithi137 ceph-mon[34099]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi105=a;smithi105=d;smithi105=g;smithi105=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:59:51.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:51 smithi137 ceph-mon[34099]: Saving service mds.all spec with placement smithi105=a;smithi105=d;smithi105=g;smithi105=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l;count:12 2024-09-16T01:59:51.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:51 smithi137 ceph-mon[34099]: Deploying daemon mds.l on smithi190 2024-09-16T01:59:52.484 INFO:journalctl@ceph.mds.l.smithi190.stdout:Sep 16 01:59:52 smithi190 systemd[1]: Starting Ceph mds.l for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T01:59:52.648 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:52 smithi105 ceph-mon[31197]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:59:52.648 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:52 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2741080299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:59:52.765 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:52 smithi190 ceph-mon[34762]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:59:52.765 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:52 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/2741080299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:59:52.766 INFO:journalctl@ceph.mds.l.smithi190.stdout:Sep 16 01:59:52 smithi190 podman[63685]: 2024-09-16 01:59:52.510287843 +0000 UTC m=+0.015803736 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T01:59:52.766 INFO:journalctl@ceph.mds.l.smithi190.stdout:Sep 16 01:59:52 smithi190 podman[63685]: 2024-09-16 01:59:52.633687807 +0000 UTC m=+0.139203697 container create 24708c1f20096d70b332937f7b552076a8b6ba54b6b6dfa0980e901c9ac40e9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-l, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True) 2024-09-16T01:59:52.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:52 smithi137 ceph-mon[34099]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:59:52.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:52 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/2741080299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:59:53.343 INFO:journalctl@ceph.mds.l.smithi190.stdout:Sep 16 01:59:52 smithi190 podman[63685]: 2024-09-16 01:59:52.8948402 +0000 UTC m=+0.400356089 container init 24708c1f20096d70b332937f7b552076a8b6ba54b6b6dfa0980e901c9ac40e9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-l, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-16T01:59:53.343 INFO:journalctl@ceph.mds.l.smithi190.stdout:Sep 16 01:59:52 smithi190 podman[63685]: 2024-09-16 01:59:52.897557744 +0000 UTC m=+0.403073638 container start 24708c1f20096d70b332937f7b552076a8b6ba54b6b6dfa0980e901c9ac40e9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-l, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True) 2024-09-16T01:59:53.343 INFO:journalctl@ceph.mds.l.smithi190.stdout:Sep 16 01:59:52 smithi190 ceph-mds[63704]: -- 172.21.15.190:0/3071099845 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55db63e9b860 con 0x55db63e78800 2024-09-16T01:59:53.343 INFO:journalctl@ceph.mds.l.smithi190.stdout:Sep 16 01:59:52 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-l[63700]: starting mds.l at 2024-09-16T01:59:53.343 INFO:journalctl@ceph.mds.l.smithi190.stdout:Sep 16 01:59:52 smithi190 bash[63685]: 24708c1f20096d70b332937f7b552076a8b6ba54b6b6dfa0980e901c9ac40e9d 2024-09-16T01:59:53.343 INFO:journalctl@ceph.mds.l.smithi190.stdout:Sep 16 01:59:52 smithi190 systemd[1]: Started Ceph mds.l for 423b7a2a-73ce-11ef-bceb-c7b262605968. 2024-09-16T01:59:54.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:53 smithi137 ceph-mon[34099]: mds.? [v2:172.21.15.190:6832/683519097,v1:172.21.15.190:6833/683519097] up:boot 2024-09-16T01:59:54.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:53 smithi137 ceph-mon[34099]: fsmap 1 up:standby 2024-09-16T01:59:54.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:53 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-16T01:59:54.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:53 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:54.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:53 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:54.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:53 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:54.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:53 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:59:54.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:53 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:59:54.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:53 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:54.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:53 smithi137 ceph-mon[34099]: Deploying daemon mds.k on smithi137 2024-09-16T01:59:54.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:53 smithi137 ceph-mon[34099]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:59:54.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:53 smithi105 ceph-mon[31197]: mds.? [v2:172.21.15.190:6832/683519097,v1:172.21.15.190:6833/683519097] up:boot 2024-09-16T01:59:54.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:53 smithi105 ceph-mon[31197]: fsmap 1 up:standby 2024-09-16T01:59:54.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:53 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-16T01:59:54.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:53 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:54.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:53 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:54.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:53 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:54.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:53 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:59:54.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:53 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:59:54.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:53 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:54.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:53 smithi105 ceph-mon[31197]: Deploying daemon mds.k on smithi137 2024-09-16T01:59:54.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:53 smithi105 ceph-mon[31197]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:59:54.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:53 smithi190 ceph-mon[34762]: mds.? [v2:172.21.15.190:6832/683519097,v1:172.21.15.190:6833/683519097] up:boot 2024-09-16T01:59:54.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:53 smithi190 ceph-mon[34762]: fsmap 1 up:standby 2024-09-16T01:59:54.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:53 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-16T01:59:54.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:53 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:54.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:53 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:54.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:53 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:54.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:53 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:59:54.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:53 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:59:54.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:53 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:54.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:53 smithi190 ceph-mon[34762]: Deploying daemon mds.k on smithi137 2024-09-16T01:59:54.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:53 smithi190 ceph-mon[34762]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:59:54.677 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:59:54.931 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:59:55.171 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 16 01:59:55 smithi137 systemd[1]: Starting Ceph mds.k for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T01:59:55.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.408+0000 7f1cf1dc0640 1 -- 172.21.15.105:0/1234770390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cec100a70 msgr2=0x7f1cec10ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:55.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.408+0000 7f1cf1dc0640 1 --2- 172.21.15.105:0/1234770390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cec100a70 0x7f1cec10ea60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4009ca0 tx=0x7f1ce402a8c0 comp rx=0 tx=0).stop 2024-09-16T01:59:55.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.408+0000 7f1cf1dc0640 1 -- 172.21.15.105:0/1234770390 shutdown_connections 2024-09-16T01:59:55.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.408+0000 7f1cf1dc0640 1 --2- 172.21.15.105:0/1234770390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cec100a70 0x7f1cec10ea60 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:55.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.408+0000 7f1cf1dc0640 1 --2- 172.21.15.105:0/1234770390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1cec1000d0 0x7f1cec100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:55.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.408+0000 7f1cf1dc0640 1 --2- 172.21.15.105:0/1234770390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1cec106ba0 0x7f1cec106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:55.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.408+0000 7f1cf1dc0640 1 -- 172.21.15.105:0/1234770390 >> 172.21.15.105:0/1234770390 conn(0x7f1cec077210 msgr2=0x7f1cec0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:55.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.408+0000 7f1cf1dc0640 1 -- 172.21.15.105:0/1234770390 shutdown_connections 2024-09-16T01:59:55.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.408+0000 7f1cf1dc0640 1 -- 172.21.15.105:0/1234770390 wait complete. 2024-09-16T01:59:55.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.409+0000 7f1cf1dc0640 1 Processor -- start 2024-09-16T01:59:55.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.409+0000 7f1cf1dc0640 1 -- start start 2024-09-16T01:59:55.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.409+0000 7f1cf1dc0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cec1000d0 0x7f1cec1012b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:55.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.409+0000 7f1cf1dc0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1cec100a70 0x7f1cec1017f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:55.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.409+0000 7f1cf1dc0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1cec106ba0 0x7f1cec101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:55.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.409+0000 7f1cf1dc0640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec111f40 con 0x7f1cec1000d0 2024-09-16T01:59:55.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.409+0000 7f1cf1dc0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec111dc0 con 0x7f1cec106ba0 2024-09-16T01:59:55.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.409+0000 7f1cf1dc0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec1120c0 con 0x7f1cec100a70 2024-09-16T01:59:55.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.409+0000 7f1ceb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cec1000d0 0x7f1cec1012b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:55.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.410+0000 7f1ceb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cec1000d0 0x7f1cec1012b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51948/0 (socket says 172.21.15.105:51948) 2024-09-16T01:59:55.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.410+0000 7f1ceb7fe640 1 -- 172.21.15.105:0/2839066491 learned_addr learned my addr 172.21.15.105:0/2839066491 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:59:55.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.410+0000 7f1cebfff640 1 --2- 172.21.15.105:0/2839066491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1cec106ba0 0x7f1cec101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:55.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.410+0000 7f1ce3fff640 1 --2- 172.21.15.105:0/2839066491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1cec100a70 0x7f1cec1017f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:55.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.410+0000 7f1ceb7fe640 1 -- 172.21.15.105:0/2839066491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1cec100a70 msgr2=0x7f1cec1017f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:55.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.410+0000 7f1ceb7fe640 1 --2- 172.21.15.105:0/2839066491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1cec100a70 0x7f1cec1017f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:55.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.410+0000 7f1ceb7fe640 1 -- 172.21.15.105:0/2839066491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1cec106ba0 msgr2=0x7f1cec101d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:55.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.410+0000 7f1ceb7fe640 1 --2- 172.21.15.105:0/2839066491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1cec106ba0 0x7f1cec101d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:55.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.410+0000 7f1ceb7fe640 1 -- 172.21.15.105:0/2839066491 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cec1a6880 con 0x7f1cec1000d0 2024-09-16T01:59:55.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.410+0000 7f1ce3fff640 1 --2- 172.21.15.105:0/2839066491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1cec100a70 0x7f1cec1017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:59:55.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.410+0000 7f1ceb7fe640 1 --2- 172.21.15.105:0/2839066491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cec1000d0 0x7f1cec1012b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc00d950 tx=0x7f1cdc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:55.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.410+0000 7f1ce97fa640 1 -- 172.21.15.105:0/2839066491 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc017030 con 0x7f1cec1000d0 2024-09-16T01:59:55.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.410+0000 7f1ce97fa640 1 -- 172.21.15.105:0/2839066491 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1cdc004020 con 0x7f1cec1000d0 2024-09-16T01:59:55.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.410+0000 7f1cf1dc0640 1 -- 172.21.15.105:0/2839066491 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cec1a6b70 con 0x7f1cec1000d0 2024-09-16T01:59:55.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.410+0000 7f1ce97fa640 1 -- 172.21.15.105:0/2839066491 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc0047a0 con 0x7f1cec1000d0 2024-09-16T01:59:55.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.410+0000 7f1cf1dc0640 1 -- 172.21.15.105:0/2839066491 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cec1a70b0 con 0x7f1cec1000d0 2024-09-16T01:59:55.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.411+0000 7f1cf1dc0640 1 -- 172.21.15.105:0/2839066491 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cec100530 con 0x7f1cec1000d0 2024-09-16T01:59:55.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.412+0000 7f1ce97fa640 1 -- 172.21.15.105:0/2839066491 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f1cdc0041d0 con 0x7f1cec1000d0 2024-09-16T01:59:55.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.413+0000 7f1ce97fa640 1 --2- 172.21.15.105:0/2839066491 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f1cd80a3240 0x7f1cd80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:55.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.413+0000 7f1ce3fff640 1 --2- 172.21.15.105:0/2839066491 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f1cd80a3240 0x7f1cd80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:55.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.413+0000 7f1ce97fa640 1 -- 172.21.15.105:0/2839066491 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f1cdc0cee90 con 0x7f1cec1000d0 2024-09-16T01:59:55.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.414+0000 7f1ce3fff640 1 --2- 172.21.15.105:0/2839066491 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f1cd80a3240 0x7f1cd80a5700 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1cec102d40 tx=0x7f1cd4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:55.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.417+0000 7f1ce97fa640 1 -- 172.21.15.105:0/2839066491 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1cdc09c050 con 0x7f1cec1000d0 2024-09-16T01:59:55.521 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 16 01:59:55 smithi137 podman[63207]: 2024-09-16 01:59:55.171601726 +0000 UTC m=+0.016400886 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T01:59:55.521 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 16 01:59:55 smithi137 podman[63207]: 2024-09-16 01:59:55.300647806 +0000 UTC m=+0.145446957 container create defb1dff59d08dc90807bcab5501b595a4da78b794ae5f88a622368317c4cac6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-16T01:59:55.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.623+0000 7f1cf1dc0640 1 -- 172.21.15.105:0/2839066491 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1cec1027e0 con 0x7f1cec1000d0 2024-09-16T01:59:55.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.623+0000 7f1ce97fa640 1 -- 172.21.15.105:0/2839066491 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1435 (secure 0 0 0) 0x7f1cdc09b950 con 0x7f1cec1000d0 2024-09-16T01:59:55.625 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:59:55.626 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":2,"btime":"2024-09-16T01:59:52:926779+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/683519097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":683519097},{"type":"v1","addr":"172.21.15.190:6833","nonce":683519097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-16T01:59:55.626 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 2 2024-09-16T01:59:55.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.625+0000 7f1cf1dc0640 1 -- 172.21.15.105:0/2839066491 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f1cd80a3240 msgr2=0x7f1cd80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:55.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.625+0000 7f1cf1dc0640 1 --2- 172.21.15.105:0/2839066491 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f1cd80a3240 0x7f1cd80a5700 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1cec102d40 tx=0x7f1cd4005f90 comp rx=0 tx=0).stop 2024-09-16T01:59:55.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.625+0000 7f1cf1dc0640 1 -- 172.21.15.105:0/2839066491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cec1000d0 msgr2=0x7f1cec1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:55.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.625+0000 7f1cf1dc0640 1 --2- 172.21.15.105:0/2839066491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cec1000d0 0x7f1cec1012b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc00d950 tx=0x7f1cdc00de10 comp rx=0 tx=0).stop 2024-09-16T01:59:55.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.625+0000 7f1cf1dc0640 1 -- 172.21.15.105:0/2839066491 shutdown_connections 2024-09-16T01:59:55.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.625+0000 7f1cf1dc0640 1 --2- 172.21.15.105:0/2839066491 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f1cd80a3240 0x7f1cd80a5700 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:55.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.625+0000 7f1cf1dc0640 1 --2- 172.21.15.105:0/2839066491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1cec100a70 0x7f1cec1017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:55.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.625+0000 7f1cf1dc0640 1 --2- 172.21.15.105:0/2839066491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1cec106ba0 0x7f1cec101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:55.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.625+0000 7f1cf1dc0640 1 --2- 172.21.15.105:0/2839066491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cec1000d0 0x7f1cec1012b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:55.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.625+0000 7f1cf1dc0640 1 -- 172.21.15.105:0/2839066491 >> 172.21.15.105:0/2839066491 conn(0x7f1cec077210 msgr2=0x7f1cec104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:55.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.625+0000 7f1cf1dc0640 1 -- 172.21.15.105:0/2839066491 shutdown_connections 2024-09-16T01:59:55.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:55.625+0000 7f1cf1dc0640 1 -- 172.21.15.105:0/2839066491 wait complete. 2024-09-16T01:59:55.921 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 16 01:59:55 smithi137 podman[63207]: 2024-09-16 01:59:55.520255981 +0000 UTC m=+0.365055137 container init defb1dff59d08dc90807bcab5501b595a4da78b794ae5f88a622368317c4cac6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-k, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-16T01:59:55.921 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 16 01:59:55 smithi137 podman[63207]: 2024-09-16 01:59:55.523264016 +0000 UTC m=+0.368063167 container start defb1dff59d08dc90807bcab5501b595a4da78b794ae5f88a622368317c4cac6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-k, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-16T01:59:55.921 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 16 01:59:55 smithi137 ceph-mds[63226]: -- 172.21.15.137:0/2179484749 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5627d33ff860 con 0x5627d33dd400 2024-09-16T01:59:55.921 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 16 01:59:55 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-k[63222]: starting mds.k at 2024-09-16T01:59:55.921 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 16 01:59:55 smithi137 bash[63207]: defb1dff59d08dc90807bcab5501b595a4da78b794ae5f88a622368317c4cac6 2024-09-16T01:59:55.922 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 16 01:59:55 smithi137 systemd[1]: Started Ceph mds.k for 423b7a2a-73ce-11ef-bceb-c7b262605968. 2024-09-16T01:59:56.681 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:56 smithi105 ceph-mon[31197]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:59:56.681 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:56 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2839066491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:59:56.681 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:56 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:56.681 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:56 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:56.681 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:56 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:56.682 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:56 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:59:56.682 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:56 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:59:56.682 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:56 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:56.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:56 smithi190 ceph-mon[34762]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:59:56.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:56 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/2839066491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:59:56.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:56 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:56.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:56 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:56.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:56 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:56.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:56 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:59:56.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:56 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:59:56.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:56 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:56.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:56 smithi137 ceph-mon[34099]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:59:56.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:56 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/2839066491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:59:56.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:56 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:56.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:56 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:56.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:56 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T01:59:56.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:56 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:59:56.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:56 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:59:56.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:56 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:57.713 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:57 smithi105 ceph-mon[31197]: Deploying daemon mds.j on smithi105 2024-09-16T01:59:57.714 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:57 smithi105 ceph-mon[31197]: mds.? [v2:172.21.15.137:6832/3025442862,v1:172.21.15.137:6833/3025442862] up:boot 2024-09-16T01:59:57.714 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:57 smithi105 ceph-mon[31197]: fsmap 2 up:standby 2024-09-16T01:59:57.714 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:57 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T01:59:57.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:57 smithi190 ceph-mon[34762]: Deploying daemon mds.j on smithi105 2024-09-16T01:59:57.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:57 smithi190 ceph-mon[34762]: mds.? [v2:172.21.15.137:6832/3025442862,v1:172.21.15.137:6833/3025442862] up:boot 2024-09-16T01:59:57.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:57 smithi190 ceph-mon[34762]: fsmap 2 up:standby 2024-09-16T01:59:57.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:57 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T01:59:57.920 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:57 smithi137 ceph-mon[34099]: Deploying daemon mds.j on smithi105 2024-09-16T01:59:57.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:57 smithi137 ceph-mon[34099]: mds.? [v2:172.21.15.137:6832/3025442862,v1:172.21.15.137:6833/3025442862] up:boot 2024-09-16T01:59:57.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:57 smithi137 ceph-mon[34099]: fsmap 2 up:standby 2024-09-16T01:59:57.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:57 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T01:59:58.295 INFO:journalctl@ceph.mds.j.smithi105.stdout:Sep 16 01:59:58 smithi105 systemd[1]: Starting Ceph mds.j for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T01:59:58.450 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:59:58.677 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:58 smithi105 ceph-mon[31197]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:59:58.677 INFO:journalctl@ceph.mds.j.smithi105.stdout:Sep 16 01:59:58 smithi105 podman[70489]: 2024-09-16 01:59:58.303677684 +0000 UTC m=+0.015925840 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T01:59:58.678 INFO:journalctl@ceph.mds.j.smithi105.stdout:Sep 16 01:59:58 smithi105 podman[70489]: 2024-09-16 01:59:58.432350907 +0000 UTC m=+0.144599048 container create d83fe50c53e3dcfc02a657e3a118ae57199201748919679b21196b49b6f9094c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-j, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-16T01:59:58.678 INFO:journalctl@ceph.mds.j.smithi105.stdout:Sep 16 01:59:58 smithi105 podman[70489]: 2024-09-16 01:59:58.67680634 +0000 UTC m=+0.389054481 container init d83fe50c53e3dcfc02a657e3a118ae57199201748919679b21196b49b6f9094c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-j, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-16T01:59:58.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:58 smithi190 ceph-mon[34762]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:59:58.851 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:59:58.920 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:58 smithi137 ceph-mon[34099]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:59:58.973 INFO:journalctl@ceph.mds.j.smithi105.stdout:Sep 16 01:59:58 smithi105 podman[70489]: 2024-09-16 01:59:58.680459106 +0000 UTC m=+0.392707250 container start d83fe50c53e3dcfc02a657e3a118ae57199201748919679b21196b49b6f9094c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-j, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-16T01:59:58.973 INFO:journalctl@ceph.mds.j.smithi105.stdout:Sep 16 01:59:58 smithi105 ceph-mds[70538]: -- 172.21.15.105:0/1596464878 <== mon.0 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b315385860 con 0x55b315363400 2024-09-16T01:59:58.973 INFO:journalctl@ceph.mds.j.smithi105.stdout:Sep 16 01:59:58 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-j[70522]: starting mds.j at 2024-09-16T01:59:58.973 INFO:journalctl@ceph.mds.j.smithi105.stdout:Sep 16 01:59:58 smithi105 bash[70489]: d83fe50c53e3dcfc02a657e3a118ae57199201748919679b21196b49b6f9094c 2024-09-16T01:59:58.974 INFO:journalctl@ceph.mds.j.smithi105.stdout:Sep 16 01:59:58 smithi105 systemd[1]: Started Ceph mds.j for 423b7a2a-73ce-11ef-bceb-c7b262605968. 2024-09-16T01:59:59.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.329+0000 7f5778dbf640 1 -- 172.21.15.105:0/1598691634 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5774103660 msgr2=0x7f577410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:59.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.329+0000 7f5778dbf640 1 --2- 172.21.15.105:0/1598691634 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5774103660 0x7f577410bb50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5764009900 tx=0x7f576402a480 comp rx=0 tx=0).stop 2024-09-16T01:59:59.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.329+0000 7f5778dbf640 1 -- 172.21.15.105:0/1598691634 shutdown_connections 2024-09-16T01:59:59.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.329+0000 7f5778dbf640 1 --2- 172.21.15.105:0/1598691634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f577410c170 0x7f577410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:59.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.329+0000 7f5778dbf640 1 --2- 172.21.15.105:0/1598691634 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5774103660 0x7f577410bb50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:59.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.329+0000 7f5778dbf640 1 --2- 172.21.15.105:0/1598691634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5774102d40 0x7f5774103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:59.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.329+0000 7f5778dbf640 1 -- 172.21.15.105:0/1598691634 >> 172.21.15.105:0/1598691634 conn(0x7f57740fc7a0 msgr2=0x7f57740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:59.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.329+0000 7f5778dbf640 1 -- 172.21.15.105:0/1598691634 shutdown_connections 2024-09-16T01:59:59.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.329+0000 7f5778dbf640 1 -- 172.21.15.105:0/1598691634 wait complete. 2024-09-16T01:59:59.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.330+0000 7f5778dbf640 1 Processor -- start 2024-09-16T01:59:59.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.330+0000 7f5778dbf640 1 -- start start 2024-09-16T01:59:59.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.330+0000 7f5778dbf640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5774102d40 0x7f57741983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:59.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.330+0000 7f5778dbf640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5774103660 0x7f5774198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:59.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.330+0000 7f5778dbf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f577410c170 0x7f577419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:59.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.330+0000 7f5778dbf640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5774111e60 con 0x7f577410c170 2024-09-16T01:59:59.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.330+0000 7f5778dbf640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5774111ce0 con 0x7f5774103660 2024-09-16T01:59:59.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.330+0000 7f5778dbf640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5774111fe0 con 0x7f5774102d40 2024-09-16T01:59:59.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.330+0000 7f5772d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f577410c170 0x7f577419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:59.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.330+0000 7f5772d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f577410c170 0x7f577419ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60940/0 (socket says 172.21.15.105:60940) 2024-09-16T01:59:59.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.330+0000 7f5772d9d640 1 -- 172.21.15.105:0/3576509908 learned_addr learned my addr 172.21.15.105:0/3576509908 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T01:59:59.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.330+0000 7f5771d9b640 1 --2- 172.21.15.105:0/3576509908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5774103660 0x7f5774198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:59.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.331+0000 7f577259c640 1 --2- 172.21.15.105:0/3576509908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5774102d40 0x7f57741983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:59.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.331+0000 7f5772d9d640 1 -- 172.21.15.105:0/3576509908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5774102d40 msgr2=0x7f57741983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:59.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.331+0000 7f5772d9d640 1 --2- 172.21.15.105:0/3576509908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5774102d40 0x7f57741983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:59.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.331+0000 7f5772d9d640 1 -- 172.21.15.105:0/3576509908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5774103660 msgr2=0x7f5774198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:59.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.331+0000 7f5772d9d640 1 --2- 172.21.15.105:0/3576509908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5774103660 0x7f5774198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:59.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.331+0000 7f5772d9d640 1 -- 172.21.15.105:0/3576509908 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f577419d3a0 con 0x7f577410c170 2024-09-16T01:59:59.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.331+0000 7f5771d9b640 1 --2- 172.21.15.105:0/3576509908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5774103660 0x7f5774198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:59:59.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.331+0000 7f5772d9d640 1 --2- 172.21.15.105:0/3576509908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f577410c170 0x7f577419ccc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f576c00b970 tx=0x7f576c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:59.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.331+0000 7f57637fe640 1 -- 172.21.15.105:0/3576509908 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c019070 con 0x7f577410c170 2024-09-16T01:59:59.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.331+0000 7f5778dbf640 1 -- 172.21.15.105:0/3576509908 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f577419d690 con 0x7f577410c170 2024-09-16T01:59:59.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.331+0000 7f57637fe640 1 -- 172.21.15.105:0/3576509908 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f576c014440 con 0x7f577410c170 2024-09-16T01:59:59.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.331+0000 7f57637fe640 1 -- 172.21.15.105:0/3576509908 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c0075d0 con 0x7f577410c170 2024-09-16T01:59:59.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.331+0000 7f5778dbf640 1 -- 172.21.15.105:0/3576509908 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57741a4f70 con 0x7f577410c170 2024-09-16T01:59:59.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.332+0000 7f5778dbf640 1 -- 172.21.15.105:0/3576509908 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5774103120 con 0x7f577410c170 2024-09-16T01:59:59.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.333+0000 7f57637fe640 1 -- 172.21.15.105:0/3576509908 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f576c01c070 con 0x7f577410c170 2024-09-16T01:59:59.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.334+0000 7f57637fe640 1 --2- 172.21.15.105:0/3576509908 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f575c0a3240 0x7f575c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:59.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.334+0000 7f577259c640 1 --2- 172.21.15.105:0/3576509908 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f575c0a3240 0x7f575c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:59.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.334+0000 7f57637fe640 1 -- 172.21.15.105:0/3576509908 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f576c0cefa0 con 0x7f577410c170 2024-09-16T01:59:59.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.334+0000 7f577259c640 1 --2- 172.21.15.105:0/3576509908 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f575c0a3240 0x7f575c0a5700 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f57680058c0 tx=0x7f5768005c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:59.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.337+0000 7f57637fe640 1 -- 172.21.15.105:0/3576509908 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f576c09c160 con 0x7f577410c170 2024-09-16T01:59:59.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.551+0000 7f5778dbf640 1 -- 172.21.15.105:0/3576509908 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f577402d7c0 con 0x7f577410c170 2024-09-16T01:59:59.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.551+0000 7f57637fe640 1 -- 172.21.15.105:0/3576509908 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3091 (secure 0 0 0) 0x7f576c09ba60 con 0x7f577410c170 2024-09-16T01:59:59.554 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T01:59:59.554 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":4,"btime":"2024-09-16T01:59:59:488665+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2057699920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2057699920},{"type":"v1","addr":"172.21.15.105:6835","nonce":2057699920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/683519097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":683519097},{"type":"v1","addr":"172.21.15.190:6833","nonce":683519097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/3025442862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3025442862},{"type":"v1","addr":"172.21.15.137:6833","nonce":3025442862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-16T01:59:59.554 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 4 2024-09-16T01:59:59.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.554+0000 7f5778dbf640 1 -- 172.21.15.105:0/3576509908 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f575c0a3240 msgr2=0x7f575c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:59.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.554+0000 7f5778dbf640 1 --2- 172.21.15.105:0/3576509908 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f575c0a3240 0x7f575c0a5700 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f57680058c0 tx=0x7f5768005c60 comp rx=0 tx=0).stop 2024-09-16T01:59:59.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.554+0000 7f5778dbf640 1 -- 172.21.15.105:0/3576509908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f577410c170 msgr2=0x7f577419ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:59.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.554+0000 7f5778dbf640 1 --2- 172.21.15.105:0/3576509908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f577410c170 0x7f577419ccc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f576c00b970 tx=0x7f576c00be30 comp rx=0 tx=0).stop 2024-09-16T01:59:59.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.554+0000 7f5778dbf640 1 -- 172.21.15.105:0/3576509908 shutdown_connections 2024-09-16T01:59:59.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.554+0000 7f5778dbf640 1 --2- 172.21.15.105:0/3576509908 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f575c0a3240 0x7f575c0a5700 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:59.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.554+0000 7f5778dbf640 1 --2- 172.21.15.105:0/3576509908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f577410c170 0x7f577419ccc0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:59.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.554+0000 7f5778dbf640 1 --2- 172.21.15.105:0/3576509908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5774103660 0x7f5774198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:59.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.554+0000 7f5778dbf640 1 --2- 172.21.15.105:0/3576509908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5774102d40 0x7f57741983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:59.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.554+0000 7f5778dbf640 1 -- 172.21.15.105:0/3576509908 >> 172.21.15.105:0/3576509908 conn(0x7f57740fc7a0 msgr2=0x7f57740fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:59.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.554+0000 7f5778dbf640 1 -- 172.21.15.105:0/3576509908 shutdown_connections 2024-09-16T01:59:59.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T01:59:59.554+0000 7f5778dbf640 1 -- 172.21.15.105:0/3576509908 wait complete. 2024-09-16T02:00:00.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:59 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:00.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:59 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:00.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:59 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:00.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:59 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:00:00.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:59 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:00:00.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:59 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:00.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:59 smithi137 ceph-mon[34099]: Deploying daemon mds.d on smithi105 2024-09-16T02:00:00.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:59 smithi137 ceph-mon[34099]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:00.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:59 smithi137 ceph-mon[34099]: mds.? [v2:172.21.15.105:6834/2057699920,v1:172.21.15.105:6835/2057699920] up:boot 2024-09-16T02:00:00.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:59 smithi137 ceph-mon[34099]: fsmap 3 up:standby 2024-09-16T02:00:00.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:59 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-16T02:00:00.173 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 01:59:59 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/3576509908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:00:00.208 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:59 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:00.209 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:59 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:00.209 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:59 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:00.209 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:59 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:00:00.209 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:59 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:00:00.209 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:59 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:00.209 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:59 smithi190 ceph-mon[34762]: Deploying daemon mds.d on smithi105 2024-09-16T02:00:00.209 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:59 smithi190 ceph-mon[34762]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:00.209 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:59 smithi190 ceph-mon[34762]: mds.? [v2:172.21.15.105:6834/2057699920,v1:172.21.15.105:6835/2057699920] up:boot 2024-09-16T02:00:00.209 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:59 smithi190 ceph-mon[34762]: fsmap 3 up:standby 2024-09-16T02:00:00.209 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:59 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-16T02:00:00.209 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 01:59:59 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/3576509908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:00:00.294 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:59 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:00.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:59 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:00.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:59 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:00.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:59 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:00:00.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:59 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:00:00.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:59 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:00.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:59 smithi105 ceph-mon[31197]: Deploying daemon mds.d on smithi105 2024-09-16T02:00:00.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:59 smithi105 ceph-mon[31197]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:00.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:59 smithi105 ceph-mon[31197]: mds.? [v2:172.21.15.105:6834/2057699920,v1:172.21.15.105:6835/2057699920] up:boot 2024-09-16T02:00:00.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:59 smithi105 ceph-mon[31197]: fsmap 3 up:standby 2024-09-16T02:00:00.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:59 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-16T02:00:00.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 01:59:59 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/3576509908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:00:01.153 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:00 smithi105 ceph-mon[31197]: overall HEALTH_OK 2024-09-16T02:00:01.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:00 smithi137 ceph-mon[34099]: overall HEALTH_OK 2024-09-16T02:00:01.342 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:00 smithi190 ceph-mon[34762]: overall HEALTH_OK 2024-09-16T02:00:01.544 INFO:journalctl@ceph.mds.d.smithi105.stdout:Sep 16 02:00:01 smithi105 systemd[1]: Starting Ceph mds.d for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:00:01.870 INFO:journalctl@ceph.mds.d.smithi105.stdout:Sep 16 02:00:01 smithi105 podman[70911]: 2024-09-16 02:00:01.504843735 +0000 UTC m=+0.015896683 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:00:01.870 INFO:journalctl@ceph.mds.d.smithi105.stdout:Sep 16 02:00:01 smithi105 podman[70911]: 2024-09-16 02:00:01.623835058 +0000 UTC m=+0.134887992 container create 66e3dc8a813a465223b7a118077795a450d6e13ed6290100fe8b6fbf8fab85ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-d, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-16T02:00:01.870 INFO:journalctl@ceph.mds.d.smithi105.stdout:Sep 16 02:00:01 smithi105 podman[70911]: 2024-09-16 02:00:01.843172799 +0000 UTC m=+0.354225739 container init 66e3dc8a813a465223b7a118077795a450d6e13ed6290100fe8b6fbf8fab85ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-d, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-16T02:00:01.870 INFO:journalctl@ceph.mds.d.smithi105.stdout:Sep 16 02:00:01 smithi105 podman[70911]: 2024-09-16 02:00:01.845951697 +0000 UTC m=+0.357004635 container start 66e3dc8a813a465223b7a118077795a450d6e13ed6290100fe8b6fbf8fab85ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-d, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-16T02:00:01.870 INFO:journalctl@ceph.mds.d.smithi105.stdout:Sep 16 02:00:01 smithi105 ceph-mds[70931]: -- 172.21.15.105:0/544901397 <== mon.0 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557aa596d860 con 0x557aa594a800 2024-09-16T02:00:02.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:01 smithi137 ceph-mon[34099]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:02.294 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:01 smithi105 ceph-mon[31197]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:02.295 INFO:journalctl@ceph.mds.d.smithi105.stdout:Sep 16 02:00:01 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-d[70927]: starting mds.d at 2024-09-16T02:00:02.295 INFO:journalctl@ceph.mds.d.smithi105.stdout:Sep 16 02:00:01 smithi105 bash[70911]: 66e3dc8a813a465223b7a118077795a450d6e13ed6290100fe8b6fbf8fab85ad 2024-09-16T02:00:02.295 INFO:journalctl@ceph.mds.d.smithi105.stdout:Sep 16 02:00:01 smithi105 systemd[1]: Started Ceph mds.d for 423b7a2a-73ce-11ef-bceb-c7b262605968. 2024-09-16T02:00:02.342 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:01 smithi190 ceph-mon[34762]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:03.294 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:03 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:03.294 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:03 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:03.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:03 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:03.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:03 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:00:03.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:03 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:00:03.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:03 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:03.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:03 smithi105 ceph-mon[31197]: Deploying daemon mds.e on smithi137 2024-09-16T02:00:03.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:03 smithi105 ceph-mon[31197]: mds.? [v2:172.21.15.105:6836/1764886948,v1:172.21.15.105:6837/1764886948] up:boot 2024-09-16T02:00:03.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:03 smithi105 ceph-mon[31197]: fsmap 4 up:standby 2024-09-16T02:00:03.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:03 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-16T02:00:03.342 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:03 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:03.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:03 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:03.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:03 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:03.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:03 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:00:03.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:03 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:00:03.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:03 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:03.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:03 smithi190 ceph-mon[34762]: Deploying daemon mds.e on smithi137 2024-09-16T02:00:03.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:03 smithi190 ceph-mon[34762]: mds.? [v2:172.21.15.105:6836/1764886948,v1:172.21.15.105:6837/1764886948] up:boot 2024-09-16T02:00:03.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:03 smithi190 ceph-mon[34762]: fsmap 4 up:standby 2024-09-16T02:00:03.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:03 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-16T02:00:03.352 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:00:03.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:03 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:03.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:03 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:03.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:03 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:03.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:03 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:00:03.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:03 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:00:03.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:03 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:03.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:03 smithi137 ceph-mon[34099]: Deploying daemon mds.e on smithi137 2024-09-16T02:00:03.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:03 smithi137 ceph-mon[34099]: mds.? [v2:172.21.15.105:6836/1764886948,v1:172.21.15.105:6837/1764886948] up:boot 2024-09-16T02:00:03.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:03 smithi137 ceph-mon[34099]: fsmap 4 up:standby 2024-09-16T02:00:03.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:03 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-16T02:00:03.607 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:00:04.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.152+0000 7f7c660fd640 1 -- 172.21.15.105:0/1818969719 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c60101400 msgr2=0x7f7c6010d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:04.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.152+0000 7f7c660fd640 1 --2- 172.21.15.105:0/1818969719 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c60101400 0x7f7c6010d930 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7c58009cd0 tx=0x7f7c5802a8e0 comp rx=0 tx=0).stop 2024-09-16T02:00:04.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.152+0000 7f7c660fd640 1 -- 172.21.15.105:0/1818969719 shutdown_connections 2024-09-16T02:00:04.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.152+0000 7f7c660fd640 1 --2- 172.21.15.105:0/1818969719 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c60101400 0x7f7c6010d930 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:04.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.152+0000 7f7c660fd640 1 --2- 172.21.15.105:0/1818969719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c60100a60 0x7f7c60100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:04.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.152+0000 7f7c660fd640 1 --2- 172.21.15.105:0/1818969719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c60106a60 0x7f7c60106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:04.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.152+0000 7f7c660fd640 1 -- 172.21.15.105:0/1818969719 >> 172.21.15.105:0/1818969719 conn(0x7f7c600fc780 msgr2=0x7f7c600feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:04.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.152+0000 7f7c660fd640 1 -- 172.21.15.105:0/1818969719 shutdown_connections 2024-09-16T02:00:04.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.152+0000 7f7c660fd640 1 -- 172.21.15.105:0/1818969719 wait complete. 2024-09-16T02:00:04.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.152+0000 7f7c660fd640 1 Processor -- start 2024-09-16T02:00:04.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.153+0000 7f7c660fd640 1 -- start start 2024-09-16T02:00:04.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.153+0000 7f7c660fd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c60100a60 0x7f7c601982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:04.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.153+0000 7f7c660fd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c60101400 0x7f7c601987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:04.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.153+0000 7f7c660fd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c60106a60 0x7f7c6019cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:04.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.153+0000 7f7c660fd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7c60110d10 con 0x7f7c60106a60 2024-09-16T02:00:04.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.153+0000 7f7c660fd640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7c60110b90 con 0x7f7c60101400 2024-09-16T02:00:04.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.153+0000 7f7c660fd640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7c60110e90 con 0x7f7c60100a60 2024-09-16T02:00:04.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.153+0000 7f7c5ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c60106a60 0x7f7c6019cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:04.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.153+0000 7f7c5ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c60106a60 0x7f7c6019cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60992/0 (socket says 172.21.15.105:60992) 2024-09-16T02:00:04.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.153+0000 7f7c5ffff640 1 -- 172.21.15.105:0/164912594 learned_addr learned my addr 172.21.15.105:0/164912594 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:00:04.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.153+0000 7f7c5effd640 1 --2- 172.21.15.105:0/164912594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c60101400 0x7f7c601987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:04.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.153+0000 7f7c5f7fe640 1 --2- 172.21.15.105:0/164912594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c60100a60 0x7f7c601982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:04.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.153+0000 7f7c5ffff640 1 -- 172.21.15.105:0/164912594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c60100a60 msgr2=0x7f7c601982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:04.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.153+0000 7f7c5ffff640 1 --2- 172.21.15.105:0/164912594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c60100a60 0x7f7c601982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:04.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.153+0000 7f7c5ffff640 1 -- 172.21.15.105:0/164912594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c60101400 msgr2=0x7f7c601987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:04.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.153+0000 7f7c5ffff640 1 --2- 172.21.15.105:0/164912594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c60101400 0x7f7c601987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:04.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.153+0000 7f7c5ffff640 1 -- 172.21.15.105:0/164912594 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c6019d260 con 0x7f7c60106a60 2024-09-16T02:00:04.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.153+0000 7f7c5ffff640 1 --2- 172.21.15.105:0/164912594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c60106a60 0x7f7c6019cb80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7c58005cc0 tx=0x7f7c58003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:04.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.153+0000 7f7c5cff9640 1 -- 172.21.15.105:0/164912594 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c58004450 con 0x7f7c60106a60 2024-09-16T02:00:04.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.153+0000 7f7c5cff9640 1 -- 172.21.15.105:0/164912594 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7c58031c90 con 0x7f7c60106a60 2024-09-16T02:00:04.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.153+0000 7f7c660fd640 1 -- 172.21.15.105:0/164912594 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c6019d4f0 con 0x7f7c60106a60 2024-09-16T02:00:04.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.154+0000 7f7c5cff9640 1 -- 172.21.15.105:0/164912594 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c58030e70 con 0x7f7c60106a60 2024-09-16T02:00:04.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.154+0000 7f7c660fd640 1 -- 172.21.15.105:0/164912594 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c601a4e30 con 0x7f7c60106a60 2024-09-16T02:00:04.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.154+0000 7f7c660fd640 1 -- 172.21.15.105:0/164912594 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c60100ec0 con 0x7f7c60106a60 2024-09-16T02:00:04.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.156+0000 7f7c5cff9640 1 -- 172.21.15.105:0/164912594 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f7c580045f0 con 0x7f7c60106a60 2024-09-16T02:00:04.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.156+0000 7f7c5cff9640 1 --2- 172.21.15.105:0/164912594 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7c4c0a3240 0x7f7c4c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:04.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.157+0000 7f7c5f7fe640 1 --2- 172.21.15.105:0/164912594 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7c4c0a3240 0x7f7c4c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:04.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.157+0000 7f7c5cff9640 1 -- 172.21.15.105:0/164912594 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f7c5802e030 con 0x7f7c60106a60 2024-09-16T02:00:04.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.157+0000 7f7c5f7fe640 1 --2- 172.21.15.105:0/164912594 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7c4c0a3240 0x7f7c4c0a5700 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7c50005fd0 tx=0x7f7c5000b510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:04.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.159+0000 7f7c5cff9640 1 -- 172.21.15.105:0/164912594 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7c5804b060 con 0x7f7c60106a60 2024-09-16T02:00:04.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:04 smithi137 ceph-mon[34099]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:04.171 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 16 02:00:04 smithi137 systemd[1]: Starting Ceph mds.e for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:00:04.294 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:04 smithi105 ceph-mon[31197]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:04.342 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:04 smithi190 ceph-mon[34762]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:04.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.358+0000 7f7c660fd640 1 -- 172.21.15.105:0/164912594 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c60109fe0 con 0x7f7c60106a60 2024-09-16T02:00:04.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.358+0000 7f7c5cff9640 1 -- 172.21.15.105:0/164912594 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3919 (secure 0 0 0) 0x7f7c580098b0 con 0x7f7c60106a60 2024-09-16T02:00:04.361 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:00:04.361 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":5,"btime":"2024-09-16T02:00:02:871916+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2057699920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2057699920},{"type":"v1","addr":"172.21.15.105:6835","nonce":2057699920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1764886948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1764886948},{"type":"v1","addr":"172.21.15.105:6837","nonce":1764886948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/683519097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":683519097},{"type":"v1","addr":"172.21.15.190:6833","nonce":683519097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/3025442862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3025442862},{"type":"v1","addr":"172.21.15.137:6833","nonce":3025442862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-16T02:00:04.361 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 5 2024-09-16T02:00:04.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.360+0000 7f7c660fd640 1 -- 172.21.15.105:0/164912594 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7c4c0a3240 msgr2=0x7f7c4c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.360+0000 7f7c660fd640 1 --2- 172.21.15.105:0/164912594 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7c4c0a3240 0x7f7c4c0a5700 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7c50005fd0 tx=0x7f7c5000b510 comp rx=0 tx=0).stop 2024-09-16T02:00:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.360+0000 7f7c660fd640 1 -- 172.21.15.105:0/164912594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c60106a60 msgr2=0x7f7c6019cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.360+0000 7f7c660fd640 1 --2- 172.21.15.105:0/164912594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c60106a60 0x7f7c6019cb80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7c58005cc0 tx=0x7f7c58003c20 comp rx=0 tx=0).stop 2024-09-16T02:00:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.360+0000 7f7c660fd640 1 -- 172.21.15.105:0/164912594 shutdown_connections 2024-09-16T02:00:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.360+0000 7f7c660fd640 1 --2- 172.21.15.105:0/164912594 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7c4c0a3240 0x7f7c4c0a5700 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.360+0000 7f7c660fd640 1 --2- 172.21.15.105:0/164912594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c60106a60 0x7f7c6019cb80 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.360+0000 7f7c660fd640 1 --2- 172.21.15.105:0/164912594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c60101400 0x7f7c601987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.360+0000 7f7c660fd640 1 --2- 172.21.15.105:0/164912594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c60100a60 0x7f7c601982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.360+0000 7f7c660fd640 1 -- 172.21.15.105:0/164912594 >> 172.21.15.105:0/164912594 conn(0x7f7c600fc780 msgr2=0x7f7c6006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.361+0000 7f7c660fd640 1 -- 172.21.15.105:0/164912594 shutdown_connections 2024-09-16T02:00:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:04.361+0000 7f7c660fd640 1 -- 172.21.15.105:0/164912594 wait complete. 2024-09-16T02:00:04.577 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 16 02:00:04 smithi137 podman[63520]: 2024-09-16 02:00:04.15376871 +0000 UTC m=+0.015935725 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:00:04.577 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 16 02:00:04 smithi137 podman[63520]: 2024-09-16 02:00:04.281288828 +0000 UTC m=+0.143455833 container create c81646a1b2254ea2204c8e7be9573c1ddf3c7a337add7d284731c309e2632ea4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T02:00:04.577 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 16 02:00:04 smithi137 podman[63520]: 2024-09-16 02:00:04.575940678 +0000 UTC m=+0.438107685 container init c81646a1b2254ea2204c8e7be9573c1ddf3c7a337add7d284731c309e2632ea4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-8921c8e) 2024-09-16T02:00:04.921 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 16 02:00:04 smithi137 podman[63520]: 2024-09-16 02:00:04.579069493 +0000 UTC m=+0.441236502 container start c81646a1b2254ea2204c8e7be9573c1ddf3c7a337add7d284731c309e2632ea4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-e, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T02:00:04.921 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 16 02:00:04 smithi137 ceph-mds[63539]: -- 172.21.15.137:0/3242246486 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55af83ba3860 con 0x55af83b81400 2024-09-16T02:00:04.921 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 16 02:00:04 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-e[63535]: starting mds.e at 2024-09-16T02:00:04.921 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 16 02:00:04 smithi137 bash[63520]: c81646a1b2254ea2204c8e7be9573c1ddf3c7a337add7d284731c309e2632ea4 2024-09-16T02:00:04.921 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 16 02:00:04 smithi137 systemd[1]: Started Ceph mds.e for 423b7a2a-73ce-11ef-bceb-c7b262605968. 2024-09-16T02:00:05.294 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:05 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/164912594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:00:05.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:05 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:05.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:05 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:05.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:05 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:05.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:05 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:00:05.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:05 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:00:05.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:05 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:05.318 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:05 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/164912594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:00:05.318 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:05 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:05.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:05 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:05.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:05 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:05.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:05 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:00:05.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:05 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:00:05.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:05 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:05.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:05 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/164912594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:00:05.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:05 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:05.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:05 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:05.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:05 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:05.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:05 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:00:05.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:05 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:00:05.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:05 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:06.294 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:06 smithi105 ceph-mon[31197]: Deploying daemon mds.f on smithi190 2024-09-16T02:00:06.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:06 smithi105 ceph-mon[31197]: mds.? [v2:172.21.15.137:6834/763783602,v1:172.21.15.137:6835/763783602] up:boot 2024-09-16T02:00:06.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:06 smithi105 ceph-mon[31197]: fsmap 5 up:standby 2024-09-16T02:00:06.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:06 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T02:00:06.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:06 smithi105 ceph-mon[31197]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:06.326 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:06 smithi190 ceph-mon[34762]: Deploying daemon mds.f on smithi190 2024-09-16T02:00:06.327 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:06 smithi190 ceph-mon[34762]: mds.? [v2:172.21.15.137:6834/763783602,v1:172.21.15.137:6835/763783602] up:boot 2024-09-16T02:00:06.327 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:06 smithi190 ceph-mon[34762]: fsmap 5 up:standby 2024-09-16T02:00:06.327 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:06 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T02:00:06.327 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:06 smithi190 ceph-mon[34762]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:06.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:06 smithi137 ceph-mon[34099]: Deploying daemon mds.f on smithi190 2024-09-16T02:00:06.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:06 smithi137 ceph-mon[34099]: mds.? [v2:172.21.15.137:6834/763783602,v1:172.21.15.137:6835/763783602] up:boot 2024-09-16T02:00:06.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:06 smithi137 ceph-mon[34099]: fsmap 5 up:standby 2024-09-16T02:00:06.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:06 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T02:00:06.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:06 smithi137 ceph-mon[34099]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:07.159 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:00:07.428 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:00:07.592 INFO:journalctl@ceph.mds.f.smithi190.stdout:Sep 16 02:00:07 smithi190 systemd[1]: Started Ceph mds.f for 423b7a2a-73ce-11ef-bceb-c7b262605968. 2024-09-16T02:00:07.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.921+0000 7ffaf6b68640 1 -- 172.21.15.105:0/744701380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffaf006a370 msgr2=0x7ffaf01009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:07.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.921+0000 7ffaf6b68640 1 --2- 172.21.15.105:0/744701380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffaf006a370 0x7ffaf01009a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ffadc009900 tx=0x7ffadc02a410 comp rx=0 tx=0).stop 2024-09-16T02:00:07.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.921+0000 7ffaf6b68640 1 -- 172.21.15.105:0/744701380 shutdown_connections 2024-09-16T02:00:07.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.921+0000 7ffaf6b68640 1 --2- 172.21.15.105:0/744701380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffaf0104e60 0x7ffaf0105310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:07.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.921+0000 7ffaf6b68640 1 --2- 172.21.15.105:0/744701380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffaf006a370 0x7ffaf01009a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:07.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.921+0000 7ffaf6b68640 1 --2- 172.21.15.105:0/744701380 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffaf0069f00 0x7ffaf0100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:07.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.921+0000 7ffaf6b68640 1 -- 172.21.15.105:0/744701380 >> 172.21.15.105:0/744701380 conn(0x7ffaf00fbee0 msgr2=0x7ffaf00fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:07.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.921+0000 7ffaf6b68640 1 -- 172.21.15.105:0/744701380 shutdown_connections 2024-09-16T02:00:07.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.921+0000 7ffaf6b68640 1 -- 172.21.15.105:0/744701380 wait complete. 2024-09-16T02:00:07.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.922+0000 7ffaf6b68640 1 Processor -- start 2024-09-16T02:00:07.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.922+0000 7ffaf6b68640 1 -- start start 2024-09-16T02:00:07.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.922+0000 7ffaf6b68640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffaf0069f00 0x7ffaf00fc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:07.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.922+0000 7ffaf6b68640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffaf006a370 0x7ffaf00fc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:07.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.922+0000 7ffaf6b68640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffaf0104e60 0x7ffaf00fce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:07.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.922+0000 7ffaf6b68640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf0110d40 con 0x7ffaf006a370 2024-09-16T02:00:07.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.922+0000 7ffaf6b68640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf0110bc0 con 0x7ffaf0104e60 2024-09-16T02:00:07.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.922+0000 7ffaf6b68640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf0110ec0 con 0x7ffaf0069f00 2024-09-16T02:00:07.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.922+0000 7ffaeffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffaf006a370 0x7ffaf00fc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:07.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.922+0000 7ffaeffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffaf006a370 0x7ffaf00fc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:32770/0 (socket says 172.21.15.105:32770) 2024-09-16T02:00:07.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.922+0000 7ffaeffff640 1 -- 172.21.15.105:0/1327000278 learned_addr learned my addr 172.21.15.105:0/1327000278 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:00:07.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.922+0000 7ffaf5105640 1 --2- 172.21.15.105:0/1327000278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffaf0104e60 0x7ffaf00fce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:07.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.922+0000 7ffaf4904640 1 --2- 172.21.15.105:0/1327000278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffaf0069f00 0x7ffaf00fc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:07.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.922+0000 7ffaeffff640 1 -- 172.21.15.105:0/1327000278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffaf0069f00 msgr2=0x7ffaf00fc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:07.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.922+0000 7ffaeffff640 1 --2- 172.21.15.105:0/1327000278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffaf0069f00 0x7ffaf00fc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:07.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.922+0000 7ffaeffff640 1 -- 172.21.15.105:0/1327000278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffaf0104e60 msgr2=0x7ffaf00fce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:07.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.922+0000 7ffaeffff640 1 --2- 172.21.15.105:0/1327000278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffaf0104e60 0x7ffaf00fce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:07.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.922+0000 7ffaeffff640 1 -- 172.21.15.105:0/1327000278 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaf01a6800 con 0x7ffaf006a370 2024-09-16T02:00:07.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.922+0000 7ffaf5105640 1 --2- 172.21.15.105:0/1327000278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffaf0104e60 0x7ffaf00fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:00:07.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.922+0000 7ffaeffff640 1 --2- 172.21.15.105:0/1327000278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffaf006a370 0x7ffaf00fc8d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ffadc005940 tx=0x7ffadc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:07.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.923+0000 7ffaedffb640 1 -- 172.21.15.105:0/1327000278 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffadc03f030 con 0x7ffaf006a370 2024-09-16T02:00:07.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.923+0000 7ffaedffb640 1 -- 172.21.15.105:0/1327000278 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffadc003cb0 con 0x7ffaf006a370 2024-09-16T02:00:07.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.923+0000 7ffaf6b68640 1 -- 172.21.15.105:0/1327000278 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaf01a6a90 con 0x7ffaf006a370 2024-09-16T02:00:07.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.923+0000 7ffaedffb640 1 -- 172.21.15.105:0/1327000278 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffadc035070 con 0x7ffaf006a370 2024-09-16T02:00:07.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.923+0000 7ffaf6b68640 1 -- 172.21.15.105:0/1327000278 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaf01a6fc0 con 0x7ffaf006a370 2024-09-16T02:00:07.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.924+0000 7ffaf6b68640 1 -- 172.21.15.105:0/1327000278 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaf0105310 con 0x7ffaf006a370 2024-09-16T02:00:07.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.926+0000 7ffaedffb640 1 -- 172.21.15.105:0/1327000278 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7ffadc042030 con 0x7ffaf006a370 2024-09-16T02:00:07.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.926+0000 7ffaedffb640 1 --2- 172.21.15.105:0/1327000278 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ffae00a32c0 0x7ffae00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:07.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.926+0000 7ffaf4904640 1 --2- 172.21.15.105:0/1327000278 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ffae00a32c0 0x7ffae00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:07.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.926+0000 7ffaedffb640 1 -- 172.21.15.105:0/1327000278 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7ffadc0efed0 con 0x7ffaf006a370 2024-09-16T02:00:07.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.927+0000 7ffaf4904640 1 --2- 172.21.15.105:0/1327000278 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ffae00a32c0 0x7ffae00a5780 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ffae4005e10 tx=0x7ffae4005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:07.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:07.929+0000 7ffaedffb640 1 -- 172.21.15.105:0/1327000278 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffadc0bd090 con 0x7ffaf006a370 2024-09-16T02:00:08.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:08.130+0000 7ffaf6b68640 1 -- 172.21.15.105:0/1327000278 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffaf01a7250 con 0x7ffaf006a370 2024-09-16T02:00:08.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:08.131+0000 7ffaedffb640 1 -- 172.21.15.105:0/1327000278 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5572 (secure 0 0 0) 0x7ffadc0bc990 con 0x7ffaf006a370 2024-09-16T02:00:08.132 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:00:08.132 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":7,"btime":"2024-09-16T02:00:07:264567+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2057699920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2057699920},{"type":"v1","addr":"172.21.15.105:6835","nonce":2057699920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1764886948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1764886948},{"type":"v1","addr":"172.21.15.105:6837","nonce":1764886948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/683519097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":683519097},{"type":"v1","addr":"172.21.15.190:6833","nonce":683519097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/3025442862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3025442862},{"type":"v1","addr":"172.21.15.137:6833","nonce":3025442862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/763783602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":763783602},{"type":"v1","addr":"172.21.15.137:6835","nonce":763783602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1851042281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1851042281},{"type":"v1","addr":"172.21.15.190:6835","nonce":1851042281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-09-16T02:00:08.133 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 7 2024-09-16T02:00:08.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:08.133+0000 7ffaf6b68640 1 -- 172.21.15.105:0/1327000278 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ffae00a32c0 msgr2=0x7ffae00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:08.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:08.133+0000 7ffaf6b68640 1 --2- 172.21.15.105:0/1327000278 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ffae00a32c0 0x7ffae00a5780 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ffae4005e10 tx=0x7ffae4005da0 comp rx=0 tx=0).stop 2024-09-16T02:00:08.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:08.133+0000 7ffaf6b68640 1 -- 172.21.15.105:0/1327000278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffaf006a370 msgr2=0x7ffaf00fc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:08.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:08.133+0000 7ffaf6b68640 1 --2- 172.21.15.105:0/1327000278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffaf006a370 0x7ffaf00fc8d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ffadc005940 tx=0x7ffadc0042f0 comp rx=0 tx=0).stop 2024-09-16T02:00:08.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:08.133+0000 7ffaf6b68640 1 -- 172.21.15.105:0/1327000278 shutdown_connections 2024-09-16T02:00:08.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:08.133+0000 7ffaf6b68640 1 --2- 172.21.15.105:0/1327000278 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ffae00a32c0 0x7ffae00a5780 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:08.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:08.133+0000 7ffaf6b68640 1 --2- 172.21.15.105:0/1327000278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffaf0104e60 0x7ffaf00fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:08.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:08.133+0000 7ffaf6b68640 1 --2- 172.21.15.105:0/1327000278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffaf006a370 0x7ffaf00fc8d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:08.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:08.133+0000 7ffaf6b68640 1 --2- 172.21.15.105:0/1327000278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffaf0069f00 0x7ffaf00fc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:08.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:08.133+0000 7ffaf6b68640 1 -- 172.21.15.105:0/1327000278 >> 172.21.15.105:0/1327000278 conn(0x7ffaf00fbee0 msgr2=0x7ffaf0109120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:08.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:08.133+0000 7ffaf6b68640 1 -- 172.21.15.105:0/1327000278 shutdown_connections 2024-09-16T02:00:08.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:08.133+0000 7ffaf6b68640 1 -- 172.21.15.105:0/1327000278 wait complete. 2024-09-16T02:00:08.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:08 smithi190 ceph-mon[34762]: mds.? [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] up:boot 2024-09-16T02:00:08.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:08 smithi190 ceph-mon[34762]: fsmap 6 up:standby 2024-09-16T02:00:08.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:08 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T02:00:08.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:08 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:08.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:08 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:08.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:08 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:08.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:08 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:00:08.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:08 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:00:08.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:08 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:08.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:08 smithi190 ceph-mon[34762]: Deploying daemon mds.c on smithi190 2024-09-16T02:00:08.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:08 smithi190 ceph-mon[34762]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:08.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:08 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/1327000278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:00:08.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:08 smithi105 ceph-mon[31197]: mds.? [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] up:boot 2024-09-16T02:00:08.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:08 smithi105 ceph-mon[31197]: fsmap 6 up:standby 2024-09-16T02:00:08.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:08 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T02:00:08.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:08 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:08.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:08 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:08.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:08 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:08.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:08 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:00:08.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:08 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:00:08.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:08 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:08.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:08 smithi105 ceph-mon[31197]: Deploying daemon mds.c on smithi190 2024-09-16T02:00:08.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:08 smithi105 ceph-mon[31197]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:08.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:08 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/1327000278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:00:08.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:08 smithi137 ceph-mon[34099]: mds.? [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] up:boot 2024-09-16T02:00:08.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:08 smithi137 ceph-mon[34099]: fsmap 6 up:standby 2024-09-16T02:00:08.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:08 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T02:00:08.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:08 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:08.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:08 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:08.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:08 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:08.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:08 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:00:08.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:08 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:00:08.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:08 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:08.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:08 smithi137 ceph-mon[34099]: Deploying daemon mds.c on smithi190 2024-09-16T02:00:08.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:08 smithi137 ceph-mon[34099]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:08.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:08 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/1327000278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:00:09.529 INFO:journalctl@ceph.mds.c.smithi190.stdout:Sep 16 02:00:09 smithi190 systemd[1]: Starting Ceph mds.c for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:00:09.838 INFO:journalctl@ceph.mds.c.smithi190.stdout:Sep 16 02:00:09 smithi190 podman[64292]: 2024-09-16 02:00:09.430841135 +0000 UTC m=+0.015938893 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:00:09.838 INFO:journalctl@ceph.mds.c.smithi190.stdout:Sep 16 02:00:09 smithi190 podman[64292]: 2024-09-16 02:00:09.593257391 +0000 UTC m=+0.178355152 container create 9877b492490bb82c4ae16ea67e2bb10da7054e81b9e4e6378320be6e8b3e5b88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-c, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-09-16T02:00:09.838 INFO:journalctl@ceph.mds.c.smithi190.stdout:Sep 16 02:00:09 smithi190 podman[64292]: 2024-09-16 02:00:09.837536633 +0000 UTC m=+0.422634391 container init 9877b492490bb82c4ae16ea67e2bb10da7054e81b9e4e6378320be6e8b3e5b88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-c, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T02:00:10.092 INFO:journalctl@ceph.mds.c.smithi190.stdout:Sep 16 02:00:09 smithi190 podman[64292]: 2024-09-16 02:00:09.840473667 +0000 UTC m=+0.425571426 container start 9877b492490bb82c4ae16ea67e2bb10da7054e81b9e4e6378320be6e8b3e5b88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-c, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-16T02:00:10.092 INFO:journalctl@ceph.mds.c.smithi190.stdout:Sep 16 02:00:09 smithi190 ceph-mds[64311]: -- 172.21.15.190:0/2072438156 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563f85b17860 con 0x563f85af5000 2024-09-16T02:00:10.092 INFO:journalctl@ceph.mds.c.smithi190.stdout:Sep 16 02:00:09 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-c[64307]: starting mds.c at 2024-09-16T02:00:10.093 INFO:journalctl@ceph.mds.c.smithi190.stdout:Sep 16 02:00:09 smithi190 bash[64292]: 9877b492490bb82c4ae16ea67e2bb10da7054e81b9e4e6378320be6e8b3e5b88 2024-09-16T02:00:10.093 INFO:journalctl@ceph.mds.c.smithi190.stdout:Sep 16 02:00:09 smithi190 systemd[1]: Started Ceph mds.c for 423b7a2a-73ce-11ef-bceb-c7b262605968. 2024-09-16T02:00:10.697 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:10 smithi137 ceph-mon[34099]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:10.697 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:10 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:10.697 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:10 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:10.697 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:10 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:10.698 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:10 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:00:10.698 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:10 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:00:10.698 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:10 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:10.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:10 smithi105 ceph-mon[31197]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:10.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:10 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:10.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:10 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:10.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:10 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:10.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:10 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:00:10.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:10 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:00:10.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:10 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:10.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:10 smithi190 ceph-mon[34762]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:10.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:10 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:10.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:10 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:10.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:10 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:10.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:10 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:00:10.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:10 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:00:10.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:10 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:10.852 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:00:11.111 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:00:11.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.580+0000 7f61b688a640 1 -- 172.21.15.105:0/2218308469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61b010c170 msgr2=0x7f61b010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:11.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.580+0000 7f61b688a640 1 --2- 172.21.15.105:0/2218308469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61b010c170 0x7f61b010e610 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f61a8009ca0 tx=0x7f61a802a950 comp rx=0 tx=0).stop 2024-09-16T02:00:11.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.580+0000 7f61b688a640 1 -- 172.21.15.105:0/2218308469 shutdown_connections 2024-09-16T02:00:11.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.580+0000 7f61b688a640 1 --2- 172.21.15.105:0/2218308469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61b010c170 0x7f61b010e610 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:11.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.580+0000 7f61b688a640 1 --2- 172.21.15.105:0/2218308469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61b0103660 0x7f61b010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:11.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.580+0000 7f61b688a640 1 --2- 172.21.15.105:0/2218308469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61b0102d40 0x7f61b0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:11.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.580+0000 7f61b688a640 1 -- 172.21.15.105:0/2218308469 >> 172.21.15.105:0/2218308469 conn(0x7f61b00fc7a0 msgr2=0x7f61b00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:11.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.580+0000 7f61b688a640 1 -- 172.21.15.105:0/2218308469 shutdown_connections 2024-09-16T02:00:11.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.580+0000 7f61b688a640 1 -- 172.21.15.105:0/2218308469 wait complete. 2024-09-16T02:00:11.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.580+0000 7f61b688a640 1 Processor -- start 2024-09-16T02:00:11.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.581+0000 7f61b688a640 1 -- start start 2024-09-16T02:00:11.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.581+0000 7f61b688a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61b0102d40 0x7f61b0198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:11.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.581+0000 7f61b688a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61b0103660 0x7f61b01989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:11.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.581+0000 7f61b688a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61b010c170 0x7f61b019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:11.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.581+0000 7f61b688a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f61b0111e60 con 0x7f61b0103660 2024-09-16T02:00:11.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.581+0000 7f61b688a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f61b0111ce0 con 0x7f61b0102d40 2024-09-16T02:00:11.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.581+0000 7f61b688a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f61b0111fe0 con 0x7f61b010c170 2024-09-16T02:00:11.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.581+0000 7f61af7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61b0103660 0x7f61b01989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:11.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.581+0000 7f61af7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61b0103660 0x7f61b01989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34920/0 (socket says 172.21.15.105:34920) 2024-09-16T02:00:11.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.581+0000 7f61af7fe640 1 -- 172.21.15.105:0/386332118 learned_addr learned my addr 172.21.15.105:0/386332118 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:00:11.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.581+0000 7f61affff640 1 --2- 172.21.15.105:0/386332118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61b0102d40 0x7f61b0198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:11.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.581+0000 7f61b4e27640 1 --2- 172.21.15.105:0/386332118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61b010c170 0x7f61b019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:11.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.581+0000 7f61af7fe640 1 -- 172.21.15.105:0/386332118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61b010c170 msgr2=0x7f61b019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:11.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.581+0000 7f61af7fe640 1 --2- 172.21.15.105:0/386332118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61b010c170 0x7f61b019cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:11.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.581+0000 7f61af7fe640 1 -- 172.21.15.105:0/386332118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61b0102d40 msgr2=0x7f61b0198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:11.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.581+0000 7f61af7fe640 1 --2- 172.21.15.105:0/386332118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61b0102d40 0x7f61b0198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:11.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.581+0000 7f61af7fe640 1 -- 172.21.15.105:0/386332118 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b019d420 con 0x7f61b0103660 2024-09-16T02:00:11.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.581+0000 7f61af7fe640 1 --2- 172.21.15.105:0/386332118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61b0103660 0x7f61b01989b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f61a400c940 tx=0x7f61a400cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:11.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.582+0000 7f61ad7fa640 1 -- 172.21.15.105:0/386332118 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a4004220 con 0x7f61b0103660 2024-09-16T02:00:11.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.582+0000 7f61ad7fa640 1 -- 172.21.15.105:0/386332118 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61a40043c0 con 0x7f61b0103660 2024-09-16T02:00:11.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.582+0000 7f61b688a640 1 -- 172.21.15.105:0/386332118 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61b019d710 con 0x7f61b0103660 2024-09-16T02:00:11.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.582+0000 7f61ad7fa640 1 -- 172.21.15.105:0/386332118 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a40047a0 con 0x7f61b0103660 2024-09-16T02:00:11.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.582+0000 7f61b688a640 1 -- 172.21.15.105:0/386332118 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61b01a4ff0 con 0x7f61b0103660 2024-09-16T02:00:11.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.583+0000 7f61b688a640 1 -- 172.21.15.105:0/386332118 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61b0103120 con 0x7f61b0103660 2024-09-16T02:00:11.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.584+0000 7f61ad7fa640 1 -- 172.21.15.105:0/386332118 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f61a40079b0 con 0x7f61b0103660 2024-09-16T02:00:11.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.584+0000 7f61ad7fa640 1 --2- 172.21.15.105:0/386332118 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f61980a3240 0x7f61980a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:11.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.585+0000 7f61affff640 1 --2- 172.21.15.105:0/386332118 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f61980a3240 0x7f61980a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:11.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.585+0000 7f61ad7fa640 1 -- 172.21.15.105:0/386332118 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f61a40cf420 con 0x7f61b0103660 2024-09-16T02:00:11.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.585+0000 7f61affff640 1 --2- 172.21.15.105:0/386332118 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f61980a3240 0x7f61980a5700 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f61a0009990 tx=0x7f61a0009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:11.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.588+0000 7f61ad7fa640 1 -- 172.21.15.105:0/386332118 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f61a4014030 con 0x7f61b0103660 2024-09-16T02:00:11.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.786+0000 7f61b688a640 1 -- 172.21.15.105:0/386332118 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61b0107240 con 0x7f61b0103660 2024-09-16T02:00:11.788 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:11 smithi105 ceph-mon[31197]: Deploying daemon mds.h on smithi137 2024-09-16T02:00:11.788 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:11 smithi105 ceph-mon[31197]: mds.? [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] up:boot 2024-09-16T02:00:11.788 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:11 smithi105 ceph-mon[31197]: fsmap 7 up:standby 2024-09-16T02:00:11.788 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:11 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T02:00:11.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.787+0000 7f61ad7fa640 1 -- 172.21.15.105:0/386332118 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6400 (secure 0 0 0) 0x7f61a409c5e0 con 0x7f61b0103660 2024-09-16T02:00:11.789 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:00:11.789 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":8,"btime":"2024-09-16T02:00:10:479286+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2057699920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2057699920},{"type":"v1","addr":"172.21.15.105:6835","nonce":2057699920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1764886948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1764886948},{"type":"v1","addr":"172.21.15.105:6837","nonce":1764886948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/683519097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":683519097},{"type":"v1","addr":"172.21.15.190:6833","nonce":683519097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/3025442862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3025442862},{"type":"v1","addr":"172.21.15.137:6833","nonce":3025442862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/763783602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":763783602},{"type":"v1","addr":"172.21.15.137:6835","nonce":763783602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1851042281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1851042281},{"type":"v1","addr":"172.21.15.190:6835","nonce":1851042281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/3181257549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3181257549},{"type":"v1","addr":"172.21.15.190:6837","nonce":3181257549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-16T02:00:11.789 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 8 2024-09-16T02:00:11.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.789+0000 7f61b688a640 1 -- 172.21.15.105:0/386332118 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f61980a3240 msgr2=0x7f61980a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:11.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.789+0000 7f61b688a640 1 --2- 172.21.15.105:0/386332118 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f61980a3240 0x7f61980a5700 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f61a0009990 tx=0x7f61a0009110 comp rx=0 tx=0).stop 2024-09-16T02:00:11.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.789+0000 7f61b688a640 1 -- 172.21.15.105:0/386332118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61b0103660 msgr2=0x7f61b01989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:11.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.789+0000 7f61b688a640 1 --2- 172.21.15.105:0/386332118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61b0103660 0x7f61b01989b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f61a400c940 tx=0x7f61a400cd40 comp rx=0 tx=0).stop 2024-09-16T02:00:11.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.790+0000 7f61b688a640 1 -- 172.21.15.105:0/386332118 shutdown_connections 2024-09-16T02:00:11.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.790+0000 7f61b688a640 1 --2- 172.21.15.105:0/386332118 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f61980a3240 0x7f61980a5700 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:11.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.790+0000 7f61b688a640 1 --2- 172.21.15.105:0/386332118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61b010c170 0x7f61b019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:11.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.790+0000 7f61b688a640 1 --2- 172.21.15.105:0/386332118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61b0103660 0x7f61b01989b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:11.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.790+0000 7f61b688a640 1 --2- 172.21.15.105:0/386332118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61b0102d40 0x7f61b0198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:11.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.790+0000 7f61b688a640 1 -- 172.21.15.105:0/386332118 >> 172.21.15.105:0/386332118 conn(0x7f61b00fc7a0 msgr2=0x7f61b00fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:11.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.790+0000 7f61b688a640 1 -- 172.21.15.105:0/386332118 shutdown_connections 2024-09-16T02:00:11.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:11.790+0000 7f61b688a640 1 -- 172.21.15.105:0/386332118 wait complete. 2024-09-16T02:00:11.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:11 smithi190 ceph-mon[34762]: Deploying daemon mds.h on smithi137 2024-09-16T02:00:11.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:11 smithi190 ceph-mon[34762]: mds.? [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] up:boot 2024-09-16T02:00:11.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:11 smithi190 ceph-mon[34762]: fsmap 7 up:standby 2024-09-16T02:00:11.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:11 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T02:00:11.871 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:11 smithi137 ceph-mon[34099]: Deploying daemon mds.h on smithi137 2024-09-16T02:00:11.871 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:11 smithi137 ceph-mon[34099]: mds.? [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] up:boot 2024-09-16T02:00:11.872 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:11 smithi137 ceph-mon[34099]: fsmap 7 up:standby 2024-09-16T02:00:11.872 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:11 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T02:00:12.171 INFO:journalctl@ceph.mds.h.smithi137.stdout:Sep 16 02:00:12 smithi137 systemd[1]: Starting Ceph mds.h for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:00:12.502 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:12 smithi105 ceph-mon[31197]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:12.502 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:12 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/386332118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:00:12.541 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:12 smithi137 ceph-mon[34099]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:12.541 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:12 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/386332118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:00:12.541 INFO:journalctl@ceph.mds.h.smithi137.stdout:Sep 16 02:00:12 smithi137 podman[63821]: 2024-09-16 02:00:12.160056511 +0000 UTC m=+0.016156898 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:00:12.541 INFO:journalctl@ceph.mds.h.smithi137.stdout:Sep 16 02:00:12 smithi137 podman[63821]: 2024-09-16 02:00:12.287536869 +0000 UTC m=+0.143637246 container create 39d3393c8f3e0536c64a84380a679453bcb8e95386e38e0aec9ed5a5bad089af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-h, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e) 2024-09-16T02:00:12.541 INFO:journalctl@ceph.mds.h.smithi137.stdout:Sep 16 02:00:12 smithi137 podman[63821]: 2024-09-16 02:00:12.540730432 +0000 UTC m=+0.396830820 container init 39d3393c8f3e0536c64a84380a679453bcb8e95386e38e0aec9ed5a5bad089af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-h, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-16T02:00:12.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:12 smithi190 ceph-mon[34762]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:12.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:12 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/386332118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:00:12.921 INFO:journalctl@ceph.mds.h.smithi137.stdout:Sep 16 02:00:12 smithi137 podman[63821]: 2024-09-16 02:00:12.543622165 +0000 UTC m=+0.399722542 container start 39d3393c8f3e0536c64a84380a679453bcb8e95386e38e0aec9ed5a5bad089af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-h, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD) 2024-09-16T02:00:12.921 INFO:journalctl@ceph.mds.h.smithi137.stdout:Sep 16 02:00:12 smithi137 ceph-mds[63840]: -- 172.21.15.137:0/292949235 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cb17d9d860 con 0x55cb17d7b400 2024-09-16T02:00:12.921 INFO:journalctl@ceph.mds.h.smithi137.stdout:Sep 16 02:00:12 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-h[63836]: starting mds.h at 2024-09-16T02:00:12.921 INFO:journalctl@ceph.mds.h.smithi137.stdout:Sep 16 02:00:12 smithi137 bash[63821]: 39d3393c8f3e0536c64a84380a679453bcb8e95386e38e0aec9ed5a5bad089af 2024-09-16T02:00:12.921 INFO:journalctl@ceph.mds.h.smithi137.stdout:Sep 16 02:00:12 smithi137 systemd[1]: Started Ceph mds.h for 423b7a2a-73ce-11ef-bceb-c7b262605968. 2024-09-16T02:00:14.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:13 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:14.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:13 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:14.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:13 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:14.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:13 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:00:14.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:13 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:00:14.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:13 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:14.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:13 smithi105 ceph-mon[31197]: Deploying daemon mds.i on smithi190 2024-09-16T02:00:14.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:13 smithi105 ceph-mon[31197]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:14.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:13 smithi105 ceph-mon[31197]: mds.? [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] up:boot 2024-09-16T02:00:14.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:13 smithi105 ceph-mon[31197]: fsmap 8 up:standby 2024-09-16T02:00:14.046 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:13 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T02:00:14.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:13 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:14.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:13 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:14.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:13 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:14.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:13 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:00:14.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:13 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:00:14.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:13 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:14.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:13 smithi190 ceph-mon[34762]: Deploying daemon mds.i on smithi190 2024-09-16T02:00:14.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:13 smithi190 ceph-mon[34762]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:14.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:13 smithi190 ceph-mon[34762]: mds.? [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] up:boot 2024-09-16T02:00:14.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:13 smithi190 ceph-mon[34762]: fsmap 8 up:standby 2024-09-16T02:00:14.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:13 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T02:00:14.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:13 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:14.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:13 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:14.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:13 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:14.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:13 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:00:14.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:13 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:00:14.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:13 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:14.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:13 smithi137 ceph-mon[34099]: Deploying daemon mds.i on smithi190 2024-09-16T02:00:14.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:13 smithi137 ceph-mon[34099]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:14.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:13 smithi137 ceph-mon[34099]: mds.? [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] up:boot 2024-09-16T02:00:14.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:13 smithi137 ceph-mon[34099]: fsmap 8 up:standby 2024-09-16T02:00:14.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:13 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T02:00:14.533 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:00:14.790 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:00:14.842 INFO:journalctl@ceph.mds.i.smithi190.stdout:Sep 16 02:00:14 smithi190 systemd[1]: Starting Ceph mds.i for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:00:15.135 INFO:journalctl@ceph.mds.i.smithi190.stdout:Sep 16 02:00:14 smithi190 podman[64590]: 2024-09-16 02:00:14.747985331 +0000 UTC m=+0.015747933 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:00:15.135 INFO:journalctl@ceph.mds.i.smithi190.stdout:Sep 16 02:00:14 smithi190 podman[64590]: 2024-09-16 02:00:14.873030199 +0000 UTC m=+0.140792844 container create 6ac8de37e3cc5191a62453fb360495e567ffe1cd3f468693f25bec08193ded58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-i, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-16T02:00:15.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.301+0000 7f3854e9e640 1 -- 172.21.15.105:0/1389882345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3850102570 msgr2=0x7f38501029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:15.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.301+0000 7f3854e9e640 1 --2- 172.21.15.105:0/1389882345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3850102570 0x7f38501029b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3844009900 tx=0x7f384402a480 comp rx=0 tx=0).stop 2024-09-16T02:00:15.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.301+0000 7f3854e9e640 1 -- 172.21.15.105:0/1389882345 shutdown_connections 2024-09-16T02:00:15.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.301+0000 7f3854e9e640 1 --2- 172.21.15.105:0/1389882345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3850102ef0 0x7f385010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:15.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.301+0000 7f3854e9e640 1 --2- 172.21.15.105:0/1389882345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3850102570 0x7f38501029b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:15.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.301+0000 7f3854e9e640 1 --2- 172.21.15.105:0/1389882345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3850100e20 0x7f3850101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:15.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.301+0000 7f3854e9e640 1 -- 172.21.15.105:0/1389882345 >> 172.21.15.105:0/1389882345 conn(0x7f38500fc7d0 msgr2=0x7f38500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:15.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.301+0000 7f3854e9e640 1 -- 172.21.15.105:0/1389882345 shutdown_connections 2024-09-16T02:00:15.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.301+0000 7f3854e9e640 1 -- 172.21.15.105:0/1389882345 wait complete. 2024-09-16T02:00:15.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.302+0000 7f3854e9e640 1 Processor -- start 2024-09-16T02:00:15.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.302+0000 7f3854e9e640 1 -- start start 2024-09-16T02:00:15.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.302+0000 7f3854e9e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3850100e20 0x7f3850199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:15.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.302+0000 7f3854e9e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3850102570 0x7f38501997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:15.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.302+0000 7f3854e9e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3850102ef0 0x7f38501a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:15.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.302+0000 7f3854e9e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3850110e00 con 0x7f3850102ef0 2024-09-16T02:00:15.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.302+0000 7f3854e9e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3850110c80 con 0x7f3850102570 2024-09-16T02:00:15.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.302+0000 7f3854e9e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3850110f80 con 0x7f3850100e20 2024-09-16T02:00:15.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.303+0000 7f384ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3850102ef0 0x7f38501a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:15.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.303+0000 7f384ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3850102ef0 0x7f38501a42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34938/0 (socket says 172.21.15.105:34938) 2024-09-16T02:00:15.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.303+0000 7f384ed9d640 1 -- 172.21.15.105:0/2805613153 learned_addr learned my addr 172.21.15.105:0/2805613153 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:00:15.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.303+0000 7f384dd9b640 1 --2- 172.21.15.105:0/2805613153 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3850102570 0x7f38501997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:15.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.303+0000 7f384e59c640 1 --2- 172.21.15.105:0/2805613153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3850100e20 0x7f3850199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:15.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.303+0000 7f384ed9d640 1 -- 172.21.15.105:0/2805613153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3850100e20 msgr2=0x7f3850199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:15.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.303+0000 7f384ed9d640 1 --2- 172.21.15.105:0/2805613153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3850100e20 0x7f3850199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:15.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.303+0000 7f384ed9d640 1 -- 172.21.15.105:0/2805613153 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3850102570 msgr2=0x7f38501997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:15.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.303+0000 7f384ed9d640 1 --2- 172.21.15.105:0/2805613153 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3850102570 0x7f38501997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:15.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.303+0000 7f384ed9d640 1 -- 172.21.15.105:0/2805613153 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38501a4830 con 0x7f3850102ef0 2024-09-16T02:00:15.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.303+0000 7f384ed9d640 1 --2- 172.21.15.105:0/2805613153 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3850102ef0 0x7f38501a42f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f384800eaa0 tx=0x7f384800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:15.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.303+0000 7f383f7fe640 1 -- 172.21.15.105:0/2805613153 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384800cdb0 con 0x7f3850102ef0 2024-09-16T02:00:15.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.303+0000 7f383f7fe640 1 -- 172.21.15.105:0/2805613153 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3848005020 con 0x7f3850102ef0 2024-09-16T02:00:15.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.303+0000 7f3854e9e640 1 -- 172.21.15.105:0/2805613153 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38501a4a00 con 0x7f3850102ef0 2024-09-16T02:00:15.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.303+0000 7f383f7fe640 1 -- 172.21.15.105:0/2805613153 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38480116d0 con 0x7f3850102ef0 2024-09-16T02:00:15.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.303+0000 7f3854e9e640 1 -- 172.21.15.105:0/2805613153 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38501a4ec0 con 0x7f3850102ef0 2024-09-16T02:00:15.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.304+0000 7f3854e9e640 1 -- 172.21.15.105:0/2805613153 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3850101200 con 0x7f3850102ef0 2024-09-16T02:00:15.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.305+0000 7f383f7fe640 1 -- 172.21.15.105:0/2805613153 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f38480051d0 con 0x7f3850102ef0 2024-09-16T02:00:15.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.306+0000 7f383f7fe640 1 --2- 172.21.15.105:0/2805613153 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f38380a3240 0x7f38380a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:15.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.306+0000 7f384e59c640 1 --2- 172.21.15.105:0/2805613153 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f38380a3240 0x7f38380a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:15.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.306+0000 7f383f7fe640 1 -- 172.21.15.105:0/2805613153 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f38480cf610 con 0x7f3850102ef0 2024-09-16T02:00:15.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.307+0000 7f384e59c640 1 --2- 172.21.15.105:0/2805613153 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f38380a3240 0x7f38380a5700 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3840005d00 tx=0x7f384000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:15.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.310+0000 7f383f7fe640 1 -- 172.21.15.105:0/2805613153 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3848016030 con 0x7f3850102ef0 2024-09-16T02:00:15.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.527+0000 7f3854e9e640 1 -- 172.21.15.105:0/2805613153 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3850109090 con 0x7f3850102ef0 2024-09-16T02:00:15.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.528+0000 7f383f7fe640 1 -- 172.21.15.105:0/2805613153 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8058 (secure 0 0 0) 0x7f384809c0e0 con 0x7f3850102ef0 2024-09-16T02:00:15.529 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:00:15.529 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":10,"btime":"2024-09-16T02:00:15:215711+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2057699920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2057699920},{"type":"v1","addr":"172.21.15.105:6835","nonce":2057699920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1764886948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1764886948},{"type":"v1","addr":"172.21.15.105:6837","nonce":1764886948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/683519097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":683519097},{"type":"v1","addr":"172.21.15.190:6833","nonce":683519097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/3025442862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3025442862},{"type":"v1","addr":"172.21.15.137:6833","nonce":3025442862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/763783602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":763783602},{"type":"v1","addr":"172.21.15.137:6835","nonce":763783602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1851042281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1851042281},{"type":"v1","addr":"172.21.15.190:6835","nonce":1851042281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/3181257549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3181257549},{"type":"v1","addr":"172.21.15.190:6837","nonce":3181257549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/2788541030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2788541030},{"type":"v1","addr":"172.21.15.137:6837","nonce":2788541030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6839/1222573537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":1222573537},{"type":"v1","addr":"172.21.15.190:6839","nonce":1222573537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-16T02:00:15.529 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 10 2024-09-16T02:00:15.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.530+0000 7f3854e9e640 1 -- 172.21.15.105:0/2805613153 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f38380a3240 msgr2=0x7f38380a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:15.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.530+0000 7f3854e9e640 1 --2- 172.21.15.105:0/2805613153 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f38380a3240 0x7f38380a5700 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3840005d00 tx=0x7f384000b410 comp rx=0 tx=0).stop 2024-09-16T02:00:15.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.530+0000 7f3854e9e640 1 -- 172.21.15.105:0/2805613153 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3850102ef0 msgr2=0x7f38501a42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:15.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.530+0000 7f3854e9e640 1 --2- 172.21.15.105:0/2805613153 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3850102ef0 0x7f38501a42f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f384800eaa0 tx=0x7f384800ef60 comp rx=0 tx=0).stop 2024-09-16T02:00:15.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.530+0000 7f3854e9e640 1 -- 172.21.15.105:0/2805613153 shutdown_connections 2024-09-16T02:00:15.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.530+0000 7f3854e9e640 1 --2- 172.21.15.105:0/2805613153 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f38380a3240 0x7f38380a5700 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:15.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.530+0000 7f3854e9e640 1 --2- 172.21.15.105:0/2805613153 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3850102ef0 0x7f38501a42f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:15.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.530+0000 7f3854e9e640 1 --2- 172.21.15.105:0/2805613153 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3850102570 0x7f38501997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:15.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.530+0000 7f3854e9e640 1 --2- 172.21.15.105:0/2805613153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3850100e20 0x7f3850199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:15.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.530+0000 7f3854e9e640 1 -- 172.21.15.105:0/2805613153 >> 172.21.15.105:0/2805613153 conn(0x7f38500fc7d0 msgr2=0x7f38500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:15.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.530+0000 7f3854e9e640 1 -- 172.21.15.105:0/2805613153 shutdown_connections 2024-09-16T02:00:15.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:15.531+0000 7f3854e9e640 1 -- 172.21.15.105:0/2805613153 wait complete. 2024-09-16T02:00:15.593 INFO:journalctl@ceph.mds.i.smithi190.stdout:Sep 16 02:00:15 smithi190 podman[64590]: 2024-09-16 02:00:15.134336555 +0000 UTC m=+0.402099145 container init 6ac8de37e3cc5191a62453fb360495e567ffe1cd3f468693f25bec08193ded58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-i, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T02:00:15.593 INFO:journalctl@ceph.mds.i.smithi190.stdout:Sep 16 02:00:15 smithi190 podman[64590]: 2024-09-16 02:00:15.137227819 +0000 UTC m=+0.404990411 container start 6ac8de37e3cc5191a62453fb360495e567ffe1cd3f468693f25bec08193ded58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-i, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-16T02:00:15.593 INFO:journalctl@ceph.mds.i.smithi190.stdout:Sep 16 02:00:15 smithi190 ceph-mds[64609]: -- 172.21.15.190:0/3671572728 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dda343d860 con 0x55dda341a800 2024-09-16T02:00:15.593 INFO:journalctl@ceph.mds.i.smithi190.stdout:Sep 16 02:00:15 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-i[64605]: starting mds.i at 2024-09-16T02:00:15.593 INFO:journalctl@ceph.mds.i.smithi190.stdout:Sep 16 02:00:15 smithi190 bash[64590]: 6ac8de37e3cc5191a62453fb360495e567ffe1cd3f468693f25bec08193ded58 2024-09-16T02:00:15.594 INFO:journalctl@ceph.mds.i.smithi190.stdout:Sep 16 02:00:15 smithi190 systemd[1]: Started Ceph mds.i for 423b7a2a-73ce-11ef-bceb-c7b262605968. 2024-09-16T02:00:16.493 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:16 smithi105 ceph-mon[31197]: mds.? [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] up:boot 2024-09-16T02:00:16.493 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:16 smithi105 ceph-mon[31197]: fsmap 9 up:standby 2024-09-16T02:00:16.493 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:16 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T02:00:16.493 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:16 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:16.494 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:16 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:16.494 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:16 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:16.494 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:16 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:00:16.494 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:16 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:00:16.494 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:16 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:16.494 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:16 smithi105 ceph-mon[31197]: Deploying daemon mds.g on smithi105 2024-09-16T02:00:16.494 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:16 smithi105 ceph-mon[31197]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:16.494 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:16 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2805613153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:00:16.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:16 smithi190 ceph-mon[34762]: mds.? [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] up:boot 2024-09-16T02:00:16.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:16 smithi190 ceph-mon[34762]: fsmap 9 up:standby 2024-09-16T02:00:16.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:16 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T02:00:16.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:16 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:16.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:16 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:16.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:16 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:16.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:16 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:00:16.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:16 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:00:16.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:16 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:16.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:16 smithi190 ceph-mon[34762]: Deploying daemon mds.g on smithi105 2024-09-16T02:00:16.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:16 smithi190 ceph-mon[34762]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:16.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:16 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/2805613153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:00:16.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:16 smithi137 ceph-mon[34099]: mds.? [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] up:boot 2024-09-16T02:00:16.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:16 smithi137 ceph-mon[34099]: fsmap 9 up:standby 2024-09-16T02:00:16.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:16 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T02:00:16.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:16 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:16.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:16 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:16.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:16 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:16.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:16 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:00:16.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:16 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:00:16.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:16 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:16.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:16 smithi137 ceph-mon[34099]: Deploying daemon mds.g on smithi105 2024-09-16T02:00:16.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:16 smithi137 ceph-mon[34099]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:16.672 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:16 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/2805613153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:00:18.045 INFO:journalctl@ceph.mds.g.smithi105.stdout:Sep 16 02:00:17 smithi105 systemd[1]: Starting Ceph mds.g for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:00:18.187 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:00:18.388 INFO:journalctl@ceph.mds.g.smithi105.stdout:Sep 16 02:00:18 smithi105 podman[71678]: 2024-09-16 02:00:18.008519774 +0000 UTC m=+0.015684888 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:00:18.389 INFO:journalctl@ceph.mds.g.smithi105.stdout:Sep 16 02:00:18 smithi105 podman[71678]: 2024-09-16 02:00:18.143500832 +0000 UTC m=+0.150665949 container create f203576a25fd05c61304502f4ac5101646f379d73872e065c6e254437cf40890 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-g, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0) 2024-09-16T02:00:18.540 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:00:18.662 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:18 smithi105 ceph-mon[31197]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:18.663 INFO:journalctl@ceph.mds.g.smithi105.stdout:Sep 16 02:00:18 smithi105 podman[71678]: 2024-09-16 02:00:18.388026649 +0000 UTC m=+0.395191761 container init f203576a25fd05c61304502f4ac5101646f379d73872e065c6e254437cf40890 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-g, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-16T02:00:18.663 INFO:journalctl@ceph.mds.g.smithi105.stdout:Sep 16 02:00:18 smithi105 podman[71678]: 2024-09-16 02:00:18.391307902 +0000 UTC m=+0.398473023 container start f203576a25fd05c61304502f4ac5101646f379d73872e065c6e254437cf40890 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-g, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-16T02:00:18.663 INFO:journalctl@ceph.mds.g.smithi105.stdout:Sep 16 02:00:18 smithi105 ceph-mds[71727]: -- 172.21.15.105:0/2108967528 <== mon.0 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bba68cb860 con 0x55bba68a9400 2024-09-16T02:00:18.663 INFO:journalctl@ceph.mds.g.smithi105.stdout:Sep 16 02:00:18 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-g[71712]: starting mds.g at 2024-09-16T02:00:18.663 INFO:journalctl@ceph.mds.g.smithi105.stdout:Sep 16 02:00:18 smithi105 bash[71678]: f203576a25fd05c61304502f4ac5101646f379d73872e065c6e254437cf40890 2024-09-16T02:00:18.663 INFO:journalctl@ceph.mds.g.smithi105.stdout:Sep 16 02:00:18 smithi105 systemd[1]: Started Ceph mds.g for 423b7a2a-73ce-11ef-bceb-c7b262605968. 2024-09-16T02:00:18.845 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:18 smithi190 ceph-mon[34762]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:18.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:18 smithi137 ceph-mon[34099]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:19.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.039+0000 7efc07e44640 1 -- 172.21.15.105:0/1865612902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc00106a80 msgr2=0x7efc00106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:19.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.039+0000 7efc07e44640 1 --2- 172.21.15.105:0/1865612902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc00106a80 0x7efc00106e60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efbf8009930 tx=0x7efbf802a420 comp rx=0 tx=0).stop 2024-09-16T02:00:19.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.039+0000 7efc07e44640 1 -- 172.21.15.105:0/1865612902 shutdown_connections 2024-09-16T02:00:19.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.039+0000 7efc07e44640 1 --2- 172.21.15.105:0/1865612902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc00100a80 0x7efc00100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:19.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.039+0000 7efc07e44640 1 --2- 172.21.15.105:0/1865612902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc00101420 0x7efc0010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:19.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.039+0000 7efc07e44640 1 --2- 172.21.15.105:0/1865612902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc00106a80 0x7efc00106e60 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:19.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.039+0000 7efc07e44640 1 -- 172.21.15.105:0/1865612902 >> 172.21.15.105:0/1865612902 conn(0x7efc000fc7a0 msgr2=0x7efc000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:19.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.039+0000 7efc07e44640 1 -- 172.21.15.105:0/1865612902 shutdown_connections 2024-09-16T02:00:19.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.039+0000 7efc07e44640 1 -- 172.21.15.105:0/1865612902 wait complete. 2024-09-16T02:00:19.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.040+0000 7efc07e44640 1 Processor -- start 2024-09-16T02:00:19.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.040+0000 7efc07e44640 1 -- start start 2024-09-16T02:00:19.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.040+0000 7efc07e44640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc00100a80 0x7efc0019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:19.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.040+0000 7efc07e44640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc00101420 0x7efc0019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:19.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.040+0000 7efc07e44640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc00106a80 0x7efc0019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:19.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.040+0000 7efc07e44640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efc0010f360 con 0x7efc00106a80 2024-09-16T02:00:19.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.040+0000 7efc07e44640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efc0010f1e0 con 0x7efc00100a80 2024-09-16T02:00:19.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.040+0000 7efc07e44640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7efc0010f4e0 con 0x7efc00101420 2024-09-16T02:00:19.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.040+0000 7efc063e1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc00106a80 0x7efc0019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:19.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.040+0000 7efc063e1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc00106a80 0x7efc0019eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34260/0 (socket says 172.21.15.105:34260) 2024-09-16T02:00:19.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.040+0000 7efc053df640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc00101420 0x7efc0019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:19.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.040+0000 7efc063e1640 1 -- 172.21.15.105:0/1156959647 learned_addr learned my addr 172.21.15.105:0/1156959647 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:00:19.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.040+0000 7efc05be0640 1 --2- 172.21.15.105:0/1156959647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc00100a80 0x7efc0019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:19.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.040+0000 7efc063e1640 1 -- 172.21.15.105:0/1156959647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc00101420 msgr2=0x7efc0019ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:19.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.040+0000 7efc063e1640 1 --2- 172.21.15.105:0/1156959647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc00101420 0x7efc0019ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:19.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.040+0000 7efc063e1640 1 -- 172.21.15.105:0/1156959647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc00100a80 msgr2=0x7efc0019a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:19.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.041+0000 7efc063e1640 1 --2- 172.21.15.105:0/1156959647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc00100a80 0x7efc0019a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:19.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.041+0000 7efc063e1640 1 -- 172.21.15.105:0/1156959647 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc0019f5b0 con 0x7efc00106a80 2024-09-16T02:00:19.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.041+0000 7efc053df640 1 --2- 172.21.15.105:0/1156959647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc00101420 0x7efc0019ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:00:19.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.041+0000 7efc063e1640 1 --2- 172.21.15.105:0/1156959647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc00106a80 0x7efc0019eed0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efbf000cce0 tx=0x7efbf000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:19.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.041+0000 7efbfeffd640 1 -- 172.21.15.105:0/1156959647 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf000f450 con 0x7efc00106a80 2024-09-16T02:00:19.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.041+0000 7efbfeffd640 1 -- 172.21.15.105:0/1156959647 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efbf000faa0 con 0x7efc00106a80 2024-09-16T02:00:19.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.041+0000 7efc07e44640 1 -- 172.21.15.105:0/1156959647 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc0019f8a0 con 0x7efc00106a80 2024-09-16T02:00:19.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.041+0000 7efbfeffd640 1 -- 172.21.15.105:0/1156959647 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf0018b70 con 0x7efc00106a80 2024-09-16T02:00:19.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.041+0000 7efc07e44640 1 -- 172.21.15.105:0/1156959647 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc001a7180 con 0x7efc00106a80 2024-09-16T02:00:19.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.042+0000 7efc07e44640 1 -- 172.21.15.105:0/1156959647 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc00100ee0 con 0x7efc00106a80 2024-09-16T02:00:19.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.043+0000 7efbfeffd640 1 -- 172.21.15.105:0/1156959647 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7efbf0018d10 con 0x7efc00106a80 2024-09-16T02:00:19.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.044+0000 7efbfeffd640 1 --2- 172.21.15.105:0/1156959647 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7efbe80a3240 0x7efbe80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:19.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.044+0000 7efbfeffd640 1 -- 172.21.15.105:0/1156959647 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7efbf00cf900 con 0x7efc00106a80 2024-09-16T02:00:19.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.044+0000 7efc05be0640 1 --2- 172.21.15.105:0/1156959647 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7efbe80a3240 0x7efbe80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:19.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.044+0000 7efc05be0640 1 --2- 172.21.15.105:0/1156959647 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7efbe80a3240 0x7efbe80a5700 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7efbf8000c00 tx=0x7efbf8032c80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:19.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.047+0000 7efbfeffd640 1 -- 172.21.15.105:0/1156959647 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efbf0014030 con 0x7efc00106a80 2024-09-16T02:00:19.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.263+0000 7efc07e44640 1 -- 172.21.15.105:0/1156959647 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc0019b960 con 0x7efc00106a80 2024-09-16T02:00:19.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.263+0000 7efbfeffd640 1 -- 172.21.15.105:0/1156959647 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8887 (secure 0 0 0) 0x7efbf009cb50 con 0x7efc00106a80 2024-09-16T02:00:19.265 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:00:19.265 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":11,"btime":"2024-09-16T02:00:18:480734+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2057699920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2057699920},{"type":"v1","addr":"172.21.15.105:6835","nonce":2057699920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1764886948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1764886948},{"type":"v1","addr":"172.21.15.105:6837","nonce":1764886948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6839/4112062361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":4112062361},{"type":"v1","addr":"172.21.15.105:6839","nonce":4112062361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/683519097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":683519097},{"type":"v1","addr":"172.21.15.190:6833","nonce":683519097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/3025442862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3025442862},{"type":"v1","addr":"172.21.15.137:6833","nonce":3025442862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/763783602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":763783602},{"type":"v1","addr":"172.21.15.137:6835","nonce":763783602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1851042281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1851042281},{"type":"v1","addr":"172.21.15.190:6835","nonce":1851042281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/3181257549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3181257549},{"type":"v1","addr":"172.21.15.190:6837","nonce":3181257549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/2788541030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2788541030},{"type":"v1","addr":"172.21.15.137:6837","nonce":2788541030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6839/1222573537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":1222573537},{"type":"v1","addr":"172.21.15.190:6839","nonce":1222573537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-16T02:00:19.265 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 11 2024-09-16T02:00:19.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.265+0000 7efc07e44640 1 -- 172.21.15.105:0/1156959647 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7efbe80a3240 msgr2=0x7efbe80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:19.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.265+0000 7efc07e44640 1 --2- 172.21.15.105:0/1156959647 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7efbe80a3240 0x7efbe80a5700 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7efbf8000c00 tx=0x7efbf8032c80 comp rx=0 tx=0).stop 2024-09-16T02:00:19.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.265+0000 7efc07e44640 1 -- 172.21.15.105:0/1156959647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc00106a80 msgr2=0x7efc0019eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:19.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.265+0000 7efc07e44640 1 --2- 172.21.15.105:0/1156959647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc00106a80 0x7efc0019eed0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efbf000cce0 tx=0x7efbf000a620 comp rx=0 tx=0).stop 2024-09-16T02:00:19.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.266+0000 7efc07e44640 1 -- 172.21.15.105:0/1156959647 shutdown_connections 2024-09-16T02:00:19.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.266+0000 7efc07e44640 1 --2- 172.21.15.105:0/1156959647 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7efbe80a3240 0x7efbe80a5700 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:19.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.266+0000 7efc07e44640 1 --2- 172.21.15.105:0/1156959647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc00101420 0x7efc0019ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:19.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.266+0000 7efc07e44640 1 --2- 172.21.15.105:0/1156959647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc00106a80 0x7efc0019eed0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:19.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.266+0000 7efc07e44640 1 --2- 172.21.15.105:0/1156959647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc00100a80 0x7efc0019a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:19.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.266+0000 7efc07e44640 1 -- 172.21.15.105:0/1156959647 >> 172.21.15.105:0/1156959647 conn(0x7efc000fc7a0 msgr2=0x7efc000fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:19.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.266+0000 7efc07e44640 1 -- 172.21.15.105:0/1156959647 shutdown_connections 2024-09-16T02:00:19.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:19.266+0000 7efc07e44640 1 -- 172.21.15.105:0/1156959647 wait complete. 2024-09-16T02:00:19.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:19 smithi105 ceph-mon[31197]: mds.? [v2:172.21.15.105:6838/4112062361,v1:172.21.15.105:6839/4112062361] up:boot 2024-09-16T02:00:19.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:19 smithi105 ceph-mon[31197]: fsmap 10 up:standby 2024-09-16T02:00:19.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:19 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-16T02:00:19.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:19 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:19.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:19 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:19.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:19 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:19.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:19 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:00:19.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:19 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:00:19.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:19 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:19.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:19 smithi105 ceph-mon[31197]: Deploying daemon mds.a on smithi105 2024-09-16T02:00:19.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:19 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/1156959647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:00:19.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:19 smithi190 ceph-mon[34762]: mds.? [v2:172.21.15.105:6838/4112062361,v1:172.21.15.105:6839/4112062361] up:boot 2024-09-16T02:00:19.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:19 smithi190 ceph-mon[34762]: fsmap 10 up:standby 2024-09-16T02:00:19.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:19 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-16T02:00:19.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:19 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:19.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:19 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:19.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:19 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:19.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:19 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:00:19.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:19 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:00:19.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:19 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:19.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:19 smithi190 ceph-mon[34762]: Deploying daemon mds.a on smithi105 2024-09-16T02:00:19.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:19 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/1156959647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:00:19.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:19 smithi137 ceph-mon[34099]: mds.? [v2:172.21.15.105:6838/4112062361,v1:172.21.15.105:6839/4112062361] up:boot 2024-09-16T02:00:19.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:19 smithi137 ceph-mon[34099]: fsmap 10 up:standby 2024-09-16T02:00:19.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:19 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-16T02:00:19.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:19 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:19.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:19 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:19.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:19 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:19.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:19 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:00:19.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:19 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:00:19.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:19 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:19.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:19 smithi137 ceph-mon[34099]: Deploying daemon mds.a on smithi105 2024-09-16T02:00:19.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:19 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/1156959647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:00:20.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:20 smithi105 ceph-mon[31197]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:20.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:20 smithi190 ceph-mon[34762]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:20.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:20 smithi137 ceph-mon[34099]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:21.545 INFO:journalctl@ceph.mds.a.smithi105.stdout:Sep 16 02:00:21 smithi105 systemd[1]: Starting Ceph mds.a for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:00:21.798 INFO:journalctl@ceph.mds.a.smithi105.stdout:Sep 16 02:00:21 smithi105 podman[72092]: 2024-09-16 02:00:21.512826051 +0000 UTC m=+0.016068995 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:00:21.798 INFO:journalctl@ceph.mds.a.smithi105.stdout:Sep 16 02:00:21 smithi105 podman[72092]: 2024-09-16 02:00:21.65087133 +0000 UTC m=+0.154114265 container create eeeb2888b1f44a64f60a5a78c19e5a4fa53ffe72f0a045c951f1651736f7f493 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True) 2024-09-16T02:00:22.294 INFO:journalctl@ceph.mds.a.smithi105.stdout:Sep 16 02:00:21 smithi105 podman[72092]: 2024-09-16 02:00:21.903953054 +0000 UTC m=+0.407195988 container init eeeb2888b1f44a64f60a5a78c19e5a4fa53ffe72f0a045c951f1651736f7f493 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-16T02:00:22.295 INFO:journalctl@ceph.mds.a.smithi105.stdout:Sep 16 02:00:21 smithi105 podman[72092]: 2024-09-16 02:00:21.906882807 +0000 UTC m=+0.410125760 container start eeeb2888b1f44a64f60a5a78c19e5a4fa53ffe72f0a045c951f1651736f7f493 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-a, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-16T02:00:22.295 INFO:journalctl@ceph.mds.a.smithi105.stdout:Sep 16 02:00:21 smithi105 ceph-mds[72112]: -- 172.21.15.105:0/3709062724 <== mon.0 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e48b395860 con 0x55e48b373000 2024-09-16T02:00:22.295 INFO:journalctl@ceph.mds.a.smithi105.stdout:Sep 16 02:00:21 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-a[72108]: starting mds.a at 2024-09-16T02:00:22.295 INFO:journalctl@ceph.mds.a.smithi105.stdout:Sep 16 02:00:22 smithi105 bash[72092]: eeeb2888b1f44a64f60a5a78c19e5a4fa53ffe72f0a045c951f1651736f7f493 2024-09-16T02:00:22.295 INFO:journalctl@ceph.mds.a.smithi105.stdout:Sep 16 02:00:22 smithi105 systemd[1]: Started Ceph mds.a for 423b7a2a-73ce-11ef-bceb-c7b262605968. 2024-09-16T02:00:22.508 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:00:22.764 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:00:22.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:22 smithi105 ceph-mon[31197]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:22.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:22 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:22.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:22 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:22.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:22 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:22.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:22 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:00:22.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:22 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:00:22.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:22 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:22.811 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:22 smithi137 ceph-mon[34099]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:22.811 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:22 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:22.811 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:22 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:22.812 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:22 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:22.812 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:22 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:00:22.812 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:22 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:00:22.812 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:22 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:22.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:22 smithi190 ceph-mon[34762]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:22.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:22 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:22.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:22 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:22.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:22 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:22.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:22 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:00:22.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:22 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:00:22.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:22 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:23.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.314+0000 7f2e12721640 1 -- 172.21.15.105:0/3920966113 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e0c101df0 msgr2=0x7f2e0c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:23.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.314+0000 7f2e12721640 1 --2- 172.21.15.105:0/3920966113 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e0c101df0 0x7f2e0c1021d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009900 tx=0x7f2dfc02a410 comp rx=0 tx=0).stop 2024-09-16T02:00:23.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.314+0000 7f2e12721640 1 -- 172.21.15.105:0/3920966113 shutdown_connections 2024-09-16T02:00:23.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.314+0000 7f2e12721640 1 --2- 172.21.15.105:0/3920966113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e0c1027a0 0x7f2e0c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:23.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.314+0000 7f2e12721640 1 --2- 172.21.15.105:0/3920966113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e0c106f20 0x7f2e0c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:23.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.314+0000 7f2e12721640 1 --2- 172.21.15.105:0/3920966113 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e0c101df0 0x7f2e0c1021d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:23.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.314+0000 7f2e12721640 1 -- 172.21.15.105:0/3920966113 >> 172.21.15.105:0/3920966113 conn(0x7f2e0c0fd8f0 msgr2=0x7f2e0c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:23.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.314+0000 7f2e12721640 1 -- 172.21.15.105:0/3920966113 shutdown_connections 2024-09-16T02:00:23.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.314+0000 7f2e12721640 1 -- 172.21.15.105:0/3920966113 wait complete. 2024-09-16T02:00:23.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.315+0000 7f2e12721640 1 Processor -- start 2024-09-16T02:00:23.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.315+0000 7f2e12721640 1 -- start start 2024-09-16T02:00:23.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.315+0000 7f2e12721640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e0c101df0 0x7f2e0c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:23.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.315+0000 7f2e12721640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e0c1027a0 0x7f2e0c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:23.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.315+0000 7f2e12721640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e0c106f20 0x7f2e0c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:23.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.315+0000 7f2e12721640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c111ed0 con 0x7f2e0c1027a0 2024-09-16T02:00:23.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.315+0000 7f2e12721640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c111d50 con 0x7f2e0c101df0 2024-09-16T02:00:23.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.315+0000 7f2e12721640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c112050 con 0x7f2e0c106f20 2024-09-16T02:00:23.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.315+0000 7f2e0b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e0c1027a0 0x7f2e0c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:23.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.315+0000 7f2e0bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e0c101df0 0x7f2e0c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:23.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.315+0000 7f2e0b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e0c1027a0 0x7f2e0c1989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34310/0 (socket says 172.21.15.105:34310) 2024-09-16T02:00:23.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.315+0000 7f2e0b7fe640 1 -- 172.21.15.105:0/3854931219 learned_addr learned my addr 172.21.15.105:0/3854931219 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:00:23.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.315+0000 7f2e10cbe640 1 --2- 172.21.15.105:0/3854931219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e0c106f20 0x7f2e0c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:23.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.315+0000 7f2e0b7fe640 1 -- 172.21.15.105:0/3854931219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e0c106f20 msgr2=0x7f2e0c19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:23.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.315+0000 7f2e0b7fe640 1 --2- 172.21.15.105:0/3854931219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e0c106f20 0x7f2e0c19cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:23.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.315+0000 7f2e0b7fe640 1 -- 172.21.15.105:0/3854931219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e0c101df0 msgr2=0x7f2e0c198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:23.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.315+0000 7f2e0b7fe640 1 --2- 172.21.15.105:0/3854931219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e0c101df0 0x7f2e0c198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:23.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.315+0000 7f2e0b7fe640 1 -- 172.21.15.105:0/3854931219 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e0c19d440 con 0x7f2e0c1027a0 2024-09-16T02:00:23.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.316+0000 7f2e0b7fe640 1 --2- 172.21.15.105:0/3854931219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e0c1027a0 0x7f2e0c1989d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2df400b4a0 tx=0x7f2df400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:23.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.316+0000 7f2e097fa640 1 -- 172.21.15.105:0/3854931219 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df4014070 con 0x7f2e0c1027a0 2024-09-16T02:00:23.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.316+0000 7f2e12721640 1 -- 172.21.15.105:0/3854931219 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e0c19d730 con 0x7f2e0c1027a0 2024-09-16T02:00:23.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.316+0000 7f2e097fa640 1 -- 172.21.15.105:0/3854931219 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2df4010ac0 con 0x7f2e0c1027a0 2024-09-16T02:00:23.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.316+0000 7f2e097fa640 1 -- 172.21.15.105:0/3854931219 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df4017c30 con 0x7f2e0c1027a0 2024-09-16T02:00:23.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.316+0000 7f2e12721640 1 -- 172.21.15.105:0/3854931219 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e0c1a5010 con 0x7f2e0c1027a0 2024-09-16T02:00:23.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.317+0000 7f2e12721640 1 -- 172.21.15.105:0/3854931219 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e0c102250 con 0x7f2e0c1027a0 2024-09-16T02:00:23.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.318+0000 7f2e097fa640 1 -- 172.21.15.105:0/3854931219 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f2df4017070 con 0x7f2e0c1027a0 2024-09-16T02:00:23.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.318+0000 7f2e097fa640 1 --2- 172.21.15.105:0/3854931219 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f2df80a3240 0x7f2df80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:23.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.319+0000 7f2e0bfff640 1 --2- 172.21.15.105:0/3854931219 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f2df80a3240 0x7f2df80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:23.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.319+0000 7f2e097fa640 1 -- 172.21.15.105:0/3854931219 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f2df40ceaa0 con 0x7f2e0c1027a0 2024-09-16T02:00:23.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.319+0000 7f2e0bfff640 1 --2- 172.21.15.105:0/3854931219 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f2df80a3240 0x7f2df80a5700 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc031440 tx=0x7f2dfc005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:23.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.323+0000 7f2e097fa640 1 -- 172.21.15.105:0/3854931219 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2df409b610 con 0x7f2e0c1027a0 2024-09-16T02:00:23.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.524+0000 7f2e12721640 1 -- 172.21.15.105:0/3854931219 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e0c109b70 con 0x7f2e0c1027a0 2024-09-16T02:00:23.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.524+0000 7f2e097fa640 1 -- 172.21.15.105:0/3854931219 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9713 (secure 0 0 0) 0x7f2df409f910 con 0x7f2e0c1027a0 2024-09-16T02:00:23.526 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:00:23.526 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":12,"btime":"2024-09-16T02:00:22:563575+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2057699920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2057699920},{"type":"v1","addr":"172.21.15.105:6835","nonce":2057699920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1764886948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1764886948},{"type":"v1","addr":"172.21.15.105:6837","nonce":1764886948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6839/4112062361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":4112062361},{"type":"v1","addr":"172.21.15.105:6839","nonce":4112062361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6841/163156514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":163156514},{"type":"v1","addr":"172.21.15.105:6841","nonce":163156514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/683519097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":683519097},{"type":"v1","addr":"172.21.15.190:6833","nonce":683519097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/3025442862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3025442862},{"type":"v1","addr":"172.21.15.137:6833","nonce":3025442862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/763783602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":763783602},{"type":"v1","addr":"172.21.15.137:6835","nonce":763783602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1851042281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1851042281},{"type":"v1","addr":"172.21.15.190:6835","nonce":1851042281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/3181257549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3181257549},{"type":"v1","addr":"172.21.15.190:6837","nonce":3181257549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/2788541030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2788541030},{"type":"v1","addr":"172.21.15.137:6837","nonce":2788541030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6839/1222573537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":1222573537},{"type":"v1","addr":"172.21.15.190:6839","nonce":1222573537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-16T02:00:23.526 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 12 2024-09-16T02:00:23.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.527+0000 7f2e12721640 1 -- 172.21.15.105:0/3854931219 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f2df80a3240 msgr2=0x7f2df80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:23.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.527+0000 7f2e12721640 1 --2- 172.21.15.105:0/3854931219 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f2df80a3240 0x7f2df80a5700 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc031440 tx=0x7f2dfc005e80 comp rx=0 tx=0).stop 2024-09-16T02:00:23.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.527+0000 7f2e12721640 1 -- 172.21.15.105:0/3854931219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e0c1027a0 msgr2=0x7f2e0c1989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:23.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.527+0000 7f2e12721640 1 --2- 172.21.15.105:0/3854931219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e0c1027a0 0x7f2e0c1989d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2df400b4a0 tx=0x7f2df400b960 comp rx=0 tx=0).stop 2024-09-16T02:00:23.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.528+0000 7f2e12721640 1 -- 172.21.15.105:0/3854931219 shutdown_connections 2024-09-16T02:00:23.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.528+0000 7f2e12721640 1 --2- 172.21.15.105:0/3854931219 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f2df80a3240 0x7f2df80a5700 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:23.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.528+0000 7f2e12721640 1 --2- 172.21.15.105:0/3854931219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e0c106f20 0x7f2e0c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:23.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.528+0000 7f2e12721640 1 --2- 172.21.15.105:0/3854931219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e0c1027a0 0x7f2e0c1989d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:23.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.528+0000 7f2e12721640 1 --2- 172.21.15.105:0/3854931219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e0c101df0 0x7f2e0c198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:23.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.528+0000 7f2e12721640 1 -- 172.21.15.105:0/3854931219 >> 172.21.15.105:0/3854931219 conn(0x7f2e0c0fd8f0 msgr2=0x7f2e0c0fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:23.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.528+0000 7f2e12721640 1 -- 172.21.15.105:0/3854931219 shutdown_connections 2024-09-16T02:00:23.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:23.528+0000 7f2e12721640 1 -- 172.21.15.105:0/3854931219 wait complete. 2024-09-16T02:00:23.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:23 smithi105 ceph-mon[31197]: Deploying daemon mds.b on smithi137 2024-09-16T02:00:23.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:23 smithi105 ceph-mon[31197]: mds.? [v2:172.21.15.105:6840/163156514,v1:172.21.15.105:6841/163156514] up:boot 2024-09-16T02:00:23.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:23 smithi105 ceph-mon[31197]: fsmap 11 up:standby 2024-09-16T02:00:23.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:23 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-16T02:00:23.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:23 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/3854931219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:00:23.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:23 smithi190 ceph-mon[34762]: Deploying daemon mds.b on smithi137 2024-09-16T02:00:23.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:23 smithi190 ceph-mon[34762]: mds.? [v2:172.21.15.105:6840/163156514,v1:172.21.15.105:6841/163156514] up:boot 2024-09-16T02:00:23.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:23 smithi190 ceph-mon[34762]: fsmap 11 up:standby 2024-09-16T02:00:23.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:23 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-16T02:00:23.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:23 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/3854931219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:00:23.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:23 smithi137 ceph-mon[34099]: Deploying daemon mds.b on smithi137 2024-09-16T02:00:23.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:23 smithi137 ceph-mon[34099]: mds.? [v2:172.21.15.105:6840/163156514,v1:172.21.15.105:6841/163156514] up:boot 2024-09-16T02:00:23.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:23 smithi137 ceph-mon[34099]: fsmap 11 up:standby 2024-09-16T02:00:23.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:23 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-16T02:00:23.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:23 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/3854931219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:00:24.334 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 16 02:00:24 smithi137 systemd[1]: Starting Ceph mds.b for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:00:24.589 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:24 smithi137 ceph-mon[34099]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:24.589 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 16 02:00:24 smithi137 podman[64138]: 2024-09-16 02:00:24.235321886 +0000 UTC m=+0.016232509 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:00:24.589 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 16 02:00:24 smithi137 podman[64138]: 2024-09-16 02:00:24.351703471 +0000 UTC m=+0.132614084 container create d260d3616179444105cafc91f4de590945867af01fe919edae1771484df65124 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-16T02:00:24.589 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 16 02:00:24 smithi137 podman[64138]: 2024-09-16 02:00:24.58772257 +0000 UTC m=+0.368633182 container init d260d3616179444105cafc91f4de590945867af01fe919edae1771484df65124 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-16T02:00:24.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:24 smithi190 ceph-mon[34762]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:24.921 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 16 02:00:24 smithi137 podman[64138]: 2024-09-16 02:00:24.590772314 +0000 UTC m=+0.371682924 container start d260d3616179444105cafc91f4de590945867af01fe919edae1771484df65124 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-16T02:00:24.921 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 16 02:00:24 smithi137 ceph-mds[64157]: -- 172.21.15.137:0/1478325617 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5632ded41860 con 0x5632ded1f000 2024-09-16T02:00:24.921 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 16 02:00:24 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-b[64153]: starting mds.b at 2024-09-16T02:00:24.921 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 16 02:00:24 smithi137 bash[64138]: d260d3616179444105cafc91f4de590945867af01fe919edae1771484df65124 2024-09-16T02:00:24.921 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 16 02:00:24 smithi137 systemd[1]: Started Ceph mds.b for 423b7a2a-73ce-11ef-bceb-c7b262605968. 2024-09-16T02:00:25.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:24 smithi105 ceph-mon[31197]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:25.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:25 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:25.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:25 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:25.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:25 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:25.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:25 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:25.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:25 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:25.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:25 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:00:25.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:25 smithi190 ceph-mon[34762]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:25.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:25 smithi190 ceph-mon[34762]: mds.? [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] up:boot 2024-09-16T02:00:25.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:25 smithi190 ceph-mon[34762]: fsmap 12 up:standby 2024-09-16T02:00:25.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:25 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T02:00:26.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:25 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:26.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:25 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:26.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:25 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:26.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:25 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:26.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:25 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:26.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:25 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:00:26.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:25 smithi105 ceph-mon[31197]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:26.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:25 smithi105 ceph-mon[31197]: mds.? [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] up:boot 2024-09-16T02:00:26.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:25 smithi105 ceph-mon[31197]: fsmap 12 up:standby 2024-09-16T02:00:26.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:25 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T02:00:26.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:25 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:26.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:25 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:26.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:25 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:26.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:25 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:26.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:25 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:26.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:25 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:00:26.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:25 smithi137 ceph-mon[34099]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:26.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:25 smithi137 ceph-mon[34099]: mds.? [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] up:boot 2024-09-16T02:00:26.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:25 smithi137 ceph-mon[34099]: fsmap 12 up:standby 2024-09-16T02:00:26.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:25 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T02:00:26.233 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:00:26.511 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:00:27.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:26.996+0000 7f3773fa4640 1 -- 172.21.15.105:0/396743696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f376c1027a0 msgr2=0x7f376c1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:27.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:26.996+0000 7f3773fa4640 1 --2- 172.21.15.105:0/396743696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f376c1027a0 0x7f376c1067f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f375c009900 tx=0x7f375c02a410 comp rx=0 tx=0).stop 2024-09-16T02:00:27.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:26.996+0000 7f3773fa4640 1 -- 172.21.15.105:0/396743696 shutdown_connections 2024-09-16T02:00:27.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:26.996+0000 7f3773fa4640 1 --2- 172.21.15.105:0/396743696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f376c106f20 0x7f376c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:27.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:26.996+0000 7f3773fa4640 1 --2- 172.21.15.105:0/396743696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f376c1027a0 0x7f376c1067f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:27.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:26.996+0000 7f3773fa4640 1 --2- 172.21.15.105:0/396743696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f376c101df0 0x7f376c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:27.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:26.996+0000 7f3773fa4640 1 -- 172.21.15.105:0/396743696 >> 172.21.15.105:0/396743696 conn(0x7f376c0fd8f0 msgr2=0x7f376c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:27.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:26.996+0000 7f3773fa4640 1 -- 172.21.15.105:0/396743696 shutdown_connections 2024-09-16T02:00:27.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:26.996+0000 7f3773fa4640 1 -- 172.21.15.105:0/396743696 wait complete. 2024-09-16T02:00:27.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:26.997+0000 7f3773fa4640 1 Processor -- start 2024-09-16T02:00:27.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:26.997+0000 7f3773fa4640 1 -- start start 2024-09-16T02:00:27.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:26.997+0000 7f3773fa4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f376c101df0 0x7f376c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:27.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:26.997+0000 7f3773fa4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f376c1027a0 0x7f376c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:27.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:26.997+0000 7f3773fa4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f376c106f20 0x7f376c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:27.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:26.997+0000 7f3773fa4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f376c111e30 con 0x7f376c1027a0 2024-09-16T02:00:27.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:26.997+0000 7f3773fa4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f376c111cb0 con 0x7f376c106f20 2024-09-16T02:00:27.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:26.997+0000 7f3773fa4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f376c111fb0 con 0x7f376c101df0 2024-09-16T02:00:27.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:26.997+0000 7f377153f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f376c1027a0 0x7f376c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:27.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:26.997+0000 7f377153f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f376c1027a0 0x7f376c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34342/0 (socket says 172.21.15.105:34342) 2024-09-16T02:00:27.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:26.997+0000 7f377153f640 1 -- 172.21.15.105:0/3503767881 learned_addr learned my addr 172.21.15.105:0/3503767881 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:00:27.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:26.997+0000 7f3771d40640 1 --2- 172.21.15.105:0/3503767881 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f376c101df0 0x7f376c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:27.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:26.997+0000 7f3772541640 1 --2- 172.21.15.105:0/3503767881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f376c106f20 0x7f376c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:27.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:26.998+0000 7f377153f640 1 -- 172.21.15.105:0/3503767881 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f376c101df0 msgr2=0x7f376c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:27.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:26.998+0000 7f377153f640 1 --2- 172.21.15.105:0/3503767881 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f376c101df0 0x7f376c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:27.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:26.998+0000 7f377153f640 1 -- 172.21.15.105:0/3503767881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f376c106f20 msgr2=0x7f376c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:27.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:26.998+0000 7f377153f640 1 --2- 172.21.15.105:0/3503767881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f376c106f20 0x7f376c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:27.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:26.998+0000 7f377153f640 1 -- 172.21.15.105:0/3503767881 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376c1a6840 con 0x7f376c1027a0 2024-09-16T02:00:27.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:26.998+0000 7f377153f640 1 --2- 172.21.15.105:0/3503767881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f376c1027a0 0x7f376c074a30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f375c005940 tx=0x7f375c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:27.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:26.998+0000 7f3762ffd640 1 -- 172.21.15.105:0/3503767881 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c03f030 con 0x7f376c1027a0 2024-09-16T02:00:27.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:26.998+0000 7f3762ffd640 1 -- 172.21.15.105:0/3503767881 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f375c003cb0 con 0x7f376c1027a0 2024-09-16T02:00:27.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:26.998+0000 7f3773fa4640 1 -- 172.21.15.105:0/3503767881 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f376c1a6ad0 con 0x7f376c1027a0 2024-09-16T02:00:27.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:26.998+0000 7f3762ffd640 1 -- 172.21.15.105:0/3503767881 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c035070 con 0x7f376c1027a0 2024-09-16T02:00:27.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:26.998+0000 7f3773fa4640 1 -- 172.21.15.105:0/3503767881 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f376c1a6fb0 con 0x7f376c1027a0 2024-09-16T02:00:27.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:26.999+0000 7f3773fa4640 1 -- 172.21.15.105:0/3503767881 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f376c1021d0 con 0x7f376c1027a0 2024-09-16T02:00:27.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:27.000+0000 7f3762ffd640 1 -- 172.21.15.105:0/3503767881 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f375c042030 con 0x7f376c1027a0 2024-09-16T02:00:27.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:27.001+0000 7f3762ffd640 1 --2- 172.21.15.105:0/3503767881 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f37580a3240 0x7f37580a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:27.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:27.001+0000 7f3771d40640 1 --2- 172.21.15.105:0/3503767881 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f37580a3240 0x7f37580a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:27.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:27.001+0000 7f3762ffd640 1 -- 172.21.15.105:0/3503767881 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f375c0efbe0 con 0x7f376c1027a0 2024-09-16T02:00:27.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:27.001+0000 7f3771d40640 1 --2- 172.21.15.105:0/3503767881 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f37580a3240 0x7f37580a5700 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3768009bd0 tx=0x7f3768005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:27.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:27.005+0000 7f3762ffd640 1 -- 172.21.15.105:0/3503767881 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f375c0bc750 con 0x7f376c1027a0 2024-09-16T02:00:27.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:27.264+0000 7f3773fa4640 1 -- 172.21.15.105:0/3503767881 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f376c10a160 con 0x7f376c1027a0 2024-09-16T02:00:27.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:27.265+0000 7f3762ffd640 1 -- 172.21.15.105:0/3503767881 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10542 (secure 0 0 0) 0x7f375c0c0a50 con 0x7f376c1027a0 2024-09-16T02:00:27.267 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:00:27.267 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":13,"btime":"2024-09-16T02:00:25:576867+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2057699920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2057699920},{"type":"v1","addr":"172.21.15.105:6835","nonce":2057699920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1764886948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1764886948},{"type":"v1","addr":"172.21.15.105:6837","nonce":1764886948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6839/4112062361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":4112062361},{"type":"v1","addr":"172.21.15.105:6839","nonce":4112062361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6841/163156514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":163156514},{"type":"v1","addr":"172.21.15.105:6841","nonce":163156514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/683519097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":683519097},{"type":"v1","addr":"172.21.15.190:6833","nonce":683519097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/3025442862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3025442862},{"type":"v1","addr":"172.21.15.137:6833","nonce":3025442862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/763783602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":763783602},{"type":"v1","addr":"172.21.15.137:6835","nonce":763783602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1851042281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1851042281},{"type":"v1","addr":"172.21.15.190:6835","nonce":1851042281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/3181257549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3181257549},{"type":"v1","addr":"172.21.15.190:6837","nonce":3181257549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/2788541030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2788541030},{"type":"v1","addr":"172.21.15.137:6837","nonce":2788541030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6839/1222573537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":1222573537},{"type":"v1","addr":"172.21.15.190:6839","nonce":1222573537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24482,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3454924009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3454924009},{"type":"v1","addr":"172.21.15.137:6839","nonce":3454924009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-16T02:00:27.267 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 13 2024-09-16T02:00:27.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:27.267+0000 7f3773fa4640 1 -- 172.21.15.105:0/3503767881 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f37580a3240 msgr2=0x7f37580a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:27.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:27.267+0000 7f3773fa4640 1 --2- 172.21.15.105:0/3503767881 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f37580a3240 0x7f37580a5700 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3768009bd0 tx=0x7f3768005f90 comp rx=0 tx=0).stop 2024-09-16T02:00:27.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:27.267+0000 7f3773fa4640 1 -- 172.21.15.105:0/3503767881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f376c1027a0 msgr2=0x7f376c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:27.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:27.267+0000 7f3773fa4640 1 --2- 172.21.15.105:0/3503767881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f376c1027a0 0x7f376c074a30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f375c005940 tx=0x7f375c0042f0 comp rx=0 tx=0).stop 2024-09-16T02:00:27.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:27.267+0000 7f3773fa4640 1 -- 172.21.15.105:0/3503767881 shutdown_connections 2024-09-16T02:00:27.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:27.267+0000 7f3773fa4640 1 --2- 172.21.15.105:0/3503767881 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f37580a3240 0x7f37580a5700 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:27.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:27.267+0000 7f3773fa4640 1 --2- 172.21.15.105:0/3503767881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f376c106f20 0x7f376c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:27.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:27.267+0000 7f3773fa4640 1 --2- 172.21.15.105:0/3503767881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f376c1027a0 0x7f376c074a30 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:27.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:27.267+0000 7f3773fa4640 1 --2- 172.21.15.105:0/3503767881 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f376c101df0 0x7f376c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:27.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:27.267+0000 7f3773fa4640 1 -- 172.21.15.105:0/3503767881 >> 172.21.15.105:0/3503767881 conn(0x7f376c0fd8f0 msgr2=0x7f376c0fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:27.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:27.268+0000 7f3773fa4640 1 -- 172.21.15.105:0/3503767881 shutdown_connections 2024-09-16T02:00:27.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:27.268+0000 7f3773fa4640 1 -- 172.21.15.105:0/3503767881 wait complete. 2024-09-16T02:00:27.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:27 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:27.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:27 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:27.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:27 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:27.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:27 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:27.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:27 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:27.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:27 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:27.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:27 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:27.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:27 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:00:27.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:27 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:27.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:27 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:00:27.545 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:27 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/3503767881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:00:27.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:27 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:27.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:27 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:27.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:27 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:27.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:27 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:27.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:27 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:27.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:27 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:27.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:27 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:27.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:27 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:00:27.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:27 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:27.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:27 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:00:27.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:27 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/3503767881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:00:27.636 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:27 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:27.636 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:27 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:27.637 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:27 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:27.637 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:27 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:27.637 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:27 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:27.637 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:27 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:27.637 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:27 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:27.637 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:27 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:00:27.637 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:27 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:27.637 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:27 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:00:27.637 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:27 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/3503767881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:00:27.979 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-16T02:00:28.473 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:28 smithi105 ceph-mon[31197]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:28.592 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:28 smithi190 ceph-mon[34762]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:28.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:28 smithi137 ceph-mon[34099]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:28.743 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:00:29.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.240+0000 7fd7366b2640 1 -- 172.21.15.105:0/4196605120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd730106f20 msgr2=0x7fd73010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:29.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.240+0000 7fd7366b2640 1 --2- 172.21.15.105:0/4196605120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd730106f20 0x7fd73010eaa0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd728009ca0 tx=0x7fd72802a8c0 comp rx=0 tx=0).stop 2024-09-16T02:00:29.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.240+0000 7fd7366b2640 1 -- 172.21.15.105:0/4196605120 shutdown_connections 2024-09-16T02:00:29.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.240+0000 7fd7366b2640 1 --2- 172.21.15.105:0/4196605120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd730106f20 0x7fd73010eaa0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:29.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.240+0000 7fd7366b2640 1 --2- 172.21.15.105:0/4196605120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7301027a0 0x7fd7301067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:29.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.240+0000 7fd7366b2640 1 --2- 172.21.15.105:0/4196605120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd730101df0 0x7fd7301021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:29.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.240+0000 7fd7366b2640 1 -- 172.21.15.105:0/4196605120 >> 172.21.15.105:0/4196605120 conn(0x7fd7300fd8b0 msgr2=0x7fd7300ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:29.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.240+0000 7fd7366b2640 1 -- 172.21.15.105:0/4196605120 shutdown_connections 2024-09-16T02:00:29.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.240+0000 7fd7366b2640 1 -- 172.21.15.105:0/4196605120 wait complete. 2024-09-16T02:00:29.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.241+0000 7fd7366b2640 1 Processor -- start 2024-09-16T02:00:29.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.241+0000 7fd7366b2640 1 -- start start 2024-09-16T02:00:29.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.241+0000 7fd7366b2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd730101df0 0x7fd7300744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:29.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.241+0000 7fd7366b2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd7301027a0 0x7fd730074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:29.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.241+0000 7fd7366b2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd730106f20 0x7fd730074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:29.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.241+0000 7fd7366b2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd730111ed0 con 0x7fd7301027a0 2024-09-16T02:00:29.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.241+0000 7fd7366b2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd730111d50 con 0x7fd730101df0 2024-09-16T02:00:29.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.241+0000 7fd7366b2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd730112050 con 0x7fd730106f20 2024-09-16T02:00:29.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.241+0000 7fd72f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd7301027a0 0x7fd730074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:29.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.241+0000 7fd72f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd7301027a0 0x7fd730074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54824/0 (socket says 172.21.15.105:54824) 2024-09-16T02:00:29.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.241+0000 7fd72f7fe640 1 -- 172.21.15.105:0/1574622902 learned_addr learned my addr 172.21.15.105:0/1574622902 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:00:29.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.241+0000 7fd72ffff640 1 --2- 172.21.15.105:0/1574622902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd730101df0 0x7fd7300744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:29.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.241+0000 7fd72f7fe640 1 -- 172.21.15.105:0/1574622902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd730106f20 msgr2=0x7fd730074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:29.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.241+0000 7fd72f7fe640 1 --2- 172.21.15.105:0/1574622902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd730106f20 0x7fd730074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:29.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.241+0000 7fd72f7fe640 1 -- 172.21.15.105:0/1574622902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd730101df0 msgr2=0x7fd7300744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:29.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.241+0000 7fd72f7fe640 1 --2- 172.21.15.105:0/1574622902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd730101df0 0x7fd7300744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:29.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.241+0000 7fd72f7fe640 1 -- 172.21.15.105:0/1574622902 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7301a6860 con 0x7fd7301027a0 2024-09-16T02:00:29.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.242+0000 7fd72f7fe640 1 --2- 172.21.15.105:0/1574622902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd7301027a0 0x7fd730074a30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd72400b4a0 tx=0x7fd72400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:29.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.242+0000 7fd72d7fa640 1 -- 172.21.15.105:0/1574622902 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd724014070 con 0x7fd7301027a0 2024-09-16T02:00:29.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.242+0000 7fd72d7fa640 1 -- 172.21.15.105:0/1574622902 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd724010ac0 con 0x7fd7301027a0 2024-09-16T02:00:29.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.242+0000 7fd7366b2640 1 -- 172.21.15.105:0/1574622902 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7301a6b50 con 0x7fd7301027a0 2024-09-16T02:00:29.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.242+0000 7fd72d7fa640 1 -- 172.21.15.105:0/1574622902 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd724017c30 con 0x7fd7301027a0 2024-09-16T02:00:29.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.242+0000 7fd7366b2640 1 -- 172.21.15.105:0/1574622902 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7301a70e0 con 0x7fd7301027a0 2024-09-16T02:00:29.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.243+0000 7fd7366b2640 1 -- 172.21.15.105:0/1574622902 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7301021d0 con 0x7fd7301027a0 2024-09-16T02:00:29.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.244+0000 7fd72d7fa640 1 -- 172.21.15.105:0/1574622902 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fd724017070 con 0x7fd7301027a0 2024-09-16T02:00:29.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.245+0000 7fd72d7fa640 1 --2- 172.21.15.105:0/1574622902 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fd7180a32c0 0x7fd7180a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:29.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.245+0000 7fd72d7fa640 1 -- 172.21.15.105:0/1574622902 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fd7240ceb50 con 0x7fd7301027a0 2024-09-16T02:00:29.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.245+0000 7fd72ffff640 1 --2- 172.21.15.105:0/1574622902 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fd7180a32c0 0x7fd7180a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:29.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.245+0000 7fd72ffff640 1 --2- 172.21.15.105:0/1574622902 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fd7180a32c0 0x7fd7180a5780 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd720009890 tx=0x7fd720008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:29.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.249+0000 7fd72d7fa640 1 -- 172.21.15.105:0/1574622902 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd72409bd10 con 0x7fd7301027a0 2024-09-16T02:00:29.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.427+0000 7fd7366b2640 1 -- 172.21.15.105:0/1574622902 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fd730109b70 con 0x7fd7301027a0 2024-09-16T02:00:29.428 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:29 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:29.429 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:29 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:29.429 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:29 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:29.429 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:29 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:29.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.427+0000 7fd72d7fa640 1 -- 172.21.15.105:0/1574622902 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v74) ==== 84+0+65 (secure 0 0 0) 0x7fd72409b610 con 0x7fd7301027a0 2024-09-16T02:00:29.429 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:00:29.429 INFO:teuthology.orchestra.run.smithi105.stdout:[ 2024-09-16T02:00:29.430 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:00:29.430 INFO:teuthology.orchestra.run.smithi105.stdout: "poolnum": 1, 2024-09-16T02:00:29.430 INFO:teuthology.orchestra.run.smithi105.stdout: "poolname": ".mgr" 2024-09-16T02:00:29.430 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:00:29.430 INFO:teuthology.orchestra.run.smithi105.stdout:] 2024-09-16T02:00:29.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.430+0000 7fd7366b2640 1 -- 172.21.15.105:0/1574622902 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fd7180a32c0 msgr2=0x7fd7180a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:29.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.430+0000 7fd7366b2640 1 --2- 172.21.15.105:0/1574622902 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fd7180a32c0 0x7fd7180a5780 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd720009890 tx=0x7fd720008040 comp rx=0 tx=0).stop 2024-09-16T02:00:29.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.430+0000 7fd7366b2640 1 -- 172.21.15.105:0/1574622902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd7301027a0 msgr2=0x7fd730074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:29.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.430+0000 7fd7366b2640 1 --2- 172.21.15.105:0/1574622902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd7301027a0 0x7fd730074a30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd72400b4a0 tx=0x7fd72400b960 comp rx=0 tx=0).stop 2024-09-16T02:00:29.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.431+0000 7fd7366b2640 1 -- 172.21.15.105:0/1574622902 shutdown_connections 2024-09-16T02:00:29.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.431+0000 7fd7366b2640 1 --2- 172.21.15.105:0/1574622902 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fd7180a32c0 0x7fd7180a5780 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:29.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.431+0000 7fd7366b2640 1 --2- 172.21.15.105:0/1574622902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd730106f20 0x7fd730074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:29.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.431+0000 7fd7366b2640 1 --2- 172.21.15.105:0/1574622902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd7301027a0 0x7fd730074a30 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:29.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.431+0000 7fd7366b2640 1 --2- 172.21.15.105:0/1574622902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd730101df0 0x7fd7300744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:29.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.431+0000 7fd7366b2640 1 -- 172.21.15.105:0/1574622902 >> 172.21.15.105:0/1574622902 conn(0x7fd7300fd8b0 msgr2=0x7fd7300fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:29.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.431+0000 7fd7366b2640 1 -- 172.21.15.105:0/1574622902 shutdown_connections 2024-09-16T02:00:29.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:29.431+0000 7fd7366b2640 1 -- 172.21.15.105:0/1574622902 wait complete. 2024-09-16T02:00:29.586 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:29 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:29.586 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:29 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:29.586 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:29 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:29.586 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:29 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:29.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:29 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:29.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:29 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:29.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:29 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:29.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:29 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:30.098 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T02:00:30.099 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T02:00:30.362 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:00:30.660 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:30 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:30.661 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:30 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/1574622902' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-16T02:00:30.661 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:30 smithi105 ceph-mon[31197]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:30.661 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:30 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:30.661 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:30 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:30.661 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:30 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:30.661 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:30 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:00:30.661 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:30 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:30.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:30 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:30.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:30 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/1574622902' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-16T02:00:30.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:30 smithi137 ceph-mon[34099]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:30.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:30 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:30.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:30 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:30.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:30 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:30.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:30 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:00:30.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:30 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:30.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:30 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:30.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:30 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/1574622902' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-16T02:00:30.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:30 smithi190 ceph-mon[34762]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:30.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:30 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:30.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:30 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:30.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:30 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:30.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:30 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:00:30.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:30 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:00:30.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.878+0000 7fa2f0b0a640 1 -- 172.21.15.105:0/3904508396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2ec102d30 msgr2=0x7fa2ec103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:30.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.878+0000 7fa2f0b0a640 1 --2- 172.21.15.105:0/3904508396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2ec102d30 0x7fa2ec103110 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009cd0 tx=0x7fa2dc02a910 comp rx=0 tx=0).stop 2024-09-16T02:00:30.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.878+0000 7fa2f0b0a640 1 -- 172.21.15.105:0/3904508396 shutdown_connections 2024-09-16T02:00:30.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.878+0000 7fa2f0b0a640 1 --2- 172.21.15.105:0/3904508396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2ec1036e0 0x7fa2ec107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:30.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.878+0000 7fa2f0b0a640 1 --2- 172.21.15.105:0/3904508396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2ec1082a0 0x7fa2ec10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:30.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.878+0000 7fa2f0b0a640 1 --2- 172.21.15.105:0/3904508396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2ec102d30 0x7fa2ec103110 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:30.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.878+0000 7fa2f0b0a640 1 -- 172.21.15.105:0/3904508396 >> 172.21.15.105:0/3904508396 conn(0x7fa2ec0fc7d0 msgr2=0x7fa2ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:30.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.878+0000 7fa2f0b0a640 1 -- 172.21.15.105:0/3904508396 shutdown_connections 2024-09-16T02:00:30.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.878+0000 7fa2f0b0a640 1 -- 172.21.15.105:0/3904508396 wait complete. 2024-09-16T02:00:30.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.878+0000 7fa2f0b0a640 1 Processor -- start 2024-09-16T02:00:30.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.878+0000 7fa2f0b0a640 1 -- start start 2024-09-16T02:00:30.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.879+0000 7fa2f0b0a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2ec102d30 0x7fa2ec1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:30.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.879+0000 7fa2f0b0a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2ec1036e0 0x7fa2ec198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:30.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.879+0000 7fa2f0b0a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2ec1082a0 0x7fa2ec19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:30.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.879+0000 7fa2f0b0a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ec111f90 con 0x7fa2ec102d30 2024-09-16T02:00:30.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.879+0000 7fa2f0b0a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ec111e10 con 0x7fa2ec1036e0 2024-09-16T02:00:30.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.879+0000 7fa2f0b0a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ec112110 con 0x7fa2ec1082a0 2024-09-16T02:00:30.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.879+0000 7fa2ea59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2ec102d30 0x7fa2ec1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:30.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.879+0000 7fa2ea59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2ec102d30 0x7fa2ec1984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54834/0 (socket says 172.21.15.105:54834) 2024-09-16T02:00:30.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.879+0000 7fa2ea59c640 1 -- 172.21.15.105:0/1133763592 learned_addr learned my addr 172.21.15.105:0/1133763592 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:00:30.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.879+0000 7fa2e9d9b640 1 --2- 172.21.15.105:0/1133763592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2ec1036e0 0x7fa2ec198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:30.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.879+0000 7fa2ead9d640 1 --2- 172.21.15.105:0/1133763592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2ec1082a0 0x7fa2ec19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:30.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.879+0000 7fa2ea59c640 1 -- 172.21.15.105:0/1133763592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2ec1082a0 msgr2=0x7fa2ec19cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:30.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.879+0000 7fa2ea59c640 1 --2- 172.21.15.105:0/1133763592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2ec1082a0 0x7fa2ec19cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:30.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.879+0000 7fa2ea59c640 1 -- 172.21.15.105:0/1133763592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2ec1036e0 msgr2=0x7fa2ec198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:30.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.879+0000 7fa2ea59c640 1 --2- 172.21.15.105:0/1133763592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2ec1036e0 0x7fa2ec198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:30.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.879+0000 7fa2ea59c640 1 -- 172.21.15.105:0/1133763592 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2ec19d470 con 0x7fa2ec102d30 2024-09-16T02:00:30.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.879+0000 7fa2ead9d640 1 --2- 172.21.15.105:0/1133763592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2ec1082a0 0x7fa2ec19cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:00:30.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.879+0000 7fa2ea59c640 1 --2- 172.21.15.105:0/1133763592 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2ec102d30 0x7fa2ec1984c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009df0 tx=0x7fa2dc003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:30.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.879+0000 7fa2db7fe640 1 -- 172.21.15.105:0/1133763592 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc004040 con 0x7fa2ec102d30 2024-09-16T02:00:30.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.879+0000 7fa2db7fe640 1 -- 172.21.15.105:0/1133763592 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2dc004690 con 0x7fa2ec102d30 2024-09-16T02:00:30.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.879+0000 7fa2db7fe640 1 -- 172.21.15.105:0/1133763592 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc030e90 con 0x7fa2ec102d30 2024-09-16T02:00:30.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.879+0000 7fa2f0b0a640 1 -- 172.21.15.105:0/1133763592 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2ec19d700 con 0x7fa2ec102d30 2024-09-16T02:00:30.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.880+0000 7fa2f0b0a640 1 -- 172.21.15.105:0/1133763592 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2ec1a5040 con 0x7fa2ec102d30 2024-09-16T02:00:30.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.880+0000 7fa2f0b0a640 1 -- 172.21.15.105:0/1133763592 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2ec06a850 con 0x7fa2ec102d30 2024-09-16T02:00:30.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.882+0000 7fa2db7fe640 1 -- 172.21.15.105:0/1133763592 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fa2dc030660 con 0x7fa2ec102d30 2024-09-16T02:00:30.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.883+0000 7fa2db7fe640 1 --2- 172.21.15.105:0/1133763592 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa2d40a3240 0x7fa2d40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:30.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.883+0000 7fa2e9d9b640 1 --2- 172.21.15.105:0/1133763592 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa2d40a3240 0x7fa2d40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:30.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.883+0000 7fa2db7fe640 1 -- 172.21.15.105:0/1133763592 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fa2dc0f0c80 con 0x7fa2ec102d30 2024-09-16T02:00:30.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.884+0000 7fa2e9d9b640 1 --2- 172.21.15.105:0/1133763592 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa2d40a3240 0x7fa2d40a5700 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec0fee70 tx=0x7fa2e0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:30.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:30.886+0000 7fa2db7fe640 1 -- 172.21.15.105:0/1133763592 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa2dc0bde40 con 0x7fa2ec102d30 2024-09-16T02:00:31.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:31.051+0000 7fa2f0b0a640 1 -- 172.21.15.105:0/1133763592 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fa2ec10a2b0 con 0x7fa2ec102d30 2024-09-16T02:00:31.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:31.418+0000 7fa2db7fe640 1 -- 172.21.15.105:0/1133763592 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v75) ==== 138+0+0 (secure 0 0 0) 0x7fa2dc0bd740 con 0x7fa2ec102d30 2024-09-16T02:00:31.421 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_metadata' created 2024-09-16T02:00:31.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:31.420+0000 7fa2f0b0a640 1 -- 172.21.15.105:0/1133763592 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa2d40a3240 msgr2=0x7fa2d40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:31.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:31.420+0000 7fa2f0b0a640 1 --2- 172.21.15.105:0/1133763592 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa2d40a3240 0x7fa2d40a5700 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec0fee70 tx=0x7fa2e0008040 comp rx=0 tx=0).stop 2024-09-16T02:00:31.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:31.420+0000 7fa2f0b0a640 1 -- 172.21.15.105:0/1133763592 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2ec102d30 msgr2=0x7fa2ec1984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:31.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:31.420+0000 7fa2f0b0a640 1 --2- 172.21.15.105:0/1133763592 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2ec102d30 0x7fa2ec1984c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009df0 tx=0x7fa2dc003ba0 comp rx=0 tx=0).stop 2024-09-16T02:00:31.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:31.420+0000 7fa2f0b0a640 1 -- 172.21.15.105:0/1133763592 shutdown_connections 2024-09-16T02:00:31.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:31.420+0000 7fa2f0b0a640 1 --2- 172.21.15.105:0/1133763592 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa2d40a3240 0x7fa2d40a5700 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:31.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:31.420+0000 7fa2f0b0a640 1 --2- 172.21.15.105:0/1133763592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2ec1082a0 0x7fa2ec19cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:31.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:31.420+0000 7fa2f0b0a640 1 --2- 172.21.15.105:0/1133763592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2ec1036e0 0x7fa2ec198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:31.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:31.420+0000 7fa2f0b0a640 1 --2- 172.21.15.105:0/1133763592 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2ec102d30 0x7fa2ec1984c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:31.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:31.420+0000 7fa2f0b0a640 1 -- 172.21.15.105:0/1133763592 >> 172.21.15.105:0/1133763592 conn(0x7fa2ec0fc7d0 msgr2=0x7fa2ec105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:31.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:31.420+0000 7fa2f0b0a640 1 -- 172.21.15.105:0/1133763592 shutdown_connections 2024-09-16T02:00:31.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:31.420+0000 7fa2f0b0a640 1 -- 172.21.15.105:0/1133763592 wait complete. 2024-09-16T02:00:31.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:31 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/1133763592' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-16T02:00:31.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:31 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/1133763592' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-16T02:00:31.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:31 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/1133763592' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-16T02:00:32.161 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T02:00:32.401 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:00:32.684 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:32 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/1133763592' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-16T02:00:32.684 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:32 smithi105 ceph-mon[31197]: osdmap e75: 12 total, 12 up, 12 in 2024-09-16T02:00:32.684 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:32 smithi105 ceph-mon[31197]: pgmap v271: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:32.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:32 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/1133763592' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-16T02:00:32.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:32 smithi190 ceph-mon[34762]: osdmap e75: 12 total, 12 up, 12 in 2024-09-16T02:00:32.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:32 smithi190 ceph-mon[34762]: pgmap v271: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:32.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.897+0000 7f53c391e640 1 -- 172.21.15.105:0/173369079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53bc105a90 msgr2=0x7f53bc105e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:32.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.897+0000 7f53c391e640 1 --2- 172.21.15.105:0/173369079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53bc105a90 0x7f53bc105e70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f53b8009900 tx=0x7f53b802a410 comp rx=0 tx=0).stop 2024-09-16T02:00:32.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.897+0000 7f53c391e640 1 -- 172.21.15.105:0/173369079 shutdown_connections 2024-09-16T02:00:32.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.897+0000 7f53c391e640 1 --2- 172.21.15.105:0/173369079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53bc101bc0 0x7f53bc10d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:32.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.897+0000 7f53c391e640 1 --2- 172.21.15.105:0/173369079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53bc101240 0x7f53bc101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:32.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.897+0000 7f53c391e640 1 --2- 172.21.15.105:0/173369079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53bc105a90 0x7f53bc105e70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:32.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.897+0000 7f53c391e640 1 -- 172.21.15.105:0/173369079 >> 172.21.15.105:0/173369079 conn(0x7f53bc077210 msgr2=0x7f53bc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:32.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.897+0000 7f53c391e640 1 -- 172.21.15.105:0/173369079 shutdown_connections 2024-09-16T02:00:32.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.897+0000 7f53c391e640 1 -- 172.21.15.105:0/173369079 wait complete. 2024-09-16T02:00:32.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.898+0000 7f53c391e640 1 Processor -- start 2024-09-16T02:00:32.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.898+0000 7f53c391e640 1 -- start start 2024-09-16T02:00:32.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.898+0000 7f53c391e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53bc101240 0x7f53bc0fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:32.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.898+0000 7f53c391e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53bc101bc0 0x7f53bc0fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:32.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.898+0000 7f53c391e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53bc105a90 0x7f53bc0fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:32.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.898+0000 7f53c391e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f53bc110ee0 con 0x7f53bc105a90 2024-09-16T02:00:32.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.898+0000 7f53c391e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f53bc110d60 con 0x7f53bc101240 2024-09-16T02:00:32.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.898+0000 7f53c391e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f53bc111060 con 0x7f53bc101bc0 2024-09-16T02:00:32.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.898+0000 7f53c1ebb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53bc105a90 0x7f53bc0fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:32.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.898+0000 7f53c1ebb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53bc105a90 0x7f53bc0fd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54854/0 (socket says 172.21.15.105:54854) 2024-09-16T02:00:32.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.898+0000 7f53c1ebb640 1 -- 172.21.15.105:0/2589838761 learned_addr learned my addr 172.21.15.105:0/2589838761 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:00:32.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.898+0000 7f53c16ba640 1 --2- 172.21.15.105:0/2589838761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53bc101240 0x7f53bc0fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:32.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.898+0000 7f53c0eb9640 1 --2- 172.21.15.105:0/2589838761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53bc101bc0 0x7f53bc0fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:32.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.899+0000 7f53c1ebb640 1 -- 172.21.15.105:0/2589838761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53bc101bc0 msgr2=0x7f53bc0fcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:32.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.899+0000 7f53c1ebb640 1 --2- 172.21.15.105:0/2589838761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53bc101bc0 0x7f53bc0fcbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:32.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.899+0000 7f53c1ebb640 1 -- 172.21.15.105:0/2589838761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53bc101240 msgr2=0x7f53bc0fc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:32.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.899+0000 7f53c1ebb640 1 --2- 172.21.15.105:0/2589838761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53bc101240 0x7f53bc0fc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:32.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.899+0000 7f53c1ebb640 1 -- 172.21.15.105:0/2589838761 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53bc1a69c0 con 0x7f53bc105a90 2024-09-16T02:00:32.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.899+0000 7f53c16ba640 1 --2- 172.21.15.105:0/2589838761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53bc101240 0x7f53bc0fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:00:32.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.899+0000 7f53c1ebb640 1 --2- 172.21.15.105:0/2589838761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53bc105a90 0x7f53bc0fd110 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f53b400b970 tx=0x7f53b400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:32.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.899+0000 7f53b27fc640 1 -- 172.21.15.105:0/2589838761 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b4019070 con 0x7f53bc105a90 2024-09-16T02:00:32.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.899+0000 7f53b27fc640 1 -- 172.21.15.105:0/2589838761 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53b4014440 con 0x7f53bc105a90 2024-09-16T02:00:32.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.899+0000 7f53c391e640 1 -- 172.21.15.105:0/2589838761 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53bc1a6cb0 con 0x7f53bc105a90 2024-09-16T02:00:32.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.899+0000 7f53b27fc640 1 -- 172.21.15.105:0/2589838761 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b40075d0 con 0x7f53bc105a90 2024-09-16T02:00:32.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.899+0000 7f53c391e640 1 -- 172.21.15.105:0/2589838761 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53bc1a7240 con 0x7f53bc105a90 2024-09-16T02:00:32.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.900+0000 7f53c391e640 1 -- 172.21.15.105:0/2589838761 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53bc101680 con 0x7f53bc105a90 2024-09-16T02:00:32.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.901+0000 7f53b27fc640 1 -- 172.21.15.105:0/2589838761 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f53b401c070 con 0x7f53bc105a90 2024-09-16T02:00:32.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.902+0000 7f53b27fc640 1 --2- 172.21.15.105:0/2589838761 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f53a80a32c0 0x7f53a80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:32.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.902+0000 7f53c16ba640 1 --2- 172.21.15.105:0/2589838761 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f53a80a32c0 0x7f53a80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:32.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.902+0000 7f53b27fc640 1 -- 172.21.15.105:0/2589838761 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7497+0+0 (secure 0 0 0) 0x7f53b40cebb0 con 0x7f53bc105a90 2024-09-16T02:00:32.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.902+0000 7f53c16ba640 1 --2- 172.21.15.105:0/2589838761 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f53a80a32c0 0x7f53a80a5780 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f53b8005ab0 tx=0x7f53b8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:32.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:32.906+0000 7f53b27fc640 1 -- 172.21.15.105:0/2589838761 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f53b409bbe0 con 0x7f53bc105a90 2024-09-16T02:00:32.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:32 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/1133763592' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-16T02:00:32.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:32 smithi137 ceph-mon[34099]: osdmap e75: 12 total, 12 up, 12 in 2024-09-16T02:00:32.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:32 smithi137 ceph-mon[34099]: pgmap v271: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:33.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:33.076+0000 7f53c391e640 1 -- 172.21.15.105:0/2589838761 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f53bc109fc0 con 0x7f53bc105a90 2024-09-16T02:00:33.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:33.432+0000 7f53b27fc640 1 -- 172.21.15.105:0/2589838761 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v77) ==== 170+0+0 (secure 0 0 0) 0x7f53b409b4e0 con 0x7f53bc105a90 2024-09-16T02:00:33.434 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data' created 2024-09-16T02:00:33.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:33.434+0000 7f53c391e640 1 -- 172.21.15.105:0/2589838761 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f53a80a32c0 msgr2=0x7f53a80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:33.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:33.434+0000 7f53c391e640 1 --2- 172.21.15.105:0/2589838761 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f53a80a32c0 0x7f53a80a5780 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f53b8005ab0 tx=0x7f53b8005a20 comp rx=0 tx=0).stop 2024-09-16T02:00:33.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:33.434+0000 7f53c391e640 1 -- 172.21.15.105:0/2589838761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53bc105a90 msgr2=0x7f53bc0fd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:33.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:33.434+0000 7f53c391e640 1 --2- 172.21.15.105:0/2589838761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53bc105a90 0x7f53bc0fd110 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f53b400b970 tx=0x7f53b400be30 comp rx=0 tx=0).stop 2024-09-16T02:00:33.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:33.435+0000 7f53c391e640 1 -- 172.21.15.105:0/2589838761 shutdown_connections 2024-09-16T02:00:33.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:33.435+0000 7f53c391e640 1 --2- 172.21.15.105:0/2589838761 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f53a80a32c0 0x7f53a80a5780 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:33.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:33.435+0000 7f53c391e640 1 --2- 172.21.15.105:0/2589838761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53bc101bc0 0x7f53bc0fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:33.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:33.435+0000 7f53c391e640 1 --2- 172.21.15.105:0/2589838761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53bc105a90 0x7f53bc0fd110 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:33.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:33.435+0000 7f53c391e640 1 --2- 172.21.15.105:0/2589838761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53bc101240 0x7f53bc0fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:33.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:33.435+0000 7f53c391e640 1 -- 172.21.15.105:0/2589838761 >> 172.21.15.105:0/2589838761 conn(0x7f53bc077210 msgr2=0x7f53bc1004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:33.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:33.435+0000 7f53c391e640 1 -- 172.21.15.105:0/2589838761 shutdown_connections 2024-09-16T02:00:33.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:33.435+0000 7f53c391e640 1 -- 172.21.15.105:0/2589838761 wait complete. 2024-09-16T02:00:33.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:33 smithi105 ceph-mon[31197]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T02:00:33.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:33 smithi105 ceph-mon[31197]: osdmap e76: 12 total, 12 up, 12 in 2024-09-16T02:00:33.796 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:33 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2589838761' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-16T02:00:33.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:33 smithi190 ceph-mon[34762]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T02:00:33.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:33 smithi190 ceph-mon[34762]: osdmap e76: 12 total, 12 up, 12 in 2024-09-16T02:00:33.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:33 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/2589838761' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-16T02:00:33.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:33 smithi137 ceph-mon[34099]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T02:00:33.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:33 smithi137 ceph-mon[34099]: osdmap e76: 12 total, 12 up, 12 in 2024-09-16T02:00:33.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:33 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/2589838761' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-16T02:00:34.162 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T02:00:34.404 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:00:34.445 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:34 smithi105 ceph-mon[31197]: pgmap v273: 33 pgs: 29 unknown, 4 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:34.445 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:34 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2589838761' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-16T02:00:34.445 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:34 smithi105 ceph-mon[31197]: osdmap e77: 12 total, 12 up, 12 in 2024-09-16T02:00:34.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:34 smithi190 ceph-mon[34762]: pgmap v273: 33 pgs: 29 unknown, 4 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:34.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:34 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/2589838761' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-16T02:00:34.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:34 smithi190 ceph-mon[34762]: osdmap e77: 12 total, 12 up, 12 in 2024-09-16T02:00:34.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:34 smithi137 ceph-mon[34099]: pgmap v273: 33 pgs: 29 unknown, 4 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:34.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:34 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/2589838761' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-16T02:00:34.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:34 smithi137 ceph-mon[34099]: osdmap e77: 12 total, 12 up, 12 in 2024-09-16T02:00:34.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:34.903+0000 7f5b59834640 1 -- 172.21.15.105:0/2870488899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b54102790 msgr2=0x7f5b541067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:34.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:34.903+0000 7f5b59834640 1 --2- 172.21.15.105:0/2870488899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b54102790 0x7f5b541067e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c009900 tx=0x7f5b3c02a410 comp rx=0 tx=0).stop 2024-09-16T02:00:34.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:34.903+0000 7f5b59834640 1 -- 172.21.15.105:0/2870488899 shutdown_connections 2024-09-16T02:00:34.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:34.903+0000 7f5b59834640 1 --2- 172.21.15.105:0/2870488899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b54106f10 0x7f5b5410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:34.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:34.903+0000 7f5b59834640 1 --2- 172.21.15.105:0/2870488899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b54102790 0x7f5b541067e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:34.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:34.903+0000 7f5b59834640 1 --2- 172.21.15.105:0/2870488899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b54101de0 0x7f5b541021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:34.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:34.903+0000 7f5b59834640 1 -- 172.21.15.105:0/2870488899 >> 172.21.15.105:0/2870488899 conn(0x7f5b540fd8c0 msgr2=0x7f5b540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:34.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:34.903+0000 7f5b59834640 1 -- 172.21.15.105:0/2870488899 shutdown_connections 2024-09-16T02:00:34.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:34.903+0000 7f5b59834640 1 -- 172.21.15.105:0/2870488899 wait complete. 2024-09-16T02:00:34.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:34.903+0000 7f5b59834640 1 Processor -- start 2024-09-16T02:00:34.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:34.904+0000 7f5b59834640 1 -- start start 2024-09-16T02:00:34.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:34.904+0000 7f5b59834640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b54101de0 0x7f5b540744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:34.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:34.904+0000 7f5b59834640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b54102790 0x7f5b54074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:34.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:34.904+0000 7f5b59834640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b54106f10 0x7f5b54074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:34.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:34.904+0000 7f5b59834640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5b54111e20 con 0x7f5b54102790 2024-09-16T02:00:34.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:34.904+0000 7f5b59834640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5b54111ca0 con 0x7f5b54106f10 2024-09-16T02:00:34.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:34.904+0000 7f5b59834640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5b54111fa0 con 0x7f5b54101de0 2024-09-16T02:00:34.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:34.904+0000 7f5b4a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b54102790 0x7f5b54074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:34.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:34.904+0000 7f5b4a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b54102790 0x7f5b54074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54862/0 (socket says 172.21.15.105:54862) 2024-09-16T02:00:34.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:34.904+0000 7f5b4a7fc640 1 -- 172.21.15.105:0/2880459800 learned_addr learned my addr 172.21.15.105:0/2880459800 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:00:34.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:34.904+0000 7f5b52ffd640 1 --2- 172.21.15.105:0/2880459800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b54101de0 0x7f5b540744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:34.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:34.904+0000 7f5b4a7fc640 1 -- 172.21.15.105:0/2880459800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b54101de0 msgr2=0x7f5b540744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:34.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:34.904+0000 7f5b4a7fc640 1 --2- 172.21.15.105:0/2880459800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b54101de0 0x7f5b540744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:34.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:34.904+0000 7f5b4a7fc640 1 -- 172.21.15.105:0/2880459800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b54106f10 msgr2=0x7f5b54074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:34.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:34.904+0000 7f5b4a7fc640 1 --2- 172.21.15.105:0/2880459800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b54106f10 0x7f5b54074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:34.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:34.904+0000 7f5b4a7fc640 1 -- 172.21.15.105:0/2880459800 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b541a6820 con 0x7f5b54102790 2024-09-16T02:00:34.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:34.904+0000 7f5b4a7fc640 1 --2- 172.21.15.105:0/2880459800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b54102790 0x7f5b54074a30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c005940 tx=0x7f5b3c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:34.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:34.905+0000 7f5b58832640 1 -- 172.21.15.105:0/2880459800 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c03f030 con 0x7f5b54102790 2024-09-16T02:00:34.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:34.905+0000 7f5b58832640 1 -- 172.21.15.105:0/2880459800 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b3c003aa0 con 0x7f5b54102790 2024-09-16T02:00:34.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:34.905+0000 7f5b58832640 1 -- 172.21.15.105:0/2880459800 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c035070 con 0x7f5b54102790 2024-09-16T02:00:34.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:34.905+0000 7f5b59834640 1 -- 172.21.15.105:0/2880459800 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b541a6ab0 con 0x7f5b54102790 2024-09-16T02:00:34.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:34.905+0000 7f5b59834640 1 -- 172.21.15.105:0/2880459800 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b541a6f40 con 0x7f5b54102790 2024-09-16T02:00:34.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:34.906+0000 7f5b59834640 1 -- 172.21.15.105:0/2880459800 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b541021c0 con 0x7f5b54102790 2024-09-16T02:00:34.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:34.907+0000 7f5b58832640 1 -- 172.21.15.105:0/2880459800 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f5b3c042030 con 0x7f5b54102790 2024-09-16T02:00:34.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:34.907+0000 7f5b58832640 1 --2- 172.21.15.105:0/2880459800 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5b400a32c0 0x7f5b400a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:34.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:34.907+0000 7f5b52ffd640 1 --2- 172.21.15.105:0/2880459800 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5b400a32c0 0x7f5b400a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:34.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:34.908+0000 7f5b58832640 1 -- 172.21.15.105:0/2880459800 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7879+0+0 (secure 0 0 0) 0x7f5b3c0efed0 con 0x7f5b54102790 2024-09-16T02:00:34.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:34.908+0000 7f5b52ffd640 1 --2- 172.21.15.105:0/2880459800 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5b400a32c0 0x7f5b400a5780 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5b440058c0 tx=0x7f5b4400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:34.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:34.912+0000 7f5b58832640 1 -- 172.21.15.105:0/2880459800 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5b3c0bcda0 con 0x7f5b54102790 2024-09-16T02:00:35.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:35.110+0000 7f5b59834640 1 -- 172.21.15.105:0/2880459800 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f5b54109b60 con 0x7f5b54102790 2024-09-16T02:00:35.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:35.447+0000 7f5b58832640 1 -- 172.21.15.105:0/2880459800 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f5b3c0bc6a0 con 0x7f5b54102790 2024-09-16T02:00:35.450 INFO:teuthology.orchestra.run.smithi105.stderr:new fs with metadata pool 2 and data pool 3 2024-09-16T02:00:35.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:35.450+0000 7f5b59834640 1 -- 172.21.15.105:0/2880459800 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5b400a32c0 msgr2=0x7f5b400a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:35.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:35.450+0000 7f5b59834640 1 --2- 172.21.15.105:0/2880459800 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5b400a32c0 0x7f5b400a5780 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5b440058c0 tx=0x7f5b4400a2d0 comp rx=0 tx=0).stop 2024-09-16T02:00:35.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:35.450+0000 7f5b59834640 1 -- 172.21.15.105:0/2880459800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b54102790 msgr2=0x7f5b54074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:35.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:35.450+0000 7f5b59834640 1 --2- 172.21.15.105:0/2880459800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b54102790 0x7f5b54074a30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c005940 tx=0x7f5b3c004270 comp rx=0 tx=0).stop 2024-09-16T02:00:35.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:35.450+0000 7f5b59834640 1 -- 172.21.15.105:0/2880459800 shutdown_connections 2024-09-16T02:00:35.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:35.450+0000 7f5b59834640 1 --2- 172.21.15.105:0/2880459800 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5b400a32c0 0x7f5b400a5780 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:35.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:35.450+0000 7f5b59834640 1 --2- 172.21.15.105:0/2880459800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b54106f10 0x7f5b54074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:35.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:35.450+0000 7f5b59834640 1 --2- 172.21.15.105:0/2880459800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b54102790 0x7f5b54074a30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:35.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:35.450+0000 7f5b59834640 1 --2- 172.21.15.105:0/2880459800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b54101de0 0x7f5b540744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:35.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:35.450+0000 7f5b59834640 1 -- 172.21.15.105:0/2880459800 >> 172.21.15.105:0/2880459800 conn(0x7f5b540fd8c0 msgr2=0x7f5b540fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:35.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:35.450+0000 7f5b59834640 1 -- 172.21.15.105:0/2880459800 shutdown_connections 2024-09-16T02:00:35.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:35.450+0000 7f5b59834640 1 -- 172.21.15.105:0/2880459800 wait complete. 2024-09-16T02:00:35.737 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:35 smithi105 ceph-mon[31197]: osdmap e78: 12 total, 12 up, 12 in 2024-09-16T02:00:35.737 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:35 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2880459800' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-16T02:00:35.737 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:35 smithi105 ceph-mon[31197]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-16T02:00:35.845 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:35 smithi190 ceph-mon[34762]: osdmap e78: 12 total, 12 up, 12 in 2024-09-16T02:00:35.845 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:35 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/2880459800' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-16T02:00:35.845 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:35 smithi190 ceph-mon[34762]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-16T02:00:35.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:35 smithi137 ceph-mon[34099]: osdmap e78: 12 total, 12 up, 12 in 2024-09-16T02:00:35.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:35 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/2880459800' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-16T02:00:35.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:35 smithi137 ceph-mon[34099]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-16T02:00:36.190 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T02:00:36.438 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:00:36.712 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:36 smithi105 ceph-mon[31197]: pgmap v276: 97 pgs: 9 creating+peering, 35 unknown, 53 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:36.712 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:36 smithi105 ceph-mon[31197]: osdmap e79: 12 total, 12 up, 12 in 2024-09-16T02:00:36.712 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:36 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2880459800' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-16T02:00:36.712 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:36 smithi105 ceph-mon[31197]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-16T02:00:36.712 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:36 smithi105 ceph-mon[31197]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-16T02:00:36.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:36 smithi190 ceph-mon[34762]: pgmap v276: 97 pgs: 9 creating+peering, 35 unknown, 53 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:36.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:36 smithi190 ceph-mon[34762]: osdmap e79: 12 total, 12 up, 12 in 2024-09-16T02:00:36.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:36 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/2880459800' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-16T02:00:36.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:36 smithi190 ceph-mon[34762]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-16T02:00:36.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:36 smithi190 ceph-mon[34762]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-16T02:00:36.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:36 smithi137 ceph-mon[34099]: pgmap v276: 97 pgs: 9 creating+peering, 35 unknown, 53 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:36.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:36 smithi137 ceph-mon[34099]: osdmap e79: 12 total, 12 up, 12 in 2024-09-16T02:00:36.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:36 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/2880459800' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-16T02:00:36.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:36 smithi137 ceph-mon[34099]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-16T02:00:36.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:36 smithi137 ceph-mon[34099]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-16T02:00:36.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.943+0000 7f3aac538640 1 -- 172.21.15.105:0/58399637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3aa4106f70 msgr2=0x7f3aa410eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:36.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.943+0000 7f3aac538640 1 --2- 172.21.15.105:0/58399637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3aa4106f70 0x7f3aa410eab0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c009ca0 tx=0x7f3a9c02a8c0 comp rx=0 tx=0).stop 2024-09-16T02:00:36.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.943+0000 7f3aac538640 1 -- 172.21.15.105:0/58399637 shutdown_connections 2024-09-16T02:00:36.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.943+0000 7f3aac538640 1 --2- 172.21.15.105:0/58399637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3aa4100eb0 0x7f3aa4106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:36.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.943+0000 7f3aac538640 1 --2- 172.21.15.105:0/58399637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3aa4106f70 0x7f3aa410eab0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:36.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.943+0000 7f3aac538640 1 --2- 172.21.15.105:0/58399637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3aa4100500 0x7f3aa41008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:36.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.943+0000 7f3aac538640 1 -- 172.21.15.105:0/58399637 >> 172.21.15.105:0/58399637 conn(0x7f3aa40fc5c0 msgr2=0x7f3aa40fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:36.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.943+0000 7f3aac538640 1 -- 172.21.15.105:0/58399637 shutdown_connections 2024-09-16T02:00:36.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.943+0000 7f3aac538640 1 -- 172.21.15.105:0/58399637 wait complete. 2024-09-16T02:00:36.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.944+0000 7f3aac538640 1 Processor -- start 2024-09-16T02:00:36.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.944+0000 7f3aac538640 1 -- start start 2024-09-16T02:00:36.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.944+0000 7f3aac538640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3aa4100500 0x7f3aa4101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:36.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.944+0000 7f3aac538640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3aa4100eb0 0x7f3aa41019c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:36.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.944+0000 7f3aac538640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3aa4106f70 0x7f3aa4101f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:36.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.944+0000 7f3aac538640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa4111e40 con 0x7f3aa4106f70 2024-09-16T02:00:36.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.944+0000 7f3aac538640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa4111cc0 con 0x7f3aa4100eb0 2024-09-16T02:00:36.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.944+0000 7f3aac538640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa4111fc0 con 0x7f3aa4100500 2024-09-16T02:00:36.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.944+0000 7f3aaaad5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3aa4106f70 0x7f3aa4101f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:36.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.944+0000 7f3aaaad5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3aa4106f70 0x7f3aa4101f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54886/0 (socket says 172.21.15.105:54886) 2024-09-16T02:00:36.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.944+0000 7f3aaaad5640 1 -- 172.21.15.105:0/2142815133 learned_addr learned my addr 172.21.15.105:0/2142815133 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:00:36.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.944+0000 7f3aaa2d4640 1 --2- 172.21.15.105:0/2142815133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3aa4100500 0x7f3aa4101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:36.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.944+0000 7f3aa9ad3640 1 --2- 172.21.15.105:0/2142815133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3aa4100eb0 0x7f3aa41019c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:36.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.944+0000 7f3aaaad5640 1 -- 172.21.15.105:0/2142815133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3aa4100500 msgr2=0x7f3aa4101480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:36.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.944+0000 7f3aaaad5640 1 --2- 172.21.15.105:0/2142815133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3aa4100500 0x7f3aa4101480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:36.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.944+0000 7f3aaaad5640 1 -- 172.21.15.105:0/2142815133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3aa4100eb0 msgr2=0x7f3aa41019c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:36.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.944+0000 7f3aaaad5640 1 --2- 172.21.15.105:0/2142815133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3aa4100eb0 0x7f3aa41019c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:36.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.944+0000 7f3aaaad5640 1 -- 172.21.15.105:0/2142815133 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa41a68a0 con 0x7f3aa4106f70 2024-09-16T02:00:36.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.944+0000 7f3aaaad5640 1 --2- 172.21.15.105:0/2142815133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3aa4106f70 0x7f3aa4101f00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c009dc0 tx=0x7f3a9c032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:36.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.945+0000 7f3a977fe640 1 -- 172.21.15.105:0/2142815133 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c003cf0 con 0x7f3aa4106f70 2024-09-16T02:00:36.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.945+0000 7f3a977fe640 1 -- 172.21.15.105:0/2142815133 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a9c0044f0 con 0x7f3aa4106f70 2024-09-16T02:00:36.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.945+0000 7f3a977fe640 1 -- 172.21.15.105:0/2142815133 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c030bf0 con 0x7f3aa4106f70 2024-09-16T02:00:36.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.945+0000 7f3aac538640 1 -- 172.21.15.105:0/2142815133 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aa41a6b30 con 0x7f3aa4106f70 2024-09-16T02:00:36.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.945+0000 7f3aac538640 1 -- 172.21.15.105:0/2142815133 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aa41a70c0 con 0x7f3aa4106f70 2024-09-16T02:00:36.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.946+0000 7f3aac538640 1 -- 172.21.15.105:0/2142815133 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3aa41008e0 con 0x7f3aa4106f70 2024-09-16T02:00:36.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.947+0000 7f3a977fe640 1 -- 172.21.15.105:0/2142815133 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f3a9c004040 con 0x7f3aa4106f70 2024-09-16T02:00:36.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.948+0000 7f3a977fe640 1 --2- 172.21.15.105:0/2142815133 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f3a900a32c0 0x7f3a900a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:36.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.948+0000 7f3aaa2d4640 1 --2- 172.21.15.105:0/2142815133 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f3a900a32c0 0x7f3a900a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:36.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.948+0000 7f3a977fe640 1 -- 172.21.15.105:0/2142815133 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f3a9c0f0f60 con 0x7f3aa4106f70 2024-09-16T02:00:36.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.948+0000 7f3aaa2d4640 1 --2- 172.21.15.105:0/2142815133 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f3a900a32c0 0x7f3a900a5780 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4069fc0 tx=0x7f3a9800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:36.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:36.951+0000 7f3a977fe640 1 -- 172.21.15.105:0/2142815133 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3a9c0bdd30 con 0x7f3aa4106f70 2024-09-16T02:00:37.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:37.119+0000 7f3aac538640 1 -- 172.21.15.105:0/2142815133 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3aa41029b0 con 0x7f3aa4106f70 2024-09-16T02:00:37.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:37.120+0000 7f3a977fe640 1 -- 172.21.15.105:0/2142815133 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22327 (secure 0 0 0) 0x7f3aa41029b0 con 0x7f3aa4106f70 2024-09-16T02:00:37.123 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:00:37.123 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":79,"fsid":"423b7a2a-73ce-11ef-bceb-c7b262605968","created":"2024-09-16T01:52:44.388033+0000","modified":"2024-09-16T02:00:35.112006+0000","last_up_change":"2024-09-16T01:59:42.218034+0000","last_in_change":"2024-09-16T01:59:18.785252+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T01:55:53.680808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T02:00:31.286117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T02:00:33.310045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a60d6c72-da8b-4b02-bb33-672be795e7f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6803","nonce":1226660290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6805","nonce":1226660290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6809","nonce":1226660290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6807","nonce":1226660290}]},"public_addr":"172.21.15.105:6803/1226660290","cluster_addr":"172.21.15.105:6805/1226660290","heartbeat_back_addr":"172.21.15.105:6809/1226660290","heartbeat_front_addr":"172.21.15.105:6807/1226660290","state":["exists","up"]},{"osd":1,"uuid":"b1d9ae84-b9a0-4a7e-b0fe-c402d46809fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6801","nonce":34753956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6803","nonce":34753956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6807","nonce":34753956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6805","nonce":34753956}]},"public_addr":"172.21.15.137:6801/34753956","cluster_addr":"172.21.15.137:6803/34753956","heartbeat_back_addr":"172.21.15.137:6807/34753956","heartbeat_front_addr":"172.21.15.137:6805/34753956","state":["exists","up"]},{"osd":2,"uuid":"137faae6-2e8b-44b5-8976-3d95a6eef1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6801","nonce":1159531918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6803","nonce":1159531918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6807","nonce":1159531918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6805","nonce":1159531918}]},"public_addr":"172.21.15.190:6801/1159531918","cluster_addr":"172.21.15.190:6803/1159531918","heartbeat_back_addr":"172.21.15.190:6807/1159531918","heartbeat_front_addr":"172.21.15.190:6805/1159531918","state":["exists","up"]},{"osd":3,"uuid":"5743c37b-5cd8-42b6-81c8-5cf5afc71e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6811","nonce":604905679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6813","nonce":604905679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6817","nonce":604905679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6815","nonce":604905679}]},"public_addr":"172.21.15.105:6811/604905679","cluster_addr":"172.21.15.105:6813/604905679","heartbeat_back_addr":"172.21.15.105:6817/604905679","heartbeat_front_addr":"172.21.15.105:6815/604905679","state":["exists","up"]},{"osd":4,"uuid":"e439d4a2-b54c-4de9-97f6-3510afe57f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6809","nonce":3758109169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6811","nonce":3758109169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6815","nonce":3758109169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6813","nonce":3758109169}]},"public_addr":"172.21.15.137:6809/3758109169","cluster_addr":"172.21.15.137:6811/3758109169","heartbeat_back_addr":"172.21.15.137:6815/3758109169","heartbeat_front_addr":"172.21.15.137:6813/3758109169","state":["exists","up"]},{"osd":5,"uuid":"d1e2a7d5-99f8-40fa-af6e-0305cc0af677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6809","nonce":2086773754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6811","nonce":2086773754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6815","nonce":2086773754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6813","nonce":2086773754}]},"public_addr":"172.21.15.190:6809/2086773754","cluster_addr":"172.21.15.190:6811/2086773754","heartbeat_back_addr":"172.21.15.190:6815/2086773754","heartbeat_front_addr":"172.21.15.190:6813/2086773754","state":["exists","up"]},{"osd":6,"uuid":"cd418159-a0f6-44c7-9213-c010b132fb56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6819","nonce":4012018725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6821","nonce":4012018725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6825","nonce":4012018725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6823","nonce":4012018725}]},"public_addr":"172.21.15.105:6819/4012018725","cluster_addr":"172.21.15.105:6821/4012018725","heartbeat_back_addr":"172.21.15.105:6825/4012018725","heartbeat_front_addr":"172.21.15.105:6823/4012018725","state":["exists","up"]},{"osd":7,"uuid":"76c15863-801e-4a72-be73-ba9df46c15b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6817","nonce":888678288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6819","nonce":888678288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6823","nonce":888678288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6821","nonce":888678288}]},"public_addr":"172.21.15.137:6817/888678288","cluster_addr":"172.21.15.137:6819/888678288","heartbeat_back_addr":"172.21.15.137:6823/888678288","heartbeat_front_addr":"172.21.15.137:6821/888678288","state":["exists","up"]},{"osd":8,"uuid":"6718597c-6f42-48ee-9493-edb2396fe707","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6817","nonce":2132638341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6819","nonce":2132638341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6823","nonce":2132638341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6821","nonce":2132638341}]},"public_addr":"172.21.15.190:6817/2132638341","cluster_addr":"172.21.15.190:6819/2132638341","heartbeat_back_addr":"172.21.15.190:6823/2132638341","heartbeat_front_addr":"172.21.15.190:6821/2132638341","state":["exists","up"]},{"osd":9,"uuid":"5178edc0-a2e8-491c-821c-6f41ec5aeb2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6827","nonce":3375034979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6829","nonce":3375034979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6833","nonce":3375034979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6831","nonce":3375034979}]},"public_addr":"172.21.15.105:6827/3375034979","cluster_addr":"172.21.15.105:6829/3375034979","heartbeat_back_addr":"172.21.15.105:6833/3375034979","heartbeat_front_addr":"172.21.15.105:6831/3375034979","state":["exists","up"]},{"osd":10,"uuid":"ad145d3e-a6e2-4dd7-bea3-bcd01eb3413e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6825","nonce":318562234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6827","nonce":318562234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6831","nonce":318562234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6829","nonce":318562234}]},"public_addr":"172.21.15.137:6825/318562234","cluster_addr":"172.21.15.137:6827/318562234","heartbeat_back_addr":"172.21.15.137:6831/318562234","heartbeat_front_addr":"172.21.15.137:6829/318562234","state":["exists","up"]},{"osd":11,"uuid":"6d44bced-3a8b-4763-a93e-0ea34acb4cb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6825","nonce":3820460689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6827","nonce":3820460689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6831","nonce":3820460689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6829","nonce":3820460689}]},"public_addr":"172.21.15.190:6825/3820460689","cluster_addr":"172.21.15.190:6827/3820460689","heartbeat_back_addr":"172.21.15.190:6831/3820460689","heartbeat_front_addr":"172.21.15.190:6829/3820460689","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:55:02.759033+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:55:26.829142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:55:51.120200+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:56:17.277870+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:56:42.313701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:07.013416+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:32.477916+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:58.643764+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:23.454908+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:48.869082+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:14.921573+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:40.908993+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:0/2453739657":"2024-09-17T01:53:39.351314+0000","172.21.15.105:6801/1469189484":"2024-09-17T01:53:39.351314+0000","172.21.15.105:0/3466166879":"2024-09-17T01:53:39.351314+0000","172.21.15.105:6801/1268348742":"2024-09-17T01:53:15.522227+0000","172.21.15.105:0/2897248478":"2024-09-17T01:53:39.351314+0000","172.21.15.105:0/3709363371":"2024-09-17T01:53:15.522227+0000","172.21.15.105:6800/1268348742":"2024-09-17T01:53:15.522227+0000","172.21.15.105:6800/1469189484":"2024-09-17T01:53:39.351314+0000","172.21.15.105:0/1198176299":"2024-09-17T01:53:15.522227+0000","172.21.15.105:0/3031917848":"2024-09-17T01:53:15.522227+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T02:00:37.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:37.122+0000 7f3aac538640 1 -- 172.21.15.105:0/2142815133 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f3a900a32c0 msgr2=0x7f3a900a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:37.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:37.122+0000 7f3aac538640 1 --2- 172.21.15.105:0/2142815133 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f3a900a32c0 0x7f3a900a5780 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4069fc0 tx=0x7f3a9800a2d0 comp rx=0 tx=0).stop 2024-09-16T02:00:37.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:37.122+0000 7f3aac538640 1 -- 172.21.15.105:0/2142815133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3aa4106f70 msgr2=0x7f3aa4101f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:37.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:37.122+0000 7f3aac538640 1 --2- 172.21.15.105:0/2142815133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3aa4106f70 0x7f3aa4101f00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c009dc0 tx=0x7f3a9c032ff0 comp rx=0 tx=0).stop 2024-09-16T02:00:37.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:37.122+0000 7f3aac538640 1 -- 172.21.15.105:0/2142815133 shutdown_connections 2024-09-16T02:00:37.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:37.122+0000 7f3aac538640 1 --2- 172.21.15.105:0/2142815133 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f3a900a32c0 0x7f3a900a5780 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:37.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:37.122+0000 7f3aac538640 1 --2- 172.21.15.105:0/2142815133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3aa4106f70 0x7f3aa4101f00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:37.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:37.122+0000 7f3aac538640 1 --2- 172.21.15.105:0/2142815133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3aa4100eb0 0x7f3aa41019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:37.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:37.122+0000 7f3aac538640 1 --2- 172.21.15.105:0/2142815133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3aa4100500 0x7f3aa4101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:37.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:37.123+0000 7f3aac538640 1 -- 172.21.15.105:0/2142815133 >> 172.21.15.105:0/2142815133 conn(0x7f3aa40fc5c0 msgr2=0x7f3aa40fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:37.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:37.123+0000 7f3aac538640 1 -- 172.21.15.105:0/2142815133 shutdown_connections 2024-09-16T02:00:37.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:37.123+0000 7f3aac538640 1 -- 172.21.15.105:0/2142815133 wait complete. 2024-09-16T02:00:37.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:37 smithi105 ceph-mon[31197]: mds.? [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] up:active 2024-09-16T02:00:37.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:37 smithi105 ceph-mon[31197]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T02:00:37.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:37 smithi105 ceph-mon[31197]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T02:00:37.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:37 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2142815133' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:00:37.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:37 smithi190 ceph-mon[34762]: mds.? [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] up:active 2024-09-16T02:00:37.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:37 smithi190 ceph-mon[34762]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T02:00:37.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:37 smithi190 ceph-mon[34762]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T02:00:37.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:37 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/2142815133' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:00:37.866 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T02:00:37.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:37 smithi137 ceph-mon[34099]: mds.? [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] up:active 2024-09-16T02:00:37.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:37 smithi137 ceph-mon[34099]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T02:00:37.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:37 smithi137 ceph-mon[34099]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T02:00:37.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:37 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/2142815133' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:00:38.114 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:00:38.541 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:38 smithi105 ceph-mon[31197]: pgmap v278: 97 pgs: 9 creating+peering, 11 unknown, 77 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 204 B/s wr, 0 op/s 2024-09-16T02:00:38.541 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:38 smithi105 ceph-mon[31197]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-16T02:00:38.541 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:38 smithi105 ceph-mon[31197]: Cluster is now healthy 2024-09-16T02:00:38.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.617+0000 7fcedd78a640 1 -- 172.21.15.105:0/1571892937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fced810c170 msgr2=0x7fced810e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:38.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.617+0000 7fcedd78a640 1 --2- 172.21.15.105:0/1571892937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fced810c170 0x7fced810e600 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fced0009900 tx=0x7fced002a480 comp rx=0 tx=0).stop 2024-09-16T02:00:38.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.618+0000 7fcedd78a640 1 -- 172.21.15.105:0/1571892937 shutdown_connections 2024-09-16T02:00:38.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.618+0000 7fcedd78a640 1 --2- 172.21.15.105:0/1571892937 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fced81036f0 0x7fced810bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:38.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.618+0000 7fcedd78a640 1 --2- 172.21.15.105:0/1571892937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fced810c170 0x7fced810e600 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:38.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.618+0000 7fcedd78a640 1 --2- 172.21.15.105:0/1571892937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fced8102dd0 0x7fced81031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:38.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.618+0000 7fcedd78a640 1 -- 172.21.15.105:0/1571892937 >> 172.21.15.105:0/1571892937 conn(0x7fced80fc790 msgr2=0x7fced80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:38.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.618+0000 7fcedd78a640 1 -- 172.21.15.105:0/1571892937 shutdown_connections 2024-09-16T02:00:38.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.618+0000 7fcedd78a640 1 -- 172.21.15.105:0/1571892937 wait complete. 2024-09-16T02:00:38.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.618+0000 7fcedd78a640 1 Processor -- start 2024-09-16T02:00:38.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.618+0000 7fcedd78a640 1 -- start start 2024-09-16T02:00:38.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.618+0000 7fcedd78a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fced8102dd0 0x7fced8198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:38.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.618+0000 7fcedd78a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fced81036f0 0x7fced8198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:38.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.618+0000 7fcedd78a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fced810c170 0x7fced819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:38.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.619+0000 7fcedd78a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fced8111ed0 con 0x7fced81036f0 2024-09-16T02:00:38.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.619+0000 7fcedd78a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fced8111d50 con 0x7fced810c170 2024-09-16T02:00:38.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.619+0000 7fcedd78a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fced8112050 con 0x7fced8102dd0 2024-09-16T02:00:38.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.619+0000 7fced67fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fced81036f0 0x7fced8198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:38.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.619+0000 7fced67fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fced81036f0 0x7fced8198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43010/0 (socket says 172.21.15.105:43010) 2024-09-16T02:00:38.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.619+0000 7fced67fc640 1 -- 172.21.15.105:0/1303031720 learned_addr learned my addr 172.21.15.105:0/1303031720 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:00:38.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.619+0000 7fced6ffd640 1 --2- 172.21.15.105:0/1303031720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fced8102dd0 0x7fced8198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:38.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.619+0000 7fced67fc640 1 -- 172.21.15.105:0/1303031720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fced8102dd0 msgr2=0x7fced8198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.619+0000 7fced67fc640 1 --2- 172.21.15.105:0/1303031720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fced8102dd0 0x7fced8198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.619+0000 7fced67fc640 1 -- 172.21.15.105:0/1303031720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fced810c170 msgr2=0x7fced819cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.619+0000 7fced67fc640 1 --2- 172.21.15.105:0/1303031720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fced810c170 0x7fced819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.619+0000 7fced67fc640 1 -- 172.21.15.105:0/1303031720 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fced819d4b0 con 0x7fced81036f0 2024-09-16T02:00:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.619+0000 7fced6ffd640 1 --2- 172.21.15.105:0/1303031720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fced8102dd0 0x7fced8198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:00:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.619+0000 7fced67fc640 1 --2- 172.21.15.105:0/1303031720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fced81036f0 0x7fced8198a40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcec000b4a0 tx=0x7fcec000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.619+0000 7fcecffff640 1 -- 172.21.15.105:0/1303031720 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec0014070 con 0x7fced81036f0 2024-09-16T02:00:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.619+0000 7fcecffff640 1 -- 172.21.15.105:0/1303031720 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcec0010ac0 con 0x7fced81036f0 2024-09-16T02:00:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.619+0000 7fcedd78a640 1 -- 172.21.15.105:0/1303031720 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fced819d7a0 con 0x7fced81036f0 2024-09-16T02:00:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.619+0000 7fcecffff640 1 -- 172.21.15.105:0/1303031720 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec0017c30 con 0x7fced81036f0 2024-09-16T02:00:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.619+0000 7fcedd78a640 1 -- 172.21.15.105:0/1303031720 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fced81a5080 con 0x7fced81036f0 2024-09-16T02:00:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.620+0000 7fcedd78a640 1 -- 172.21.15.105:0/1303031720 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fced8103230 con 0x7fced81036f0 2024-09-16T02:00:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.621+0000 7fcecffff640 1 -- 172.21.15.105:0/1303031720 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fcec0017070 con 0x7fced81036f0 2024-09-16T02:00:38.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.622+0000 7fcecffff640 1 --2- 172.21.15.105:0/1303031720 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fcec40a3240 0x7fcec40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:38.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.622+0000 7fced6ffd640 1 --2- 172.21.15.105:0/1303031720 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fcec40a3240 0x7fcec40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:38.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.622+0000 7fcecffff640 1 -- 172.21.15.105:0/1303031720 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fcec00cf040 con 0x7fced81036f0 2024-09-16T02:00:38.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.623+0000 7fced6ffd640 1 --2- 172.21.15.105:0/1303031720 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fcec40a3240 0x7fcec40a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcec8007eb0 tx=0x7fcec80073c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:38.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.626+0000 7fcecffff640 1 -- 172.21.15.105:0/1303031720 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcec009be90 con 0x7fced81036f0 2024-09-16T02:00:38.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.796+0000 7fcedd78a640 1 -- 172.21.15.105:0/1303031720 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fced8199860 con 0x7fced81036f0 2024-09-16T02:00:38.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.797+0000 7fcecffff640 1 -- 172.21.15.105:0/1303031720 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22327 (secure 0 0 0) 0x7fcec009b790 con 0x7fced81036f0 2024-09-16T02:00:38.800 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:00:38.800 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":79,"fsid":"423b7a2a-73ce-11ef-bceb-c7b262605968","created":"2024-09-16T01:52:44.388033+0000","modified":"2024-09-16T02:00:35.112006+0000","last_up_change":"2024-09-16T01:59:42.218034+0000","last_in_change":"2024-09-16T01:59:18.785252+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T01:55:53.680808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T02:00:31.286117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T02:00:33.310045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a60d6c72-da8b-4b02-bb33-672be795e7f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6803","nonce":1226660290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6805","nonce":1226660290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6809","nonce":1226660290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6807","nonce":1226660290}]},"public_addr":"172.21.15.105:6803/1226660290","cluster_addr":"172.21.15.105:6805/1226660290","heartbeat_back_addr":"172.21.15.105:6809/1226660290","heartbeat_front_addr":"172.21.15.105:6807/1226660290","state":["exists","up"]},{"osd":1,"uuid":"b1d9ae84-b9a0-4a7e-b0fe-c402d46809fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6801","nonce":34753956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6803","nonce":34753956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6807","nonce":34753956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6805","nonce":34753956}]},"public_addr":"172.21.15.137:6801/34753956","cluster_addr":"172.21.15.137:6803/34753956","heartbeat_back_addr":"172.21.15.137:6807/34753956","heartbeat_front_addr":"172.21.15.137:6805/34753956","state":["exists","up"]},{"osd":2,"uuid":"137faae6-2e8b-44b5-8976-3d95a6eef1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6801","nonce":1159531918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6803","nonce":1159531918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6807","nonce":1159531918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6805","nonce":1159531918}]},"public_addr":"172.21.15.190:6801/1159531918","cluster_addr":"172.21.15.190:6803/1159531918","heartbeat_back_addr":"172.21.15.190:6807/1159531918","heartbeat_front_addr":"172.21.15.190:6805/1159531918","state":["exists","up"]},{"osd":3,"uuid":"5743c37b-5cd8-42b6-81c8-5cf5afc71e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6811","nonce":604905679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6813","nonce":604905679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6817","nonce":604905679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6815","nonce":604905679}]},"public_addr":"172.21.15.105:6811/604905679","cluster_addr":"172.21.15.105:6813/604905679","heartbeat_back_addr":"172.21.15.105:6817/604905679","heartbeat_front_addr":"172.21.15.105:6815/604905679","state":["exists","up"]},{"osd":4,"uuid":"e439d4a2-b54c-4de9-97f6-3510afe57f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6809","nonce":3758109169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6811","nonce":3758109169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6815","nonce":3758109169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6813","nonce":3758109169}]},"public_addr":"172.21.15.137:6809/3758109169","cluster_addr":"172.21.15.137:6811/3758109169","heartbeat_back_addr":"172.21.15.137:6815/3758109169","heartbeat_front_addr":"172.21.15.137:6813/3758109169","state":["exists","up"]},{"osd":5,"uuid":"d1e2a7d5-99f8-40fa-af6e-0305cc0af677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6809","nonce":2086773754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6811","nonce":2086773754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6815","nonce":2086773754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6813","nonce":2086773754}]},"public_addr":"172.21.15.190:6809/2086773754","cluster_addr":"172.21.15.190:6811/2086773754","heartbeat_back_addr":"172.21.15.190:6815/2086773754","heartbeat_front_addr":"172.21.15.190:6813/2086773754","state":["exists","up"]},{"osd":6,"uuid":"cd418159-a0f6-44c7-9213-c010b132fb56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6819","nonce":4012018725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6821","nonce":4012018725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6825","nonce":4012018725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6823","nonce":4012018725}]},"public_addr":"172.21.15.105:6819/4012018725","cluster_addr":"172.21.15.105:6821/4012018725","heartbeat_back_addr":"172.21.15.105:6825/4012018725","heartbeat_front_addr":"172.21.15.105:6823/4012018725","state":["exists","up"]},{"osd":7,"uuid":"76c15863-801e-4a72-be73-ba9df46c15b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6817","nonce":888678288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6819","nonce":888678288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6823","nonce":888678288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6821","nonce":888678288}]},"public_addr":"172.21.15.137:6817/888678288","cluster_addr":"172.21.15.137:6819/888678288","heartbeat_back_addr":"172.21.15.137:6823/888678288","heartbeat_front_addr":"172.21.15.137:6821/888678288","state":["exists","up"]},{"osd":8,"uuid":"6718597c-6f42-48ee-9493-edb2396fe707","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6817","nonce":2132638341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6819","nonce":2132638341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6823","nonce":2132638341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6821","nonce":2132638341}]},"public_addr":"172.21.15.190:6817/2132638341","cluster_addr":"172.21.15.190:6819/2132638341","heartbeat_back_addr":"172.21.15.190:6823/2132638341","heartbeat_front_addr":"172.21.15.190:6821/2132638341","state":["exists","up"]},{"osd":9,"uuid":"5178edc0-a2e8-491c-821c-6f41ec5aeb2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6827","nonce":3375034979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6829","nonce":3375034979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6833","nonce":3375034979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6831","nonce":3375034979}]},"public_addr":"172.21.15.105:6827/3375034979","cluster_addr":"172.21.15.105:6829/3375034979","heartbeat_back_addr":"172.21.15.105:6833/3375034979","heartbeat_front_addr":"172.21.15.105:6831/3375034979","state":["exists","up"]},{"osd":10,"uuid":"ad145d3e-a6e2-4dd7-bea3-bcd01eb3413e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6825","nonce":318562234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6827","nonce":318562234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6831","nonce":318562234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6829","nonce":318562234}]},"public_addr":"172.21.15.137:6825/318562234","cluster_addr":"172.21.15.137:6827/318562234","heartbeat_back_addr":"172.21.15.137:6831/318562234","heartbeat_front_addr":"172.21.15.137:6829/318562234","state":["exists","up"]},{"osd":11,"uuid":"6d44bced-3a8b-4763-a93e-0ea34acb4cb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6825","nonce":3820460689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6827","nonce":3820460689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6831","nonce":3820460689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6829","nonce":3820460689}]},"public_addr":"172.21.15.190:6825/3820460689","cluster_addr":"172.21.15.190:6827/3820460689","heartbeat_back_addr":"172.21.15.190:6831/3820460689","heartbeat_front_addr":"172.21.15.190:6829/3820460689","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:55:02.759033+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:55:26.829142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:55:51.120200+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:56:17.277870+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:56:42.313701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:07.013416+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:32.477916+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:58.643764+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:23.454908+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:48.869082+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:14.921573+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:40.908993+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:0/2453739657":"2024-09-17T01:53:39.351314+0000","172.21.15.105:6801/1469189484":"2024-09-17T01:53:39.351314+0000","172.21.15.105:0/3466166879":"2024-09-17T01:53:39.351314+0000","172.21.15.105:6801/1268348742":"2024-09-17T01:53:15.522227+0000","172.21.15.105:0/2897248478":"2024-09-17T01:53:39.351314+0000","172.21.15.105:0/3709363371":"2024-09-17T01:53:15.522227+0000","172.21.15.105:6800/1268348742":"2024-09-17T01:53:15.522227+0000","172.21.15.105:6800/1469189484":"2024-09-17T01:53:39.351314+0000","172.21.15.105:0/1198176299":"2024-09-17T01:53:15.522227+0000","172.21.15.105:0/3031917848":"2024-09-17T01:53:15.522227+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T02:00:38.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.800+0000 7fcedd78a640 1 -- 172.21.15.105:0/1303031720 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fcec40a3240 msgr2=0x7fcec40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:38.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.800+0000 7fcedd78a640 1 --2- 172.21.15.105:0/1303031720 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fcec40a3240 0x7fcec40a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcec8007eb0 tx=0x7fcec80073c0 comp rx=0 tx=0).stop 2024-09-16T02:00:38.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.800+0000 7fcedd78a640 1 -- 172.21.15.105:0/1303031720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fced81036f0 msgr2=0x7fced8198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:38.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.800+0000 7fcedd78a640 1 --2- 172.21.15.105:0/1303031720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fced81036f0 0x7fced8198a40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcec000b4a0 tx=0x7fcec000b960 comp rx=0 tx=0).stop 2024-09-16T02:00:38.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.800+0000 7fcedd78a640 1 -- 172.21.15.105:0/1303031720 shutdown_connections 2024-09-16T02:00:38.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.800+0000 7fcedd78a640 1 --2- 172.21.15.105:0/1303031720 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fcec40a3240 0x7fcec40a5700 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:38.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.800+0000 7fcedd78a640 1 --2- 172.21.15.105:0/1303031720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fced810c170 0x7fced819cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:38.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.800+0000 7fcedd78a640 1 --2- 172.21.15.105:0/1303031720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fced81036f0 0x7fced8198a40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:38.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.800+0000 7fcedd78a640 1 --2- 172.21.15.105:0/1303031720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fced8102dd0 0x7fced8198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:38.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.800+0000 7fcedd78a640 1 -- 172.21.15.105:0/1303031720 >> 172.21.15.105:0/1303031720 conn(0x7fced80fc790 msgr2=0x7fced80fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:38.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.800+0000 7fcedd78a640 1 -- 172.21.15.105:0/1303031720 shutdown_connections 2024-09-16T02:00:38.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:38.800+0000 7fcedd78a640 1 -- 172.21.15.105:0/1303031720 wait complete. 2024-09-16T02:00:38.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:38 smithi190 ceph-mon[34762]: pgmap v278: 97 pgs: 9 creating+peering, 11 unknown, 77 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 204 B/s wr, 0 op/s 2024-09-16T02:00:38.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:38 smithi190 ceph-mon[34762]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-16T02:00:38.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:38 smithi190 ceph-mon[34762]: Cluster is now healthy 2024-09-16T02:00:38.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:38 smithi137 ceph-mon[34099]: pgmap v278: 97 pgs: 9 creating+peering, 11 unknown, 77 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 204 B/s wr, 0 op/s 2024-09-16T02:00:38.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:38 smithi137 ceph-mon[34099]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-16T02:00:38.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:38 smithi137 ceph-mon[34099]: Cluster is now healthy 2024-09-16T02:00:39.513 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:39 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/1303031720' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:00:39.541 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-16T02:00:39.785 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:00:39.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:39 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/1303031720' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:00:39.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:39 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/1303031720' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:00:40.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.263+0000 7f5cfcf30640 1 -- 172.21.15.105:0/4237486435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5cf8102570 msgr2=0x7f5cf81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:40.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.263+0000 7f5cfcf30640 1 --2- 172.21.15.105:0/4237486435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5cf8102570 0x7f5cf81029b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0009900 tx=0x7f5ce002a480 comp rx=0 tx=0).stop 2024-09-16T02:00:40.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.263+0000 7f5cfcf30640 1 -- 172.21.15.105:0/4237486435 shutdown_connections 2024-09-16T02:00:40.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.263+0000 7f5cfcf30640 1 --2- 172.21.15.105:0/4237486435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5cf8102ef0 0x7f5cf810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:40.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.263+0000 7f5cfcf30640 1 --2- 172.21.15.105:0/4237486435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5cf8102570 0x7f5cf81029b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:40.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.263+0000 7f5cfcf30640 1 --2- 172.21.15.105:0/4237486435 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cf8100e20 0x7f5cf8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:40.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.263+0000 7f5cfcf30640 1 -- 172.21.15.105:0/4237486435 >> 172.21.15.105:0/4237486435 conn(0x7f5cf80fc7d0 msgr2=0x7f5cf80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:40.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.263+0000 7f5cfcf30640 1 -- 172.21.15.105:0/4237486435 shutdown_connections 2024-09-16T02:00:40.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.263+0000 7f5cfcf30640 1 -- 172.21.15.105:0/4237486435 wait complete. 2024-09-16T02:00:40.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.263+0000 7f5cfcf30640 1 Processor -- start 2024-09-16T02:00:40.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.264+0000 7f5cfcf30640 1 -- start start 2024-09-16T02:00:40.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.264+0000 7f5cfcf30640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5cf8100e20 0x7f5cf80fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:40.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.264+0000 7f5cfcf30640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cf8102570 0x7f5cf80fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:40.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.264+0000 7f5cfcf30640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5cf8102ef0 0x7f5cf80fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:40.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.264+0000 7f5cfcf30640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf8110e50 con 0x7f5cf8102ef0 2024-09-16T02:00:40.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.264+0000 7f5cfcf30640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf8110cd0 con 0x7f5cf8100e20 2024-09-16T02:00:40.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.264+0000 7f5cfcf30640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf8110fd0 con 0x7f5cf8102570 2024-09-16T02:00:40.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.264+0000 7f5cf6d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5cf8102ef0 0x7f5cf80fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:40.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.264+0000 7f5cf6d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5cf8102ef0 0x7f5cf80fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43026/0 (socket says 172.21.15.105:43026) 2024-09-16T02:00:40.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.264+0000 7f5cf6d9d640 1 -- 172.21.15.105:0/4231434310 learned_addr learned my addr 172.21.15.105:0/4231434310 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:00:40.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.264+0000 7f5cf5d9b640 1 --2- 172.21.15.105:0/4231434310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cf8102570 0x7f5cf80fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:40.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.264+0000 7f5cf659c640 1 --2- 172.21.15.105:0/4231434310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5cf8100e20 0x7f5cf80fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:40.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.264+0000 7f5cf6d9d640 1 -- 172.21.15.105:0/4231434310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cf8102570 msgr2=0x7f5cf80fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:40.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.264+0000 7f5cf6d9d640 1 --2- 172.21.15.105:0/4231434310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cf8102570 0x7f5cf80fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:40.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.264+0000 7f5cf6d9d640 1 -- 172.21.15.105:0/4231434310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5cf8100e20 msgr2=0x7f5cf80fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:40.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.264+0000 7f5cf6d9d640 1 --2- 172.21.15.105:0/4231434310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5cf8100e20 0x7f5cf80fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:40.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.264+0000 7f5cf6d9d640 1 -- 172.21.15.105:0/4231434310 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf81a69a0 con 0x7f5cf8102ef0 2024-09-16T02:00:40.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.264+0000 7f5cf659c640 1 --2- 172.21.15.105:0/4231434310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5cf8100e20 0x7f5cf80fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:00:40.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.264+0000 7f5cf6d9d640 1 --2- 172.21.15.105:0/4231434310 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5cf8102ef0 0x7f5cf80fd700 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5ce800c970 tx=0x7f5ce800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:40.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.264+0000 7f5cef7fe640 1 -- 172.21.15.105:0/4231434310 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8007bb0 con 0x7f5cf8102ef0 2024-09-16T02:00:40.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.264+0000 7f5cef7fe640 1 -- 172.21.15.105:0/4231434310 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ce8007d50 con 0x7f5cf8102ef0 2024-09-16T02:00:40.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.264+0000 7f5cfcf30640 1 -- 172.21.15.105:0/4231434310 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cf81a6c90 con 0x7f5cf8102ef0 2024-09-16T02:00:40.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.264+0000 7f5cef7fe640 1 -- 172.21.15.105:0/4231434310 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8004e10 con 0x7f5cf8102ef0 2024-09-16T02:00:40.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.265+0000 7f5cfcf30640 1 -- 172.21.15.105:0/4231434310 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cf81a7180 con 0x7f5cf8102ef0 2024-09-16T02:00:40.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.265+0000 7f5cfcf30640 1 -- 172.21.15.105:0/4231434310 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cf8101200 con 0x7f5cf8102ef0 2024-09-16T02:00:40.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.266+0000 7f5cef7fe640 1 -- 172.21.15.105:0/4231434310 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f5ce8005020 con 0x7f5cf8102ef0 2024-09-16T02:00:40.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.267+0000 7f5cef7fe640 1 --2- 172.21.15.105:0/4231434310 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5ce40a3240 0x7f5ce40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:40.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.267+0000 7f5cf659c640 1 --2- 172.21.15.105:0/4231434310 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5ce40a3240 0x7f5ce40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:40.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.267+0000 7f5cef7fe640 1 -- 172.21.15.105:0/4231434310 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f5ce80cf540 con 0x7f5cf8102ef0 2024-09-16T02:00:40.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.268+0000 7f5cf659c640 1 --2- 172.21.15.105:0/4231434310 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5ce40a3240 0x7f5ce40a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0007eb0 tx=0x7f5cf00073c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:40.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.271+0000 7f5cef7fe640 1 -- 172.21.15.105:0/4231434310 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5ce8014030 con 0x7f5cf8102ef0 2024-09-16T02:00:40.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:40.478+0000 7f5cfcf30640 1 -- 172.21.15.105:0/4231434310 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f5cf810a0a0 con 0x7f5cf8102ef0 2024-09-16T02:00:40.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:40 smithi105 ceph-mon[31197]: pgmap v279: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 2 op/s 2024-09-16T02:00:40.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:40 smithi190 ceph-mon[34762]: pgmap v279: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 2 op/s 2024-09-16T02:00:40.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:40 smithi137 ceph-mon[34099]: pgmap v279: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 2 op/s 2024-09-16T02:00:40.921 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 16 02:00:40 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-b[64153]: 2024-09-16T02:00:40.584+0000 7fa83671b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T02:00:41.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:41.471+0000 7f5cef7fe640 1 -- 172.21.15.105:0/4231434310 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f5ce809c3b0 con 0x7f5cf8102ef0 2024-09-16T02:00:41.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:41.474+0000 7f5cfcf30640 1 -- 172.21.15.105:0/4231434310 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5ce40a3240 msgr2=0x7f5ce40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:41.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:41.474+0000 7f5cfcf30640 1 --2- 172.21.15.105:0/4231434310 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5ce40a3240 0x7f5ce40a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0007eb0 tx=0x7f5cf00073c0 comp rx=0 tx=0).stop 2024-09-16T02:00:41.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:41.474+0000 7f5cfcf30640 1 -- 172.21.15.105:0/4231434310 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5cf8102ef0 msgr2=0x7f5cf80fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:41.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:41.474+0000 7f5cfcf30640 1 --2- 172.21.15.105:0/4231434310 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5cf8102ef0 0x7f5cf80fd700 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5ce800c970 tx=0x7f5ce800ce30 comp rx=0 tx=0).stop 2024-09-16T02:00:41.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:41.474+0000 7f5cfcf30640 1 -- 172.21.15.105:0/4231434310 shutdown_connections 2024-09-16T02:00:41.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:41.474+0000 7f5cfcf30640 1 --2- 172.21.15.105:0/4231434310 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5ce40a3240 0x7f5ce40a5700 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:41.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:41.474+0000 7f5cfcf30640 1 --2- 172.21.15.105:0/4231434310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cf8102570 0x7f5cf80fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:41.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:41.474+0000 7f5cfcf30640 1 --2- 172.21.15.105:0/4231434310 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5cf8102ef0 0x7f5cf80fd700 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:41.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:41.474+0000 7f5cfcf30640 1 --2- 172.21.15.105:0/4231434310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5cf8100e20 0x7f5cf80fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:41.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:41.474+0000 7f5cfcf30640 1 -- 172.21.15.105:0/4231434310 >> 172.21.15.105:0/4231434310 conn(0x7f5cf80fc7d0 msgr2=0x7f5cf806a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:41.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:41.474+0000 7f5cfcf30640 1 -- 172.21.15.105:0/4231434310 shutdown_connections 2024-09-16T02:00:41.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:41.474+0000 7f5cfcf30640 1 -- 172.21.15.105:0/4231434310 wait complete. 2024-09-16T02:00:41.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:41 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/4231434310' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-16T02:00:41.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:41 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/4231434310' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-16T02:00:41.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:41 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/4231434310' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-16T02:00:42.206 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-09-16T02:00:42.206 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-16T02:00:42.455 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:00:42.475 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:42 smithi105 ceph-mon[31197]: pgmap v280: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-16T02:00:42.475 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:42 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/4231434310' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-16T02:00:42.475 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:42 smithi105 ceph-mon[31197]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T02:00:42.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:42 smithi190 ceph-mon[34762]: pgmap v280: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-16T02:00:42.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:42 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/4231434310' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-16T02:00:42.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:42 smithi190 ceph-mon[34762]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T02:00:42.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:42 smithi137 ceph-mon[34099]: pgmap v280: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-16T02:00:42.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:42 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/4231434310' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-16T02:00:42.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:42 smithi137 ceph-mon[34099]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T02:00:42.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.942+0000 7fdcaa143640 1 -- 172.21.15.105:0/3166411227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdca4102790 msgr2=0x7fdca41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:42.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.942+0000 7fdcaa143640 1 --2- 172.21.15.105:0/3166411227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdca4102790 0x7fdca41067e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdc98009900 tx=0x7fdc9802a410 comp rx=0 tx=0).stop 2024-09-16T02:00:42.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.942+0000 7fdcaa143640 1 -- 172.21.15.105:0/3166411227 shutdown_connections 2024-09-16T02:00:42.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.942+0000 7fdcaa143640 1 --2- 172.21.15.105:0/3166411227 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdca4106f10 0x7fdca410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:42.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.942+0000 7fdcaa143640 1 --2- 172.21.15.105:0/3166411227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdca4102790 0x7fdca41067e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:42.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.942+0000 7fdcaa143640 1 --2- 172.21.15.105:0/3166411227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdca4101de0 0x7fdca41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:42.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.942+0000 7fdcaa143640 1 -- 172.21.15.105:0/3166411227 >> 172.21.15.105:0/3166411227 conn(0x7fdca40fd8c0 msgr2=0x7fdca40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:42.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.942+0000 7fdcaa143640 1 -- 172.21.15.105:0/3166411227 shutdown_connections 2024-09-16T02:00:42.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.942+0000 7fdcaa143640 1 -- 172.21.15.105:0/3166411227 wait complete. 2024-09-16T02:00:42.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.942+0000 7fdcaa143640 1 Processor -- start 2024-09-16T02:00:42.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.943+0000 7fdcaa143640 1 -- start start 2024-09-16T02:00:42.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.943+0000 7fdcaa143640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdca4101de0 0x7fdca4198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:42.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.943+0000 7fdcaa143640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdca4102790 0x7fdca4198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:42.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.943+0000 7fdcaa143640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdca4106f10 0x7fdca419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:42.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.943+0000 7fdcaa143640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdca4111e60 con 0x7fdca4101de0 2024-09-16T02:00:42.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.943+0000 7fdcaa143640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdca4111ce0 con 0x7fdca4106f10 2024-09-16T02:00:42.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.943+0000 7fdcaa143640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdca4111fe0 con 0x7fdca4102790 2024-09-16T02:00:42.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.943+0000 7fdca37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdca4101de0 0x7fdca4198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:42.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.943+0000 7fdca37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdca4101de0 0x7fdca4198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43038/0 (socket says 172.21.15.105:43038) 2024-09-16T02:00:42.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.943+0000 7fdca37fe640 1 -- 172.21.15.105:0/3079414967 learned_addr learned my addr 172.21.15.105:0/3079414967 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:00:42.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.943+0000 7fdca3fff640 1 --2- 172.21.15.105:0/3079414967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdca4106f10 0x7fdca419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:42.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.943+0000 7fdca2ffd640 1 --2- 172.21.15.105:0/3079414967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdca4102790 0x7fdca4198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:42.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.943+0000 7fdca37fe640 1 -- 172.21.15.105:0/3079414967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdca4102790 msgr2=0x7fdca4198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:42.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.943+0000 7fdca37fe640 1 --2- 172.21.15.105:0/3079414967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdca4102790 0x7fdca4198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:42.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.943+0000 7fdca37fe640 1 -- 172.21.15.105:0/3079414967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdca4106f10 msgr2=0x7fdca419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:42.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.943+0000 7fdca37fe640 1 --2- 172.21.15.105:0/3079414967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdca4106f10 0x7fdca419ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:42.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.943+0000 7fdca37fe640 1 -- 172.21.15.105:0/3079414967 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca419d3b0 con 0x7fdca4101de0 2024-09-16T02:00:42.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.943+0000 7fdca2ffd640 1 --2- 172.21.15.105:0/3079414967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdca4102790 0x7fdca4198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:00:42.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.943+0000 7fdca37fe640 1 --2- 172.21.15.105:0/3079414967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdca4101de0 0x7fdca4198400 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdc9400c970 tx=0x7fdc9400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:42.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.944+0000 7fdca0ff9640 1 -- 172.21.15.105:0/3079414967 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc94007bb0 con 0x7fdca4101de0 2024-09-16T02:00:42.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.944+0000 7fdca0ff9640 1 -- 172.21.15.105:0/3079414967 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc94007d50 con 0x7fdca4101de0 2024-09-16T02:00:42.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.944+0000 7fdca0ff9640 1 -- 172.21.15.105:0/3079414967 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9400f450 con 0x7fdca4101de0 2024-09-16T02:00:42.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.944+0000 7fdcaa143640 1 -- 172.21.15.105:0/3079414967 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdca419d6a0 con 0x7fdca4101de0 2024-09-16T02:00:42.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.944+0000 7fdcaa143640 1 -- 172.21.15.105:0/3079414967 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdca41a4f80 con 0x7fdca4101de0 2024-09-16T02:00:42.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.945+0000 7fdcaa143640 1 -- 172.21.15.105:0/3079414967 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdca41021c0 con 0x7fdca4101de0 2024-09-16T02:00:42.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.946+0000 7fdca0ff9640 1 -- 172.21.15.105:0/3079414967 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fdc94004f00 con 0x7fdca4101de0 2024-09-16T02:00:42.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.946+0000 7fdca0ff9640 1 --2- 172.21.15.105:0/3079414967 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fdc8c0a32c0 0x7fdc8c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:42.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.947+0000 7fdca2ffd640 1 --2- 172.21.15.105:0/3079414967 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fdc8c0a32c0 0x7fdc8c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:42.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.947+0000 7fdca0ff9640 1 -- 172.21.15.105:0/3079414967 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fdc940cf8c0 con 0x7fdca4101de0 2024-09-16T02:00:42.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.947+0000 7fdca2ffd640 1 --2- 172.21.15.105:0/3079414967 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fdc8c0a32c0 0x7fdc8c0a5780 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdc98031440 tx=0x7fdc980095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:42.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:42.950+0000 7fdca0ff9640 1 -- 172.21.15.105:0/3079414967 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdc94014030 con 0x7fdca4101de0 2024-09-16T02:00:43.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:43.155+0000 7fdcaa143640 1 -- 172.21.15.105:0/3079414967 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fdca402d7c0 con 0x7fdca4101de0 2024-09-16T02:00:43.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:43.478+0000 7fdca0ff9640 1 -- 172.21.15.105:0/3079414967 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7fdc9409c7b0 con 0x7fdca4101de0 2024-09-16T02:00:43.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:43.480+0000 7fdcaa143640 1 -- 172.21.15.105:0/3079414967 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fdc8c0a32c0 msgr2=0x7fdc8c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:43.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:43.480+0000 7fdcaa143640 1 --2- 172.21.15.105:0/3079414967 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fdc8c0a32c0 0x7fdc8c0a5780 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdc98031440 tx=0x7fdc980095f0 comp rx=0 tx=0).stop 2024-09-16T02:00:43.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:43.480+0000 7fdcaa143640 1 -- 172.21.15.105:0/3079414967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdca4101de0 msgr2=0x7fdca4198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:43.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:43.480+0000 7fdcaa143640 1 --2- 172.21.15.105:0/3079414967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdca4101de0 0x7fdca4198400 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdc9400c970 tx=0x7fdc9400ce30 comp rx=0 tx=0).stop 2024-09-16T02:00:43.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:43.480+0000 7fdcaa143640 1 -- 172.21.15.105:0/3079414967 shutdown_connections 2024-09-16T02:00:43.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:43.480+0000 7fdcaa143640 1 --2- 172.21.15.105:0/3079414967 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fdc8c0a32c0 0x7fdc8c0a5780 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:43.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:43.480+0000 7fdcaa143640 1 --2- 172.21.15.105:0/3079414967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdca4102790 0x7fdca4198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:43.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:43.480+0000 7fdcaa143640 1 --2- 172.21.15.105:0/3079414967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdca4106f10 0x7fdca419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:43.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:43.480+0000 7fdcaa143640 1 --2- 172.21.15.105:0/3079414967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdca4101de0 0x7fdca4198400 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:43.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:43.480+0000 7fdcaa143640 1 -- 172.21.15.105:0/3079414967 >> 172.21.15.105:0/3079414967 conn(0x7fdca40fd8c0 msgr2=0x7fdca40fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:43.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:43.481+0000 7fdcaa143640 1 -- 172.21.15.105:0/3079414967 shutdown_connections 2024-09-16T02:00:43.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:43.481+0000 7fdcaa143640 1 -- 172.21.15.105:0/3079414967 wait complete. 2024-09-16T02:00:43.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:43 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/3079414967' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-16T02:00:43.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:43 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/3079414967' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-16T02:00:43.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:43 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/3079414967' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-16T02:00:44.186 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-16T02:00:44.429 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:00:44.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:44 smithi105 ceph-mon[31197]: pgmap v281: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-09-16T02:00:44.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:44 smithi105 ceph-mon[31197]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-16T02:00:44.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:44 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/3079414967' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-16T02:00:44.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:44 smithi105 ceph-mon[31197]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-16T02:00:44.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:44 smithi105 ceph-mon[31197]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T02:00:44.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:44 smithi105 ceph-mon[31197]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-16T02:00:44.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:44 smithi105 ceph-mon[31197]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-16T02:00:44.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:44 smithi190 ceph-mon[34762]: pgmap v281: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-09-16T02:00:44.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:44 smithi190 ceph-mon[34762]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-16T02:00:44.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:44 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/3079414967' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-16T02:00:44.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:44 smithi190 ceph-mon[34762]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-16T02:00:44.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:44 smithi190 ceph-mon[34762]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T02:00:44.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:44 smithi190 ceph-mon[34762]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-16T02:00:44.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:44 smithi190 ceph-mon[34762]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-16T02:00:44.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:44 smithi137 ceph-mon[34099]: pgmap v281: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-09-16T02:00:44.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:44 smithi137 ceph-mon[34099]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-16T02:00:44.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:44 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/3079414967' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-16T02:00:44.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:44 smithi137 ceph-mon[34099]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-16T02:00:44.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:44 smithi137 ceph-mon[34099]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T02:00:44.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:44 smithi137 ceph-mon[34099]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-16T02:00:44.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:44 smithi137 ceph-mon[34099]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-16T02:00:44.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.953+0000 7f80e1362640 1 -- 172.21.15.105:0/263583789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80dc1027a0 msgr2=0x7f80dc1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:44.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.953+0000 7f80e1362640 1 --2- 172.21.15.105:0/263583789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80dc1027a0 0x7f80dc1067f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f80d0009900 tx=0x7f80d002a410 comp rx=0 tx=0).stop 2024-09-16T02:00:44.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.953+0000 7f80e1362640 1 -- 172.21.15.105:0/263583789 shutdown_connections 2024-09-16T02:00:44.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.953+0000 7f80e1362640 1 --2- 172.21.15.105:0/263583789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80dc106f20 0x7f80dc10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:44.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.953+0000 7f80e1362640 1 --2- 172.21.15.105:0/263583789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80dc1027a0 0x7f80dc1067f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:44.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.953+0000 7f80e1362640 1 --2- 172.21.15.105:0/263583789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80dc101df0 0x7f80dc1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:44.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.953+0000 7f80e1362640 1 -- 172.21.15.105:0/263583789 >> 172.21.15.105:0/263583789 conn(0x7f80dc0fd8b0 msgr2=0x7f80dc0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:44.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.953+0000 7f80e1362640 1 -- 172.21.15.105:0/263583789 shutdown_connections 2024-09-16T02:00:44.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.953+0000 7f80e1362640 1 -- 172.21.15.105:0/263583789 wait complete. 2024-09-16T02:00:44.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.954+0000 7f80e1362640 1 Processor -- start 2024-09-16T02:00:44.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.954+0000 7f80e1362640 1 -- start start 2024-09-16T02:00:44.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.954+0000 7f80e1362640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80dc101df0 0x7f80dc198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:44.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.954+0000 7f80e1362640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80dc1027a0 0x7f80dc1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:44.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.954+0000 7f80e1362640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80dc106f20 0x7f80dc19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:44.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.954+0000 7f80e1362640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f80dc111e50 con 0x7f80dc106f20 2024-09-16T02:00:44.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.954+0000 7f80e1362640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f80dc111cd0 con 0x7f80dc101df0 2024-09-16T02:00:44.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.954+0000 7f80e1362640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f80dc111fd0 con 0x7f80dc1027a0 2024-09-16T02:00:44.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.954+0000 7f80db7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80dc106f20 0x7f80dc19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:44.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.954+0000 7f80db7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80dc106f20 0x7f80dc19cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43058/0 (socket says 172.21.15.105:43058) 2024-09-16T02:00:44.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.954+0000 7f80db7fe640 1 -- 172.21.15.105:0/2173864961 learned_addr learned my addr 172.21.15.105:0/2173864961 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:00:44.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.954+0000 7f80daffd640 1 --2- 172.21.15.105:0/2173864961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80dc101df0 0x7f80dc198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:44.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.954+0000 7f80da7fc640 1 --2- 172.21.15.105:0/2173864961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80dc1027a0 0x7f80dc1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:44.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.954+0000 7f80db7fe640 1 -- 172.21.15.105:0/2173864961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80dc1027a0 msgr2=0x7f80dc1988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:44.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.954+0000 7f80db7fe640 1 --2- 172.21.15.105:0/2173864961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80dc1027a0 0x7f80dc1988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:44.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.954+0000 7f80db7fe640 1 -- 172.21.15.105:0/2173864961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80dc101df0 msgr2=0x7f80dc198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:44.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.954+0000 7f80db7fe640 1 --2- 172.21.15.105:0/2173864961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80dc101df0 0x7f80dc198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:44.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.954+0000 7f80db7fe640 1 -- 172.21.15.105:0/2173864961 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80dc19d320 con 0x7f80dc106f20 2024-09-16T02:00:44.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.954+0000 7f80daffd640 1 --2- 172.21.15.105:0/2173864961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80dc101df0 0x7f80dc198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:00:44.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.955+0000 7f80db7fe640 1 --2- 172.21.15.105:0/2173864961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80dc106f20 0x7f80dc19cc40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f80d400eaa0 tx=0x7f80d400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:44.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.955+0000 7f80c3fff640 1 -- 172.21.15.105:0/2173864961 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d400cdb0 con 0x7f80dc106f20 2024-09-16T02:00:44.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.955+0000 7f80c3fff640 1 -- 172.21.15.105:0/2173864961 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80d4005020 con 0x7f80dc106f20 2024-09-16T02:00:44.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.955+0000 7f80e1362640 1 -- 172.21.15.105:0/2173864961 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80dc19d610 con 0x7f80dc106f20 2024-09-16T02:00:44.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.955+0000 7f80c3fff640 1 -- 172.21.15.105:0/2173864961 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d40116d0 con 0x7f80dc106f20 2024-09-16T02:00:44.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.955+0000 7f80e1362640 1 -- 172.21.15.105:0/2173864961 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80dc1a4ef0 con 0x7f80dc106f20 2024-09-16T02:00:44.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.956+0000 7f80e1362640 1 -- 172.21.15.105:0/2173864961 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80dc1021d0 con 0x7f80dc106f20 2024-09-16T02:00:44.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.957+0000 7f80c3fff640 1 -- 172.21.15.105:0/2173864961 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f80d40051d0 con 0x7f80dc106f20 2024-09-16T02:00:44.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.958+0000 7f80c3fff640 1 --2- 172.21.15.105:0/2173864961 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f80c40a3240 0x7f80c40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:44.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.958+0000 7f80daffd640 1 --2- 172.21.15.105:0/2173864961 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f80c40a3240 0x7f80c40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:44.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.958+0000 7f80c3fff640 1 -- 172.21.15.105:0/2173864961 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f80d40cfe00 con 0x7f80dc106f20 2024-09-16T02:00:44.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.958+0000 7f80daffd640 1 --2- 172.21.15.105:0/2173864961 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f80c40a3240 0x7f80c40a5700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f80cc009930 tx=0x7f80cc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:44.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:44.961+0000 7f80c3fff640 1 -- 172.21.15.105:0/2173864961 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f80d4016030 con 0x7f80dc106f20 2024-09-16T02:00:45.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:45.161+0000 7f80e1362640 1 -- 172.21.15.105:0/2173864961 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f80dc1996d0 con 0x7f80dc106f20 2024-09-16T02:00:45.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:45.503+0000 7f80c3fff640 1 -- 172.21.15.105:0/2173864961 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f80dc1996d0 con 0x7f80dc106f20 2024-09-16T02:00:45.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:45.505+0000 7f80e1362640 1 -- 172.21.15.105:0/2173864961 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f80c40a3240 msgr2=0x7f80c40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:45.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:45.505+0000 7f80e1362640 1 --2- 172.21.15.105:0/2173864961 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f80c40a3240 0x7f80c40a5700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f80cc009930 tx=0x7f80cc008040 comp rx=0 tx=0).stop 2024-09-16T02:00:45.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:45.505+0000 7f80e1362640 1 -- 172.21.15.105:0/2173864961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80dc106f20 msgr2=0x7f80dc19cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:45.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:45.505+0000 7f80e1362640 1 --2- 172.21.15.105:0/2173864961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80dc106f20 0x7f80dc19cc40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f80d400eaa0 tx=0x7f80d400ef60 comp rx=0 tx=0).stop 2024-09-16T02:00:45.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:45.505+0000 7f80e1362640 1 -- 172.21.15.105:0/2173864961 shutdown_connections 2024-09-16T02:00:45.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:45.505+0000 7f80e1362640 1 --2- 172.21.15.105:0/2173864961 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f80c40a3240 0x7f80c40a5700 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:45.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:45.505+0000 7f80e1362640 1 --2- 172.21.15.105:0/2173864961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80dc1027a0 0x7f80dc1988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:45.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:45.505+0000 7f80e1362640 1 --2- 172.21.15.105:0/2173864961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80dc106f20 0x7f80dc19cc40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:45.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:45.505+0000 7f80e1362640 1 --2- 172.21.15.105:0/2173864961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80dc101df0 0x7f80dc198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:45.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:45.505+0000 7f80e1362640 1 -- 172.21.15.105:0/2173864961 >> 172.21.15.105:0/2173864961 conn(0x7f80dc0fd8b0 msgr2=0x7f80dc0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:45.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:45.505+0000 7f80e1362640 1 -- 172.21.15.105:0/2173864961 shutdown_connections 2024-09-16T02:00:45.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:45.506+0000 7f80e1362640 1 -- 172.21.15.105:0/2173864961 wait complete. 2024-09-16T02:00:45.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:45 smithi105 ceph-mon[31197]: mds.? [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] up:active 2024-09-16T02:00:45.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:45 smithi105 ceph-mon[31197]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-16T02:00:45.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:45 smithi105 ceph-mon[31197]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-16T02:00:45.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:45 smithi105 ceph-mon[31197]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-16T02:00:45.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:45 smithi105 ceph-mon[31197]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-16T02:00:45.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:45 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2173864961' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-16T02:00:45.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:45 smithi190 ceph-mon[34762]: mds.? [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] up:active 2024-09-16T02:00:45.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:45 smithi190 ceph-mon[34762]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-16T02:00:45.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:45 smithi190 ceph-mon[34762]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-16T02:00:45.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:45 smithi190 ceph-mon[34762]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-16T02:00:45.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:45 smithi190 ceph-mon[34762]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-16T02:00:45.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:45 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/2173864961' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-16T02:00:45.921 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 16 02:00:45 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-b[64153]: 2024-09-16T02:00:45.584+0000 7fa83671b640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-16T02:00:45.921 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 16 02:00:45 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-b[64153]: 2024-09-16T02:00:45.584+0000 7fa83671b640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-16T02:00:45.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:45 smithi137 ceph-mon[34099]: mds.? [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] up:active 2024-09-16T02:00:45.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:45 smithi137 ceph-mon[34099]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-16T02:00:45.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:45 smithi137 ceph-mon[34099]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-16T02:00:45.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:45 smithi137 ceph-mon[34099]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-16T02:00:45.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:45 smithi137 ceph-mon[34099]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-16T02:00:45.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:45 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/2173864961' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-16T02:00:46.198 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-16T02:00:46.445 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:00:46.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:46 smithi105 ceph-mon[31197]: pgmap v282: 97 pgs: 97 active+clean; 580 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 8 op/s 2024-09-16T02:00:46.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:46 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2173864961' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-16T02:00:46.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:46 smithi105 ceph-mon[31197]: mds.? [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] up:active 2024-09-16T02:00:46.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:46 smithi105 ceph-mon[31197]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-16T02:00:46.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:46 smithi105 ceph-mon[31197]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-16T02:00:46.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:46 smithi105 ceph-mon[31197]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-16T02:00:46.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:46 smithi105 ceph-mon[31197]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-16T02:00:46.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:46 smithi190 ceph-mon[34762]: pgmap v282: 97 pgs: 97 active+clean; 580 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 8 op/s 2024-09-16T02:00:46.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:46 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/2173864961' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-16T02:00:46.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:46 smithi190 ceph-mon[34762]: mds.? [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] up:active 2024-09-16T02:00:46.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:46 smithi190 ceph-mon[34762]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-16T02:00:46.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:46 smithi190 ceph-mon[34762]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-16T02:00:46.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:46 smithi190 ceph-mon[34762]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-16T02:00:46.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:46 smithi190 ceph-mon[34762]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-16T02:00:46.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:46 smithi137 ceph-mon[34099]: pgmap v282: 97 pgs: 97 active+clean; 580 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 8 op/s 2024-09-16T02:00:46.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:46 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/2173864961' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-16T02:00:46.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:46 smithi137 ceph-mon[34099]: mds.? [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] up:active 2024-09-16T02:00:46.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:46 smithi137 ceph-mon[34099]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-16T02:00:46.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:46 smithi137 ceph-mon[34099]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-16T02:00:46.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:46 smithi137 ceph-mon[34099]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-16T02:00:46.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:46 smithi137 ceph-mon[34099]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-16T02:00:46.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.968+0000 7f7b59c54640 1 -- 172.21.15.105:0/3901965788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b54106f10 msgr2=0x7f7b5410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:46.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.968+0000 7f7b59c54640 1 --2- 172.21.15.105:0/3901965788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b54106f10 0x7f7b5410ea90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c009ca0 tx=0x7f7b4c02a8c0 comp rx=0 tx=0).stop 2024-09-16T02:00:46.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.968+0000 7f7b59c54640 1 -- 172.21.15.105:0/3901965788 shutdown_connections 2024-09-16T02:00:46.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.968+0000 7f7b59c54640 1 --2- 172.21.15.105:0/3901965788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b54106f10 0x7f7b5410ea90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:46.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.968+0000 7f7b59c54640 1 --2- 172.21.15.105:0/3901965788 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b54102790 0x7f7b541067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:46.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.968+0000 7f7b59c54640 1 --2- 172.21.15.105:0/3901965788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b54101de0 0x7f7b541021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:46.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.968+0000 7f7b59c54640 1 -- 172.21.15.105:0/3901965788 >> 172.21.15.105:0/3901965788 conn(0x7f7b540fd8c0 msgr2=0x7f7b540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:46.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.968+0000 7f7b59c54640 1 -- 172.21.15.105:0/3901965788 shutdown_connections 2024-09-16T02:00:46.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.968+0000 7f7b59c54640 1 -- 172.21.15.105:0/3901965788 wait complete. 2024-09-16T02:00:46.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.969+0000 7f7b59c54640 1 Processor -- start 2024-09-16T02:00:46.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.969+0000 7f7b59c54640 1 -- start start 2024-09-16T02:00:46.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.969+0000 7f7b59c54640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b54101de0 0x7f7b541982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:46.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.969+0000 7f7b59c54640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b54102790 0x7f7b541987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:46.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.969+0000 7f7b59c54640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b54106f10 0x7f7b5419cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:46.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.969+0000 7f7b59c54640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7b54111e20 con 0x7f7b54106f10 2024-09-16T02:00:46.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.969+0000 7f7b59c54640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7b54111ca0 con 0x7f7b54101de0 2024-09-16T02:00:46.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.969+0000 7f7b59c54640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7b54111fa0 con 0x7f7b54102790 2024-09-16T02:00:46.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.969+0000 7f7b53fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b54106f10 0x7f7b5419cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:46.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.969+0000 7f7b53fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b54106f10 0x7f7b5419cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43090/0 (socket says 172.21.15.105:43090) 2024-09-16T02:00:46.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.969+0000 7f7b53fff640 1 -- 172.21.15.105:0/2890623725 learned_addr learned my addr 172.21.15.105:0/2890623725 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:00:46.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.969+0000 7f7b52ffd640 1 --2- 172.21.15.105:0/2890623725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b54102790 0x7f7b541987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:46.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.969+0000 7f7b537fe640 1 --2- 172.21.15.105:0/2890623725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b54101de0 0x7f7b541982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:46.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.969+0000 7f7b53fff640 1 -- 172.21.15.105:0/2890623725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b54102790 msgr2=0x7f7b541987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:46.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.969+0000 7f7b53fff640 1 --2- 172.21.15.105:0/2890623725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b54102790 0x7f7b541987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:46.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.969+0000 7f7b53fff640 1 -- 172.21.15.105:0/2890623725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b54101de0 msgr2=0x7f7b541982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:46.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.969+0000 7f7b53fff640 1 --2- 172.21.15.105:0/2890623725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b54101de0 0x7f7b541982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:46.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.969+0000 7f7b53fff640 1 -- 172.21.15.105:0/2890623725 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b5419d250 con 0x7f7b54106f10 2024-09-16T02:00:46.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.969+0000 7f7b537fe640 1 --2- 172.21.15.105:0/2890623725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b54101de0 0x7f7b541982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:00:46.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.970+0000 7f7b53fff640 1 --2- 172.21.15.105:0/2890623725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b54106f10 0x7f7b5419cb70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c009dc0 tx=0x7f7b4c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:46.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.970+0000 7f7b50ff9640 1 -- 172.21.15.105:0/2890623725 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c004040 con 0x7f7b54106f10 2024-09-16T02:00:46.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.970+0000 7f7b50ff9640 1 -- 172.21.15.105:0/2890623725 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b4c004690 con 0x7f7b54106f10 2024-09-16T02:00:46.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.970+0000 7f7b59c54640 1 -- 172.21.15.105:0/2890623725 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b5419d4e0 con 0x7f7b54106f10 2024-09-16T02:00:46.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.970+0000 7f7b50ff9640 1 -- 172.21.15.105:0/2890623725 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c030c90 con 0x7f7b54106f10 2024-09-16T02:00:46.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.970+0000 7f7b59c54640 1 -- 172.21.15.105:0/2890623725 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b541a4e20 con 0x7f7b54106f10 2024-09-16T02:00:46.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.971+0000 7f7b59c54640 1 -- 172.21.15.105:0/2890623725 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b541021c0 con 0x7f7b54106f10 2024-09-16T02:00:46.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.972+0000 7f7b50ff9640 1 -- 172.21.15.105:0/2890623725 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f7b4c0041e0 con 0x7f7b54106f10 2024-09-16T02:00:46.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.973+0000 7f7b50ff9640 1 --2- 172.21.15.105:0/2890623725 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7b3c0a32c0 0x7f7b3c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:46.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.973+0000 7f7b537fe640 1 --2- 172.21.15.105:0/2890623725 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7b3c0a32c0 0x7f7b3c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:46.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.973+0000 7f7b50ff9640 1 -- 172.21.15.105:0/2890623725 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f7b4c0f0f30 con 0x7f7b54106f10 2024-09-16T02:00:46.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.973+0000 7f7b537fe640 1 --2- 172.21.15.105:0/2890623725 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7b3c0a32c0 0x7f7b3c0a5780 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7b440058c0 tx=0x7f7b44005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:46.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:46.977+0000 7f7b50ff9640 1 -- 172.21.15.105:0/2890623725 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7b4c0bdec0 con 0x7f7b54106f10 2024-09-16T02:00:47.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:47.177+0000 7f7b59c54640 1 -- 172.21.15.105:0/2890623725 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f7b54109bc0 con 0x7f7b54106f10 2024-09-16T02:00:47.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:47.522+0000 7f7b50ff9640 1 -- 172.21.15.105:0/2890623725 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7f7b4c0bd7c0 con 0x7f7b54106f10 2024-09-16T02:00:47.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:47.525+0000 7f7b59c54640 1 -- 172.21.15.105:0/2890623725 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7b3c0a32c0 msgr2=0x7f7b3c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:47.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:47.525+0000 7f7b59c54640 1 --2- 172.21.15.105:0/2890623725 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7b3c0a32c0 0x7f7b3c0a5780 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7b440058c0 tx=0x7f7b44005ec0 comp rx=0 tx=0).stop 2024-09-16T02:00:47.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:47.525+0000 7f7b59c54640 1 -- 172.21.15.105:0/2890623725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b54106f10 msgr2=0x7f7b5419cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:47.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:47.525+0000 7f7b59c54640 1 --2- 172.21.15.105:0/2890623725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b54106f10 0x7f7b5419cb70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c009dc0 tx=0x7f7b4c003ba0 comp rx=0 tx=0).stop 2024-09-16T02:00:47.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:47.525+0000 7f7b59c54640 1 -- 172.21.15.105:0/2890623725 shutdown_connections 2024-09-16T02:00:47.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:47.525+0000 7f7b59c54640 1 --2- 172.21.15.105:0/2890623725 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7b3c0a32c0 0x7f7b3c0a5780 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:47.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:47.525+0000 7f7b59c54640 1 --2- 172.21.15.105:0/2890623725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b54102790 0x7f7b541987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:47.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:47.525+0000 7f7b59c54640 1 --2- 172.21.15.105:0/2890623725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b54106f10 0x7f7b5419cb70 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:47.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:47.525+0000 7f7b59c54640 1 --2- 172.21.15.105:0/2890623725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b54101de0 0x7f7b541982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:47.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:47.525+0000 7f7b59c54640 1 -- 172.21.15.105:0/2890623725 >> 172.21.15.105:0/2890623725 conn(0x7f7b540fd8c0 msgr2=0x7f7b540feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:47.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:47.525+0000 7f7b59c54640 1 -- 172.21.15.105:0/2890623725 shutdown_connections 2024-09-16T02:00:47.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:47.525+0000 7f7b59c54640 1 -- 172.21.15.105:0/2890623725 wait complete. 2024-09-16T02:00:47.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:47 smithi105 ceph-mon[31197]: mds.? [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] up:active 2024-09-16T02:00:47.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:47 smithi105 ceph-mon[31197]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-16T02:00:47.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:47 smithi105 ceph-mon[31197]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-16T02:00:47.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:47 smithi105 ceph-mon[31197]: Cluster is now healthy 2024-09-16T02:00:47.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:47 smithi105 ceph-mon[31197]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-09-16T02:00:47.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:47 smithi105 ceph-mon[31197]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-16T02:00:47.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:47 smithi105 ceph-mon[31197]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-16T02:00:47.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:47 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2890623725' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-16T02:00:47.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:47 smithi190 ceph-mon[34762]: mds.? [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] up:active 2024-09-16T02:00:47.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:47 smithi190 ceph-mon[34762]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-16T02:00:47.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:47 smithi190 ceph-mon[34762]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-16T02:00:47.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:47 smithi190 ceph-mon[34762]: Cluster is now healthy 2024-09-16T02:00:47.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:47 smithi190 ceph-mon[34762]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-09-16T02:00:47.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:47 smithi190 ceph-mon[34762]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-16T02:00:47.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:47 smithi190 ceph-mon[34762]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-16T02:00:47.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:47 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/2890623725' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-16T02:00:47.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:47 smithi137 ceph-mon[34099]: mds.? [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] up:active 2024-09-16T02:00:47.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:47 smithi137 ceph-mon[34099]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-16T02:00:47.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:47 smithi137 ceph-mon[34099]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-16T02:00:47.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:47 smithi137 ceph-mon[34099]: Cluster is now healthy 2024-09-16T02:00:47.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:47 smithi137 ceph-mon[34099]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-09-16T02:00:47.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:47 smithi137 ceph-mon[34099]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-16T02:00:47.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:47 smithi137 ceph-mon[34099]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-16T02:00:47.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:47 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/2890623725' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-16T02:00:48.272 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-09-16T02:00:48.273 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-16T02:00:48.518 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:00:48.790 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:48 smithi190 ceph-mon[34762]: pgmap v283: 97 pgs: 97 active+clean; 580 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s wr, 9 op/s 2024-09-16T02:00:48.790 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:48 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/2890623725' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-16T02:00:48.790 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:48 smithi190 ceph-mon[34762]: mds.? [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] up:active 2024-09-16T02:00:48.790 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:48 smithi190 ceph-mon[34762]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-16T02:00:48.790 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:48 smithi190 ceph-mon[34762]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T02:00:48.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:48 smithi105 ceph-mon[31197]: pgmap v283: 97 pgs: 97 active+clean; 580 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s wr, 9 op/s 2024-09-16T02:00:48.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:48 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2890623725' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-16T02:00:48.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:48 smithi105 ceph-mon[31197]: mds.? [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] up:active 2024-09-16T02:00:48.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:48 smithi105 ceph-mon[31197]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-16T02:00:48.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:48 smithi105 ceph-mon[31197]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T02:00:48.822 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:48 smithi137 ceph-mon[34099]: pgmap v283: 97 pgs: 97 active+clean; 580 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s wr, 9 op/s 2024-09-16T02:00:48.822 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:48 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/2890623725' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-16T02:00:48.822 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:48 smithi137 ceph-mon[34099]: mds.? [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] up:active 2024-09-16T02:00:48.822 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:48 smithi137 ceph-mon[34099]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-16T02:00:48.822 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:48 smithi137 ceph-mon[34099]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T02:00:49.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.013+0000 7fc84fa21640 1 -- 172.21.15.105:0/2032868099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc848102790 msgr2=0x7fc8481067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:49.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.013+0000 7fc84fa21640 1 --2- 172.21.15.105:0/2032868099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc848102790 0x7fc8481067e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc838009900 tx=0x7fc83802a410 comp rx=0 tx=0).stop 2024-09-16T02:00:49.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.013+0000 7fc84fa21640 1 -- 172.21.15.105:0/2032868099 shutdown_connections 2024-09-16T02:00:49.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.013+0000 7fc84fa21640 1 --2- 172.21.15.105:0/2032868099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc848106f10 0x7fc84810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:49.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.013+0000 7fc84fa21640 1 --2- 172.21.15.105:0/2032868099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc848102790 0x7fc8481067e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:49.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.013+0000 7fc84fa21640 1 --2- 172.21.15.105:0/2032868099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc848101de0 0x7fc8481021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:49.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.013+0000 7fc84fa21640 1 -- 172.21.15.105:0/2032868099 >> 172.21.15.105:0/2032868099 conn(0x7fc8480fd8c0 msgr2=0x7fc8480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:49.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.013+0000 7fc84fa21640 1 -- 172.21.15.105:0/2032868099 shutdown_connections 2024-09-16T02:00:49.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.013+0000 7fc84fa21640 1 -- 172.21.15.105:0/2032868099 wait complete. 2024-09-16T02:00:49.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.014+0000 7fc84fa21640 1 Processor -- start 2024-09-16T02:00:49.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.014+0000 7fc84fa21640 1 -- start start 2024-09-16T02:00:49.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.014+0000 7fc84fa21640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc848101de0 0x7fc8480744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:49.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.014+0000 7fc84fa21640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc848102790 0x7fc848074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:49.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.014+0000 7fc84fa21640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc848106f10 0x7fc848074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:49.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.014+0000 7fc84fa21640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc848111e60 con 0x7fc848106f10 2024-09-16T02:00:49.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.014+0000 7fc84fa21640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc848111ce0 con 0x7fc848102790 2024-09-16T02:00:49.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.014+0000 7fc84fa21640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc848111fe0 con 0x7fc848101de0 2024-09-16T02:00:49.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.014+0000 7fc84dfbe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc848106f10 0x7fc848074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:49.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.014+0000 7fc84dfbe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc848106f10 0x7fc848074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58322/0 (socket says 172.21.15.105:58322) 2024-09-16T02:00:49.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.014+0000 7fc84dfbe640 1 -- 172.21.15.105:0/3512558375 learned_addr learned my addr 172.21.15.105:0/3512558375 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:00:49.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.014+0000 7fc84d7bd640 1 --2- 172.21.15.105:0/3512558375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc848101de0 0x7fc8480744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:49.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.014+0000 7fc84cfbc640 1 --2- 172.21.15.105:0/3512558375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc848102790 0x7fc848074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:49.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.014+0000 7fc84dfbe640 1 -- 172.21.15.105:0/3512558375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc848101de0 msgr2=0x7fc8480744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:49.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.014+0000 7fc84dfbe640 1 --2- 172.21.15.105:0/3512558375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc848101de0 0x7fc8480744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:49.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.014+0000 7fc84dfbe640 1 -- 172.21.15.105:0/3512558375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc848102790 msgr2=0x7fc848074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:49.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.014+0000 7fc84dfbe640 1 --2- 172.21.15.105:0/3512558375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc848102790 0x7fc848074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:49.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.014+0000 7fc84dfbe640 1 -- 172.21.15.105:0/3512558375 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8481a68a0 con 0x7fc848106f10 2024-09-16T02:00:49.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.014+0000 7fc84dfbe640 1 --2- 172.21.15.105:0/3512558375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc848106f10 0x7fc848074f70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc84000eaa0 tx=0x7fc84000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:49.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.015+0000 7fc83e7fc640 1 -- 172.21.15.105:0/3512558375 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc840004d20 con 0x7fc848106f10 2024-09-16T02:00:49.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.015+0000 7fc83e7fc640 1 -- 172.21.15.105:0/3512558375 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc840005020 con 0x7fc848106f10 2024-09-16T02:00:49.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.015+0000 7fc84fa21640 1 -- 172.21.15.105:0/3512558375 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8481a6b90 con 0x7fc848106f10 2024-09-16T02:00:49.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.015+0000 7fc83e7fc640 1 -- 172.21.15.105:0/3512558375 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8400116b0 con 0x7fc848106f10 2024-09-16T02:00:49.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.015+0000 7fc84fa21640 1 -- 172.21.15.105:0/3512558375 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8481a70a0 con 0x7fc848106f10 2024-09-16T02:00:49.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.016+0000 7fc84fa21640 1 -- 172.21.15.105:0/3512558375 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8481021c0 con 0x7fc848106f10 2024-09-16T02:00:49.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.017+0000 7fc83e7fc640 1 -- 172.21.15.105:0/3512558375 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fc8400051d0 con 0x7fc848106f10 2024-09-16T02:00:49.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.017+0000 7fc83e7fc640 1 --2- 172.21.15.105:0/3512558375 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc8340a3240 0x7fc8340a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:49.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.017+0000 7fc84d7bd640 1 --2- 172.21.15.105:0/3512558375 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc8340a3240 0x7fc8340a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:49.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.017+0000 7fc83e7fc640 1 -- 172.21.15.105:0/3512558375 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fc8400cf9f0 con 0x7fc848106f10 2024-09-16T02:00:49.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.018+0000 7fc84d7bd640 1 --2- 172.21.15.105:0/3512558375 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc8340a3240 0x7fc8340a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc844009990 tx=0x7fc844009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:49.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.021+0000 7fc83e7fc640 1 -- 172.21.15.105:0/3512558375 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc840016030 con 0x7fc848106f10 2024-09-16T02:00:49.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.210+0000 7fc84fa21640 1 -- 172.21.15.105:0/3512558375 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc84810a160 con 0x7fc8340a3240 2024-09-16T02:00:49.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.232+0000 7fc83e7fc640 1 -- 172.21.15.105:0/3512558375 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc8440095e0 con 0x7fc8340a3240 2024-09-16T02:00:49.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.234+0000 7fc84fa21640 1 -- 172.21.15.105:0/3512558375 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc8340a3240 msgr2=0x7fc8340a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:49.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.234+0000 7fc84fa21640 1 --2- 172.21.15.105:0/3512558375 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc8340a3240 0x7fc8340a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc844009990 tx=0x7fc844009110 comp rx=0 tx=0).stop 2024-09-16T02:00:49.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.234+0000 7fc84fa21640 1 -- 172.21.15.105:0/3512558375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc848106f10 msgr2=0x7fc848074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:49.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.234+0000 7fc84fa21640 1 --2- 172.21.15.105:0/3512558375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc848106f10 0x7fc848074f70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc84000eaa0 tx=0x7fc84000eea0 comp rx=0 tx=0).stop 2024-09-16T02:00:49.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.234+0000 7fc84fa21640 1 -- 172.21.15.105:0/3512558375 shutdown_connections 2024-09-16T02:00:49.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.234+0000 7fc84fa21640 1 --2- 172.21.15.105:0/3512558375 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc8340a3240 0x7fc8340a5700 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:49.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.234+0000 7fc84fa21640 1 --2- 172.21.15.105:0/3512558375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc848106f10 0x7fc848074f70 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:49.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.234+0000 7fc84fa21640 1 --2- 172.21.15.105:0/3512558375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc848102790 0x7fc848074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:49.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.234+0000 7fc84fa21640 1 --2- 172.21.15.105:0/3512558375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc848101de0 0x7fc8480744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:49.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.234+0000 7fc84fa21640 1 -- 172.21.15.105:0/3512558375 >> 172.21.15.105:0/3512558375 conn(0x7fc8480fd8c0 msgr2=0x7fc8480fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:49.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.235+0000 7fc84fa21640 1 -- 172.21.15.105:0/3512558375 shutdown_connections 2024-09-16T02:00:49.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:49.235+0000 7fc84fa21640 1 -- 172.21.15.105:0/3512558375 wait complete. 2024-09-16T02:00:49.545 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 02:00:49 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T02:00:49.214+0000 7f21ed775640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T02:00:49.545 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 02:00:49 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T02:00:49.214+0000 7f21ed775640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T02:00:49.545 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 02:00:49 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T02:00:49.214+0000 7f21ed775640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T02:00:49.545 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 02:00:49 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T02:00:49.214+0000 7f21ed775640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T02:00:49.545 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 02:00:49 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T02:00:49.214+0000 7f21ed775640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T02:00:49.938 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-16T02:00:50.180 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:00:50.641 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:50 smithi105 ceph-mon[31197]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:00:50.641 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:50 smithi105 ceph-mon[31197]: pgmap v284: 97 pgs: 97 active+clean; 582 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 3.9 KiB/s wr, 14 op/s 2024-09-16T02:00:50.641 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:50 smithi105 ceph-mon[31197]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-16T02:00:50.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.724+0000 7ff827d52640 1 -- 172.21.15.105:0/1660594922 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff820103660 msgr2=0x7ff82010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:50.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.724+0000 7ff827d52640 1 --2- 172.21.15.105:0/1660594922 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff820103660 0x7ff82010bb50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff814009900 tx=0x7ff81402a480 comp rx=0 tx=0).stop 2024-09-16T02:00:50.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.724+0000 7ff827d52640 1 -- 172.21.15.105:0/1660594922 shutdown_connections 2024-09-16T02:00:50.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.724+0000 7ff827d52640 1 --2- 172.21.15.105:0/1660594922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff82010c170 0x7ff82010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:50.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.724+0000 7ff827d52640 1 --2- 172.21.15.105:0/1660594922 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff820103660 0x7ff82010bb50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:50.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.724+0000 7ff827d52640 1 --2- 172.21.15.105:0/1660594922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff820102d40 0x7ff820103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:50.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.724+0000 7ff827d52640 1 -- 172.21.15.105:0/1660594922 >> 172.21.15.105:0/1660594922 conn(0x7ff8200fc7a0 msgr2=0x7ff8200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:50.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.724+0000 7ff827d52640 1 -- 172.21.15.105:0/1660594922 shutdown_connections 2024-09-16T02:00:50.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.724+0000 7ff827d52640 1 -- 172.21.15.105:0/1660594922 wait complete. 2024-09-16T02:00:50.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.724+0000 7ff827d52640 1 Processor -- start 2024-09-16T02:00:50.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.724+0000 7ff827d52640 1 -- start start 2024-09-16T02:00:50.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.725+0000 7ff827d52640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff820102d40 0x7ff820198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:50.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.725+0000 7ff827d52640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff820103660 0x7ff820198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:50.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.725+0000 7ff827d52640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff82010c170 0x7ff82019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:50.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.725+0000 7ff827d52640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff820111e60 con 0x7ff820103660 2024-09-16T02:00:50.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.725+0000 7ff827d52640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff820111ce0 con 0x7ff820102d40 2024-09-16T02:00:50.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.725+0000 7ff827d52640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff820111fe0 con 0x7ff82010c170 2024-09-16T02:00:50.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.725+0000 7ff8252ed640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff820103660 0x7ff820198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:50.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.725+0000 7ff8252ed640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff820103660 0x7ff820198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58356/0 (socket says 172.21.15.105:58356) 2024-09-16T02:00:50.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.725+0000 7ff8252ed640 1 -- 172.21.15.105:0/2781462612 learned_addr learned my addr 172.21.15.105:0/2781462612 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:00:50.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.725+0000 7ff825aee640 1 --2- 172.21.15.105:0/2781462612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff820102d40 0x7ff820198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:50.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.725+0000 7ff8262ef640 1 --2- 172.21.15.105:0/2781462612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff82010c170 0x7ff82019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:50.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.725+0000 7ff8252ed640 1 -- 172.21.15.105:0/2781462612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff82010c170 msgr2=0x7ff82019ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:50.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.725+0000 7ff8252ed640 1 --2- 172.21.15.105:0/2781462612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff82010c170 0x7ff82019ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:50.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.725+0000 7ff8252ed640 1 -- 172.21.15.105:0/2781462612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff820102d40 msgr2=0x7ff820198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:50.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.725+0000 7ff8252ed640 1 --2- 172.21.15.105:0/2781462612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff820102d40 0x7ff820198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:50.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.725+0000 7ff8252ed640 1 -- 172.21.15.105:0/2781462612 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff82019d3b0 con 0x7ff820103660 2024-09-16T02:00:50.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.725+0000 7ff8252ed640 1 --2- 172.21.15.105:0/2781462612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff820103660 0x7ff820198940 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff814005940 tx=0x7ff8140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:50.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.725+0000 7ff81effd640 1 -- 172.21.15.105:0/2781462612 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81403f030 con 0x7ff820103660 2024-09-16T02:00:50.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.725+0000 7ff81effd640 1 -- 172.21.15.105:0/2781462612 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff814003cb0 con 0x7ff820103660 2024-09-16T02:00:50.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.725+0000 7ff81effd640 1 -- 172.21.15.105:0/2781462612 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814035070 con 0x7ff820103660 2024-09-16T02:00:50.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.725+0000 7ff827d52640 1 -- 172.21.15.105:0/2781462612 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff82019d640 con 0x7ff820103660 2024-09-16T02:00:50.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.725+0000 7ff827d52640 1 -- 172.21.15.105:0/2781462612 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8201a4f80 con 0x7ff820103660 2024-09-16T02:00:50.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.726+0000 7ff827d52640 1 -- 172.21.15.105:0/2781462612 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff82006b070 con 0x7ff820103660 2024-09-16T02:00:50.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.727+0000 7ff81effd640 1 -- 172.21.15.105:0/2781462612 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7ff814042030 con 0x7ff820103660 2024-09-16T02:00:50.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.728+0000 7ff81effd640 1 --2- 172.21.15.105:0/2781462612 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff8080a32d0 0x7ff8080a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:50.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.728+0000 7ff825aee640 1 --2- 172.21.15.105:0/2781462612 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff8080a32d0 0x7ff8080a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:50.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.728+0000 7ff81effd640 1 -- 172.21.15.105:0/2781462612 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7ff8140f0110 con 0x7ff820103660 2024-09-16T02:00:50.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.728+0000 7ff825aee640 1 --2- 172.21.15.105:0/2781462612 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff8080a32d0 0x7ff8080a5790 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff81000a400 tx=0x7ff81000d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:50.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.732+0000 7ff81effd640 1 -- 172.21.15.105:0/2781462612 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff8140bce80 con 0x7ff820103660 2024-09-16T02:00:50.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:50 smithi190 ceph-mon[34762]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:00:50.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:50 smithi190 ceph-mon[34762]: pgmap v284: 97 pgs: 97 active+clean; 582 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 3.9 KiB/s wr, 14 op/s 2024-09-16T02:00:50.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:50 smithi190 ceph-mon[34762]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-16T02:00:50.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:50 smithi137 ceph-mon[34099]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:00:50.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:50 smithi137 ceph-mon[34099]: pgmap v284: 97 pgs: 97 active+clean; 582 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 3.9 KiB/s wr, 14 op/s 2024-09-16T02:00:50.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:50 smithi137 ceph-mon[34099]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-16T02:00:50.921 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 16 02:00:50 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-b[64153]: 2024-09-16T02:00:50.584+0000 7fa83671b640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-09-16T02:00:50.921 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 16 02:00:50 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-b[64153]: 2024-09-16T02:00:50.584+0000 7fa83671b640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-09-16T02:00:50.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.922+0000 7ff827d52640 1 -- 172.21.15.105:0/2781462612 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7ff820062630 con 0x7ff8080a32d0 2024-09-16T02:00:50.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.944+0000 7ff81effd640 1 -- 172.21.15.105:0/2781462612 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff820062630 con 0x7ff8080a32d0 2024-09-16T02:00:50.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.946+0000 7ff827d52640 1 -- 172.21.15.105:0/2781462612 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff8080a32d0 msgr2=0x7ff8080a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:50.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.946+0000 7ff827d52640 1 --2- 172.21.15.105:0/2781462612 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff8080a32d0 0x7ff8080a5790 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff81000a400 tx=0x7ff81000d040 comp rx=0 tx=0).stop 2024-09-16T02:00:50.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.946+0000 7ff827d52640 1 -- 172.21.15.105:0/2781462612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff820103660 msgr2=0x7ff820198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:50.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.946+0000 7ff827d52640 1 --2- 172.21.15.105:0/2781462612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff820103660 0x7ff820198940 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff814005940 tx=0x7ff8140042f0 comp rx=0 tx=0).stop 2024-09-16T02:00:50.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.946+0000 7ff827d52640 1 -- 172.21.15.105:0/2781462612 shutdown_connections 2024-09-16T02:00:50.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.946+0000 7ff827d52640 1 --2- 172.21.15.105:0/2781462612 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff8080a32d0 0x7ff8080a5790 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:50.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.946+0000 7ff827d52640 1 --2- 172.21.15.105:0/2781462612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff82010c170 0x7ff82019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:50.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.946+0000 7ff827d52640 1 --2- 172.21.15.105:0/2781462612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff820103660 0x7ff820198940 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:50.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.946+0000 7ff827d52640 1 --2- 172.21.15.105:0/2781462612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff820102d40 0x7ff820198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:50.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.946+0000 7ff827d52640 1 -- 172.21.15.105:0/2781462612 >> 172.21.15.105:0/2781462612 conn(0x7ff8200fc7a0 msgr2=0x7ff8200fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:50.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.946+0000 7ff827d52640 1 -- 172.21.15.105:0/2781462612 shutdown_connections 2024-09-16T02:00:50.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:50.947+0000 7ff827d52640 1 -- 172.21.15.105:0/2781462612 wait complete. 2024-09-16T02:00:51.778 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-16T02:00:52.025 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:00:52.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.516+0000 7fb5f8d0d640 1 -- 172.21.15.105:0/435608247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5f4106a80 msgr2=0x7fb5f4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:52.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.516+0000 7fb5f8d0d640 1 --2- 172.21.15.105:0/435608247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5f4106a80 0x7fb5f4106e60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4009930 tx=0x7fb5e402a420 comp rx=0 tx=0).stop 2024-09-16T02:00:52.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.516+0000 7fb5f8d0d640 1 -- 172.21.15.105:0/435608247 shutdown_connections 2024-09-16T02:00:52.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.516+0000 7fb5f8d0d640 1 --2- 172.21.15.105:0/435608247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5f4101420 0x7fb5f410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:52.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.516+0000 7fb5f8d0d640 1 --2- 172.21.15.105:0/435608247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5f4100a80 0x7fb5f4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:52.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.516+0000 7fb5f8d0d640 1 --2- 172.21.15.105:0/435608247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5f4106a80 0x7fb5f4106e60 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:52.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.516+0000 7fb5f8d0d640 1 -- 172.21.15.105:0/435608247 >> 172.21.15.105:0/435608247 conn(0x7fb5f40fc7a0 msgr2=0x7fb5f40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:52.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.516+0000 7fb5f8d0d640 1 -- 172.21.15.105:0/435608247 shutdown_connections 2024-09-16T02:00:52.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.516+0000 7fb5f8d0d640 1 -- 172.21.15.105:0/435608247 wait complete. 2024-09-16T02:00:52.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.517+0000 7fb5f8d0d640 1 Processor -- start 2024-09-16T02:00:52.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.517+0000 7fb5f8d0d640 1 -- start start 2024-09-16T02:00:52.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.517+0000 7fb5f8d0d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5f4100a80 0x7fb5f41972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:52.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.517+0000 7fb5f8d0d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5f4101420 0x7fb5f4197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:52.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.517+0000 7fb5f8d0d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5f4106a80 0x7fb5f419bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:52.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.517+0000 7fb5f8d0d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f4110d30 con 0x7fb5f4106a80 2024-09-16T02:00:52.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.517+0000 7fb5f8d0d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f4110bb0 con 0x7fb5f4100a80 2024-09-16T02:00:52.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.517+0000 7fb5f8d0d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f4110eb0 con 0x7fb5f4101420 2024-09-16T02:00:52.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.517+0000 7fb5f2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5f4106a80 0x7fb5f419bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:52.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.517+0000 7fb5f2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5f4106a80 0x7fb5f419bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58380/0 (socket says 172.21.15.105:58380) 2024-09-16T02:00:52.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.517+0000 7fb5f2d9d640 1 -- 172.21.15.105:0/2200742782 learned_addr learned my addr 172.21.15.105:0/2200742782 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:00:52.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.517+0000 7fb5f259c640 1 --2- 172.21.15.105:0/2200742782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5f4100a80 0x7fb5f41972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:52.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.517+0000 7fb5ebfff640 1 --2- 172.21.15.105:0/2200742782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5f4101420 0x7fb5f4197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:52.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.517+0000 7fb5f2d9d640 1 -- 172.21.15.105:0/2200742782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5f4101420 msgr2=0x7fb5f4197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:52.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.517+0000 7fb5f2d9d640 1 --2- 172.21.15.105:0/2200742782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5f4101420 0x7fb5f4197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:52.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.517+0000 7fb5f2d9d640 1 -- 172.21.15.105:0/2200742782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5f4100a80 msgr2=0x7fb5f41972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:52.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.517+0000 7fb5f2d9d640 1 --2- 172.21.15.105:0/2200742782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5f4100a80 0x7fb5f41972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:52.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.517+0000 7fb5f2d9d640 1 -- 172.21.15.105:0/2200742782 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f419c280 con 0x7fb5f4106a80 2024-09-16T02:00:52.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.518+0000 7fb5f2d9d640 1 --2- 172.21.15.105:0/2200742782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5f4106a80 0x7fb5f419bba0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00cad0 tx=0x7fb5ec00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:52.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.518+0000 7fb5eb7fe640 1 -- 172.21.15.105:0/2200742782 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec00ae20 con 0x7fb5f4106a80 2024-09-16T02:00:52.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.518+0000 7fb5eb7fe640 1 -- 172.21.15.105:0/2200742782 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5ec005020 con 0x7fb5f4106a80 2024-09-16T02:00:52.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.518+0000 7fb5f8d0d640 1 -- 172.21.15.105:0/2200742782 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5f419c570 con 0x7fb5f4106a80 2024-09-16T02:00:52.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.518+0000 7fb5eb7fe640 1 -- 172.21.15.105:0/2200742782 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec00f6b0 con 0x7fb5f4106a80 2024-09-16T02:00:52.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.518+0000 7fb5f8d0d640 1 -- 172.21.15.105:0/2200742782 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5f41a3e50 con 0x7fb5f4106a80 2024-09-16T02:00:52.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.519+0000 7fb5f8d0d640 1 -- 172.21.15.105:0/2200742782 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5f4100ee0 con 0x7fb5f4106a80 2024-09-16T02:00:52.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.521+0000 7fb5eb7fe640 1 -- 172.21.15.105:0/2200742782 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fb5ec0051d0 con 0x7fb5f4106a80 2024-09-16T02:00:52.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.521+0000 7fb5eb7fe640 1 --2- 172.21.15.105:0/2200742782 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb5e00a32d0 0x7fb5e00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:52.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.521+0000 7fb5f259c640 1 --2- 172.21.15.105:0/2200742782 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb5e00a32d0 0x7fb5e00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:52.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.521+0000 7fb5eb7fe640 1 -- 172.21.15.105:0/2200742782 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fb5ec0cfc60 con 0x7fb5f4106a80 2024-09-16T02:00:52.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.522+0000 7fb5f259c640 1 --2- 172.21.15.105:0/2200742782 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb5e00a32d0 0x7fb5e00a5790 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4009900 tx=0x7fb5e4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:52.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.525+0000 7fb5eb7fe640 1 -- 172.21.15.105:0/2200742782 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb5ec014030 con 0x7fb5f4106a80 2024-09-16T02:00:52.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.715+0000 7fb5f8d0d640 1 -- 172.21.15.105:0/2200742782 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fb5f4062630 con 0x7fb5e00a32d0 2024-09-16T02:00:52.718 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:52 smithi105 ceph-mon[31197]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:00:52.718 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:52 smithi105 ceph-mon[31197]: pgmap v285: 97 pgs: 97 active+clean; 585 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 4.2 KiB/s wr, 20 op/s 2024-09-16T02:00:52.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.721+0000 7fb5eb7fe640 1 -- 172.21.15.105:0/2200742782 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fb5f4062630 con 0x7fb5e00a32d0 2024-09-16T02:00:52.723 INFO:teuthology.orchestra.run.smithi105.stdout:/volumes/qa/sv_0/d0aa8553-5106-4f12-b555-f9d49c86e9ca 2024-09-16T02:00:52.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.723+0000 7fb5f8d0d640 1 -- 172.21.15.105:0/2200742782 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb5e00a32d0 msgr2=0x7fb5e00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:52.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.723+0000 7fb5f8d0d640 1 --2- 172.21.15.105:0/2200742782 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb5e00a32d0 0x7fb5e00a5790 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4009900 tx=0x7fb5e4000bc0 comp rx=0 tx=0).stop 2024-09-16T02:00:52.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.723+0000 7fb5f8d0d640 1 -- 172.21.15.105:0/2200742782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5f4106a80 msgr2=0x7fb5f419bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:52.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.723+0000 7fb5f8d0d640 1 --2- 172.21.15.105:0/2200742782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5f4106a80 0x7fb5f419bba0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00cad0 tx=0x7fb5ec00cf90 comp rx=0 tx=0).stop 2024-09-16T02:00:52.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.723+0000 7fb5f8d0d640 1 -- 172.21.15.105:0/2200742782 shutdown_connections 2024-09-16T02:00:52.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.723+0000 7fb5f8d0d640 1 --2- 172.21.15.105:0/2200742782 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb5e00a32d0 0x7fb5e00a5790 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:52.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.723+0000 7fb5f8d0d640 1 --2- 172.21.15.105:0/2200742782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5f4101420 0x7fb5f4197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:52.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.723+0000 7fb5f8d0d640 1 --2- 172.21.15.105:0/2200742782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5f4106a80 0x7fb5f419bba0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:52.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.723+0000 7fb5f8d0d640 1 --2- 172.21.15.105:0/2200742782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5f4100a80 0x7fb5f41972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:52.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.723+0000 7fb5f8d0d640 1 -- 172.21.15.105:0/2200742782 >> 172.21.15.105:0/2200742782 conn(0x7fb5f40fc7a0 msgr2=0x7fb5f40fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:52.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.723+0000 7fb5f8d0d640 1 -- 172.21.15.105:0/2200742782 shutdown_connections 2024-09-16T02:00:52.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:52.723+0000 7fb5f8d0d640 1 -- 172.21.15.105:0/2200742782 wait complete. 2024-09-16T02:00:52.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:52 smithi190 ceph-mon[34762]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:00:52.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:52 smithi190 ceph-mon[34762]: pgmap v285: 97 pgs: 97 active+clean; 585 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 4.2 KiB/s wr, 20 op/s 2024-09-16T02:00:52.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:52 smithi137 ceph-mon[34099]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:00:52.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:52 smithi137 ceph-mon[34099]: pgmap v285: 97 pgs: 97 active+clean; 585 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 4.2 KiB/s wr, 20 op/s 2024-09-16T02:00:53.543 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-16T02:00:53.783 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:00:54.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.288+0000 7f54412e2640 1 -- 172.21.15.105:0/29345919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f543c1082a0 msgr2=0x7f543c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:54.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.288+0000 7f54412e2640 1 --2- 172.21.15.105:0/29345919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f543c1082a0 0x7f543c10ec40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5434009cd0 tx=0x7f543402a8e0 comp rx=0 tx=0).stop 2024-09-16T02:00:54.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.288+0000 7f54412e2640 1 -- 172.21.15.105:0/29345919 shutdown_connections 2024-09-16T02:00:54.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.288+0000 7f54412e2640 1 --2- 172.21.15.105:0/29345919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f543c1082a0 0x7f543c10ec40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:54.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.288+0000 7f54412e2640 1 --2- 172.21.15.105:0/29345919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f543c1036e0 0x7f543c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:54.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.289+0000 7f54412e2640 1 --2- 172.21.15.105:0/29345919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f543c102d30 0x7f543c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:54.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.289+0000 7f54412e2640 1 -- 172.21.15.105:0/29345919 >> 172.21.15.105:0/29345919 conn(0x7f543c0fc7d0 msgr2=0x7f543c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:54.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.289+0000 7f54412e2640 1 -- 172.21.15.105:0/29345919 shutdown_connections 2024-09-16T02:00:54.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.289+0000 7f54412e2640 1 -- 172.21.15.105:0/29345919 wait complete. 2024-09-16T02:00:54.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.289+0000 7f54412e2640 1 Processor -- start 2024-09-16T02:00:54.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.289+0000 7f54412e2640 1 -- start start 2024-09-16T02:00:54.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.290+0000 7f54412e2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f543c102d30 0x7f543c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:54.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.290+0000 7f54412e2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f543c1036e0 0x7f543c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:54.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.290+0000 7f54412e2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f543c1082a0 0x7f543c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:54.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.290+0000 7f54412e2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f543c111fb0 con 0x7f543c1082a0 2024-09-16T02:00:54.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.290+0000 7f54412e2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f543c111e30 con 0x7f543c1036e0 2024-09-16T02:00:54.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.290+0000 7f54412e2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f543c112130 con 0x7f543c102d30 2024-09-16T02:00:54.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.290+0000 7f543b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f543c1082a0 0x7f543c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:54.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.290+0000 7f543b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f543c1082a0 0x7f543c19cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58406/0 (socket says 172.21.15.105:58406) 2024-09-16T02:00:54.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.290+0000 7f543b7fe640 1 -- 172.21.15.105:0/1676446970 learned_addr learned my addr 172.21.15.105:0/1676446970 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:00:54.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.290+0000 7f543affd640 1 --2- 172.21.15.105:0/1676446970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f543c102d30 0x7f543c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:54.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.290+0000 7f54327fc640 1 --2- 172.21.15.105:0/1676446970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f543c1036e0 0x7f543c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:54.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.290+0000 7f543b7fe640 1 -- 172.21.15.105:0/1676446970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f543c102d30 msgr2=0x7f543c198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:54.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.290+0000 7f543b7fe640 1 --2- 172.21.15.105:0/1676446970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f543c102d30 0x7f543c198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:54.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.290+0000 7f543b7fe640 1 -- 172.21.15.105:0/1676446970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f543c1036e0 msgr2=0x7f543c1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:54.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.290+0000 7f543b7fe640 1 --2- 172.21.15.105:0/1676446970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f543c1036e0 0x7f543c1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:54.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.290+0000 7f543b7fe640 1 -- 172.21.15.105:0/1676446970 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f543c19d420 con 0x7f543c1082a0 2024-09-16T02:00:54.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.290+0000 7f543affd640 1 --2- 172.21.15.105:0/1676446970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f543c102d30 0x7f543c198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:00:54.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.290+0000 7f543b7fe640 1 --2- 172.21.15.105:0/1676446970 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f543c1082a0 0x7f543c19cd40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5434005cc0 tx=0x7f5434003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:54.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.290+0000 7f5438ff9640 1 -- 172.21.15.105:0/1676446970 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5434004240 con 0x7f543c1082a0 2024-09-16T02:00:54.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.290+0000 7f5438ff9640 1 -- 172.21.15.105:0/1676446970 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54340305a0 con 0x7f543c1082a0 2024-09-16T02:00:54.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.290+0000 7f54412e2640 1 -- 172.21.15.105:0/1676446970 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f543c19d6b0 con 0x7f543c1082a0 2024-09-16T02:00:54.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.290+0000 7f5438ff9640 1 -- 172.21.15.105:0/1676446970 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54340466c0 con 0x7f543c1082a0 2024-09-16T02:00:54.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.291+0000 7f54412e2640 1 -- 172.21.15.105:0/1676446970 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f543c1a4ff0 con 0x7f543c1082a0 2024-09-16T02:00:54.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.292+0000 7f54412e2640 1 -- 172.21.15.105:0/1676446970 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f543c103110 con 0x7f543c1082a0 2024-09-16T02:00:54.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.293+0000 7f5438ff9640 1 -- 172.21.15.105:0/1676446970 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f5434030ac0 con 0x7f543c1082a0 2024-09-16T02:00:54.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.293+0000 7f5438ff9640 1 --2- 172.21.15.105:0/1676446970 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f54280a32d0 0x7f54280a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:54.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.293+0000 7f543affd640 1 --2- 172.21.15.105:0/1676446970 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f54280a32d0 0x7f54280a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:54.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.293+0000 7f5438ff9640 1 -- 172.21.15.105:0/1676446970 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f543402e030 con 0x7f543c1082a0 2024-09-16T02:00:54.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.294+0000 7f543affd640 1 --2- 172.21.15.105:0/1676446970 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f54280a32d0 0x7f54280a5790 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f542c009930 tx=0x7f542c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:54.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.296+0000 7f5438ff9640 1 -- 172.21.15.105:0/1676446970 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f543404b060 con 0x7f543c1082a0 2024-09-16T02:00:54.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.487+0000 7f54412e2640 1 -- 172.21.15.105:0/1676446970 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f543c062630 con 0x7f54280a32d0 2024-09-16T02:00:54.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.512+0000 7f5438ff9640 1 -- 172.21.15.105:0/1676446970 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f543c062630 con 0x7f54280a32d0 2024-09-16T02:00:54.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.514+0000 7f54412e2640 1 -- 172.21.15.105:0/1676446970 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f54280a32d0 msgr2=0x7f54280a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:54.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.514+0000 7f54412e2640 1 --2- 172.21.15.105:0/1676446970 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f54280a32d0 0x7f54280a5790 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f542c009930 tx=0x7f542c008040 comp rx=0 tx=0).stop 2024-09-16T02:00:54.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.514+0000 7f54412e2640 1 -- 172.21.15.105:0/1676446970 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f543c1082a0 msgr2=0x7f543c19cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:54.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.514+0000 7f54412e2640 1 --2- 172.21.15.105:0/1676446970 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f543c1082a0 0x7f543c19cd40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5434005cc0 tx=0x7f5434003ba0 comp rx=0 tx=0).stop 2024-09-16T02:00:54.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.514+0000 7f54412e2640 1 -- 172.21.15.105:0/1676446970 shutdown_connections 2024-09-16T02:00:54.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.514+0000 7f54412e2640 1 --2- 172.21.15.105:0/1676446970 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f54280a32d0 0x7f54280a5790 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:54.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.514+0000 7f54412e2640 1 --2- 172.21.15.105:0/1676446970 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f543c1082a0 0x7f543c19cd40 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:54.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.514+0000 7f54412e2640 1 --2- 172.21.15.105:0/1676446970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f543c1036e0 0x7f543c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:54.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.514+0000 7f54412e2640 1 --2- 172.21.15.105:0/1676446970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f543c102d30 0x7f543c198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:54.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.514+0000 7f54412e2640 1 -- 172.21.15.105:0/1676446970 >> 172.21.15.105:0/1676446970 conn(0x7f543c0fc7d0 msgr2=0x7f543c105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:54.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.514+0000 7f54412e2640 1 -- 172.21.15.105:0/1676446970 shutdown_connections 2024-09-16T02:00:54.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:54.514+0000 7f54412e2640 1 -- 172.21.15.105:0/1676446970 wait complete. 2024-09-16T02:00:54.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:54 smithi105 ceph-mon[31197]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:00:54.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:54 smithi105 ceph-mon[31197]: pgmap v286: 97 pgs: 97 active+clean; 605 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.2 KiB/s wr, 20 op/s 2024-09-16T02:00:54.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:54 smithi190 ceph-mon[34762]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:00:54.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:54 smithi190 ceph-mon[34762]: pgmap v286: 97 pgs: 97 active+clean; 605 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.2 KiB/s wr, 20 op/s 2024-09-16T02:00:54.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:54 smithi137 ceph-mon[34099]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:00:54.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:54 smithi137 ceph-mon[34099]: pgmap v286: 97 pgs: 97 active+clean; 605 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.2 KiB/s wr, 20 op/s 2024-09-16T02:00:55.194 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-16T02:00:55.440 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:00:55.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:55 smithi105 ceph-mon[31197]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:00:55.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:55 smithi190 ceph-mon[34762]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:00:55.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:55 smithi137 ceph-mon[34099]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:00:55.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.919+0000 7f43b6518640 1 -- 172.21.15.105:0/2760189832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43b0102d30 msgr2=0x7f43b0103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:55.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.919+0000 7f43b6518640 1 --2- 172.21.15.105:0/2760189832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43b0102d30 0x7f43b0103110 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f43a0009900 tx=0x7f43a002a480 comp rx=0 tx=0).stop 2024-09-16T02:00:55.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.919+0000 7f43b6518640 1 -- 172.21.15.105:0/2760189832 shutdown_connections 2024-09-16T02:00:55.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.919+0000 7f43b6518640 1 --2- 172.21.15.105:0/2760189832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43b010c160 0x7f43b010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:55.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.919+0000 7f43b6518640 1 --2- 172.21.15.105:0/2760189832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43b0103650 0x7f43b010bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:55.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.919+0000 7f43b6518640 1 --2- 172.21.15.105:0/2760189832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43b0102d30 0x7f43b0103110 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:55.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.920+0000 7f43b6518640 1 -- 172.21.15.105:0/2760189832 >> 172.21.15.105:0/2760189832 conn(0x7f43b00fc790 msgr2=0x7f43b00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:55.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.920+0000 7f43b6518640 1 -- 172.21.15.105:0/2760189832 shutdown_connections 2024-09-16T02:00:55.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.920+0000 7f43b6518640 1 -- 172.21.15.105:0/2760189832 wait complete. 2024-09-16T02:00:55.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.920+0000 7f43b6518640 1 Processor -- start 2024-09-16T02:00:55.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.920+0000 7f43b6518640 1 -- start start 2024-09-16T02:00:55.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.920+0000 7f43b6518640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43b0102d30 0x7f43b01983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:55.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.920+0000 7f43b6518640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43b0103650 0x7f43b01988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:55.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.920+0000 7f43b6518640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43b010c160 0x7f43b019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:55.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.920+0000 7f43b6518640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f43b0111e50 con 0x7f43b010c160 2024-09-16T02:00:55.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.920+0000 7f43b6518640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f43b0111cd0 con 0x7f43b0102d30 2024-09-16T02:00:55.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.920+0000 7f43b6518640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f43b0111fd0 con 0x7f43b0103650 2024-09-16T02:00:55.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.921+0000 7f43b4ab5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43b010c160 0x7f43b019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:55.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.921+0000 7f43b4ab5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43b010c160 0x7f43b019cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58418/0 (socket says 172.21.15.105:58418) 2024-09-16T02:00:55.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.921+0000 7f43b4ab5640 1 -- 172.21.15.105:0/429769687 learned_addr learned my addr 172.21.15.105:0/429769687 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:00:55.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.921+0000 7f43a7fff640 1 --2- 172.21.15.105:0/429769687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43b0103650 0x7f43b01988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:55.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.921+0000 7f43b4ab5640 1 -- 172.21.15.105:0/429769687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43b0103650 msgr2=0x7f43b01988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:55.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.921+0000 7f43b4ab5640 1 --2- 172.21.15.105:0/429769687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43b0103650 0x7f43b01988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:55.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.921+0000 7f43b4ab5640 1 -- 172.21.15.105:0/429769687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43b0102d30 msgr2=0x7f43b01983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:55.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.921+0000 7f43affff640 1 --2- 172.21.15.105:0/429769687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43b0102d30 0x7f43b01983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:55.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.921+0000 7f43b4ab5640 1 --2- 172.21.15.105:0/429769687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43b0102d30 0x7f43b01983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:55.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.921+0000 7f43b4ab5640 1 -- 172.21.15.105:0/429769687 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43b019d360 con 0x7f43b010c160 2024-09-16T02:00:55.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.921+0000 7f43affff640 1 --2- 172.21.15.105:0/429769687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43b0102d30 0x7f43b01983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:00:55.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.921+0000 7f43b4ab5640 1 --2- 172.21.15.105:0/429769687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43b010c160 0x7f43b019cc80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f43a800b970 tx=0x7f43a800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:55.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.921+0000 7f43adffb640 1 -- 172.21.15.105:0/429769687 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a8019070 con 0x7f43b010c160 2024-09-16T02:00:55.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.921+0000 7f43b6518640 1 -- 172.21.15.105:0/429769687 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43b019d650 con 0x7f43b010c160 2024-09-16T02:00:55.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.921+0000 7f43adffb640 1 -- 172.21.15.105:0/429769687 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43a8014440 con 0x7f43b010c160 2024-09-16T02:00:55.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.921+0000 7f43adffb640 1 -- 172.21.15.105:0/429769687 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a80075d0 con 0x7f43b010c160 2024-09-16T02:00:55.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.921+0000 7f43b6518640 1 -- 172.21.15.105:0/429769687 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43b01a4f30 con 0x7f43b010c160 2024-09-16T02:00:55.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.922+0000 7f43b6518640 1 -- 172.21.15.105:0/429769687 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43b0103110 con 0x7f43b010c160 2024-09-16T02:00:55.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.923+0000 7f43adffb640 1 -- 172.21.15.105:0/429769687 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f43a801c070 con 0x7f43b010c160 2024-09-16T02:00:55.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.924+0000 7f43adffb640 1 --2- 172.21.15.105:0/429769687 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f439c0a32d0 0x7f439c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:55.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.924+0000 7f43affff640 1 --2- 172.21.15.105:0/429769687 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f439c0a32d0 0x7f439c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:55.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.924+0000 7f43adffb640 1 -- 172.21.15.105:0/429769687 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f43a80cee20 con 0x7f43b010c160 2024-09-16T02:00:55.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.924+0000 7f43affff640 1 --2- 172.21.15.105:0/429769687 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f439c0a32d0 0x7f439c0a5790 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f43a0000c00 tx=0x7f43a0031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:55.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:55.928+0000 7f43adffb640 1 -- 172.21.15.105:0/429769687 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f43a809bca0 con 0x7f43b010c160 2024-09-16T02:00:56.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:56.120+0000 7f43b6518640 1 -- 172.21.15.105:0/429769687 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f43b0062630 con 0x7f439c0a32d0 2024-09-16T02:00:56.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:56.125+0000 7f43adffb640 1 -- 172.21.15.105:0/429769687 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f43b0062630 con 0x7f439c0a32d0 2024-09-16T02:00:56.127 INFO:teuthology.orchestra.run.smithi105.stdout:/volumes/qa/sv_1/76c0fdfc-77d5-46cc-9800-f0081eea5ad2 2024-09-16T02:00:56.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:56.127+0000 7f43b6518640 1 -- 172.21.15.105:0/429769687 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f439c0a32d0 msgr2=0x7f439c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:56.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:56.127+0000 7f43b6518640 1 --2- 172.21.15.105:0/429769687 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f439c0a32d0 0x7f439c0a5790 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f43a0000c00 tx=0x7f43a0031400 comp rx=0 tx=0).stop 2024-09-16T02:00:56.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:56.127+0000 7f43b6518640 1 -- 172.21.15.105:0/429769687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43b010c160 msgr2=0x7f43b019cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:56.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:56.127+0000 7f43b6518640 1 --2- 172.21.15.105:0/429769687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43b010c160 0x7f43b019cc80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f43a800b970 tx=0x7f43a800be30 comp rx=0 tx=0).stop 2024-09-16T02:00:56.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:56.127+0000 7f43b6518640 1 -- 172.21.15.105:0/429769687 shutdown_connections 2024-09-16T02:00:56.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:56.127+0000 7f43b6518640 1 --2- 172.21.15.105:0/429769687 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f439c0a32d0 0x7f439c0a5790 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:56.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:56.127+0000 7f43b6518640 1 --2- 172.21.15.105:0/429769687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43b0103650 0x7f43b01988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:56.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:56.127+0000 7f43b6518640 1 --2- 172.21.15.105:0/429769687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43b010c160 0x7f43b019cc80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:56.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:56.127+0000 7f43b6518640 1 --2- 172.21.15.105:0/429769687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43b0102d30 0x7f43b01983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:56.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:56.127+0000 7f43b6518640 1 -- 172.21.15.105:0/429769687 >> 172.21.15.105:0/429769687 conn(0x7f43b00fc790 msgr2=0x7f43b00fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:56.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:56.127+0000 7f43b6518640 1 -- 172.21.15.105:0/429769687 shutdown_connections 2024-09-16T02:00:56.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:56.127+0000 7f43b6518640 1 -- 172.21.15.105:0/429769687 wait complete. 2024-09-16T02:00:56.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:56 smithi105 ceph-mon[31197]: pgmap v287: 97 pgs: 97 active+clean; 605 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.2 KiB/s wr, 22 op/s 2024-09-16T02:00:56.835 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:00:56.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:56 smithi190 ceph-mon[34762]: pgmap v287: 97 pgs: 97 active+clean; 605 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.2 KiB/s wr, 22 op/s 2024-09-16T02:00:56.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:56 smithi137 ceph-mon[34099]: pgmap v287: 97 pgs: 97 active+clean; 605 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.2 KiB/s wr, 22 op/s 2024-09-16T02:00:57.078 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:00:57.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.607+0000 7f392f6fe640 1 -- 172.21.15.105:0/3217725187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3928103660 msgr2=0x7f392810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:57.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.607+0000 7f392f6fe640 1 --2- 172.21.15.105:0/3217725187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3928103660 0x7f392810bb50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3924009900 tx=0x7f392402a480 comp rx=0 tx=0).stop 2024-09-16T02:00:57.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.607+0000 7f392f6fe640 1 -- 172.21.15.105:0/3217725187 shutdown_connections 2024-09-16T02:00:57.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.607+0000 7f392f6fe640 1 --2- 172.21.15.105:0/3217725187 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f392810c170 0x7f392810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:57.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.607+0000 7f392f6fe640 1 --2- 172.21.15.105:0/3217725187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3928103660 0x7f392810bb50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:57.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.607+0000 7f392f6fe640 1 --2- 172.21.15.105:0/3217725187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3928102d40 0x7f3928103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:57.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.607+0000 7f392f6fe640 1 -- 172.21.15.105:0/3217725187 >> 172.21.15.105:0/3217725187 conn(0x7f39280fc7a0 msgr2=0x7f39280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:57.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.607+0000 7f392f6fe640 1 -- 172.21.15.105:0/3217725187 shutdown_connections 2024-09-16T02:00:57.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.607+0000 7f392f6fe640 1 -- 172.21.15.105:0/3217725187 wait complete. 2024-09-16T02:00:57.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.608+0000 7f392f6fe640 1 Processor -- start 2024-09-16T02:00:57.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.608+0000 7f392f6fe640 1 -- start start 2024-09-16T02:00:57.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.608+0000 7f392f6fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3928102d40 0x7f39281982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:57.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.608+0000 7f392f6fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3928103660 0x7f3928198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:57.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.608+0000 7f392f6fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f392810c170 0x7f392819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:57.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.608+0000 7f392f6fe640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3928111e20 con 0x7f392810c170 2024-09-16T02:00:57.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.608+0000 7f392f6fe640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3928111ca0 con 0x7f3928102d40 2024-09-16T02:00:57.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.608+0000 7f392f6fe640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3928111fa0 con 0x7f3928103660 2024-09-16T02:00:57.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.608+0000 7f392dc9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f392810c170 0x7f392819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:57.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.608+0000 7f392dc9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f392810c170 0x7f392819cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58434/0 (socket says 172.21.15.105:58434) 2024-09-16T02:00:57.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.608+0000 7f392dc9b640 1 -- 172.21.15.105:0/2318711184 learned_addr learned my addr 172.21.15.105:0/2318711184 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:00:57.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.608+0000 7f392d49a640 1 --2- 172.21.15.105:0/2318711184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3928102d40 0x7f39281982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:57.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.608+0000 7f392cc99640 1 --2- 172.21.15.105:0/2318711184 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3928103660 0x7f3928198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:57.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.608+0000 7f392dc9b640 1 -- 172.21.15.105:0/2318711184 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3928103660 msgr2=0x7f3928198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:57.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.608+0000 7f392dc9b640 1 --2- 172.21.15.105:0/2318711184 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3928103660 0x7f3928198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:57.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.608+0000 7f392dc9b640 1 -- 172.21.15.105:0/2318711184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3928102d40 msgr2=0x7f39281982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:57.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.608+0000 7f392dc9b640 1 --2- 172.21.15.105:0/2318711184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3928102d40 0x7f39281982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:57.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.608+0000 7f392dc9b640 1 -- 172.21.15.105:0/2318711184 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f392819d290 con 0x7f392810c170 2024-09-16T02:00:57.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.608+0000 7f392cc99640 1 --2- 172.21.15.105:0/2318711184 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3928103660 0x7f3928198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:00:57.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.608+0000 7f392d49a640 1 --2- 172.21.15.105:0/2318711184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3928102d40 0x7f39281982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:00:57.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.608+0000 7f392dc9b640 1 --2- 172.21.15.105:0/2318711184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f392810c170 0x7f392819cbb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f392000cce0 tx=0x7f392000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:57.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.609+0000 7f391e7fc640 1 -- 172.21.15.105:0/2318711184 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392000f450 con 0x7f392810c170 2024-09-16T02:00:57.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.609+0000 7f391e7fc640 1 -- 172.21.15.105:0/2318711184 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f392000faa0 con 0x7f392810c170 2024-09-16T02:00:57.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.609+0000 7f392f6fe640 1 -- 172.21.15.105:0/2318711184 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f392819d580 con 0x7f392810c170 2024-09-16T02:00:57.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.609+0000 7f391e7fc640 1 -- 172.21.15.105:0/2318711184 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3920018b70 con 0x7f392810c170 2024-09-16T02:00:57.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.609+0000 7f392f6fe640 1 -- 172.21.15.105:0/2318711184 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39281a4e60 con 0x7f392810c170 2024-09-16T02:00:57.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.610+0000 7f392f6fe640 1 -- 172.21.15.105:0/2318711184 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3928103120 con 0x7f392810c170 2024-09-16T02:00:57.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.611+0000 7f391e7fc640 1 -- 172.21.15.105:0/2318711184 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f3920018d10 con 0x7f392810c170 2024-09-16T02:00:57.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.611+0000 7f391e7fc640 1 --2- 172.21.15.105:0/2318711184 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f39140a3350 0x7f39140a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:57.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.612+0000 7f392d49a640 1 --2- 172.21.15.105:0/2318711184 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f39140a3350 0x7f39140a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:57.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.612+0000 7f391e7fc640 1 -- 172.21.15.105:0/2318711184 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f39200cfce0 con 0x7f392810c170 2024-09-16T02:00:57.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.612+0000 7f392d49a640 1 --2- 172.21.15.105:0/2318711184 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f39140a3350 0x7f39140a5810 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f39180058c0 tx=0x7f3918005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:57.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.615+0000 7f391e7fc640 1 -- 172.21.15.105:0/2318711184 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3920014030 con 0x7f392810c170 2024-09-16T02:00:57.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:57 smithi105 ceph-mon[31197]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:00:57.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.818+0000 7f392f6fe640 1 -- 172.21.15.105:0/2318711184 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f392802d7c0 con 0x7f392810c170 2024-09-16T02:00:57.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.818+0000 7f391e7fc640 1 -- 172.21.15.105:0/2318711184 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11991 (secure 0 0 0) 0x7f392009ca50 con 0x7f392810c170 2024-09-16T02:00:57.820 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:00:57.820 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":27,"btime":"2024-09-16T02:00:47:526725+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2057699920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2057699920},{"type":"v1","addr":"172.21.15.105:6835","nonce":2057699920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1764886948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1764886948},{"type":"v1","addr":"172.21.15.105:6837","nonce":1764886948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T02:00:35.112266+0000","modified":"2024-09-16T02:00:47.526704+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24469,"mds_2":24461,"mds_3":24457,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/4112062361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":4112062361},{"type":"v1","addr":"172.21.15.105:6839","nonce":4112062361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6841/163156514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":163156514},{"type":"v1","addr":"172.21.15.105:6841","nonce":163156514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/683519097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":683519097},{"type":"v1","addr":"172.21.15.190:6833","nonce":683519097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3025442862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3025442862},{"type":"v1","addr":"172.21.15.137:6833","nonce":3025442862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/763783602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":763783602},{"type":"v1","addr":"172.21.15.137:6835","nonce":763783602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.190:6835/1851042281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1851042281},{"type":"v1","addr":"172.21.15.190:6835","nonce":1851042281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/3181257549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3181257549},{"type":"v1","addr":"172.21.15.190:6837","nonce":3181257549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6837/2788541030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2788541030},{"type":"v1","addr":"172.21.15.137:6837","nonce":2788541030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.190:6839/1222573537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":1222573537},{"type":"v1","addr":"172.21.15.190:6839","nonce":1222573537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/3454924009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3454924009},{"type":"v1","addr":"172.21.15.137:6839","nonce":3454924009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24469,24461,24457,24445]},"id":1}]} 2024-09-16T02:00:57.820 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 27 2024-09-16T02:00:57.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.820+0000 7f392f6fe640 1 -- 172.21.15.105:0/2318711184 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f39140a3350 msgr2=0x7f39140a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:57.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.820+0000 7f392f6fe640 1 --2- 172.21.15.105:0/2318711184 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f39140a3350 0x7f39140a5810 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f39180058c0 tx=0x7f3918005ec0 comp rx=0 tx=0).stop 2024-09-16T02:00:57.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.820+0000 7f392f6fe640 1 -- 172.21.15.105:0/2318711184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f392810c170 msgr2=0x7f392819cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:57.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.820+0000 7f392f6fe640 1 --2- 172.21.15.105:0/2318711184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f392810c170 0x7f392819cbb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f392000cce0 tx=0x7f392000a620 comp rx=0 tx=0).stop 2024-09-16T02:00:57.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.820+0000 7f392f6fe640 1 -- 172.21.15.105:0/2318711184 shutdown_connections 2024-09-16T02:00:57.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.820+0000 7f392f6fe640 1 --2- 172.21.15.105:0/2318711184 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f39140a3350 0x7f39140a5810 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:57.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.820+0000 7f392f6fe640 1 --2- 172.21.15.105:0/2318711184 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3928103660 0x7f3928198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:57.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.820+0000 7f392f6fe640 1 --2- 172.21.15.105:0/2318711184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f392810c170 0x7f392819cbb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:57.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.820+0000 7f392f6fe640 1 --2- 172.21.15.105:0/2318711184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3928102d40 0x7f39281982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:57.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.820+0000 7f392f6fe640 1 -- 172.21.15.105:0/2318711184 >> 172.21.15.105:0/2318711184 conn(0x7f39280fc7a0 msgr2=0x7f39280fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:57.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.821+0000 7f392f6fe640 1 -- 172.21.15.105:0/2318711184 shutdown_connections 2024-09-16T02:00:57.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:57.821+0000 7f392f6fe640 1 -- 172.21.15.105:0/2318711184 wait complete. 2024-09-16T02:00:57.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:57 smithi190 ceph-mon[34762]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:00:57.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:57 smithi137 ceph-mon[34099]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:00:58.664 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T02:00:58.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:58 smithi190 ceph-mon[34762]: pgmap v288: 97 pgs: 97 active+clean; 605 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.7 KiB/s wr, 19 op/s 2024-09-16T02:00:58.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:00:58 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/2318711184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:00:58.907 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:00:58.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:58 smithi137 ceph-mon[34099]: pgmap v288: 97 pgs: 97 active+clean; 605 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.7 KiB/s wr, 19 op/s 2024-09-16T02:00:58.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:00:58 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/2318711184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:00:59.028 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:58 smithi105 ceph-mon[31197]: pgmap v288: 97 pgs: 97 active+clean; 605 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.7 KiB/s wr, 19 op/s 2024-09-16T02:00:59.028 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:00:58 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2318711184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:00:59.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.418+0000 7fcc581f7640 1 -- 172.21.15.105:0/3111516325 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc50101420 msgr2=0x7fcc5010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:59.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.418+0000 7fcc581f7640 1 --2- 172.21.15.105:0/3111516325 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc50101420 0x7fcc5010d950 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009cd0 tx=0x7fcc4802a8e0 comp rx=0 tx=0).stop 2024-09-16T02:00:59.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.418+0000 7fcc581f7640 1 -- 172.21.15.105:0/3111516325 shutdown_connections 2024-09-16T02:00:59.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.418+0000 7fcc581f7640 1 --2- 172.21.15.105:0/3111516325 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc50101420 0x7fcc5010d950 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:59.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.418+0000 7fcc581f7640 1 --2- 172.21.15.105:0/3111516325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc50100a80 0x7fcc50100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:59.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.418+0000 7fcc581f7640 1 --2- 172.21.15.105:0/3111516325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc50106a80 0x7fcc50106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:59.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.418+0000 7fcc581f7640 1 -- 172.21.15.105:0/3111516325 >> 172.21.15.105:0/3111516325 conn(0x7fcc500fc7a0 msgr2=0x7fcc500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:59.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.418+0000 7fcc581f7640 1 -- 172.21.15.105:0/3111516325 shutdown_connections 2024-09-16T02:00:59.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.418+0000 7fcc581f7640 1 -- 172.21.15.105:0/3111516325 wait complete. 2024-09-16T02:00:59.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.419+0000 7fcc581f7640 1 Processor -- start 2024-09-16T02:00:59.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.419+0000 7fcc581f7640 1 -- start start 2024-09-16T02:00:59.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.419+0000 7fcc581f7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc50100a80 0x7fcc50198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:59.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.419+0000 7fcc581f7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc50101420 0x7fcc501988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:59.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.419+0000 7fcc581f7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc50106a80 0x7fcc5019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:59.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.419+0000 7fcc581f7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcc50110d30 con 0x7fcc50100a80 2024-09-16T02:00:59.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.419+0000 7fcc581f7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcc50110bb0 con 0x7fcc50101420 2024-09-16T02:00:59.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.419+0000 7fcc581f7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcc50110eb0 con 0x7fcc50106a80 2024-09-16T02:00:59.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.419+0000 7fcc55f93640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc50100a80 0x7fcc50198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:59.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.419+0000 7fcc55f93640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc50100a80 0x7fcc50198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51150/0 (socket says 172.21.15.105:51150) 2024-09-16T02:00:59.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.419+0000 7fcc55f93640 1 -- 172.21.15.105:0/2002852536 learned_addr learned my addr 172.21.15.105:0/2002852536 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:00:59.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.419+0000 7fcc56794640 1 --2- 172.21.15.105:0/2002852536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc50106a80 0x7fcc5019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:59.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.419+0000 7fcc55792640 1 --2- 172.21.15.105:0/2002852536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc50101420 0x7fcc501988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:59.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.419+0000 7fcc55f93640 1 -- 172.21.15.105:0/2002852536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc50106a80 msgr2=0x7fcc5019cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:59.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.419+0000 7fcc55f93640 1 --2- 172.21.15.105:0/2002852536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc50106a80 0x7fcc5019cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:59.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.419+0000 7fcc55f93640 1 -- 172.21.15.105:0/2002852536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc50101420 msgr2=0x7fcc501988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:59.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.419+0000 7fcc55f93640 1 --2- 172.21.15.105:0/2002852536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc50101420 0x7fcc501988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:59.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.419+0000 7fcc55f93640 1 -- 172.21.15.105:0/2002852536 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc5019d310 con 0x7fcc50100a80 2024-09-16T02:00:59.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.419+0000 7fcc55f93640 1 --2- 172.21.15.105:0/2002852536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc50100a80 0x7fcc50198360 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcc4400c890 tx=0x7fcc4400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:59.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.420+0000 7fcc42ffd640 1 -- 172.21.15.105:0/2002852536 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc44004220 con 0x7fcc50100a80 2024-09-16T02:00:59.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.420+0000 7fcc42ffd640 1 -- 172.21.15.105:0/2002852536 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc440043c0 con 0x7fcc50100a80 2024-09-16T02:00:59.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.420+0000 7fcc581f7640 1 -- 172.21.15.105:0/2002852536 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc5019d600 con 0x7fcc50100a80 2024-09-16T02:00:59.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.420+0000 7fcc42ffd640 1 -- 172.21.15.105:0/2002852536 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc44004a00 con 0x7fcc50100a80 2024-09-16T02:00:59.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.420+0000 7fcc581f7640 1 -- 172.21.15.105:0/2002852536 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc501a4ee0 con 0x7fcc50100a80 2024-09-16T02:00:59.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.421+0000 7fcc581f7640 1 -- 172.21.15.105:0/2002852536 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc50100ee0 con 0x7fcc50100a80 2024-09-16T02:00:59.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.422+0000 7fcc42ffd640 1 -- 172.21.15.105:0/2002852536 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fcc44004ba0 con 0x7fcc50100a80 2024-09-16T02:00:59.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.422+0000 7fcc42ffd640 1 --2- 172.21.15.105:0/2002852536 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fcc3c0a3350 0x7fcc3c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:59.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.423+0000 7fcc55792640 1 --2- 172.21.15.105:0/2002852536 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fcc3c0a3350 0x7fcc3c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:59.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.423+0000 7fcc42ffd640 1 -- 172.21.15.105:0/2002852536 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fcc440cf9c0 con 0x7fcc50100a80 2024-09-16T02:00:59.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.423+0000 7fcc55792640 1 --2- 172.21.15.105:0/2002852536 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fcc3c0a3350 0x7fcc3c0a5810 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcc501051c0 tx=0x7fcc4c00a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:59.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.426+0000 7fcc42ffd640 1 -- 172.21.15.105:0/2002852536 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcc44014030 con 0x7fcc50100a80 2024-09-16T02:00:59.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.596+0000 7fcc581f7640 1 -- 172.21.15.105:0/2002852536 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcc50109a70 con 0x7fcc50100a80 2024-09-16T02:00:59.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.597+0000 7fcc42ffd640 1 -- 172.21.15.105:0/2002852536 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22327 (secure 0 0 0) 0x7fcc4409c730 con 0x7fcc50100a80 2024-09-16T02:00:59.599 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:00:59.599 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":79,"fsid":"423b7a2a-73ce-11ef-bceb-c7b262605968","created":"2024-09-16T01:52:44.388033+0000","modified":"2024-09-16T02:00:35.112006+0000","last_up_change":"2024-09-16T01:59:42.218034+0000","last_in_change":"2024-09-16T01:59:18.785252+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T01:55:53.680808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T02:00:31.286117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T02:00:33.310045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a60d6c72-da8b-4b02-bb33-672be795e7f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6803","nonce":1226660290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6805","nonce":1226660290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6809","nonce":1226660290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6807","nonce":1226660290}]},"public_addr":"172.21.15.105:6803/1226660290","cluster_addr":"172.21.15.105:6805/1226660290","heartbeat_back_addr":"172.21.15.105:6809/1226660290","heartbeat_front_addr":"172.21.15.105:6807/1226660290","state":["exists","up"]},{"osd":1,"uuid":"b1d9ae84-b9a0-4a7e-b0fe-c402d46809fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6801","nonce":34753956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6803","nonce":34753956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6807","nonce":34753956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6805","nonce":34753956}]},"public_addr":"172.21.15.137:6801/34753956","cluster_addr":"172.21.15.137:6803/34753956","heartbeat_back_addr":"172.21.15.137:6807/34753956","heartbeat_front_addr":"172.21.15.137:6805/34753956","state":["exists","up"]},{"osd":2,"uuid":"137faae6-2e8b-44b5-8976-3d95a6eef1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6801","nonce":1159531918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6803","nonce":1159531918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6807","nonce":1159531918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6805","nonce":1159531918}]},"public_addr":"172.21.15.190:6801/1159531918","cluster_addr":"172.21.15.190:6803/1159531918","heartbeat_back_addr":"172.21.15.190:6807/1159531918","heartbeat_front_addr":"172.21.15.190:6805/1159531918","state":["exists","up"]},{"osd":3,"uuid":"5743c37b-5cd8-42b6-81c8-5cf5afc71e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6811","nonce":604905679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6813","nonce":604905679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6817","nonce":604905679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6815","nonce":604905679}]},"public_addr":"172.21.15.105:6811/604905679","cluster_addr":"172.21.15.105:6813/604905679","heartbeat_back_addr":"172.21.15.105:6817/604905679","heartbeat_front_addr":"172.21.15.105:6815/604905679","state":["exists","up"]},{"osd":4,"uuid":"e439d4a2-b54c-4de9-97f6-3510afe57f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6809","nonce":3758109169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6811","nonce":3758109169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6815","nonce":3758109169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6813","nonce":3758109169}]},"public_addr":"172.21.15.137:6809/3758109169","cluster_addr":"172.21.15.137:6811/3758109169","heartbeat_back_addr":"172.21.15.137:6815/3758109169","heartbeat_front_addr":"172.21.15.137:6813/3758109169","state":["exists","up"]},{"osd":5,"uuid":"d1e2a7d5-99f8-40fa-af6e-0305cc0af677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6809","nonce":2086773754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6811","nonce":2086773754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6815","nonce":2086773754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6813","nonce":2086773754}]},"public_addr":"172.21.15.190:6809/2086773754","cluster_addr":"172.21.15.190:6811/2086773754","heartbeat_back_addr":"172.21.15.190:6815/2086773754","heartbeat_front_addr":"172.21.15.190:6813/2086773754","state":["exists","up"]},{"osd":6,"uuid":"cd418159-a0f6-44c7-9213-c010b132fb56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6819","nonce":4012018725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6821","nonce":4012018725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6825","nonce":4012018725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6823","nonce":4012018725}]},"public_addr":"172.21.15.105:6819/4012018725","cluster_addr":"172.21.15.105:6821/4012018725","heartbeat_back_addr":"172.21.15.105:6825/4012018725","heartbeat_front_addr":"172.21.15.105:6823/4012018725","state":["exists","up"]},{"osd":7,"uuid":"76c15863-801e-4a72-be73-ba9df46c15b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6817","nonce":888678288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6819","nonce":888678288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6823","nonce":888678288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6821","nonce":888678288}]},"public_addr":"172.21.15.137:6817/888678288","cluster_addr":"172.21.15.137:6819/888678288","heartbeat_back_addr":"172.21.15.137:6823/888678288","heartbeat_front_addr":"172.21.15.137:6821/888678288","state":["exists","up"]},{"osd":8,"uuid":"6718597c-6f42-48ee-9493-edb2396fe707","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6817","nonce":2132638341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6819","nonce":2132638341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6823","nonce":2132638341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6821","nonce":2132638341}]},"public_addr":"172.21.15.190:6817/2132638341","cluster_addr":"172.21.15.190:6819/2132638341","heartbeat_back_addr":"172.21.15.190:6823/2132638341","heartbeat_front_addr":"172.21.15.190:6821/2132638341","state":["exists","up"]},{"osd":9,"uuid":"5178edc0-a2e8-491c-821c-6f41ec5aeb2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6827","nonce":3375034979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6829","nonce":3375034979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6833","nonce":3375034979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6831","nonce":3375034979}]},"public_addr":"172.21.15.105:6827/3375034979","cluster_addr":"172.21.15.105:6829/3375034979","heartbeat_back_addr":"172.21.15.105:6833/3375034979","heartbeat_front_addr":"172.21.15.105:6831/3375034979","state":["exists","up"]},{"osd":10,"uuid":"ad145d3e-a6e2-4dd7-bea3-bcd01eb3413e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6825","nonce":318562234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6827","nonce":318562234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6831","nonce":318562234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6829","nonce":318562234}]},"public_addr":"172.21.15.137:6825/318562234","cluster_addr":"172.21.15.137:6827/318562234","heartbeat_back_addr":"172.21.15.137:6831/318562234","heartbeat_front_addr":"172.21.15.137:6829/318562234","state":["exists","up"]},{"osd":11,"uuid":"6d44bced-3a8b-4763-a93e-0ea34acb4cb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6825","nonce":3820460689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6827","nonce":3820460689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6831","nonce":3820460689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6829","nonce":3820460689}]},"public_addr":"172.21.15.190:6825/3820460689","cluster_addr":"172.21.15.190:6827/3820460689","heartbeat_back_addr":"172.21.15.190:6831/3820460689","heartbeat_front_addr":"172.21.15.190:6829/3820460689","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:55:02.759033+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:55:26.829142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:55:51.120200+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:56:17.277870+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:56:42.313701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:07.013416+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:32.477916+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:58.643764+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:23.454908+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:48.869082+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:14.921573+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:40.908993+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:0/2453739657":"2024-09-17T01:53:39.351314+0000","172.21.15.105:6801/1469189484":"2024-09-17T01:53:39.351314+0000","172.21.15.105:0/3466166879":"2024-09-17T01:53:39.351314+0000","172.21.15.105:6801/1268348742":"2024-09-17T01:53:15.522227+0000","172.21.15.105:0/2897248478":"2024-09-17T01:53:39.351314+0000","172.21.15.105:0/3709363371":"2024-09-17T01:53:15.522227+0000","172.21.15.105:6800/1268348742":"2024-09-17T01:53:15.522227+0000","172.21.15.105:6800/1469189484":"2024-09-17T01:53:39.351314+0000","172.21.15.105:0/1198176299":"2024-09-17T01:53:15.522227+0000","172.21.15.105:0/3031917848":"2024-09-17T01:53:15.522227+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T02:00:59.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.599+0000 7fcc581f7640 1 -- 172.21.15.105:0/2002852536 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fcc3c0a3350 msgr2=0x7fcc3c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:59.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.599+0000 7fcc581f7640 1 --2- 172.21.15.105:0/2002852536 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fcc3c0a3350 0x7fcc3c0a5810 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcc501051c0 tx=0x7fcc4c00a4c0 comp rx=0 tx=0).stop 2024-09-16T02:00:59.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.599+0000 7fcc581f7640 1 -- 172.21.15.105:0/2002852536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc50100a80 msgr2=0x7fcc50198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:59.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.599+0000 7fcc581f7640 1 --2- 172.21.15.105:0/2002852536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc50100a80 0x7fcc50198360 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcc4400c890 tx=0x7fcc4400cd50 comp rx=0 tx=0).stop 2024-09-16T02:00:59.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.599+0000 7fcc581f7640 1 -- 172.21.15.105:0/2002852536 shutdown_connections 2024-09-16T02:00:59.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.599+0000 7fcc581f7640 1 --2- 172.21.15.105:0/2002852536 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fcc3c0a3350 0x7fcc3c0a5810 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:59.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.600+0000 7fcc581f7640 1 --2- 172.21.15.105:0/2002852536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc50106a80 0x7fcc5019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:59.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.600+0000 7fcc581f7640 1 --2- 172.21.15.105:0/2002852536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc50101420 0x7fcc501988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:59.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.600+0000 7fcc581f7640 1 --2- 172.21.15.105:0/2002852536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc50100a80 0x7fcc50198360 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:59.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.600+0000 7fcc581f7640 1 -- 172.21.15.105:0/2002852536 >> 172.21.15.105:0/2002852536 conn(0x7fcc500fc7a0 msgr2=0x7fcc5006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:59.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.600+0000 7fcc581f7640 1 -- 172.21.15.105:0/2002852536 shutdown_connections 2024-09-16T02:00:59.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:00:59.600+0000 7fcc581f7640 1 -- 172.21.15.105:0/2002852536 wait complete. 2024-09-16T02:01:00.408 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-16T02:01:00.409 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T02:01:00.658 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:00.778 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:00 smithi105 ceph-mon[31197]: pgmap v289: 97 pgs: 97 active+clean; 629 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 KiB/s wr, 20 op/s 2024-09-16T02:01:00.779 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:00 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2002852536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:01:00.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:00 smithi190 ceph-mon[34762]: pgmap v289: 97 pgs: 97 active+clean; 629 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 KiB/s wr, 20 op/s 2024-09-16T02:01:00.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:00 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/2002852536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:01:00.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:00 smithi137 ceph-mon[34099]: pgmap v289: 97 pgs: 97 active+clean; 629 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 KiB/s wr, 20 op/s 2024-09-16T02:01:00.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:00 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/2002852536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:01:01.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.128+0000 7f74245ba640 1 -- 172.21.15.105:0/3821477941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f741c100a80 msgr2=0x7f741c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:01.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.128+0000 7f74245ba640 1 --2- 172.21.15.105:0/3821477941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f741c100a80 0x7f741c100ee0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7410009930 tx=0x7f741002a420 comp rx=0 tx=0).stop 2024-09-16T02:01:01.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.128+0000 7f74245ba640 1 -- 172.21.15.105:0/3821477941 shutdown_connections 2024-09-16T02:01:01.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.128+0000 7f74245ba640 1 --2- 172.21.15.105:0/3821477941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f741c101420 0x7f741c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:01.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.128+0000 7f74245ba640 1 --2- 172.21.15.105:0/3821477941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f741c100a80 0x7f741c100ee0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:01.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.128+0000 7f74245ba640 1 --2- 172.21.15.105:0/3821477941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f741c106a80 0x7f741c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:01.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.128+0000 7f74245ba640 1 -- 172.21.15.105:0/3821477941 >> 172.21.15.105:0/3821477941 conn(0x7f741c0fc7a0 msgr2=0x7f741c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:01.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.128+0000 7f74245ba640 1 -- 172.21.15.105:0/3821477941 shutdown_connections 2024-09-16T02:01:01.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.128+0000 7f74245ba640 1 -- 172.21.15.105:0/3821477941 wait complete. 2024-09-16T02:01:01.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.128+0000 7f74245ba640 1 Processor -- start 2024-09-16T02:01:01.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.129+0000 7f74245ba640 1 -- start start 2024-09-16T02:01:01.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.129+0000 7f74245ba640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f741c100a80 0x7f741c1972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:01.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.129+0000 7f74245ba640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f741c101420 0x7f741c197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:01.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.129+0000 7f74245ba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f741c106a80 0x7f741c19bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:01.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.129+0000 7f74245ba640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f741c110d80 con 0x7f741c106a80 2024-09-16T02:01:01.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.129+0000 7f74245ba640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f741c110c00 con 0x7f741c101420 2024-09-16T02:01:01.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.129+0000 7f74245ba640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f741c110f00 con 0x7f741c100a80 2024-09-16T02:01:01.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.129+0000 7f7422b57640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f741c106a80 0x7f741c19bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:01.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.129+0000 7f7422b57640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f741c106a80 0x7f741c19bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51178/0 (socket says 172.21.15.105:51178) 2024-09-16T02:01:01.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.129+0000 7f7422b57640 1 -- 172.21.15.105:0/3920606360 learned_addr learned my addr 172.21.15.105:0/3920606360 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:01:01.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.129+0000 7f7421b55640 1 --2- 172.21.15.105:0/3920606360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f741c101420 0x7f741c197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:01.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.129+0000 7f7422356640 1 --2- 172.21.15.105:0/3920606360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f741c100a80 0x7f741c1972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:01.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.129+0000 7f7422b57640 1 -- 172.21.15.105:0/3920606360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f741c100a80 msgr2=0x7f741c1972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:01.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.129+0000 7f7422b57640 1 --2- 172.21.15.105:0/3920606360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f741c100a80 0x7f741c1972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:01.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.129+0000 7f7422b57640 1 -- 172.21.15.105:0/3920606360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f741c101420 msgr2=0x7f741c197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:01.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.129+0000 7f7422b57640 1 --2- 172.21.15.105:0/3920606360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f741c101420 0x7f741c197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:01.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.129+0000 7f7422b57640 1 -- 172.21.15.105:0/3920606360 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f741c19c290 con 0x7f741c106a80 2024-09-16T02:01:01.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.129+0000 7f7422b57640 1 --2- 172.21.15.105:0/3920606360 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f741c106a80 0x7f741c19bbb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f740c00ebb0 tx=0x7f740c007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:01.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.129+0000 7f741b7fe640 1 -- 172.21.15.105:0/3920606360 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740c007ca0 con 0x7f741c106a80 2024-09-16T02:01:01.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.129+0000 7f741b7fe640 1 -- 172.21.15.105:0/3920606360 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f740c007e40 con 0x7f741c106a80 2024-09-16T02:01:01.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.129+0000 7f741b7fe640 1 -- 172.21.15.105:0/3920606360 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740c0057d0 con 0x7f741c106a80 2024-09-16T02:01:01.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.129+0000 7f74245ba640 1 -- 172.21.15.105:0/3920606360 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f741c19c520 con 0x7f741c106a80 2024-09-16T02:01:01.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.130+0000 7f74245ba640 1 -- 172.21.15.105:0/3920606360 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f741c19c9e0 con 0x7f741c106a80 2024-09-16T02:01:01.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.131+0000 7f74245ba640 1 -- 172.21.15.105:0/3920606360 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f741c100ee0 con 0x7f741c106a80 2024-09-16T02:01:01.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.131+0000 7f741b7fe640 1 -- 172.21.15.105:0/3920606360 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f740c004a40 con 0x7f741c106a80 2024-09-16T02:01:01.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.132+0000 7f741b7fe640 1 --2- 172.21.15.105:0/3920606360 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f74040a32d0 0x7f74040a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:01.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.132+0000 7f7422356640 1 --2- 172.21.15.105:0/3920606360 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f74040a32d0 0x7f74040a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:01.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.132+0000 7f741b7fe640 1 -- 172.21.15.105:0/3920606360 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f740c0d0eb0 con 0x7f741c106a80 2024-09-16T02:01:01.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.133+0000 7f7422356640 1 --2- 172.21.15.105:0/3920606360 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f74040a32d0 0x7f74040a5790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7414005d90 tx=0x7f7414005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:01.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.136+0000 7f741b7fe640 1 -- 172.21.15.105:0/3920606360 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f741c19c9e0 con 0x7f741c106a80 2024-09-16T02:01:01.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.298+0000 7f74245ba640 1 -- 172.21.15.105:0/3920606360 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f741c108b30 con 0x7f74040a32d0 2024-09-16T02:01:01.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.305+0000 7f741b7fe640 1 -- 172.21.15.105:0/3920606360 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258911 (secure 0 0 0) 0x7f741c108b30 con 0x7f74040a32d0 2024-09-16T02:01:01.309 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:01:01.310 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-09-16T02:01:01.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.308+0000 7f74245ba640 1 -- 172.21.15.105:0/3920606360 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f74040a32d0 msgr2=0x7f74040a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:01.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.308+0000 7f74245ba640 1 --2- 172.21.15.105:0/3920606360 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f74040a32d0 0x7f74040a5790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7414005d90 tx=0x7f7414005d20 comp rx=0 tx=0).stop 2024-09-16T02:01:01.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.308+0000 7f74245ba640 1 -- 172.21.15.105:0/3920606360 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f741c106a80 msgr2=0x7f741c19bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:01.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.308+0000 7f74245ba640 1 --2- 172.21.15.105:0/3920606360 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f741c106a80 0x7f741c19bbb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f740c00ebb0 tx=0x7f740c007590 comp rx=0 tx=0).stop 2024-09-16T02:01:01.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.308+0000 7f74245ba640 1 -- 172.21.15.105:0/3920606360 shutdown_connections 2024-09-16T02:01:01.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.308+0000 7f74245ba640 1 --2- 172.21.15.105:0/3920606360 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f74040a32d0 0x7f74040a5790 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:01.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.308+0000 7f74245ba640 1 --2- 172.21.15.105:0/3920606360 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f741c106a80 0x7f741c19bbb0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:01.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.308+0000 7f74245ba640 1 --2- 172.21.15.105:0/3920606360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f741c101420 0x7f741c197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:01.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.308+0000 7f74245ba640 1 --2- 172.21.15.105:0/3920606360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f741c100a80 0x7f741c1972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:01.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.308+0000 7f74245ba640 1 -- 172.21.15.105:0/3920606360 >> 172.21.15.105:0/3920606360 conn(0x7f741c0fc7a0 msgr2=0x7f741c0fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:01.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.308+0000 7f74245ba640 1 -- 172.21.15.105:0/3920606360 shutdown_connections 2024-09-16T02:01:01.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:01.308+0000 7f74245ba640 1 -- 172.21.15.105:0/3920606360 wait complete. 2024-09-16T02:01:02.029 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-09-16T02:00:59.438084+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":644219,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":263,"num_read_kb":238,"num_write":409,"num_write_kb":3055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":336,"ondisk_log_size":336,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364052,"kb_used_data":10644,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577772,"statfs":{"total":1151940427776,"available":1151567638528,"internally_reserved":0,"allocated":10899456,"data_stored":6872491,"data_compressed":107436,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":49889,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":137,"num_read_kb":129,"num_write":114,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003173"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.463390+0000","last_change":"2024-09-16T02:00:34.454121+0000","last_active":"2024-09-16T02:00:35.463390+0000","last_peered":"2024-09-16T02:00:35.463390+0000","last_clean":"2024-09-16T02:00:35.463390+0000","last_became_active":"2024-09-16T02:00:34.453810+0000","last_became_peered":"2024-09-16T02:00:34.453810+0000","last_unstale":"2024-09-16T02:00:35.463390+0000","last_undegraded":"2024-09-16T02:00:35.463390+0000","last_fullsized":"2024-09-16T02:00:35.463390+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:42:38.781928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458143+0000","last_change":"2024-09-16T02:00:34.462020+0000","last_active":"2024-09-16T02:00:35.458143+0000","last_peered":"2024-09-16T02:00:35.458143+0000","last_clean":"2024-09-16T02:00:35.458143+0000","last_became_active":"2024-09-16T02:00:34.461680+0000","last_became_peered":"2024-09-16T02:00:34.461680+0000","last_unstale":"2024-09-16T02:00:35.458143+0000","last_undegraded":"2024-09-16T02:00:35.458143+0000","last_fullsized":"2024-09-16T02:00:35.458143+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:00:14.493643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458302+0000","last_change":"2024-09-16T02:00:34.453034+0000","last_active":"2024-09-16T02:00:35.458302+0000","last_peered":"2024-09-16T02:00:35.458302+0000","last_clean":"2024-09-16T02:00:35.458302+0000","last_became_active":"2024-09-16T02:00:34.452719+0000","last_became_peered":"2024-09-16T02:00:34.452719+0000","last_unstale":"2024-09-16T02:00:35.458302+0000","last_undegraded":"2024-09-16T02:00:35.458302+0000","last_fullsized":"2024-09-16T02:00:35.458302+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:48:52.478501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458864+0000","last_change":"2024-09-16T02:00:34.458156+0000","last_active":"2024-09-16T02:00:35.458864+0000","last_peered":"2024-09-16T02:00:35.458864+0000","last_clean":"2024-09-16T02:00:35.458864+0000","last_became_active":"2024-09-16T02:00:34.457831+0000","last_became_peered":"2024-09-16T02:00:34.457831+0000","last_unstale":"2024-09-16T02:00:35.458864+0000","last_undegraded":"2024-09-16T02:00:35.458864+0000","last_fullsized":"2024-09-16T02:00:35.458864+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:35:36.609046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458845+0000","last_change":"2024-09-16T02:00:34.448638+0000","last_active":"2024-09-16T02:00:35.458845+0000","last_peered":"2024-09-16T02:00:35.458845+0000","last_clean":"2024-09-16T02:00:35.458845+0000","last_became_active":"2024-09-16T02:00:34.447993+0000","last_became_peered":"2024-09-16T02:00:34.447993+0000","last_unstale":"2024-09-16T02:00:35.458845+0000","last_undegraded":"2024-09-16T02:00:35.458845+0000","last_fullsized":"2024-09-16T02:00:35.458845+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:33:04.832053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.459310+0000","last_change":"2024-09-16T02:00:34.462651+0000","last_active":"2024-09-16T02:00:35.459310+0000","last_peered":"2024-09-16T02:00:35.459310+0000","last_clean":"2024-09-16T02:00:35.459310+0000","last_became_active":"2024-09-16T02:00:34.461837+0000","last_became_peered":"2024-09-16T02:00:34.461837+0000","last_unstale":"2024-09-16T02:00:35.459310+0000","last_undegraded":"2024-09-16T02:00:35.459310+0000","last_fullsized":"2024-09-16T02:00:35.459310+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:52:13.733591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458629+0000","last_change":"2024-09-16T02:00:34.455362+0000","last_active":"2024-09-16T02:00:35.458629+0000","last_peered":"2024-09-16T02:00:35.458629+0000","last_clean":"2024-09-16T02:00:35.458629+0000","last_became_active":"2024-09-16T02:00:34.455029+0000","last_became_peered":"2024-09-16T02:00:34.455029+0000","last_unstale":"2024-09-16T02:00:35.458629+0000","last_undegraded":"2024-09-16T02:00:35.458629+0000","last_fullsized":"2024-09-16T02:00:35.458629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:14:08.316845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458205+0000","last_change":"2024-09-16T02:00:34.459578+0000","last_active":"2024-09-16T02:00:35.458205+0000","last_peered":"2024-09-16T02:00:35.458205+0000","last_clean":"2024-09-16T02:00:35.458205+0000","last_became_active":"2024-09-16T02:00:34.459288+0000","last_became_peered":"2024-09-16T02:00:34.459288+0000","last_unstale":"2024-09-16T02:00:35.458205+0000","last_undegraded":"2024-09-16T02:00:35.458205+0000","last_fullsized":"2024-09-16T02:00:35.458205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:52:28.974276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.447603+0000","last_change":"2024-09-16T02:00:34.463011+0000","last_active":"2024-09-16T02:00:35.447603+0000","last_peered":"2024-09-16T02:00:35.447603+0000","last_clean":"2024-09-16T02:00:35.447603+0000","last_became_active":"2024-09-16T02:00:34.462668+0000","last_became_peered":"2024-09-16T02:00:34.462668+0000","last_unstale":"2024-09-16T02:00:35.447603+0000","last_undegraded":"2024-09-16T02:00:35.447603+0000","last_fullsized":"2024-09-16T02:00:35.447603+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:06:31.655869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458464+0000","last_change":"2024-09-16T02:00:34.462241+0000","last_active":"2024-09-16T02:00:35.458464+0000","last_peered":"2024-09-16T02:00:35.458464+0000","last_clean":"2024-09-16T02:00:35.458464+0000","last_became_active":"2024-09-16T02:00:34.461925+0000","last_became_peered":"2024-09-16T02:00:34.461925+0000","last_unstale":"2024-09-16T02:00:35.458464+0000","last_undegraded":"2024-09-16T02:00:35.458464+0000","last_fullsized":"2024-09-16T02:00:35.458464+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:48:02.414471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458170+0000","last_change":"2024-09-16T02:00:34.466132+0000","last_active":"2024-09-16T02:00:35.458170+0000","last_peered":"2024-09-16T02:00:35.458170+0000","last_clean":"2024-09-16T02:00:35.458170+0000","last_became_active":"2024-09-16T02:00:34.465750+0000","last_became_peered":"2024-09-16T02:00:34.465750+0000","last_unstale":"2024-09-16T02:00:35.458170+0000","last_undegraded":"2024-09-16T02:00:35.458170+0000","last_fullsized":"2024-09-16T02:00:35.458170+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:07:33.072432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458928+0000","last_change":"2024-09-16T02:00:34.461759+0000","last_active":"2024-09-16T02:00:35.458928+0000","last_peered":"2024-09-16T02:00:35.458928+0000","last_clean":"2024-09-16T02:00:35.458928+0000","last_became_active":"2024-09-16T02:00:34.461385+0000","last_became_peered":"2024-09-16T02:00:34.461385+0000","last_unstale":"2024-09-16T02:00:35.458928+0000","last_undegraded":"2024-09-16T02:00:35.458928+0000","last_fullsized":"2024-09-16T02:00:35.458928+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:00:24.179172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.446281+0000","last_change":"2024-09-16T02:00:34.456584+0000","last_active":"2024-09-16T02:00:35.446281+0000","last_peered":"2024-09-16T02:00:35.446281+0000","last_clean":"2024-09-16T02:00:35.446281+0000","last_became_active":"2024-09-16T02:00:34.456257+0000","last_became_peered":"2024-09-16T02:00:34.456257+0000","last_unstale":"2024-09-16T02:00:35.446281+0000","last_undegraded":"2024-09-16T02:00:35.446281+0000","last_fullsized":"2024-09-16T02:00:35.446281+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:26:45.835435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458070+0000","last_change":"2024-09-16T02:00:34.460680+0000","last_active":"2024-09-16T02:00:35.458070+0000","last_peered":"2024-09-16T02:00:35.458070+0000","last_clean":"2024-09-16T02:00:35.458070+0000","last_became_active":"2024-09-16T02:00:34.460161+0000","last_became_peered":"2024-09-16T02:00:34.460161+0000","last_unstale":"2024-09-16T02:00:35.458070+0000","last_undegraded":"2024-09-16T02:00:35.458070+0000","last_fullsized":"2024-09-16T02:00:35.458070+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:38:19.848420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458271+0000","last_change":"2024-09-16T02:00:34.454606+0000","last_active":"2024-09-16T02:00:35.458271+0000","last_peered":"2024-09-16T02:00:35.458271+0000","last_clean":"2024-09-16T02:00:35.458271+0000","last_became_active":"2024-09-16T02:00:34.454293+0000","last_became_peered":"2024-09-16T02:00:34.454293+0000","last_unstale":"2024-09-16T02:00:35.458271+0000","last_undegraded":"2024-09-16T02:00:35.458271+0000","last_fullsized":"2024-09-16T02:00:35.458271+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:55:54.466176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.446992+0000","last_change":"2024-09-16T02:00:34.452940+0000","last_active":"2024-09-16T02:00:35.446992+0000","last_peered":"2024-09-16T02:00:35.446992+0000","last_clean":"2024-09-16T02:00:35.446992+0000","last_became_active":"2024-09-16T02:00:34.452560+0000","last_became_peered":"2024-09-16T02:00:34.452560+0000","last_unstale":"2024-09-16T02:00:35.446992+0000","last_undegraded":"2024-09-16T02:00:35.446992+0000","last_fullsized":"2024-09-16T02:00:35.446992+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:42:11.564264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.463540+0000","last_change":"2024-09-16T02:00:34.453457+0000","last_active":"2024-09-16T02:00:35.463540+0000","last_peered":"2024-09-16T02:00:35.463540+0000","last_clean":"2024-09-16T02:00:35.463540+0000","last_became_active":"2024-09-16T02:00:34.453148+0000","last_became_peered":"2024-09-16T02:00:34.453148+0000","last_unstale":"2024-09-16T02:00:35.463540+0000","last_undegraded":"2024-09-16T02:00:35.463540+0000","last_fullsized":"2024-09-16T02:00:35.463540+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:28.456892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458927+0000","last_change":"2024-09-16T02:00:34.457057+0000","last_active":"2024-09-16T02:00:35.458927+0000","last_peered":"2024-09-16T02:00:35.458927+0000","last_clean":"2024-09-16T02:00:35.458927+0000","last_became_active":"2024-09-16T02:00:34.456751+0000","last_became_peered":"2024-09-16T02:00:34.456751+0000","last_unstale":"2024-09-16T02:00:35.458927+0000","last_undegraded":"2024-09-16T02:00:35.458927+0000","last_fullsized":"2024-09-16T02:00:35.458927+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:32:32.207510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458938+0000","last_change":"2024-09-16T02:00:34.448434+0000","last_active":"2024-09-16T02:00:35.458938+0000","last_peered":"2024-09-16T02:00:35.458938+0000","last_clean":"2024-09-16T02:00:35.458938+0000","last_became_active":"2024-09-16T02:00:34.448056+0000","last_became_peered":"2024-09-16T02:00:34.448056+0000","last_unstale":"2024-09-16T02:00:35.458938+0000","last_undegraded":"2024-09-16T02:00:35.458938+0000","last_fullsized":"2024-09-16T02:00:35.458938+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:15:28.162159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.461643+0000","last_change":"2024-09-16T02:00:34.459499+0000","last_active":"2024-09-16T02:00:35.461643+0000","last_peered":"2024-09-16T02:00:35.461643+0000","last_clean":"2024-09-16T02:00:35.461643+0000","last_became_active":"2024-09-16T02:00:34.459017+0000","last_became_peered":"2024-09-16T02:00:34.459017+0000","last_unstale":"2024-09-16T02:00:35.461643+0000","last_undegraded":"2024-09-16T02:00:35.461643+0000","last_fullsized":"2024-09-16T02:00:35.461643+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:41:36.866100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.446287+0000","last_change":"2024-09-16T02:00:34.448640+0000","last_active":"2024-09-16T02:00:35.446287+0000","last_peered":"2024-09-16T02:00:35.446287+0000","last_clean":"2024-09-16T02:00:35.446287+0000","last_became_active":"2024-09-16T02:00:34.447737+0000","last_became_peered":"2024-09-16T02:00:34.447737+0000","last_unstale":"2024-09-16T02:00:35.446287+0000","last_undegraded":"2024-09-16T02:00:35.446287+0000","last_fullsized":"2024-09-16T02:00:35.446287+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:20.192650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.447240+0000","last_change":"2024-09-16T02:00:34.451718+0000","last_active":"2024-09-16T02:00:35.447240+0000","last_peered":"2024-09-16T02:00:35.447240+0000","last_clean":"2024-09-16T02:00:35.447240+0000","last_became_active":"2024-09-16T02:00:34.451389+0000","last_became_peered":"2024-09-16T02:00:34.451389+0000","last_unstale":"2024-09-16T02:00:35.447240+0000","last_undegraded":"2024-09-16T02:00:35.447240+0000","last_fullsized":"2024-09-16T02:00:35.447240+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:13:19.296178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.457950+0000","last_change":"2024-09-16T02:00:34.458379+0000","last_active":"2024-09-16T02:00:35.457950+0000","last_peered":"2024-09-16T02:00:35.457950+0000","last_clean":"2024-09-16T02:00:35.457950+0000","last_became_active":"2024-09-16T02:00:34.458038+0000","last_became_peered":"2024-09-16T02:00:34.458038+0000","last_unstale":"2024-09-16T02:00:35.457950+0000","last_undegraded":"2024-09-16T02:00:35.457950+0000","last_fullsized":"2024-09-16T02:00:35.457950+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:34.669470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.446886+0000","last_change":"2024-09-16T02:00:34.450927+0000","last_active":"2024-09-16T02:00:35.446886+0000","last_peered":"2024-09-16T02:00:35.446886+0000","last_clean":"2024-09-16T02:00:35.446886+0000","last_became_active":"2024-09-16T02:00:34.447669+0000","last_became_peered":"2024-09-16T02:00:34.447669+0000","last_unstale":"2024-09-16T02:00:35.446886+0000","last_undegraded":"2024-09-16T02:00:35.446886+0000","last_fullsized":"2024-09-16T02:00:35.446886+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:57.861633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.459622+0000","last_change":"2024-09-16T02:00:34.454827+0000","last_active":"2024-09-16T02:00:35.459622+0000","last_peered":"2024-09-16T02:00:35.459622+0000","last_clean":"2024-09-16T02:00:35.459622+0000","last_became_active":"2024-09-16T02:00:34.454549+0000","last_became_peered":"2024-09-16T02:00:34.454549+0000","last_unstale":"2024-09-16T02:00:35.459622+0000","last_undegraded":"2024-09-16T02:00:35.459622+0000","last_fullsized":"2024-09-16T02:00:35.459622+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:53:45.440860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.461087+0000","last_change":"2024-09-16T02:00:34.455121+0000","last_active":"2024-09-16T02:00:35.461087+0000","last_peered":"2024-09-16T02:00:35.461087+0000","last_clean":"2024-09-16T02:00:35.461087+0000","last_became_active":"2024-09-16T02:00:34.454665+0000","last_became_peered":"2024-09-16T02:00:34.454665+0000","last_unstale":"2024-09-16T02:00:35.461087+0000","last_undegraded":"2024-09-16T02:00:35.461087+0000","last_fullsized":"2024-09-16T02:00:35.461087+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:31:42.771230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.462593+0000","last_change":"2024-09-16T02:00:34.458777+0000","last_active":"2024-09-16T02:00:35.462593+0000","last_peered":"2024-09-16T02:00:35.462593+0000","last_clean":"2024-09-16T02:00:35.462593+0000","last_became_active":"2024-09-16T02:00:34.457954+0000","last_became_peered":"2024-09-16T02:00:34.457954+0000","last_unstale":"2024-09-16T02:00:35.462593+0000","last_undegraded":"2024-09-16T02:00:35.462593+0000","last_fullsized":"2024-09-16T02:00:35.462593+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:58:28.032293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.446190+0000","last_change":"2024-09-16T02:00:34.444712+0000","last_active":"2024-09-16T02:00:35.446190+0000","last_peered":"2024-09-16T02:00:35.446190+0000","last_clean":"2024-09-16T02:00:35.446190+0000","last_became_active":"2024-09-16T02:00:34.444421+0000","last_became_peered":"2024-09-16T02:00:34.444421+0000","last_unstale":"2024-09-16T02:00:35.446190+0000","last_undegraded":"2024-09-16T02:00:35.446190+0000","last_fullsized":"2024-09-16T02:00:35.446190+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:36.557558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458606+0000","last_change":"2024-09-16T02:00:34.454570+0000","last_active":"2024-09-16T02:00:35.458606+0000","last_peered":"2024-09-16T02:00:35.458606+0000","last_clean":"2024-09-16T02:00:35.458606+0000","last_became_active":"2024-09-16T02:00:34.454246+0000","last_became_peered":"2024-09-16T02:00:34.454246+0000","last_unstale":"2024-09-16T02:00:35.458606+0000","last_undegraded":"2024-09-16T02:00:35.458606+0000","last_fullsized":"2024-09-16T02:00:35.458606+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:36:18.634907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458979+0000","last_change":"2024-09-16T02:00:34.449680+0000","last_active":"2024-09-16T02:00:35.458979+0000","last_peered":"2024-09-16T02:00:35.458979+0000","last_clean":"2024-09-16T02:00:35.458979+0000","last_became_active":"2024-09-16T02:00:34.449184+0000","last_became_peered":"2024-09-16T02:00:34.449184+0000","last_unstale":"2024-09-16T02:00:35.458979+0000","last_undegraded":"2024-09-16T02:00:35.458979+0000","last_fullsized":"2024-09-16T02:00:35.458979+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:28:34.867820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458803+0000","last_change":"2024-09-16T02:00:34.455362+0000","last_active":"2024-09-16T02:00:35.458803+0000","last_peered":"2024-09-16T02:00:35.458803+0000","last_clean":"2024-09-16T02:00:35.458803+0000","last_became_active":"2024-09-16T02:00:34.455018+0000","last_became_peered":"2024-09-16T02:00:34.455018+0000","last_unstale":"2024-09-16T02:00:35.458803+0000","last_undegraded":"2024-09-16T02:00:35.458803+0000","last_fullsized":"2024-09-16T02:00:35.458803+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:38.278418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.462886+0000","last_change":"2024-09-16T02:00:34.459426+0000","last_active":"2024-09-16T02:00:35.462886+0000","last_peered":"2024-09-16T02:00:35.462886+0000","last_clean":"2024-09-16T02:00:35.462886+0000","last_became_active":"2024-09-16T02:00:34.459082+0000","last_became_peered":"2024-09-16T02:00:34.459082+0000","last_unstale":"2024-09-16T02:00:35.462886+0000","last_undegraded":"2024-09-16T02:00:35.462886+0000","last_fullsized":"2024-09-16T02:00:35.462886+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:59:42.226065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.459009+0000","last_change":"2024-09-16T02:00:34.450936+0000","last_active":"2024-09-16T02:00:35.459009+0000","last_peered":"2024-09-16T02:00:35.459009+0000","last_clean":"2024-09-16T02:00:35.459009+0000","last_became_active":"2024-09-16T02:00:34.450503+0000","last_became_peered":"2024-09-16T02:00:34.450503+0000","last_unstale":"2024-09-16T02:00:35.459009+0000","last_undegraded":"2024-09-16T02:00:35.459009+0000","last_fullsized":"2024-09-16T02:00:35.459009+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:35:55.806193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'5","reported_seq":37,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:57.551569+0000","last_change":"2024-09-16T02:00:32.443193+0000","last_active":"2024-09-16T02:00:57.551569+0000","last_peered":"2024-09-16T02:00:57.551569+0000","last_clean":"2024-09-16T02:00:57.551569+0000","last_became_active":"2024-09-16T02:00:32.442863+0000","last_became_peered":"2024-09-16T02:00:32.442863+0000","last_unstale":"2024-09-16T02:00:57.551569+0000","last_undegraded":"2024-09-16T02:00:57.551569+0000","last_fullsized":"2024-09-16T02:00:57.551569+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:19.734946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.447318+0000","last_change":"2024-09-16T02:00:34.454819+0000","last_active":"2024-09-16T02:00:35.447318+0000","last_peered":"2024-09-16T02:00:35.447318+0000","last_clean":"2024-09-16T02:00:35.447318+0000","last_became_active":"2024-09-16T02:00:34.454492+0000","last_became_peered":"2024-09-16T02:00:34.454492+0000","last_unstale":"2024-09-16T02:00:35.447318+0000","last_undegraded":"2024-09-16T02:00:35.447318+0000","last_fullsized":"2024-09-16T02:00:35.447318+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:11.479330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:45.518640+0000","last_change":"2024-09-16T02:00:32.438666+0000","last_active":"2024-09-16T02:00:45.518640+0000","last_peered":"2024-09-16T02:00:45.518640+0000","last_clean":"2024-09-16T02:00:45.518640+0000","last_became_active":"2024-09-16T02:00:32.438324+0000","last_became_peered":"2024-09-16T02:00:32.438324+0000","last_unstale":"2024-09-16T02:00:45.518640+0000","last_undegraded":"2024-09-16T02:00:45.518640+0000","last_fullsized":"2024-09-16T02:00:45.518640+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:56:28.203977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:44.506262+0000","last_change":"2024-09-16T02:00:32.438530+0000","last_active":"2024-09-16T02:00:44.506262+0000","last_peered":"2024-09-16T02:00:44.506262+0000","last_clean":"2024-09-16T02:00:44.506262+0000","last_became_active":"2024-09-16T02:00:32.438238+0000","last_became_peered":"2024-09-16T02:00:32.438238+0000","last_unstale":"2024-09-16T02:00:44.506262+0000","last_undegraded":"2024-09-16T02:00:44.506262+0000","last_fullsized":"2024-09-16T02:00:44.506262+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:52:32.799701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458329+0000","last_change":"2024-09-16T02:00:34.456299+0000","last_active":"2024-09-16T02:00:35.458329+0000","last_peered":"2024-09-16T02:00:35.458329+0000","last_clean":"2024-09-16T02:00:35.458329+0000","last_became_active":"2024-09-16T02:00:34.455925+0000","last_became_peered":"2024-09-16T02:00:34.455925+0000","last_unstale":"2024-09-16T02:00:35.458329+0000","last_undegraded":"2024-09-16T02:00:35.458329+0000","last_fullsized":"2024-09-16T02:00:35.458329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:33:27.162664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:46.527624+0000","last_change":"2024-09-16T02:00:32.442029+0000","last_active":"2024-09-16T02:00:46.527624+0000","last_peered":"2024-09-16T02:00:46.527624+0000","last_clean":"2024-09-16T02:00:46.527624+0000","last_became_active":"2024-09-16T02:00:32.441705+0000","last_became_peered":"2024-09-16T02:00:32.441705+0000","last_unstale":"2024-09-16T02:00:46.527624+0000","last_undegraded":"2024-09-16T02:00:46.527624+0000","last_fullsized":"2024-09-16T02:00:46.527624+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:50:54.725768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458032+0000","last_change":"2024-09-16T02:00:34.457833+0000","last_active":"2024-09-16T02:00:35.458032+0000","last_peered":"2024-09-16T02:00:35.458032+0000","last_clean":"2024-09-16T02:00:35.458032+0000","last_became_active":"2024-09-16T02:00:34.457453+0000","last_became_peered":"2024-09-16T02:00:34.457453+0000","last_unstale":"2024-09-16T02:00:35.458032+0000","last_undegraded":"2024-09-16T02:00:35.458032+0000","last_fullsized":"2024-09-16T02:00:35.458032+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:21.777574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":44,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:57.550881+0000","last_change":"2024-09-16T02:00:32.441341+0000","last_active":"2024-09-16T02:00:57.550881+0000","last_peered":"2024-09-16T02:00:57.550881+0000","last_clean":"2024-09-16T02:00:57.550881+0000","last_became_active":"2024-09-16T02:00:32.441019+0000","last_became_peered":"2024-09-16T02:00:32.441019+0000","last_unstale":"2024-09-16T02:00:57.550881+0000","last_undegraded":"2024-09-16T02:00:57.550881+0000","last_fullsized":"2024-09-16T02:00:57.550881+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:10:10.171714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.447426+0000","last_change":"2024-09-16T02:00:34.461746+0000","last_active":"2024-09-16T02:00:35.447426+0000","last_peered":"2024-09-16T02:00:35.447426+0000","last_clean":"2024-09-16T02:00:35.447426+0000","last_became_active":"2024-09-16T02:00:34.461441+0000","last_became_peered":"2024-09-16T02:00:34.461441+0000","last_unstale":"2024-09-16T02:00:35.447426+0000","last_undegraded":"2024-09-16T02:00:35.447426+0000","last_fullsized":"2024-09-16T02:00:35.447426+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:28:14.519159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:46.527397+0000","last_change":"2024-09-16T02:00:32.443129+0000","last_active":"2024-09-16T02:00:46.527397+0000","last_peered":"2024-09-16T02:00:46.527397+0000","last_clean":"2024-09-16T02:00:46.527397+0000","last_became_active":"2024-09-16T02:00:32.442772+0000","last_became_peered":"2024-09-16T02:00:32.442772+0000","last_unstale":"2024-09-16T02:00:46.527397+0000","last_undegraded":"2024-09-16T02:00:46.527397+0000","last_fullsized":"2024-09-16T02:00:46.527397+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:10:03.964583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458972+0000","last_change":"2024-09-16T02:00:34.457449+0000","last_active":"2024-09-16T02:00:35.458972+0000","last_peered":"2024-09-16T02:00:35.458972+0000","last_clean":"2024-09-16T02:00:35.458972+0000","last_became_active":"2024-09-16T02:00:34.457110+0000","last_became_peered":"2024-09-16T02:00:34.457110+0000","last_unstale":"2024-09-16T02:00:35.458972+0000","last_undegraded":"2024-09-16T02:00:35.458972+0000","last_fullsized":"2024-09-16T02:00:35.458972+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:35.662503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:45.516635+0000","last_change":"2024-09-16T02:00:32.587419+0000","last_active":"2024-09-16T02:00:45.516635+0000","last_peered":"2024-09-16T02:00:45.516635+0000","last_clean":"2024-09-16T02:00:45.516635+0000","last_became_active":"2024-09-16T02:00:32.587037+0000","last_became_peered":"2024-09-16T02:00:32.587037+0000","last_unstale":"2024-09-16T02:00:45.516635+0000","last_undegraded":"2024-09-16T02:00:45.516635+0000","last_fullsized":"2024-09-16T02:00:45.516635+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:11.835505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.447483+0000","last_change":"2024-09-16T02:00:34.456371+0000","last_active":"2024-09-16T02:00:35.447483+0000","last_peered":"2024-09-16T02:00:35.447483+0000","last_clean":"2024-09-16T02:00:35.447483+0000","last_became_active":"2024-09-16T02:00:34.456013+0000","last_became_peered":"2024-09-16T02:00:34.456013+0000","last_unstale":"2024-09-16T02:00:35.447483+0000","last_undegraded":"2024-09-16T02:00:35.447483+0000","last_fullsized":"2024-09-16T02:00:35.447483+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:30:15.847189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'24","reported_seq":70,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:57.552471+0000","last_change":"2024-09-16T02:00:32.444922+0000","last_active":"2024-09-16T02:00:57.552471+0000","last_peered":"2024-09-16T02:00:57.552471+0000","last_clean":"2024-09-16T02:00:57.552471+0000","last_became_active":"2024-09-16T02:00:32.444540+0000","last_became_peered":"2024-09-16T02:00:32.444540+0000","last_unstale":"2024-09-16T02:00:57.552471+0000","last_undegraded":"2024-09-16T02:00:57.552471+0000","last_fullsized":"2024-09-16T02:00:57.552471+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:59.183952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46802,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":28,"num_read_kb":59,"num_write":26,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458532+0000","last_change":"2024-09-16T02:00:34.453366+0000","last_active":"2024-09-16T02:00:35.458532+0000","last_peered":"2024-09-16T02:00:35.458532+0000","last_clean":"2024-09-16T02:00:35.458532+0000","last_became_active":"2024-09-16T02:00:34.453048+0000","last_became_peered":"2024-09-16T02:00:34.453048+0000","last_unstale":"2024-09-16T02:00:35.458532+0000","last_undegraded":"2024-09-16T02:00:35.458532+0000","last_fullsized":"2024-09-16T02:00:35.458532+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:15:52.721893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:47.535399+0000","last_change":"2024-09-16T02:00:32.445516+0000","last_active":"2024-09-16T02:00:47.535399+0000","last_peered":"2024-09-16T02:00:47.535399+0000","last_clean":"2024-09-16T02:00:47.535399+0000","last_became_active":"2024-09-16T02:00:32.445092+0000","last_became_peered":"2024-09-16T02:00:32.445092+0000","last_unstale":"2024-09-16T02:00:47.535399+0000","last_undegraded":"2024-09-16T02:00:47.535399+0000","last_fullsized":"2024-09-16T02:00:47.535399+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:54.274692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.446077+0000","last_change":"2024-09-16T02:00:34.448450+0000","last_active":"2024-09-16T02:00:35.446077+0000","last_peered":"2024-09-16T02:00:35.446077+0000","last_clean":"2024-09-16T02:00:35.446077+0000","last_became_active":"2024-09-16T02:00:34.448116+0000","last_became_peered":"2024-09-16T02:00:34.448116+0000","last_unstale":"2024-09-16T02:00:35.446077+0000","last_undegraded":"2024-09-16T02:00:35.446077+0000","last_fullsized":"2024-09-16T02:00:35.446077+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:12:34.834798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:46.527344+0000","last_change":"2024-09-16T02:00:32.445427+0000","last_active":"2024-09-16T02:00:46.527344+0000","last_peered":"2024-09-16T02:00:46.527344+0000","last_clean":"2024-09-16T02:00:46.527344+0000","last_became_active":"2024-09-16T02:00:32.445058+0000","last_became_peered":"2024-09-16T02:00:32.445058+0000","last_unstale":"2024-09-16T02:00:46.527344+0000","last_undegraded":"2024-09-16T02:00:46.527344+0000","last_fullsized":"2024-09-16T02:00:46.527344+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:35:34.701331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.459848+0000","last_change":"2024-09-16T02:00:34.458479+0000","last_active":"2024-09-16T02:00:35.459848+0000","last_peered":"2024-09-16T02:00:35.459848+0000","last_clean":"2024-09-16T02:00:35.459848+0000","last_became_active":"2024-09-16T02:00:34.458135+0000","last_became_peered":"2024-09-16T02:00:34.458135+0000","last_unstale":"2024-09-16T02:00:35.459848+0000","last_undegraded":"2024-09-16T02:00:35.459848+0000","last_fullsized":"2024-09-16T02:00:35.459848+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:37:40.188861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'6","reported_seq":34,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:52.545291+0000","last_change":"2024-09-16T02:00:32.440319+0000","last_active":"2024-09-16T02:00:52.545291+0000","last_peered":"2024-09-16T02:00:52.545291+0000","last_clean":"2024-09-16T02:00:52.545291+0000","last_became_active":"2024-09-16T02:00:32.440005+0000","last_became_peered":"2024-09-16T02:00:32.440005+0000","last_unstale":"2024-09-16T02:00:52.545291+0000","last_undegraded":"2024-09-16T02:00:52.545291+0000","last_fullsized":"2024-09-16T02:00:52.545291+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:07:33.876577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458153+0000","last_change":"2024-09-16T02:00:34.460468+0000","last_active":"2024-09-16T02:00:35.458153+0000","last_peered":"2024-09-16T02:00:35.458153+0000","last_clean":"2024-09-16T02:00:35.458153+0000","last_became_active":"2024-09-16T02:00:34.459190+0000","last_became_peered":"2024-09-16T02:00:34.459190+0000","last_unstale":"2024-09-16T02:00:35.458153+0000","last_undegraded":"2024-09-16T02:00:35.458153+0000","last_fullsized":"2024-09-16T02:00:35.458153+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:46:43.654991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:46.528118+0000","last_change":"2024-09-16T02:00:32.587700+0000","last_active":"2024-09-16T02:00:46.528118+0000","last_peered":"2024-09-16T02:00:46.528118+0000","last_clean":"2024-09-16T02:00:46.528118+0000","last_became_active":"2024-09-16T02:00:32.586964+0000","last_became_peered":"2024-09-16T02:00:32.586964+0000","last_unstale":"2024-09-16T02:00:46.528118+0000","last_undegraded":"2024-09-16T02:00:46.528118+0000","last_fullsized":"2024-09-16T02:00:46.528118+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:41.309918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.446915+0000","last_change":"2024-09-16T02:00:34.455652+0000","last_active":"2024-09-16T02:00:35.446915+0000","last_peered":"2024-09-16T02:00:35.446915+0000","last_clean":"2024-09-16T02:00:35.446915+0000","last_became_active":"2024-09-16T02:00:34.455306+0000","last_became_peered":"2024-09-16T02:00:34.455306+0000","last_unstale":"2024-09-16T02:00:35.446915+0000","last_undegraded":"2024-09-16T02:00:35.446915+0000","last_fullsized":"2024-09-16T02:00:35.446915+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:44:09.505096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:46.502399+0000","last_change":"2024-09-16T02:00:32.441443+0000","last_active":"2024-09-16T02:00:46.502399+0000","last_peered":"2024-09-16T02:00:46.502399+0000","last_clean":"2024-09-16T02:00:46.502399+0000","last_became_active":"2024-09-16T02:00:32.441073+0000","last_became_peered":"2024-09-16T02:00:32.441073+0000","last_unstale":"2024-09-16T02:00:46.502399+0000","last_undegraded":"2024-09-16T02:00:46.502399+0000","last_fullsized":"2024-09-16T02:00:46.502399+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:16.865970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.462234+0000","last_change":"2024-09-16T02:00:34.458108+0000","last_active":"2024-09-16T02:00:35.462234+0000","last_peered":"2024-09-16T02:00:35.462234+0000","last_clean":"2024-09-16T02:00:35.462234+0000","last_became_active":"2024-09-16T02:00:34.457766+0000","last_became_peered":"2024-09-16T02:00:34.457766+0000","last_unstale":"2024-09-16T02:00:35.462234+0000","last_undegraded":"2024-09-16T02:00:35.462234+0000","last_fullsized":"2024-09-16T02:00:35.462234+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:21:54.410438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:46.526168+0000","last_change":"2024-09-16T02:00:32.441728+0000","last_active":"2024-09-16T02:00:46.526168+0000","last_peered":"2024-09-16T02:00:46.526168+0000","last_clean":"2024-09-16T02:00:46.526168+0000","last_became_active":"2024-09-16T02:00:32.441388+0000","last_became_peered":"2024-09-16T02:00:32.441388+0000","last_unstale":"2024-09-16T02:00:46.526168+0000","last_undegraded":"2024-09-16T02:00:46.526168+0000","last_fullsized":"2024-09-16T02:00:46.526168+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:18:24.412876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.460483+0000","last_change":"2024-09-16T02:00:34.461889+0000","last_active":"2024-09-16T02:00:35.460483+0000","last_peered":"2024-09-16T02:00:35.460483+0000","last_clean":"2024-09-16T02:00:35.460483+0000","last_became_active":"2024-09-16T02:00:34.461583+0000","last_became_peered":"2024-09-16T02:00:34.461583+0000","last_unstale":"2024-09-16T02:00:35.460483+0000","last_undegraded":"2024-09-16T02:00:35.460483+0000","last_fullsized":"2024-09-16T02:00:35.460483+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:32.598422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:46.526605+0000","last_change":"2024-09-16T02:00:32.445201+0000","last_active":"2024-09-16T02:00:46.526605+0000","last_peered":"2024-09-16T02:00:46.526605+0000","last_clean":"2024-09-16T02:00:46.526605+0000","last_became_active":"2024-09-16T02:00:32.444722+0000","last_became_peered":"2024-09-16T02:00:32.444722+0000","last_unstale":"2024-09-16T02:00:46.526605+0000","last_undegraded":"2024-09-16T02:00:46.526605+0000","last_fullsized":"2024-09-16T02:00:46.526605+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:41:11.657028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'1","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.479926+0000","last_change":"2024-09-16T02:00:34.456108+0000","last_active":"2024-09-16T02:00:35.479926+0000","last_peered":"2024-09-16T02:00:35.479926+0000","last_clean":"2024-09-16T02:00:35.479926+0000","last_became_active":"2024-09-16T02:00:34.455718+0000","last_became_peered":"2024-09-16T02:00:34.455718+0000","last_unstale":"2024-09-16T02:00:35.479926+0000","last_undegraded":"2024-09-16T02:00:35.479926+0000","last_fullsized":"2024-09-16T02:00:35.479926+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:27:09.201182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'5","reported_seq":37,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:57.551325+0000","last_change":"2024-09-16T02:00:32.860059+0000","last_active":"2024-09-16T02:00:57.551325+0000","last_peered":"2024-09-16T02:00:57.551325+0000","last_clean":"2024-09-16T02:00:57.551325+0000","last_became_active":"2024-09-16T02:00:32.859617+0000","last_became_peered":"2024-09-16T02:00:32.859617+0000","last_unstale":"2024-09-16T02:00:57.551325+0000","last_undegraded":"2024-09-16T02:00:57.551325+0000","last_fullsized":"2024-09-16T02:00:57.551325+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:41:50.248194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.459816+0000","last_change":"2024-09-16T02:00:34.454344+0000","last_active":"2024-09-16T02:00:35.459816+0000","last_peered":"2024-09-16T02:00:35.459816+0000","last_clean":"2024-09-16T02:00:35.459816+0000","last_became_active":"2024-09-16T02:00:34.454027+0000","last_became_peered":"2024-09-16T02:00:34.454027+0000","last_unstale":"2024-09-16T02:00:35.459816+0000","last_undegraded":"2024-09-16T02:00:35.459816+0000","last_fullsized":"2024-09-16T02:00:35.459816+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:33.020365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":330,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458425+0000","last_change":"2024-09-16T01:59:19.798159+0000","last_active":"2024-09-16T02:00:35.458425+0000","last_peered":"2024-09-16T02:00:35.458425+0000","last_clean":"2024-09-16T02:00:35.458425+0000","last_became_active":"2024-09-16T01:59:19.797688+0000","last_became_peered":"2024-09-16T01:59:19.797688+0000","last_unstale":"2024-09-16T02:00:35.458425+0000","last_undegraded":"2024-09-16T02:00:35.458425+0000","last_fullsized":"2024-09-16T02:00:35.458425+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:55:54.312750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:55:54.312750+0000","last_clean_scrub_stamp":"2024-09-16T01:55:54.312750+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:00:36.860158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:47.534936+0000","last_change":"2024-09-16T02:00:32.441925+0000","last_active":"2024-09-16T02:00:47.534936+0000","last_peered":"2024-09-16T02:00:47.534936+0000","last_clean":"2024-09-16T02:00:47.534936+0000","last_became_active":"2024-09-16T02:00:32.441565+0000","last_became_peered":"2024-09-16T02:00:32.441565+0000","last_unstale":"2024-09-16T02:00:47.534936+0000","last_undegraded":"2024-09-16T02:00:47.534936+0000","last_fullsized":"2024-09-16T02:00:47.534936+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:23:09.755034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.460113+0000","last_change":"2024-09-16T02:00:34.452986+0000","last_active":"2024-09-16T02:00:35.460113+0000","last_peered":"2024-09-16T02:00:35.460113+0000","last_clean":"2024-09-16T02:00:35.460113+0000","last_became_active":"2024-09-16T02:00:34.452688+0000","last_became_peered":"2024-09-16T02:00:34.452688+0000","last_unstale":"2024-09-16T02:00:35.460113+0000","last_undegraded":"2024-09-16T02:00:35.460113+0000","last_fullsized":"2024-09-16T02:00:35.460113+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:12.958209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:44.506336+0000","last_change":"2024-09-16T02:00:32.443964+0000","last_active":"2024-09-16T02:00:44.506336+0000","last_peered":"2024-09-16T02:00:44.506336+0000","last_clean":"2024-09-16T02:00:44.506336+0000","last_became_active":"2024-09-16T02:00:32.443623+0000","last_became_peered":"2024-09-16T02:00:32.443623+0000","last_unstale":"2024-09-16T02:00:44.506336+0000","last_undegraded":"2024-09-16T02:00:44.506336+0000","last_fullsized":"2024-09-16T02:00:44.506336+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:30:34.406024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.446017+0000","last_change":"2024-09-16T02:00:34.449615+0000","last_active":"2024-09-16T02:00:35.446017+0000","last_peered":"2024-09-16T02:00:35.446017+0000","last_clean":"2024-09-16T02:00:35.446017+0000","last_became_active":"2024-09-16T02:00:34.449109+0000","last_became_peered":"2024-09-16T02:00:34.449109+0000","last_unstale":"2024-09-16T02:00:35.446017+0000","last_undegraded":"2024-09-16T02:00:35.446017+0000","last_fullsized":"2024-09-16T02:00:35.446017+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:45.036177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:53.545994+0000","last_change":"2024-09-16T02:00:32.587107+0000","last_active":"2024-09-16T02:00:53.545994+0000","last_peered":"2024-09-16T02:00:53.545994+0000","last_clean":"2024-09-16T02:00:53.545994+0000","last_became_active":"2024-09-16T02:00:32.586777+0000","last_became_peered":"2024-09-16T02:00:32.586777+0000","last_unstale":"2024-09-16T02:00:53.545994+0000","last_undegraded":"2024-09-16T02:00:53.545994+0000","last_fullsized":"2024-09-16T02:00:53.545994+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:03.968987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.447491+0000","last_change":"2024-09-16T02:00:34.453500+0000","last_active":"2024-09-16T02:00:35.447491+0000","last_peered":"2024-09-16T02:00:35.447491+0000","last_clean":"2024-09-16T02:00:35.447491+0000","last_became_active":"2024-09-16T02:00:34.453175+0000","last_became_peered":"2024-09-16T02:00:34.453175+0000","last_unstale":"2024-09-16T02:00:35.447491+0000","last_undegraded":"2024-09-16T02:00:35.447491+0000","last_fullsized":"2024-09-16T02:00:35.447491+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:52.666456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'5","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:53.545521+0000","last_change":"2024-09-16T02:00:32.443201+0000","last_active":"2024-09-16T02:00:53.545521+0000","last_peered":"2024-09-16T02:00:53.545521+0000","last_clean":"2024-09-16T02:00:53.545521+0000","last_became_active":"2024-09-16T02:00:32.442778+0000","last_became_peered":"2024-09-16T02:00:32.442778+0000","last_unstale":"2024-09-16T02:00:53.545521+0000","last_undegraded":"2024-09-16T02:00:53.545521+0000","last_fullsized":"2024-09-16T02:00:53.545521+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:43.933817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458153+0000","last_change":"2024-09-16T02:00:34.463000+0000","last_active":"2024-09-16T02:00:35.458153+0000","last_peered":"2024-09-16T02:00:35.458153+0000","last_clean":"2024-09-16T02:00:35.458153+0000","last_became_active":"2024-09-16T02:00:34.461890+0000","last_became_peered":"2024-09-16T02:00:34.461890+0000","last_unstale":"2024-09-16T02:00:35.458153+0000","last_undegraded":"2024-09-16T02:00:35.458153+0000","last_fullsized":"2024-09-16T02:00:35.458153+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:21:15.161295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:36.132963+0000","last_change":"2024-09-16T02:00:32.586933+0000","last_active":"2024-09-16T02:00:36.132963+0000","last_peered":"2024-09-16T02:00:36.132963+0000","last_clean":"2024-09-16T02:00:36.132963+0000","last_became_active":"2024-09-16T02:00:32.586565+0000","last_became_peered":"2024-09-16T02:00:32.586565+0000","last_unstale":"2024-09-16T02:00:36.132963+0000","last_undegraded":"2024-09-16T02:00:36.132963+0000","last_fullsized":"2024-09-16T02:00:36.132963+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:45:07.130649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:37.582252+0000","last_change":"2024-09-16T02:00:34.460805+0000","last_active":"2024-09-16T02:00:37.582252+0000","last_peered":"2024-09-16T02:00:37.582252+0000","last_clean":"2024-09-16T02:00:37.582252+0000","last_became_active":"2024-09-16T02:00:34.460482+0000","last_became_peered":"2024-09-16T02:00:34.460482+0000","last_unstale":"2024-09-16T02:00:37.582252+0000","last_undegraded":"2024-09-16T02:00:37.582252+0000","last_fullsized":"2024-09-16T02:00:37.582252+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:07:51.565053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:46.527428+0000","last_change":"2024-09-16T02:00:32.587516+0000","last_active":"2024-09-16T02:00:46.527428+0000","last_peered":"2024-09-16T02:00:46.527428+0000","last_clean":"2024-09-16T02:00:46.527428+0000","last_became_active":"2024-09-16T02:00:32.587172+0000","last_became_peered":"2024-09-16T02:00:32.587172+0000","last_unstale":"2024-09-16T02:00:46.527428+0000","last_undegraded":"2024-09-16T02:00:46.527428+0000","last_fullsized":"2024-09-16T02:00:46.527428+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:52:29.018290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.446340+0000","last_change":"2024-09-16T02:00:34.446021+0000","last_active":"2024-09-16T02:00:35.446340+0000","last_peered":"2024-09-16T02:00:35.446340+0000","last_clean":"2024-09-16T02:00:35.446340+0000","last_became_active":"2024-09-16T02:00:34.445732+0000","last_became_peered":"2024-09-16T02:00:34.445732+0000","last_unstale":"2024-09-16T02:00:35.446340+0000","last_undegraded":"2024-09-16T02:00:35.446340+0000","last_fullsized":"2024-09-16T02:00:35.446340+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:51:17.960792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'10","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:51.457810+0000","last_change":"2024-09-16T02:00:32.439750+0000","last_active":"2024-09-16T02:00:51.457810+0000","last_peered":"2024-09-16T02:00:51.457810+0000","last_clean":"2024-09-16T02:00:51.457810+0000","last_became_active":"2024-09-16T02:00:32.439438+0000","last_became_peered":"2024-09-16T02:00:32.439438+0000","last_unstale":"2024-09-16T02:00:51.457810+0000","last_undegraded":"2024-09-16T02:00:51.457810+0000","last_fullsized":"2024-09-16T02:00:51.457810+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:50.764733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.459772+0000","last_change":"2024-09-16T02:00:34.453810+0000","last_active":"2024-09-16T02:00:35.459772+0000","last_peered":"2024-09-16T02:00:35.459772+0000","last_clean":"2024-09-16T02:00:35.459772+0000","last_became_active":"2024-09-16T02:00:34.453421+0000","last_became_peered":"2024-09-16T02:00:34.453421+0000","last_unstale":"2024-09-16T02:00:35.459772+0000","last_undegraded":"2024-09-16T02:00:35.459772+0000","last_fullsized":"2024-09-16T02:00:35.459772+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:23:50.405749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:46.525899+0000","last_change":"2024-09-16T02:00:32.860565+0000","last_active":"2024-09-16T02:00:46.525899+0000","last_peered":"2024-09-16T02:00:46.525899+0000","last_clean":"2024-09-16T02:00:46.525899+0000","last_became_active":"2024-09-16T02:00:32.860130+0000","last_became_peered":"2024-09-16T02:00:32.860130+0000","last_unstale":"2024-09-16T02:00:46.525899+0000","last_undegraded":"2024-09-16T02:00:46.525899+0000","last_fullsized":"2024-09-16T02:00:46.525899+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:09:55.759883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458637+0000","last_change":"2024-09-16T02:00:34.454033+0000","last_active":"2024-09-16T02:00:35.458637+0000","last_peered":"2024-09-16T02:00:35.458637+0000","last_clean":"2024-09-16T02:00:35.458637+0000","last_became_active":"2024-09-16T02:00:34.453725+0000","last_became_peered":"2024-09-16T02:00:34.453725+0000","last_unstale":"2024-09-16T02:00:35.458637+0000","last_undegraded":"2024-09-16T02:00:35.458637+0000","last_fullsized":"2024-09-16T02:00:35.458637+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:15.702752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:43.496005+0000","last_change":"2024-09-16T02:00:32.587115+0000","last_active":"2024-09-16T02:00:43.496005+0000","last_peered":"2024-09-16T02:00:43.496005+0000","last_clean":"2024-09-16T02:00:43.496005+0000","last_became_active":"2024-09-16T02:00:32.586835+0000","last_became_peered":"2024-09-16T02:00:32.586835+0000","last_unstale":"2024-09-16T02:00:43.496005+0000","last_undegraded":"2024-09-16T02:00:43.496005+0000","last_fullsized":"2024-09-16T02:00:43.496005+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:12:58.820005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458137+0000","last_change":"2024-09-16T02:00:34.457673+0000","last_active":"2024-09-16T02:00:35.458137+0000","last_peered":"2024-09-16T02:00:35.458137+0000","last_clean":"2024-09-16T02:00:35.458137+0000","last_became_active":"2024-09-16T02:00:34.457331+0000","last_became_peered":"2024-09-16T02:00:34.457331+0000","last_unstale":"2024-09-16T02:00:35.458137+0000","last_undegraded":"2024-09-16T02:00:35.458137+0000","last_fullsized":"2024-09-16T02:00:35.458137+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:12:16.100036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:46.526274+0000","last_change":"2024-09-16T02:00:32.437528+0000","last_active":"2024-09-16T02:00:46.526274+0000","last_peered":"2024-09-16T02:00:46.526274+0000","last_clean":"2024-09-16T02:00:46.526274+0000","last_became_active":"2024-09-16T02:00:32.437080+0000","last_became_peered":"2024-09-16T02:00:32.437080+0000","last_unstale":"2024-09-16T02:00:46.526274+0000","last_undegraded":"2024-09-16T02:00:46.526274+0000","last_fullsized":"2024-09-16T02:00:46.526274+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:59:33.908026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.463059+0000","last_change":"2024-09-16T02:00:34.453504+0000","last_active":"2024-09-16T02:00:35.463059+0000","last_peered":"2024-09-16T02:00:35.463059+0000","last_clean":"2024-09-16T02:00:35.463059+0000","last_became_active":"2024-09-16T02:00:34.453212+0000","last_became_peered":"2024-09-16T02:00:34.453212+0000","last_unstale":"2024-09-16T02:00:35.463059+0000","last_undegraded":"2024-09-16T02:00:35.463059+0000","last_fullsized":"2024-09-16T02:00:35.463059+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:00:56.414548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:45.518572+0000","last_change":"2024-09-16T02:00:32.442282+0000","last_active":"2024-09-16T02:00:45.518572+0000","last_peered":"2024-09-16T02:00:45.518572+0000","last_clean":"2024-09-16T02:00:45.518572+0000","last_became_active":"2024-09-16T02:00:32.441910+0000","last_became_peered":"2024-09-16T02:00:32.441910+0000","last_unstale":"2024-09-16T02:00:45.518572+0000","last_undegraded":"2024-09-16T02:00:45.518572+0000","last_fullsized":"2024-09-16T02:00:45.518572+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:01:46.045021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.459686+0000","last_change":"2024-09-16T02:00:34.460177+0000","last_active":"2024-09-16T02:00:35.459686+0000","last_peered":"2024-09-16T02:00:35.459686+0000","last_clean":"2024-09-16T02:00:35.459686+0000","last_became_active":"2024-09-16T02:00:34.459874+0000","last_became_peered":"2024-09-16T02:00:34.459874+0000","last_unstale":"2024-09-16T02:00:35.459686+0000","last_undegraded":"2024-09-16T02:00:35.459686+0000","last_fullsized":"2024-09-16T02:00:35.459686+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:35.855591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:43.496920+0000","last_change":"2024-09-16T02:00:32.443441+0000","last_active":"2024-09-16T02:00:43.496920+0000","last_peered":"2024-09-16T02:00:43.496920+0000","last_clean":"2024-09-16T02:00:43.496920+0000","last_became_active":"2024-09-16T02:00:32.443095+0000","last_became_peered":"2024-09-16T02:00:32.443095+0000","last_unstale":"2024-09-16T02:00:43.496920+0000","last_undegraded":"2024-09-16T02:00:43.496920+0000","last_fullsized":"2024-09-16T02:00:43.496920+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:16.680798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.459028+0000","last_change":"2024-09-16T02:00:34.452964+0000","last_active":"2024-09-16T02:00:35.459028+0000","last_peered":"2024-09-16T02:00:35.459028+0000","last_clean":"2024-09-16T02:00:35.459028+0000","last_became_active":"2024-09-16T02:00:34.452617+0000","last_became_peered":"2024-09-16T02:00:34.452617+0000","last_unstale":"2024-09-16T02:00:35.459028+0000","last_undegraded":"2024-09-16T02:00:35.459028+0000","last_fullsized":"2024-09-16T02:00:35.459028+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:04.617970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.447223+0000","last_change":"2024-09-16T02:00:34.456585+0000","last_active":"2024-09-16T02:00:35.447223+0000","last_peered":"2024-09-16T02:00:35.447223+0000","last_clean":"2024-09-16T02:00:35.447223+0000","last_became_active":"2024-09-16T02:00:34.456199+0000","last_became_peered":"2024-09-16T02:00:34.456199+0000","last_unstale":"2024-09-16T02:00:35.447223+0000","last_undegraded":"2024-09-16T02:00:35.447223+0000","last_fullsized":"2024-09-16T02:00:35.447223+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:37:54.376785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:46.527336+0000","last_change":"2024-09-16T02:00:32.446324+0000","last_active":"2024-09-16T02:00:46.527336+0000","last_peered":"2024-09-16T02:00:46.527336+0000","last_clean":"2024-09-16T02:00:46.527336+0000","last_became_active":"2024-09-16T02:00:32.441840+0000","last_became_peered":"2024-09-16T02:00:32.441840+0000","last_unstale":"2024-09-16T02:00:46.527336+0000","last_undegraded":"2024-09-16T02:00:46.527336+0000","last_fullsized":"2024-09-16T02:00:46.527336+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:46:08.601738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.447200+0000","last_change":"2024-09-16T02:00:34.460925+0000","last_active":"2024-09-16T02:00:35.447200+0000","last_peered":"2024-09-16T02:00:35.447200+0000","last_clean":"2024-09-16T02:00:35.447200+0000","last_became_active":"2024-09-16T02:00:34.460581+0000","last_became_peered":"2024-09-16T02:00:34.460581+0000","last_unstale":"2024-09-16T02:00:35.447200+0000","last_undegraded":"2024-09-16T02:00:35.447200+0000","last_fullsized":"2024-09-16T02:00:35.447200+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:19:44.460695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:47.534682+0000","last_change":"2024-09-16T02:00:32.443877+0000","last_active":"2024-09-16T02:00:47.534682+0000","last_peered":"2024-09-16T02:00:47.534682+0000","last_clean":"2024-09-16T02:00:47.534682+0000","last_became_active":"2024-09-16T02:00:32.443416+0000","last_became_peered":"2024-09-16T02:00:32.443416+0000","last_unstale":"2024-09-16T02:00:47.534682+0000","last_undegraded":"2024-09-16T02:00:47.534682+0000","last_fullsized":"2024-09-16T02:00:47.534682+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:36:57.367232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458846+0000","last_change":"2024-09-16T02:00:34.453048+0000","last_active":"2024-09-16T02:00:35.458846+0000","last_peered":"2024-09-16T02:00:35.458846+0000","last_clean":"2024-09-16T02:00:35.458846+0000","last_became_active":"2024-09-16T02:00:34.452752+0000","last_became_peered":"2024-09-16T02:00:34.452752+0000","last_unstale":"2024-09-16T02:00:35.458846+0000","last_undegraded":"2024-09-16T02:00:35.458846+0000","last_fullsized":"2024-09-16T02:00:35.458846+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:55.970195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.463149+0000","last_change":"2024-09-16T02:00:32.586974+0000","last_active":"2024-09-16T02:00:35.463149+0000","last_peered":"2024-09-16T02:00:35.463149+0000","last_clean":"2024-09-16T02:00:35.463149+0000","last_became_active":"2024-09-16T02:00:32.586644+0000","last_became_peered":"2024-09-16T02:00:32.586644+0000","last_unstale":"2024-09-16T02:00:35.463149+0000","last_undegraded":"2024-09-16T02:00:35.463149+0000","last_fullsized":"2024-09-16T02:00:35.463149+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:52.913287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.460108+0000","last_change":"2024-09-16T02:00:34.447548+0000","last_active":"2024-09-16T02:00:35.460108+0000","last_peered":"2024-09-16T02:00:35.460108+0000","last_clean":"2024-09-16T02:00:35.460108+0000","last_became_active":"2024-09-16T02:00:34.447207+0000","last_became_peered":"2024-09-16T02:00:34.447207+0000","last_unstale":"2024-09-16T02:00:35.460108+0000","last_undegraded":"2024-09-16T02:00:35.460108+0000","last_fullsized":"2024-09-16T02:00:35.460108+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:22:02.494343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":41,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:54.546306+0000","last_change":"2024-09-16T02:00:32.441129+0000","last_active":"2024-09-16T02:00:54.546306+0000","last_peered":"2024-09-16T02:00:54.546306+0000","last_clean":"2024-09-16T02:00:54.546306+0000","last_became_active":"2024-09-16T02:00:32.440843+0000","last_became_peered":"2024-09-16T02:00:32.440843+0000","last_unstale":"2024-09-16T02:00:54.546306+0000","last_undegraded":"2024-09-16T02:00:54.546306+0000","last_fullsized":"2024-09-16T02:00:54.546306+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:17.721740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":53617,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":137,"num_read_kb":129,"num_write":226,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":520192,"data_stored":155114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281576,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612626,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":864256,"data_stored":414884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841559,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1007616,"data_stored":1053108,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005083,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":414814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234016,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":415192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462949,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":937984,"data_stored":1007712,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626475,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":872448,"data_stored":414531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855408,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":1006997,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018932,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":414785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247866,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":414356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444096,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":942080,"data_stored":462466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":897024,"data_stored":437776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":415870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":42817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":48774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":48931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T02:01:02.035 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T02:01:02.281 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:02.732 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:02 smithi105 ceph-mon[31197]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:01:02.732 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:02 smithi105 ceph-mon[31197]: pgmap v290: 97 pgs: 97 active+clean; 629 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 KiB/s wr, 17 op/s 2024-09-16T02:01:02.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.810+0000 7efe8baa4640 1 -- 172.21.15.105:0/3144535929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe84102570 msgr2=0x7efe841029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:02.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.810+0000 7efe8baa4640 1 --2- 172.21.15.105:0/3144535929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe84102570 0x7efe841029b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efe78009900 tx=0x7efe7802a480 comp rx=0 tx=0).stop 2024-09-16T02:01:02.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.810+0000 7efe8baa4640 1 -- 172.21.15.105:0/3144535929 shutdown_connections 2024-09-16T02:01:02.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.810+0000 7efe8baa4640 1 --2- 172.21.15.105:0/3144535929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe84102ef0 0x7efe8410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:02.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.810+0000 7efe8baa4640 1 --2- 172.21.15.105:0/3144535929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe84102570 0x7efe841029b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:02.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.810+0000 7efe8baa4640 1 --2- 172.21.15.105:0/3144535929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe84100e20 0x7efe84101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:02.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.810+0000 7efe8baa4640 1 -- 172.21.15.105:0/3144535929 >> 172.21.15.105:0/3144535929 conn(0x7efe840fc7d0 msgr2=0x7efe840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:02.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.810+0000 7efe8baa4640 1 -- 172.21.15.105:0/3144535929 shutdown_connections 2024-09-16T02:01:02.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.810+0000 7efe8baa4640 1 -- 172.21.15.105:0/3144535929 wait complete. 2024-09-16T02:01:02.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.811+0000 7efe8baa4640 1 Processor -- start 2024-09-16T02:01:02.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.811+0000 7efe8baa4640 1 -- start start 2024-09-16T02:01:02.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.811+0000 7efe8baa4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe84100e20 0x7efe841984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:02.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.811+0000 7efe8baa4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe84102570 0x7efe841989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:02.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.811+0000 7efe8baa4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe84102ef0 0x7efe8419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:02.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.811+0000 7efe8baa4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efe84111f50 con 0x7efe84102ef0 2024-09-16T02:01:02.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.811+0000 7efe8baa4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efe84111dd0 con 0x7efe84102570 2024-09-16T02:01:02.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.811+0000 7efe8baa4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7efe841120d0 con 0x7efe84100e20 2024-09-16T02:01:02.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.811+0000 7efe8a041640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe84102ef0 0x7efe8419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:02.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.811+0000 7efe8a041640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe84102ef0 0x7efe8419cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51196/0 (socket says 172.21.15.105:51196) 2024-09-16T02:01:02.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.811+0000 7efe8a041640 1 -- 172.21.15.105:0/1114731193 learned_addr learned my addr 172.21.15.105:0/1114731193 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:01:02.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.811+0000 7efe89840640 1 --2- 172.21.15.105:0/1114731193 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe84100e20 0x7efe841984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:02.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.811+0000 7efe8903f640 1 --2- 172.21.15.105:0/1114731193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe84102570 0x7efe841989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:02.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.811+0000 7efe8a041640 1 -- 172.21.15.105:0/1114731193 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe84100e20 msgr2=0x7efe841984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:02.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.811+0000 7efe8a041640 1 --2- 172.21.15.105:0/1114731193 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe84100e20 0x7efe841984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:02.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.811+0000 7efe8a041640 1 -- 172.21.15.105:0/1114731193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe84102570 msgr2=0x7efe841989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:02.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.811+0000 7efe8a041640 1 --2- 172.21.15.105:0/1114731193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe84102570 0x7efe841989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:02.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.811+0000 7efe8a041640 1 -- 172.21.15.105:0/1114731193 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe8419d460 con 0x7efe84102ef0 2024-09-16T02:01:02.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.811+0000 7efe8903f640 1 --2- 172.21.15.105:0/1114731193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe84102570 0x7efe841989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:01:02.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.811+0000 7efe8a041640 1 --2- 172.21.15.105:0/1114731193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe84102ef0 0x7efe8419cd80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7efe7400cad0 tx=0x7efe7400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:02.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.812+0000 7efe82ffd640 1 -- 172.21.15.105:0/1114731193 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7400ae20 con 0x7efe84102ef0 2024-09-16T02:01:02.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.812+0000 7efe8baa4640 1 -- 172.21.15.105:0/1114731193 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe8419d750 con 0x7efe84102ef0 2024-09-16T02:01:02.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.812+0000 7efe82ffd640 1 -- 172.21.15.105:0/1114731193 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe74005020 con 0x7efe84102ef0 2024-09-16T02:01:02.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.812+0000 7efe82ffd640 1 -- 172.21.15.105:0/1114731193 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7400f6b0 con 0x7efe84102ef0 2024-09-16T02:01:02.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.812+0000 7efe8baa4640 1 -- 172.21.15.105:0/1114731193 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe841a5030 con 0x7efe84102ef0 2024-09-16T02:01:02.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.813+0000 7efe8baa4640 1 -- 172.21.15.105:0/1114731193 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe84101200 con 0x7efe84102ef0 2024-09-16T02:01:02.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.814+0000 7efe82ffd640 1 -- 172.21.15.105:0/1114731193 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7efe740051d0 con 0x7efe84102ef0 2024-09-16T02:01:02.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.814+0000 7efe82ffd640 1 --2- 172.21.15.105:0/1114731193 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7efe6c0a32d0 0x7efe6c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:02.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.815+0000 7efe89840640 1 --2- 172.21.15.105:0/1114731193 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7efe6c0a32d0 0x7efe6c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:02.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.815+0000 7efe82ffd640 1 -- 172.21.15.105:0/1114731193 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7efe740cfe10 con 0x7efe84102ef0 2024-09-16T02:01:02.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.815+0000 7efe89840640 1 --2- 172.21.15.105:0/1114731193 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7efe6c0a32d0 0x7efe6c0a5790 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efe7c005fd0 tx=0x7efe7c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:02.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.818+0000 7efe82ffd640 1 -- 172.21.15.105:0/1114731193 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efe74014030 con 0x7efe84102ef0 2024-09-16T02:01:02.845 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:02 smithi190 ceph-mon[34762]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:01:02.845 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:02 smithi190 ceph-mon[34762]: pgmap v290: 97 pgs: 97 active+clean; 629 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 KiB/s wr, 17 op/s 2024-09-16T02:01:02.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:02 smithi137 ceph-mon[34099]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:01:02.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:02 smithi137 ceph-mon[34099]: pgmap v290: 97 pgs: 97 active+clean; 629 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 KiB/s wr, 17 op/s 2024-09-16T02:01:02.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.980+0000 7efe8baa4640 1 -- 172.21.15.105:0/1114731193 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efe84109100 con 0x7efe6c0a32d0 2024-09-16T02:01:02.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.986+0000 7efe82ffd640 1 -- 172.21.15.105:0/1114731193 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258912 (secure 0 0 0) 0x7efe7c003890 con 0x7efe6c0a32d0 2024-09-16T02:01:02.989 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:01:02.990 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-09-16T02:01:02.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.989+0000 7efe8baa4640 1 -- 172.21.15.105:0/1114731193 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7efe6c0a32d0 msgr2=0x7efe6c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:02.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.989+0000 7efe8baa4640 1 --2- 172.21.15.105:0/1114731193 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7efe6c0a32d0 0x7efe6c0a5790 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efe7c005fd0 tx=0x7efe7c00b390 comp rx=0 tx=0).stop 2024-09-16T02:01:02.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.989+0000 7efe8baa4640 1 -- 172.21.15.105:0/1114731193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe84102ef0 msgr2=0x7efe8419cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:02.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.989+0000 7efe8baa4640 1 --2- 172.21.15.105:0/1114731193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe84102ef0 0x7efe8419cd80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7efe7400cad0 tx=0x7efe7400cf90 comp rx=0 tx=0).stop 2024-09-16T02:01:02.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.989+0000 7efe8baa4640 1 -- 172.21.15.105:0/1114731193 shutdown_connections 2024-09-16T02:01:02.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.989+0000 7efe8baa4640 1 --2- 172.21.15.105:0/1114731193 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7efe6c0a32d0 0x7efe6c0a5790 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:02.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.989+0000 7efe8baa4640 1 --2- 172.21.15.105:0/1114731193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe84102ef0 0x7efe8419cd80 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:02.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.989+0000 7efe8baa4640 1 --2- 172.21.15.105:0/1114731193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe84102570 0x7efe841989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:02.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.989+0000 7efe8baa4640 1 --2- 172.21.15.105:0/1114731193 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe84100e20 0x7efe841984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:02.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.989+0000 7efe8baa4640 1 -- 172.21.15.105:0/1114731193 >> 172.21.15.105:0/1114731193 conn(0x7efe840fc7d0 msgr2=0x7efe840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:02.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.989+0000 7efe8baa4640 1 -- 172.21.15.105:0/1114731193 shutdown_connections 2024-09-16T02:01:02.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:02.989+0000 7efe8baa4640 1 -- 172.21.15.105:0/1114731193 wait complete. 2024-09-16T02:01:03.833 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2024-09-16T02:01:01.438558+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":644219,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":282,"num_read_kb":247,"num_write":415,"num_write_kb":3055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":339,"ondisk_log_size":339,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364076,"kb_used_data":10668,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577748,"statfs":{"total":1151940427776,"available":1151567613952,"internally_reserved":0,"allocated":10924032,"data_stored":6896961,"data_compressed":107436,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":47833,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":135,"num_read_kb":122,"num_write":81,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003108"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.463390+0000","last_change":"2024-09-16T02:00:34.454121+0000","last_active":"2024-09-16T02:00:35.463390+0000","last_peered":"2024-09-16T02:00:35.463390+0000","last_clean":"2024-09-16T02:00:35.463390+0000","last_became_active":"2024-09-16T02:00:34.453810+0000","last_became_peered":"2024-09-16T02:00:34.453810+0000","last_unstale":"2024-09-16T02:00:35.463390+0000","last_undegraded":"2024-09-16T02:00:35.463390+0000","last_fullsized":"2024-09-16T02:00:35.463390+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:42:38.781928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458143+0000","last_change":"2024-09-16T02:00:34.462020+0000","last_active":"2024-09-16T02:00:35.458143+0000","last_peered":"2024-09-16T02:00:35.458143+0000","last_clean":"2024-09-16T02:00:35.458143+0000","last_became_active":"2024-09-16T02:00:34.461680+0000","last_became_peered":"2024-09-16T02:00:34.461680+0000","last_unstale":"2024-09-16T02:00:35.458143+0000","last_undegraded":"2024-09-16T02:00:35.458143+0000","last_fullsized":"2024-09-16T02:00:35.458143+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:00:14.493643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458302+0000","last_change":"2024-09-16T02:00:34.453034+0000","last_active":"2024-09-16T02:00:35.458302+0000","last_peered":"2024-09-16T02:00:35.458302+0000","last_clean":"2024-09-16T02:00:35.458302+0000","last_became_active":"2024-09-16T02:00:34.452719+0000","last_became_peered":"2024-09-16T02:00:34.452719+0000","last_unstale":"2024-09-16T02:00:35.458302+0000","last_undegraded":"2024-09-16T02:00:35.458302+0000","last_fullsized":"2024-09-16T02:00:35.458302+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:48:52.478501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458864+0000","last_change":"2024-09-16T02:00:34.458156+0000","last_active":"2024-09-16T02:00:35.458864+0000","last_peered":"2024-09-16T02:00:35.458864+0000","last_clean":"2024-09-16T02:00:35.458864+0000","last_became_active":"2024-09-16T02:00:34.457831+0000","last_became_peered":"2024-09-16T02:00:34.457831+0000","last_unstale":"2024-09-16T02:00:35.458864+0000","last_undegraded":"2024-09-16T02:00:35.458864+0000","last_fullsized":"2024-09-16T02:00:35.458864+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:35:36.609046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458845+0000","last_change":"2024-09-16T02:00:34.448638+0000","last_active":"2024-09-16T02:00:35.458845+0000","last_peered":"2024-09-16T02:00:35.458845+0000","last_clean":"2024-09-16T02:00:35.458845+0000","last_became_active":"2024-09-16T02:00:34.447993+0000","last_became_peered":"2024-09-16T02:00:34.447993+0000","last_unstale":"2024-09-16T02:00:35.458845+0000","last_undegraded":"2024-09-16T02:00:35.458845+0000","last_fullsized":"2024-09-16T02:00:35.458845+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:33:04.832053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.459310+0000","last_change":"2024-09-16T02:00:34.462651+0000","last_active":"2024-09-16T02:00:35.459310+0000","last_peered":"2024-09-16T02:00:35.459310+0000","last_clean":"2024-09-16T02:00:35.459310+0000","last_became_active":"2024-09-16T02:00:34.461837+0000","last_became_peered":"2024-09-16T02:00:34.461837+0000","last_unstale":"2024-09-16T02:00:35.459310+0000","last_undegraded":"2024-09-16T02:00:35.459310+0000","last_fullsized":"2024-09-16T02:00:35.459310+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:52:13.733591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458629+0000","last_change":"2024-09-16T02:00:34.455362+0000","last_active":"2024-09-16T02:00:35.458629+0000","last_peered":"2024-09-16T02:00:35.458629+0000","last_clean":"2024-09-16T02:00:35.458629+0000","last_became_active":"2024-09-16T02:00:34.455029+0000","last_became_peered":"2024-09-16T02:00:34.455029+0000","last_unstale":"2024-09-16T02:00:35.458629+0000","last_undegraded":"2024-09-16T02:00:35.458629+0000","last_fullsized":"2024-09-16T02:00:35.458629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:14:08.316845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458205+0000","last_change":"2024-09-16T02:00:34.459578+0000","last_active":"2024-09-16T02:00:35.458205+0000","last_peered":"2024-09-16T02:00:35.458205+0000","last_clean":"2024-09-16T02:00:35.458205+0000","last_became_active":"2024-09-16T02:00:34.459288+0000","last_became_peered":"2024-09-16T02:00:34.459288+0000","last_unstale":"2024-09-16T02:00:35.458205+0000","last_undegraded":"2024-09-16T02:00:35.458205+0000","last_fullsized":"2024-09-16T02:00:35.458205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:52:28.974276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.447603+0000","last_change":"2024-09-16T02:00:34.463011+0000","last_active":"2024-09-16T02:00:35.447603+0000","last_peered":"2024-09-16T02:00:35.447603+0000","last_clean":"2024-09-16T02:00:35.447603+0000","last_became_active":"2024-09-16T02:00:34.462668+0000","last_became_peered":"2024-09-16T02:00:34.462668+0000","last_unstale":"2024-09-16T02:00:35.447603+0000","last_undegraded":"2024-09-16T02:00:35.447603+0000","last_fullsized":"2024-09-16T02:00:35.447603+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:06:31.655869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458464+0000","last_change":"2024-09-16T02:00:34.462241+0000","last_active":"2024-09-16T02:00:35.458464+0000","last_peered":"2024-09-16T02:00:35.458464+0000","last_clean":"2024-09-16T02:00:35.458464+0000","last_became_active":"2024-09-16T02:00:34.461925+0000","last_became_peered":"2024-09-16T02:00:34.461925+0000","last_unstale":"2024-09-16T02:00:35.458464+0000","last_undegraded":"2024-09-16T02:00:35.458464+0000","last_fullsized":"2024-09-16T02:00:35.458464+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:48:02.414471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458170+0000","last_change":"2024-09-16T02:00:34.466132+0000","last_active":"2024-09-16T02:00:35.458170+0000","last_peered":"2024-09-16T02:00:35.458170+0000","last_clean":"2024-09-16T02:00:35.458170+0000","last_became_active":"2024-09-16T02:00:34.465750+0000","last_became_peered":"2024-09-16T02:00:34.465750+0000","last_unstale":"2024-09-16T02:00:35.458170+0000","last_undegraded":"2024-09-16T02:00:35.458170+0000","last_fullsized":"2024-09-16T02:00:35.458170+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:07:33.072432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458928+0000","last_change":"2024-09-16T02:00:34.461759+0000","last_active":"2024-09-16T02:00:35.458928+0000","last_peered":"2024-09-16T02:00:35.458928+0000","last_clean":"2024-09-16T02:00:35.458928+0000","last_became_active":"2024-09-16T02:00:34.461385+0000","last_became_peered":"2024-09-16T02:00:34.461385+0000","last_unstale":"2024-09-16T02:00:35.458928+0000","last_undegraded":"2024-09-16T02:00:35.458928+0000","last_fullsized":"2024-09-16T02:00:35.458928+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:00:24.179172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.446281+0000","last_change":"2024-09-16T02:00:34.456584+0000","last_active":"2024-09-16T02:00:35.446281+0000","last_peered":"2024-09-16T02:00:35.446281+0000","last_clean":"2024-09-16T02:00:35.446281+0000","last_became_active":"2024-09-16T02:00:34.456257+0000","last_became_peered":"2024-09-16T02:00:34.456257+0000","last_unstale":"2024-09-16T02:00:35.446281+0000","last_undegraded":"2024-09-16T02:00:35.446281+0000","last_fullsized":"2024-09-16T02:00:35.446281+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:26:45.835435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458070+0000","last_change":"2024-09-16T02:00:34.460680+0000","last_active":"2024-09-16T02:00:35.458070+0000","last_peered":"2024-09-16T02:00:35.458070+0000","last_clean":"2024-09-16T02:00:35.458070+0000","last_became_active":"2024-09-16T02:00:34.460161+0000","last_became_peered":"2024-09-16T02:00:34.460161+0000","last_unstale":"2024-09-16T02:00:35.458070+0000","last_undegraded":"2024-09-16T02:00:35.458070+0000","last_fullsized":"2024-09-16T02:00:35.458070+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:38:19.848420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458271+0000","last_change":"2024-09-16T02:00:34.454606+0000","last_active":"2024-09-16T02:00:35.458271+0000","last_peered":"2024-09-16T02:00:35.458271+0000","last_clean":"2024-09-16T02:00:35.458271+0000","last_became_active":"2024-09-16T02:00:34.454293+0000","last_became_peered":"2024-09-16T02:00:34.454293+0000","last_unstale":"2024-09-16T02:00:35.458271+0000","last_undegraded":"2024-09-16T02:00:35.458271+0000","last_fullsized":"2024-09-16T02:00:35.458271+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:55:54.466176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.446992+0000","last_change":"2024-09-16T02:00:34.452940+0000","last_active":"2024-09-16T02:00:35.446992+0000","last_peered":"2024-09-16T02:00:35.446992+0000","last_clean":"2024-09-16T02:00:35.446992+0000","last_became_active":"2024-09-16T02:00:34.452560+0000","last_became_peered":"2024-09-16T02:00:34.452560+0000","last_unstale":"2024-09-16T02:00:35.446992+0000","last_undegraded":"2024-09-16T02:00:35.446992+0000","last_fullsized":"2024-09-16T02:00:35.446992+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:42:11.564264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.463540+0000","last_change":"2024-09-16T02:00:34.453457+0000","last_active":"2024-09-16T02:00:35.463540+0000","last_peered":"2024-09-16T02:00:35.463540+0000","last_clean":"2024-09-16T02:00:35.463540+0000","last_became_active":"2024-09-16T02:00:34.453148+0000","last_became_peered":"2024-09-16T02:00:34.453148+0000","last_unstale":"2024-09-16T02:00:35.463540+0000","last_undegraded":"2024-09-16T02:00:35.463540+0000","last_fullsized":"2024-09-16T02:00:35.463540+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:28.456892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458927+0000","last_change":"2024-09-16T02:00:34.457057+0000","last_active":"2024-09-16T02:00:35.458927+0000","last_peered":"2024-09-16T02:00:35.458927+0000","last_clean":"2024-09-16T02:00:35.458927+0000","last_became_active":"2024-09-16T02:00:34.456751+0000","last_became_peered":"2024-09-16T02:00:34.456751+0000","last_unstale":"2024-09-16T02:00:35.458927+0000","last_undegraded":"2024-09-16T02:00:35.458927+0000","last_fullsized":"2024-09-16T02:00:35.458927+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:32:32.207510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458938+0000","last_change":"2024-09-16T02:00:34.448434+0000","last_active":"2024-09-16T02:00:35.458938+0000","last_peered":"2024-09-16T02:00:35.458938+0000","last_clean":"2024-09-16T02:00:35.458938+0000","last_became_active":"2024-09-16T02:00:34.448056+0000","last_became_peered":"2024-09-16T02:00:34.448056+0000","last_unstale":"2024-09-16T02:00:35.458938+0000","last_undegraded":"2024-09-16T02:00:35.458938+0000","last_fullsized":"2024-09-16T02:00:35.458938+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:15:28.162159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.461643+0000","last_change":"2024-09-16T02:00:34.459499+0000","last_active":"2024-09-16T02:00:35.461643+0000","last_peered":"2024-09-16T02:00:35.461643+0000","last_clean":"2024-09-16T02:00:35.461643+0000","last_became_active":"2024-09-16T02:00:34.459017+0000","last_became_peered":"2024-09-16T02:00:34.459017+0000","last_unstale":"2024-09-16T02:00:35.461643+0000","last_undegraded":"2024-09-16T02:00:35.461643+0000","last_fullsized":"2024-09-16T02:00:35.461643+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:41:36.866100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.446287+0000","last_change":"2024-09-16T02:00:34.448640+0000","last_active":"2024-09-16T02:00:35.446287+0000","last_peered":"2024-09-16T02:00:35.446287+0000","last_clean":"2024-09-16T02:00:35.446287+0000","last_became_active":"2024-09-16T02:00:34.447737+0000","last_became_peered":"2024-09-16T02:00:34.447737+0000","last_unstale":"2024-09-16T02:00:35.446287+0000","last_undegraded":"2024-09-16T02:00:35.446287+0000","last_fullsized":"2024-09-16T02:00:35.446287+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:20.192650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.447240+0000","last_change":"2024-09-16T02:00:34.451718+0000","last_active":"2024-09-16T02:00:35.447240+0000","last_peered":"2024-09-16T02:00:35.447240+0000","last_clean":"2024-09-16T02:00:35.447240+0000","last_became_active":"2024-09-16T02:00:34.451389+0000","last_became_peered":"2024-09-16T02:00:34.451389+0000","last_unstale":"2024-09-16T02:00:35.447240+0000","last_undegraded":"2024-09-16T02:00:35.447240+0000","last_fullsized":"2024-09-16T02:00:35.447240+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:13:19.296178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.457950+0000","last_change":"2024-09-16T02:00:34.458379+0000","last_active":"2024-09-16T02:00:35.457950+0000","last_peered":"2024-09-16T02:00:35.457950+0000","last_clean":"2024-09-16T02:00:35.457950+0000","last_became_active":"2024-09-16T02:00:34.458038+0000","last_became_peered":"2024-09-16T02:00:34.458038+0000","last_unstale":"2024-09-16T02:00:35.457950+0000","last_undegraded":"2024-09-16T02:00:35.457950+0000","last_fullsized":"2024-09-16T02:00:35.457950+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:34.669470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.446886+0000","last_change":"2024-09-16T02:00:34.450927+0000","last_active":"2024-09-16T02:00:35.446886+0000","last_peered":"2024-09-16T02:00:35.446886+0000","last_clean":"2024-09-16T02:00:35.446886+0000","last_became_active":"2024-09-16T02:00:34.447669+0000","last_became_peered":"2024-09-16T02:00:34.447669+0000","last_unstale":"2024-09-16T02:00:35.446886+0000","last_undegraded":"2024-09-16T02:00:35.446886+0000","last_fullsized":"2024-09-16T02:00:35.446886+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:57.861633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.459622+0000","last_change":"2024-09-16T02:00:34.454827+0000","last_active":"2024-09-16T02:00:35.459622+0000","last_peered":"2024-09-16T02:00:35.459622+0000","last_clean":"2024-09-16T02:00:35.459622+0000","last_became_active":"2024-09-16T02:00:34.454549+0000","last_became_peered":"2024-09-16T02:00:34.454549+0000","last_unstale":"2024-09-16T02:00:35.459622+0000","last_undegraded":"2024-09-16T02:00:35.459622+0000","last_fullsized":"2024-09-16T02:00:35.459622+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:53:45.440860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.461087+0000","last_change":"2024-09-16T02:00:34.455121+0000","last_active":"2024-09-16T02:00:35.461087+0000","last_peered":"2024-09-16T02:00:35.461087+0000","last_clean":"2024-09-16T02:00:35.461087+0000","last_became_active":"2024-09-16T02:00:34.454665+0000","last_became_peered":"2024-09-16T02:00:34.454665+0000","last_unstale":"2024-09-16T02:00:35.461087+0000","last_undegraded":"2024-09-16T02:00:35.461087+0000","last_fullsized":"2024-09-16T02:00:35.461087+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:31:42.771230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.462593+0000","last_change":"2024-09-16T02:00:34.458777+0000","last_active":"2024-09-16T02:00:35.462593+0000","last_peered":"2024-09-16T02:00:35.462593+0000","last_clean":"2024-09-16T02:00:35.462593+0000","last_became_active":"2024-09-16T02:00:34.457954+0000","last_became_peered":"2024-09-16T02:00:34.457954+0000","last_unstale":"2024-09-16T02:00:35.462593+0000","last_undegraded":"2024-09-16T02:00:35.462593+0000","last_fullsized":"2024-09-16T02:00:35.462593+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:58:28.032293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.446190+0000","last_change":"2024-09-16T02:00:34.444712+0000","last_active":"2024-09-16T02:00:35.446190+0000","last_peered":"2024-09-16T02:00:35.446190+0000","last_clean":"2024-09-16T02:00:35.446190+0000","last_became_active":"2024-09-16T02:00:34.444421+0000","last_became_peered":"2024-09-16T02:00:34.444421+0000","last_unstale":"2024-09-16T02:00:35.446190+0000","last_undegraded":"2024-09-16T02:00:35.446190+0000","last_fullsized":"2024-09-16T02:00:35.446190+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:36.557558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458606+0000","last_change":"2024-09-16T02:00:34.454570+0000","last_active":"2024-09-16T02:00:35.458606+0000","last_peered":"2024-09-16T02:00:35.458606+0000","last_clean":"2024-09-16T02:00:35.458606+0000","last_became_active":"2024-09-16T02:00:34.454246+0000","last_became_peered":"2024-09-16T02:00:34.454246+0000","last_unstale":"2024-09-16T02:00:35.458606+0000","last_undegraded":"2024-09-16T02:00:35.458606+0000","last_fullsized":"2024-09-16T02:00:35.458606+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:36:18.634907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458979+0000","last_change":"2024-09-16T02:00:34.449680+0000","last_active":"2024-09-16T02:00:35.458979+0000","last_peered":"2024-09-16T02:00:35.458979+0000","last_clean":"2024-09-16T02:00:35.458979+0000","last_became_active":"2024-09-16T02:00:34.449184+0000","last_became_peered":"2024-09-16T02:00:34.449184+0000","last_unstale":"2024-09-16T02:00:35.458979+0000","last_undegraded":"2024-09-16T02:00:35.458979+0000","last_fullsized":"2024-09-16T02:00:35.458979+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:28:34.867820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458803+0000","last_change":"2024-09-16T02:00:34.455362+0000","last_active":"2024-09-16T02:00:35.458803+0000","last_peered":"2024-09-16T02:00:35.458803+0000","last_clean":"2024-09-16T02:00:35.458803+0000","last_became_active":"2024-09-16T02:00:34.455018+0000","last_became_peered":"2024-09-16T02:00:34.455018+0000","last_unstale":"2024-09-16T02:00:35.458803+0000","last_undegraded":"2024-09-16T02:00:35.458803+0000","last_fullsized":"2024-09-16T02:00:35.458803+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:38.278418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.462886+0000","last_change":"2024-09-16T02:00:34.459426+0000","last_active":"2024-09-16T02:00:35.462886+0000","last_peered":"2024-09-16T02:00:35.462886+0000","last_clean":"2024-09-16T02:00:35.462886+0000","last_became_active":"2024-09-16T02:00:34.459082+0000","last_became_peered":"2024-09-16T02:00:34.459082+0000","last_unstale":"2024-09-16T02:00:35.462886+0000","last_undegraded":"2024-09-16T02:00:35.462886+0000","last_fullsized":"2024-09-16T02:00:35.462886+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:59:42.226065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.459009+0000","last_change":"2024-09-16T02:00:34.450936+0000","last_active":"2024-09-16T02:00:35.459009+0000","last_peered":"2024-09-16T02:00:35.459009+0000","last_clean":"2024-09-16T02:00:35.459009+0000","last_became_active":"2024-09-16T02:00:34.450503+0000","last_became_peered":"2024-09-16T02:00:34.450503+0000","last_unstale":"2024-09-16T02:00:35.459009+0000","last_undegraded":"2024-09-16T02:00:35.459009+0000","last_fullsized":"2024-09-16T02:00:35.459009+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:35:55.806193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'5","reported_seq":37,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:57.551569+0000","last_change":"2024-09-16T02:00:32.443193+0000","last_active":"2024-09-16T02:00:57.551569+0000","last_peered":"2024-09-16T02:00:57.551569+0000","last_clean":"2024-09-16T02:00:57.551569+0000","last_became_active":"2024-09-16T02:00:32.442863+0000","last_became_peered":"2024-09-16T02:00:32.442863+0000","last_unstale":"2024-09-16T02:00:57.551569+0000","last_undegraded":"2024-09-16T02:00:57.551569+0000","last_fullsized":"2024-09-16T02:00:57.551569+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:19.734946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.447318+0000","last_change":"2024-09-16T02:00:34.454819+0000","last_active":"2024-09-16T02:00:35.447318+0000","last_peered":"2024-09-16T02:00:35.447318+0000","last_clean":"2024-09-16T02:00:35.447318+0000","last_became_active":"2024-09-16T02:00:34.454492+0000","last_became_peered":"2024-09-16T02:00:34.454492+0000","last_unstale":"2024-09-16T02:00:35.447318+0000","last_undegraded":"2024-09-16T02:00:35.447318+0000","last_fullsized":"2024-09-16T02:00:35.447318+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:11.479330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:45.518640+0000","last_change":"2024-09-16T02:00:32.438666+0000","last_active":"2024-09-16T02:00:45.518640+0000","last_peered":"2024-09-16T02:00:45.518640+0000","last_clean":"2024-09-16T02:00:45.518640+0000","last_became_active":"2024-09-16T02:00:32.438324+0000","last_became_peered":"2024-09-16T02:00:32.438324+0000","last_unstale":"2024-09-16T02:00:45.518640+0000","last_undegraded":"2024-09-16T02:00:45.518640+0000","last_fullsized":"2024-09-16T02:00:45.518640+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:56:28.203977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:44.506262+0000","last_change":"2024-09-16T02:00:32.438530+0000","last_active":"2024-09-16T02:00:44.506262+0000","last_peered":"2024-09-16T02:00:44.506262+0000","last_clean":"2024-09-16T02:00:44.506262+0000","last_became_active":"2024-09-16T02:00:32.438238+0000","last_became_peered":"2024-09-16T02:00:32.438238+0000","last_unstale":"2024-09-16T02:00:44.506262+0000","last_undegraded":"2024-09-16T02:00:44.506262+0000","last_fullsized":"2024-09-16T02:00:44.506262+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:52:32.799701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458329+0000","last_change":"2024-09-16T02:00:34.456299+0000","last_active":"2024-09-16T02:00:35.458329+0000","last_peered":"2024-09-16T02:00:35.458329+0000","last_clean":"2024-09-16T02:00:35.458329+0000","last_became_active":"2024-09-16T02:00:34.455925+0000","last_became_peered":"2024-09-16T02:00:34.455925+0000","last_unstale":"2024-09-16T02:00:35.458329+0000","last_undegraded":"2024-09-16T02:00:35.458329+0000","last_fullsized":"2024-09-16T02:00:35.458329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:33:27.162664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:46.527624+0000","last_change":"2024-09-16T02:00:32.442029+0000","last_active":"2024-09-16T02:00:46.527624+0000","last_peered":"2024-09-16T02:00:46.527624+0000","last_clean":"2024-09-16T02:00:46.527624+0000","last_became_active":"2024-09-16T02:00:32.441705+0000","last_became_peered":"2024-09-16T02:00:32.441705+0000","last_unstale":"2024-09-16T02:00:46.527624+0000","last_undegraded":"2024-09-16T02:00:46.527624+0000","last_fullsized":"2024-09-16T02:00:46.527624+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:50:54.725768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458032+0000","last_change":"2024-09-16T02:00:34.457833+0000","last_active":"2024-09-16T02:00:35.458032+0000","last_peered":"2024-09-16T02:00:35.458032+0000","last_clean":"2024-09-16T02:00:35.458032+0000","last_became_active":"2024-09-16T02:00:34.457453+0000","last_became_peered":"2024-09-16T02:00:34.457453+0000","last_unstale":"2024-09-16T02:00:35.458032+0000","last_undegraded":"2024-09-16T02:00:35.458032+0000","last_fullsized":"2024-09-16T02:00:35.458032+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:21.777574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":44,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:57.550881+0000","last_change":"2024-09-16T02:00:32.441341+0000","last_active":"2024-09-16T02:00:57.550881+0000","last_peered":"2024-09-16T02:00:57.550881+0000","last_clean":"2024-09-16T02:00:57.550881+0000","last_became_active":"2024-09-16T02:00:32.441019+0000","last_became_peered":"2024-09-16T02:00:32.441019+0000","last_unstale":"2024-09-16T02:00:57.550881+0000","last_undegraded":"2024-09-16T02:00:57.550881+0000","last_fullsized":"2024-09-16T02:00:57.550881+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:10:10.171714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.447426+0000","last_change":"2024-09-16T02:00:34.461746+0000","last_active":"2024-09-16T02:00:35.447426+0000","last_peered":"2024-09-16T02:00:35.447426+0000","last_clean":"2024-09-16T02:00:35.447426+0000","last_became_active":"2024-09-16T02:00:34.461441+0000","last_became_peered":"2024-09-16T02:00:34.461441+0000","last_unstale":"2024-09-16T02:00:35.447426+0000","last_undegraded":"2024-09-16T02:00:35.447426+0000","last_fullsized":"2024-09-16T02:00:35.447426+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:28:14.519159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:46.527397+0000","last_change":"2024-09-16T02:00:32.443129+0000","last_active":"2024-09-16T02:00:46.527397+0000","last_peered":"2024-09-16T02:00:46.527397+0000","last_clean":"2024-09-16T02:00:46.527397+0000","last_became_active":"2024-09-16T02:00:32.442772+0000","last_became_peered":"2024-09-16T02:00:32.442772+0000","last_unstale":"2024-09-16T02:00:46.527397+0000","last_undegraded":"2024-09-16T02:00:46.527397+0000","last_fullsized":"2024-09-16T02:00:46.527397+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:10:03.964583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458972+0000","last_change":"2024-09-16T02:00:34.457449+0000","last_active":"2024-09-16T02:00:35.458972+0000","last_peered":"2024-09-16T02:00:35.458972+0000","last_clean":"2024-09-16T02:00:35.458972+0000","last_became_active":"2024-09-16T02:00:34.457110+0000","last_became_peered":"2024-09-16T02:00:34.457110+0000","last_unstale":"2024-09-16T02:00:35.458972+0000","last_undegraded":"2024-09-16T02:00:35.458972+0000","last_fullsized":"2024-09-16T02:00:35.458972+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:35.662503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:45.516635+0000","last_change":"2024-09-16T02:00:32.587419+0000","last_active":"2024-09-16T02:00:45.516635+0000","last_peered":"2024-09-16T02:00:45.516635+0000","last_clean":"2024-09-16T02:00:45.516635+0000","last_became_active":"2024-09-16T02:00:32.587037+0000","last_became_peered":"2024-09-16T02:00:32.587037+0000","last_unstale":"2024-09-16T02:00:45.516635+0000","last_undegraded":"2024-09-16T02:00:45.516635+0000","last_fullsized":"2024-09-16T02:00:45.516635+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:11.835505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.447483+0000","last_change":"2024-09-16T02:00:34.456371+0000","last_active":"2024-09-16T02:00:35.447483+0000","last_peered":"2024-09-16T02:00:35.447483+0000","last_clean":"2024-09-16T02:00:35.447483+0000","last_became_active":"2024-09-16T02:00:34.456013+0000","last_became_peered":"2024-09-16T02:00:34.456013+0000","last_unstale":"2024-09-16T02:00:35.447483+0000","last_undegraded":"2024-09-16T02:00:35.447483+0000","last_fullsized":"2024-09-16T02:00:35.447483+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:30:15.847189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'24","reported_seq":70,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:57.552471+0000","last_change":"2024-09-16T02:00:32.444922+0000","last_active":"2024-09-16T02:00:57.552471+0000","last_peered":"2024-09-16T02:00:57.552471+0000","last_clean":"2024-09-16T02:00:57.552471+0000","last_became_active":"2024-09-16T02:00:32.444540+0000","last_became_peered":"2024-09-16T02:00:32.444540+0000","last_unstale":"2024-09-16T02:00:57.552471+0000","last_undegraded":"2024-09-16T02:00:57.552471+0000","last_fullsized":"2024-09-16T02:00:57.552471+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:59.183952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46802,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":28,"num_read_kb":59,"num_write":26,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458532+0000","last_change":"2024-09-16T02:00:34.453366+0000","last_active":"2024-09-16T02:00:35.458532+0000","last_peered":"2024-09-16T02:00:35.458532+0000","last_clean":"2024-09-16T02:00:35.458532+0000","last_became_active":"2024-09-16T02:00:34.453048+0000","last_became_peered":"2024-09-16T02:00:34.453048+0000","last_unstale":"2024-09-16T02:00:35.458532+0000","last_undegraded":"2024-09-16T02:00:35.458532+0000","last_fullsized":"2024-09-16T02:00:35.458532+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:15:52.721893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:47.535399+0000","last_change":"2024-09-16T02:00:32.445516+0000","last_active":"2024-09-16T02:00:47.535399+0000","last_peered":"2024-09-16T02:00:47.535399+0000","last_clean":"2024-09-16T02:00:47.535399+0000","last_became_active":"2024-09-16T02:00:32.445092+0000","last_became_peered":"2024-09-16T02:00:32.445092+0000","last_unstale":"2024-09-16T02:00:47.535399+0000","last_undegraded":"2024-09-16T02:00:47.535399+0000","last_fullsized":"2024-09-16T02:00:47.535399+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:54.274692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.446077+0000","last_change":"2024-09-16T02:00:34.448450+0000","last_active":"2024-09-16T02:00:35.446077+0000","last_peered":"2024-09-16T02:00:35.446077+0000","last_clean":"2024-09-16T02:00:35.446077+0000","last_became_active":"2024-09-16T02:00:34.448116+0000","last_became_peered":"2024-09-16T02:00:34.448116+0000","last_unstale":"2024-09-16T02:00:35.446077+0000","last_undegraded":"2024-09-16T02:00:35.446077+0000","last_fullsized":"2024-09-16T02:00:35.446077+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:12:34.834798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:46.527344+0000","last_change":"2024-09-16T02:00:32.445427+0000","last_active":"2024-09-16T02:00:46.527344+0000","last_peered":"2024-09-16T02:00:46.527344+0000","last_clean":"2024-09-16T02:00:46.527344+0000","last_became_active":"2024-09-16T02:00:32.445058+0000","last_became_peered":"2024-09-16T02:00:32.445058+0000","last_unstale":"2024-09-16T02:00:46.527344+0000","last_undegraded":"2024-09-16T02:00:46.527344+0000","last_fullsized":"2024-09-16T02:00:46.527344+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:35:34.701331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.459848+0000","last_change":"2024-09-16T02:00:34.458479+0000","last_active":"2024-09-16T02:00:35.459848+0000","last_peered":"2024-09-16T02:00:35.459848+0000","last_clean":"2024-09-16T02:00:35.459848+0000","last_became_active":"2024-09-16T02:00:34.458135+0000","last_became_peered":"2024-09-16T02:00:34.458135+0000","last_unstale":"2024-09-16T02:00:35.459848+0000","last_undegraded":"2024-09-16T02:00:35.459848+0000","last_fullsized":"2024-09-16T02:00:35.459848+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:37:40.188861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'6","reported_seq":39,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:57.551703+0000","last_change":"2024-09-16T02:00:32.440319+0000","last_active":"2024-09-16T02:00:57.551703+0000","last_peered":"2024-09-16T02:00:57.551703+0000","last_clean":"2024-09-16T02:00:57.551703+0000","last_became_active":"2024-09-16T02:00:32.440005+0000","last_became_peered":"2024-09-16T02:00:32.440005+0000","last_unstale":"2024-09-16T02:00:57.551703+0000","last_undegraded":"2024-09-16T02:00:57.551703+0000","last_fullsized":"2024-09-16T02:00:57.551703+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:07:33.876577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458153+0000","last_change":"2024-09-16T02:00:34.460468+0000","last_active":"2024-09-16T02:00:35.458153+0000","last_peered":"2024-09-16T02:00:35.458153+0000","last_clean":"2024-09-16T02:00:35.458153+0000","last_became_active":"2024-09-16T02:00:34.459190+0000","last_became_peered":"2024-09-16T02:00:34.459190+0000","last_unstale":"2024-09-16T02:00:35.458153+0000","last_undegraded":"2024-09-16T02:00:35.458153+0000","last_fullsized":"2024-09-16T02:00:35.458153+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:46:43.654991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:46.528118+0000","last_change":"2024-09-16T02:00:32.587700+0000","last_active":"2024-09-16T02:00:46.528118+0000","last_peered":"2024-09-16T02:00:46.528118+0000","last_clean":"2024-09-16T02:00:46.528118+0000","last_became_active":"2024-09-16T02:00:32.586964+0000","last_became_peered":"2024-09-16T02:00:32.586964+0000","last_unstale":"2024-09-16T02:00:46.528118+0000","last_undegraded":"2024-09-16T02:00:46.528118+0000","last_fullsized":"2024-09-16T02:00:46.528118+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:41.309918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.446915+0000","last_change":"2024-09-16T02:00:34.455652+0000","last_active":"2024-09-16T02:00:35.446915+0000","last_peered":"2024-09-16T02:00:35.446915+0000","last_clean":"2024-09-16T02:00:35.446915+0000","last_became_active":"2024-09-16T02:00:34.455306+0000","last_became_peered":"2024-09-16T02:00:34.455306+0000","last_unstale":"2024-09-16T02:00:35.446915+0000","last_undegraded":"2024-09-16T02:00:35.446915+0000","last_fullsized":"2024-09-16T02:00:35.446915+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:44:09.505096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:46.502399+0000","last_change":"2024-09-16T02:00:32.441443+0000","last_active":"2024-09-16T02:00:46.502399+0000","last_peered":"2024-09-16T02:00:46.502399+0000","last_clean":"2024-09-16T02:00:46.502399+0000","last_became_active":"2024-09-16T02:00:32.441073+0000","last_became_peered":"2024-09-16T02:00:32.441073+0000","last_unstale":"2024-09-16T02:00:46.502399+0000","last_undegraded":"2024-09-16T02:00:46.502399+0000","last_fullsized":"2024-09-16T02:00:46.502399+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:16.865970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.462234+0000","last_change":"2024-09-16T02:00:34.458108+0000","last_active":"2024-09-16T02:00:35.462234+0000","last_peered":"2024-09-16T02:00:35.462234+0000","last_clean":"2024-09-16T02:00:35.462234+0000","last_became_active":"2024-09-16T02:00:34.457766+0000","last_became_peered":"2024-09-16T02:00:34.457766+0000","last_unstale":"2024-09-16T02:00:35.462234+0000","last_undegraded":"2024-09-16T02:00:35.462234+0000","last_fullsized":"2024-09-16T02:00:35.462234+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:21:54.410438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:46.526168+0000","last_change":"2024-09-16T02:00:32.441728+0000","last_active":"2024-09-16T02:00:46.526168+0000","last_peered":"2024-09-16T02:00:46.526168+0000","last_clean":"2024-09-16T02:00:46.526168+0000","last_became_active":"2024-09-16T02:00:32.441388+0000","last_became_peered":"2024-09-16T02:00:32.441388+0000","last_unstale":"2024-09-16T02:00:46.526168+0000","last_undegraded":"2024-09-16T02:00:46.526168+0000","last_fullsized":"2024-09-16T02:00:46.526168+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:18:24.412876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.460483+0000","last_change":"2024-09-16T02:00:34.461889+0000","last_active":"2024-09-16T02:00:35.460483+0000","last_peered":"2024-09-16T02:00:35.460483+0000","last_clean":"2024-09-16T02:00:35.460483+0000","last_became_active":"2024-09-16T02:00:34.461583+0000","last_became_peered":"2024-09-16T02:00:34.461583+0000","last_unstale":"2024-09-16T02:00:35.460483+0000","last_undegraded":"2024-09-16T02:00:35.460483+0000","last_fullsized":"2024-09-16T02:00:35.460483+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:32.598422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:46.526605+0000","last_change":"2024-09-16T02:00:32.445201+0000","last_active":"2024-09-16T02:00:46.526605+0000","last_peered":"2024-09-16T02:00:46.526605+0000","last_clean":"2024-09-16T02:00:46.526605+0000","last_became_active":"2024-09-16T02:00:32.444722+0000","last_became_peered":"2024-09-16T02:00:32.444722+0000","last_unstale":"2024-09-16T02:00:46.526605+0000","last_undegraded":"2024-09-16T02:00:46.526605+0000","last_fullsized":"2024-09-16T02:00:46.526605+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:41:11.657028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'1","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.479926+0000","last_change":"2024-09-16T02:00:34.456108+0000","last_active":"2024-09-16T02:00:35.479926+0000","last_peered":"2024-09-16T02:00:35.479926+0000","last_clean":"2024-09-16T02:00:35.479926+0000","last_became_active":"2024-09-16T02:00:34.455718+0000","last_became_peered":"2024-09-16T02:00:34.455718+0000","last_unstale":"2024-09-16T02:00:35.479926+0000","last_undegraded":"2024-09-16T02:00:35.479926+0000","last_fullsized":"2024-09-16T02:00:35.479926+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:27:09.201182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'5","reported_seq":37,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:57.551325+0000","last_change":"2024-09-16T02:00:32.860059+0000","last_active":"2024-09-16T02:00:57.551325+0000","last_peered":"2024-09-16T02:00:57.551325+0000","last_clean":"2024-09-16T02:00:57.551325+0000","last_became_active":"2024-09-16T02:00:32.859617+0000","last_became_peered":"2024-09-16T02:00:32.859617+0000","last_unstale":"2024-09-16T02:00:57.551325+0000","last_undegraded":"2024-09-16T02:00:57.551325+0000","last_fullsized":"2024-09-16T02:00:57.551325+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:41:50.248194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.459816+0000","last_change":"2024-09-16T02:00:34.454344+0000","last_active":"2024-09-16T02:00:35.459816+0000","last_peered":"2024-09-16T02:00:35.459816+0000","last_clean":"2024-09-16T02:00:35.459816+0000","last_became_active":"2024-09-16T02:00:34.454027+0000","last_became_peered":"2024-09-16T02:00:34.454027+0000","last_unstale":"2024-09-16T02:00:35.459816+0000","last_undegraded":"2024-09-16T02:00:35.459816+0000","last_fullsized":"2024-09-16T02:00:35.459816+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:33.020365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":330,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458425+0000","last_change":"2024-09-16T01:59:19.798159+0000","last_active":"2024-09-16T02:00:35.458425+0000","last_peered":"2024-09-16T02:00:35.458425+0000","last_clean":"2024-09-16T02:00:35.458425+0000","last_became_active":"2024-09-16T01:59:19.797688+0000","last_became_peered":"2024-09-16T01:59:19.797688+0000","last_unstale":"2024-09-16T02:00:35.458425+0000","last_undegraded":"2024-09-16T02:00:35.458425+0000","last_fullsized":"2024-09-16T02:00:35.458425+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:55:54.312750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:55:54.312750+0000","last_clean_scrub_stamp":"2024-09-16T01:55:54.312750+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:00:36.860158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:47.534936+0000","last_change":"2024-09-16T02:00:32.441925+0000","last_active":"2024-09-16T02:00:47.534936+0000","last_peered":"2024-09-16T02:00:47.534936+0000","last_clean":"2024-09-16T02:00:47.534936+0000","last_became_active":"2024-09-16T02:00:32.441565+0000","last_became_peered":"2024-09-16T02:00:32.441565+0000","last_unstale":"2024-09-16T02:00:47.534936+0000","last_undegraded":"2024-09-16T02:00:47.534936+0000","last_fullsized":"2024-09-16T02:00:47.534936+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:23:09.755034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.460113+0000","last_change":"2024-09-16T02:00:34.452986+0000","last_active":"2024-09-16T02:00:35.460113+0000","last_peered":"2024-09-16T02:00:35.460113+0000","last_clean":"2024-09-16T02:00:35.460113+0000","last_became_active":"2024-09-16T02:00:34.452688+0000","last_became_peered":"2024-09-16T02:00:34.452688+0000","last_unstale":"2024-09-16T02:00:35.460113+0000","last_undegraded":"2024-09-16T02:00:35.460113+0000","last_fullsized":"2024-09-16T02:00:35.460113+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:12.958209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:44.506336+0000","last_change":"2024-09-16T02:00:32.443964+0000","last_active":"2024-09-16T02:00:44.506336+0000","last_peered":"2024-09-16T02:00:44.506336+0000","last_clean":"2024-09-16T02:00:44.506336+0000","last_became_active":"2024-09-16T02:00:32.443623+0000","last_became_peered":"2024-09-16T02:00:32.443623+0000","last_unstale":"2024-09-16T02:00:44.506336+0000","last_undegraded":"2024-09-16T02:00:44.506336+0000","last_fullsized":"2024-09-16T02:00:44.506336+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:30:34.406024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.446017+0000","last_change":"2024-09-16T02:00:34.449615+0000","last_active":"2024-09-16T02:00:35.446017+0000","last_peered":"2024-09-16T02:00:35.446017+0000","last_clean":"2024-09-16T02:00:35.446017+0000","last_became_active":"2024-09-16T02:00:34.449109+0000","last_became_peered":"2024-09-16T02:00:34.449109+0000","last_unstale":"2024-09-16T02:00:35.446017+0000","last_undegraded":"2024-09-16T02:00:35.446017+0000","last_fullsized":"2024-09-16T02:00:35.446017+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:45.036177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:57.551217+0000","last_change":"2024-09-16T02:00:32.587107+0000","last_active":"2024-09-16T02:00:57.551217+0000","last_peered":"2024-09-16T02:00:57.551217+0000","last_clean":"2024-09-16T02:00:57.551217+0000","last_became_active":"2024-09-16T02:00:32.586777+0000","last_became_peered":"2024-09-16T02:00:32.586777+0000","last_unstale":"2024-09-16T02:00:57.551217+0000","last_undegraded":"2024-09-16T02:00:57.551217+0000","last_fullsized":"2024-09-16T02:00:57.551217+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:03.968987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.447491+0000","last_change":"2024-09-16T02:00:34.453500+0000","last_active":"2024-09-16T02:00:35.447491+0000","last_peered":"2024-09-16T02:00:35.447491+0000","last_clean":"2024-09-16T02:00:35.447491+0000","last_became_active":"2024-09-16T02:00:34.453175+0000","last_became_peered":"2024-09-16T02:00:34.453175+0000","last_unstale":"2024-09-16T02:00:35.447491+0000","last_undegraded":"2024-09-16T02:00:35.447491+0000","last_fullsized":"2024-09-16T02:00:35.447491+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:52.666456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'5","reported_seq":37,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:58.551852+0000","last_change":"2024-09-16T02:00:32.443201+0000","last_active":"2024-09-16T02:00:58.551852+0000","last_peered":"2024-09-16T02:00:58.551852+0000","last_clean":"2024-09-16T02:00:58.551852+0000","last_became_active":"2024-09-16T02:00:32.442778+0000","last_became_peered":"2024-09-16T02:00:32.442778+0000","last_unstale":"2024-09-16T02:00:58.551852+0000","last_undegraded":"2024-09-16T02:00:58.551852+0000","last_fullsized":"2024-09-16T02:00:58.551852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:43.933817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458153+0000","last_change":"2024-09-16T02:00:34.463000+0000","last_active":"2024-09-16T02:00:35.458153+0000","last_peered":"2024-09-16T02:00:35.458153+0000","last_clean":"2024-09-16T02:00:35.458153+0000","last_became_active":"2024-09-16T02:00:34.461890+0000","last_became_peered":"2024-09-16T02:00:34.461890+0000","last_unstale":"2024-09-16T02:00:35.458153+0000","last_undegraded":"2024-09-16T02:00:35.458153+0000","last_fullsized":"2024-09-16T02:00:35.458153+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:21:15.161295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:36.132963+0000","last_change":"2024-09-16T02:00:32.586933+0000","last_active":"2024-09-16T02:00:36.132963+0000","last_peered":"2024-09-16T02:00:36.132963+0000","last_clean":"2024-09-16T02:00:36.132963+0000","last_became_active":"2024-09-16T02:00:32.586565+0000","last_became_peered":"2024-09-16T02:00:32.586565+0000","last_unstale":"2024-09-16T02:00:36.132963+0000","last_undegraded":"2024-09-16T02:00:36.132963+0000","last_fullsized":"2024-09-16T02:00:36.132963+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:45:07.130649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:37.582252+0000","last_change":"2024-09-16T02:00:34.460805+0000","last_active":"2024-09-16T02:00:37.582252+0000","last_peered":"2024-09-16T02:00:37.582252+0000","last_clean":"2024-09-16T02:00:37.582252+0000","last_became_active":"2024-09-16T02:00:34.460482+0000","last_became_peered":"2024-09-16T02:00:34.460482+0000","last_unstale":"2024-09-16T02:00:37.582252+0000","last_undegraded":"2024-09-16T02:00:37.582252+0000","last_fullsized":"2024-09-16T02:00:37.582252+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:07:51.565053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:46.527428+0000","last_change":"2024-09-16T02:00:32.587516+0000","last_active":"2024-09-16T02:00:46.527428+0000","last_peered":"2024-09-16T02:00:46.527428+0000","last_clean":"2024-09-16T02:00:46.527428+0000","last_became_active":"2024-09-16T02:00:32.587172+0000","last_became_peered":"2024-09-16T02:00:32.587172+0000","last_unstale":"2024-09-16T02:00:46.527428+0000","last_undegraded":"2024-09-16T02:00:46.527428+0000","last_fullsized":"2024-09-16T02:00:46.527428+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:52:29.018290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.446340+0000","last_change":"2024-09-16T02:00:34.446021+0000","last_active":"2024-09-16T02:00:35.446340+0000","last_peered":"2024-09-16T02:00:35.446340+0000","last_clean":"2024-09-16T02:00:35.446340+0000","last_became_active":"2024-09-16T02:00:34.445732+0000","last_became_peered":"2024-09-16T02:00:34.445732+0000","last_unstale":"2024-09-16T02:00:35.446340+0000","last_undegraded":"2024-09-16T02:00:35.446340+0000","last_fullsized":"2024-09-16T02:00:35.446340+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:51:17.960792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'13","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:58.458959+0000","last_change":"2024-09-16T02:00:32.439750+0000","last_active":"2024-09-16T02:00:58.458959+0000","last_peered":"2024-09-16T02:00:58.458959+0000","last_clean":"2024-09-16T02:00:58.458959+0000","last_became_active":"2024-09-16T02:00:32.439438+0000","last_became_peered":"2024-09-16T02:00:32.439438+0000","last_unstale":"2024-09-16T02:00:58.458959+0000","last_undegraded":"2024-09-16T02:00:58.458959+0000","last_fullsized":"2024-09-16T02:00:58.458959+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:50.764733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.459772+0000","last_change":"2024-09-16T02:00:34.453810+0000","last_active":"2024-09-16T02:00:35.459772+0000","last_peered":"2024-09-16T02:00:35.459772+0000","last_clean":"2024-09-16T02:00:35.459772+0000","last_became_active":"2024-09-16T02:00:34.453421+0000","last_became_peered":"2024-09-16T02:00:34.453421+0000","last_unstale":"2024-09-16T02:00:35.459772+0000","last_undegraded":"2024-09-16T02:00:35.459772+0000","last_fullsized":"2024-09-16T02:00:35.459772+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:23:50.405749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:46.525899+0000","last_change":"2024-09-16T02:00:32.860565+0000","last_active":"2024-09-16T02:00:46.525899+0000","last_peered":"2024-09-16T02:00:46.525899+0000","last_clean":"2024-09-16T02:00:46.525899+0000","last_became_active":"2024-09-16T02:00:32.860130+0000","last_became_peered":"2024-09-16T02:00:32.860130+0000","last_unstale":"2024-09-16T02:00:46.525899+0000","last_undegraded":"2024-09-16T02:00:46.525899+0000","last_fullsized":"2024-09-16T02:00:46.525899+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:09:55.759883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458637+0000","last_change":"2024-09-16T02:00:34.454033+0000","last_active":"2024-09-16T02:00:35.458637+0000","last_peered":"2024-09-16T02:00:35.458637+0000","last_clean":"2024-09-16T02:00:35.458637+0000","last_became_active":"2024-09-16T02:00:34.453725+0000","last_became_peered":"2024-09-16T02:00:34.453725+0000","last_unstale":"2024-09-16T02:00:35.458637+0000","last_undegraded":"2024-09-16T02:00:35.458637+0000","last_fullsized":"2024-09-16T02:00:35.458637+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:15.702752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:43.496005+0000","last_change":"2024-09-16T02:00:32.587115+0000","last_active":"2024-09-16T02:00:43.496005+0000","last_peered":"2024-09-16T02:00:43.496005+0000","last_clean":"2024-09-16T02:00:43.496005+0000","last_became_active":"2024-09-16T02:00:32.586835+0000","last_became_peered":"2024-09-16T02:00:32.586835+0000","last_unstale":"2024-09-16T02:00:43.496005+0000","last_undegraded":"2024-09-16T02:00:43.496005+0000","last_fullsized":"2024-09-16T02:00:43.496005+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:12:58.820005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458137+0000","last_change":"2024-09-16T02:00:34.457673+0000","last_active":"2024-09-16T02:00:35.458137+0000","last_peered":"2024-09-16T02:00:35.458137+0000","last_clean":"2024-09-16T02:00:35.458137+0000","last_became_active":"2024-09-16T02:00:34.457331+0000","last_became_peered":"2024-09-16T02:00:34.457331+0000","last_unstale":"2024-09-16T02:00:35.458137+0000","last_undegraded":"2024-09-16T02:00:35.458137+0000","last_fullsized":"2024-09-16T02:00:35.458137+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:12:16.100036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:46.526274+0000","last_change":"2024-09-16T02:00:32.437528+0000","last_active":"2024-09-16T02:00:46.526274+0000","last_peered":"2024-09-16T02:00:46.526274+0000","last_clean":"2024-09-16T02:00:46.526274+0000","last_became_active":"2024-09-16T02:00:32.437080+0000","last_became_peered":"2024-09-16T02:00:32.437080+0000","last_unstale":"2024-09-16T02:00:46.526274+0000","last_undegraded":"2024-09-16T02:00:46.526274+0000","last_fullsized":"2024-09-16T02:00:46.526274+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:59:33.908026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.463059+0000","last_change":"2024-09-16T02:00:34.453504+0000","last_active":"2024-09-16T02:00:35.463059+0000","last_peered":"2024-09-16T02:00:35.463059+0000","last_clean":"2024-09-16T02:00:35.463059+0000","last_became_active":"2024-09-16T02:00:34.453212+0000","last_became_peered":"2024-09-16T02:00:34.453212+0000","last_unstale":"2024-09-16T02:00:35.463059+0000","last_undegraded":"2024-09-16T02:00:35.463059+0000","last_fullsized":"2024-09-16T02:00:35.463059+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:00:56.414548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:45.518572+0000","last_change":"2024-09-16T02:00:32.442282+0000","last_active":"2024-09-16T02:00:45.518572+0000","last_peered":"2024-09-16T02:00:45.518572+0000","last_clean":"2024-09-16T02:00:45.518572+0000","last_became_active":"2024-09-16T02:00:32.441910+0000","last_became_peered":"2024-09-16T02:00:32.441910+0000","last_unstale":"2024-09-16T02:00:45.518572+0000","last_undegraded":"2024-09-16T02:00:45.518572+0000","last_fullsized":"2024-09-16T02:00:45.518572+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:01:46.045021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.459686+0000","last_change":"2024-09-16T02:00:34.460177+0000","last_active":"2024-09-16T02:00:35.459686+0000","last_peered":"2024-09-16T02:00:35.459686+0000","last_clean":"2024-09-16T02:00:35.459686+0000","last_became_active":"2024-09-16T02:00:34.459874+0000","last_became_peered":"2024-09-16T02:00:34.459874+0000","last_unstale":"2024-09-16T02:00:35.459686+0000","last_undegraded":"2024-09-16T02:00:35.459686+0000","last_fullsized":"2024-09-16T02:00:35.459686+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:35.855591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:43.496920+0000","last_change":"2024-09-16T02:00:32.443441+0000","last_active":"2024-09-16T02:00:43.496920+0000","last_peered":"2024-09-16T02:00:43.496920+0000","last_clean":"2024-09-16T02:00:43.496920+0000","last_became_active":"2024-09-16T02:00:32.443095+0000","last_became_peered":"2024-09-16T02:00:32.443095+0000","last_unstale":"2024-09-16T02:00:43.496920+0000","last_undegraded":"2024-09-16T02:00:43.496920+0000","last_fullsized":"2024-09-16T02:00:43.496920+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:16.680798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.459028+0000","last_change":"2024-09-16T02:00:34.452964+0000","last_active":"2024-09-16T02:00:35.459028+0000","last_peered":"2024-09-16T02:00:35.459028+0000","last_clean":"2024-09-16T02:00:35.459028+0000","last_became_active":"2024-09-16T02:00:34.452617+0000","last_became_peered":"2024-09-16T02:00:34.452617+0000","last_unstale":"2024-09-16T02:00:35.459028+0000","last_undegraded":"2024-09-16T02:00:35.459028+0000","last_fullsized":"2024-09-16T02:00:35.459028+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:04.617970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.447223+0000","last_change":"2024-09-16T02:00:34.456585+0000","last_active":"2024-09-16T02:00:35.447223+0000","last_peered":"2024-09-16T02:00:35.447223+0000","last_clean":"2024-09-16T02:00:35.447223+0000","last_became_active":"2024-09-16T02:00:34.456199+0000","last_became_peered":"2024-09-16T02:00:34.456199+0000","last_unstale":"2024-09-16T02:00:35.447223+0000","last_undegraded":"2024-09-16T02:00:35.447223+0000","last_fullsized":"2024-09-16T02:00:35.447223+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:37:54.376785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:46.527336+0000","last_change":"2024-09-16T02:00:32.446324+0000","last_active":"2024-09-16T02:00:46.527336+0000","last_peered":"2024-09-16T02:00:46.527336+0000","last_clean":"2024-09-16T02:00:46.527336+0000","last_became_active":"2024-09-16T02:00:32.441840+0000","last_became_peered":"2024-09-16T02:00:32.441840+0000","last_unstale":"2024-09-16T02:00:46.527336+0000","last_undegraded":"2024-09-16T02:00:46.527336+0000","last_fullsized":"2024-09-16T02:00:46.527336+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:46:08.601738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.447200+0000","last_change":"2024-09-16T02:00:34.460925+0000","last_active":"2024-09-16T02:00:35.447200+0000","last_peered":"2024-09-16T02:00:35.447200+0000","last_clean":"2024-09-16T02:00:35.447200+0000","last_became_active":"2024-09-16T02:00:34.460581+0000","last_became_peered":"2024-09-16T02:00:34.460581+0000","last_unstale":"2024-09-16T02:00:35.447200+0000","last_undegraded":"2024-09-16T02:00:35.447200+0000","last_fullsized":"2024-09-16T02:00:35.447200+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:19:44.460695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:47.534682+0000","last_change":"2024-09-16T02:00:32.443877+0000","last_active":"2024-09-16T02:00:47.534682+0000","last_peered":"2024-09-16T02:00:47.534682+0000","last_clean":"2024-09-16T02:00:47.534682+0000","last_became_active":"2024-09-16T02:00:32.443416+0000","last_became_peered":"2024-09-16T02:00:32.443416+0000","last_unstale":"2024-09-16T02:00:47.534682+0000","last_undegraded":"2024-09-16T02:00:47.534682+0000","last_fullsized":"2024-09-16T02:00:47.534682+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:36:57.367232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458846+0000","last_change":"2024-09-16T02:00:34.453048+0000","last_active":"2024-09-16T02:00:35.458846+0000","last_peered":"2024-09-16T02:00:35.458846+0000","last_clean":"2024-09-16T02:00:35.458846+0000","last_became_active":"2024-09-16T02:00:34.452752+0000","last_became_peered":"2024-09-16T02:00:34.452752+0000","last_unstale":"2024-09-16T02:00:35.458846+0000","last_undegraded":"2024-09-16T02:00:35.458846+0000","last_fullsized":"2024-09-16T02:00:35.458846+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:55.970195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.463149+0000","last_change":"2024-09-16T02:00:32.586974+0000","last_active":"2024-09-16T02:00:35.463149+0000","last_peered":"2024-09-16T02:00:35.463149+0000","last_clean":"2024-09-16T02:00:35.463149+0000","last_became_active":"2024-09-16T02:00:32.586644+0000","last_became_peered":"2024-09-16T02:00:32.586644+0000","last_unstale":"2024-09-16T02:00:35.463149+0000","last_undegraded":"2024-09-16T02:00:35.463149+0000","last_fullsized":"2024-09-16T02:00:35.463149+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:52.913287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.460108+0000","last_change":"2024-09-16T02:00:34.447548+0000","last_active":"2024-09-16T02:00:35.460108+0000","last_peered":"2024-09-16T02:00:35.460108+0000","last_clean":"2024-09-16T02:00:35.460108+0000","last_became_active":"2024-09-16T02:00:34.447207+0000","last_became_peered":"2024-09-16T02:00:34.447207+0000","last_unstale":"2024-09-16T02:00:35.460108+0000","last_undegraded":"2024-09-16T02:00:35.460108+0000","last_fullsized":"2024-09-16T02:00:35.460108+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:22:02.494343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":46,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:59.552376+0000","last_change":"2024-09-16T02:00:32.441129+0000","last_active":"2024-09-16T02:00:59.552376+0000","last_peered":"2024-09-16T02:00:59.552376+0000","last_clean":"2024-09-16T02:00:59.552376+0000","last_became_active":"2024-09-16T02:00:32.440843+0000","last_became_peered":"2024-09-16T02:00:32.440843+0000","last_unstale":"2024-09-16T02:00:59.552376+0000","last_undegraded":"2024-09-16T02:00:59.552376+0000","last_fullsized":"2024-09-16T02:00:59.552376+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:17.721740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":53617,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":156,"num_read_kb":138,"num_write":232,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":528384,"data_stored":160851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":200,"ondisk_log_size":200,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281576,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612626,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":864256,"data_stored":414884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841559,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1007616,"data_stored":1053108,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005084,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":414814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234017,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":415192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462950,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":937984,"data_stored":1007712,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626476,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":872448,"data_stored":414531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855408,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":1006997,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018933,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":414785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247867,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":414356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444096,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":942080,"data_stored":462466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":921600,"data_stored":462246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":415870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":48554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":48774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":48931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T02:01:03.837 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-16T02:01:03.837 INFO:tasks.cephadm:Setting up client nodes... 2024-09-16T02:01:03.838 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-16T02:01:04.086 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:04.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.633+0000 7f249949a640 1 -- 172.21.15.105:0/1863169872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2494100ad0 msgr2=0x7f2494100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:04.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.633+0000 7f249949a640 1 --2- 172.21.15.105:0/1863169872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2494100ad0 0x7f2494100ed0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2484009900 tx=0x7f248402a410 comp rx=0 tx=0).stop 2024-09-16T02:01:04.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.633+0000 7f249949a640 1 -- 172.21.15.105:0/1863169872 shutdown_connections 2024-09-16T02:01:04.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.633+0000 7f249949a640 1 --2- 172.21.15.105:0/1863169872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2494101cd0 0x7f2494102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:04.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.633+0000 7f249949a640 1 --2- 172.21.15.105:0/1863169872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2494102690 0x7f2494108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:04.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.633+0000 7f249949a640 1 --2- 172.21.15.105:0/1863169872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2494100ad0 0x7f2494100ed0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:04.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.633+0000 7f249949a640 1 -- 172.21.15.105:0/1863169872 >> 172.21.15.105:0/1863169872 conn(0x7f24940fc280 msgr2=0x7f24940fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:04.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.633+0000 7f249949a640 1 -- 172.21.15.105:0/1863169872 shutdown_connections 2024-09-16T02:01:04.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.633+0000 7f249949a640 1 -- 172.21.15.105:0/1863169872 wait complete. 2024-09-16T02:01:04.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.634+0000 7f249949a640 1 Processor -- start 2024-09-16T02:01:04.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.634+0000 7f249949a640 1 -- start start 2024-09-16T02:01:04.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.634+0000 7f249949a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2494100ad0 0x7f2494104a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:04.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.634+0000 7f249949a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2494101cd0 0x7f24941030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:04.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.634+0000 7f249949a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2494102690 0x7f24941035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:04.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.634+0000 7f249949a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f249410c680 con 0x7f2494102690 2024-09-16T02:01:04.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.634+0000 7f249949a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f249410c500 con 0x7f2494100ad0 2024-09-16T02:01:04.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.634+0000 7f249949a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f249410c800 con 0x7f2494101cd0 2024-09-16T02:01:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.634+0000 7f24937fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2494102690 0x7f24941035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.634+0000 7f24937fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2494102690 0x7f24941035f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51222/0 (socket says 172.21.15.105:51222) 2024-09-16T02:01:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.634+0000 7f24937fe640 1 -- 172.21.15.105:0/2415876745 learned_addr learned my addr 172.21.15.105:0/2415876745 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:01:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.634+0000 7f248bfff640 1 --2- 172.21.15.105:0/2415876745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2494101cd0 0x7f24941030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.634+0000 7f2492ffd640 1 --2- 172.21.15.105:0/2415876745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2494100ad0 0x7f2494104a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.634+0000 7f24937fe640 1 -- 172.21.15.105:0/2415876745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2494101cd0 msgr2=0x7f24941030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.634+0000 7f24937fe640 1 --2- 172.21.15.105:0/2415876745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2494101cd0 0x7f24941030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.634+0000 7f24937fe640 1 -- 172.21.15.105:0/2415876745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2494100ad0 msgr2=0x7f2494104a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.634+0000 7f24937fe640 1 --2- 172.21.15.105:0/2415876745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2494100ad0 0x7f2494104a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.634+0000 7f24937fe640 1 -- 172.21.15.105:0/2415876745 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2494103eb0 con 0x7f2494102690 2024-09-16T02:01:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.635+0000 7f24937fe640 1 --2- 172.21.15.105:0/2415876745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2494102690 0x7f24941035f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f248c00b970 tx=0x7f248c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.635+0000 7f2490ff9640 1 -- 172.21.15.105:0/2415876745 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248c019070 con 0x7f2494102690 2024-09-16T02:01:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.635+0000 7f2490ff9640 1 -- 172.21.15.105:0/2415876745 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f248c014440 con 0x7f2494102690 2024-09-16T02:01:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.635+0000 7f249949a640 1 -- 172.21.15.105:0/2415876745 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24941a2090 con 0x7f2494102690 2024-09-16T02:01:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.635+0000 7f2490ff9640 1 -- 172.21.15.105:0/2415876745 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248c0075d0 con 0x7f2494102690 2024-09-16T02:01:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.635+0000 7f249949a640 1 -- 172.21.15.105:0/2415876745 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24941a25d0 con 0x7f2494102690 2024-09-16T02:01:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.636+0000 7f249949a640 1 -- 172.21.15.105:0/2415876745 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24941a2860 con 0x7f2494102690 2024-09-16T02:01:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.637+0000 7f2490ff9640 1 -- 172.21.15.105:0/2415876745 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f248c01c070 con 0x7f2494102690 2024-09-16T02:01:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.638+0000 7f2490ff9640 1 --2- 172.21.15.105:0/2415876745 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f24800a32d0 0x7f24800a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.638+0000 7f2492ffd640 1 --2- 172.21.15.105:0/2415876745 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f24800a32d0 0x7f24800a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.638+0000 7f2490ff9640 1 -- 172.21.15.105:0/2415876745 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f248c0cf360 con 0x7f2494102690 2024-09-16T02:01:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.638+0000 7f2492ffd640 1 --2- 172.21.15.105:0/2415876745 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f24800a32d0 0x7f24800a5790 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2484000c00 tx=0x7f24840314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.641+0000 7f2490ff9640 1 -- 172.21.15.105:0/2415876745 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f248c09c0e0 con 0x7f2494102690 2024-09-16T02:01:04.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:04 smithi105 ceph-mon[31197]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:01:04.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:04 smithi105 ceph-mon[31197]: pgmap v291: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-09-16T02:01:04.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:04 smithi190 ceph-mon[34762]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:01:04.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:04 smithi190 ceph-mon[34762]: pgmap v291: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-09-16T02:01:04.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.872+0000 7f249949a640 1 -- 172.21.15.105:0/2415876745 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f24941a2bf0 con 0x7f2494102690 2024-09-16T02:01:04.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.876+0000 7f2490ff9640 1 -- 172.21.15.105:0/2415876745 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f248c09b9e0 con 0x7f2494102690 2024-09-16T02:01:04.877 INFO:teuthology.orchestra.run.smithi105.stdout:[client.0] 2024-09-16T02:01:04.877 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQBgkedmVaAHNBAAAdkBO4QA8XOoOI13C2OpWg== 2024-09-16T02:01:04.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.878+0000 7f249949a640 1 -- 172.21.15.105:0/2415876745 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f24800a32d0 msgr2=0x7f24800a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:04.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.878+0000 7f249949a640 1 --2- 172.21.15.105:0/2415876745 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f24800a32d0 0x7f24800a5790 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2484000c00 tx=0x7f24840314a0 comp rx=0 tx=0).stop 2024-09-16T02:01:04.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.878+0000 7f249949a640 1 -- 172.21.15.105:0/2415876745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2494102690 msgr2=0x7f24941035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:04.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.878+0000 7f249949a640 1 --2- 172.21.15.105:0/2415876745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2494102690 0x7f24941035f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f248c00b970 tx=0x7f248c00be30 comp rx=0 tx=0).stop 2024-09-16T02:01:04.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.878+0000 7f249949a640 1 -- 172.21.15.105:0/2415876745 shutdown_connections 2024-09-16T02:01:04.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.878+0000 7f249949a640 1 --2- 172.21.15.105:0/2415876745 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f24800a32d0 0x7f24800a5790 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:04.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.878+0000 7f249949a640 1 --2- 172.21.15.105:0/2415876745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2494101cd0 0x7f24941030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:04.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.878+0000 7f249949a640 1 --2- 172.21.15.105:0/2415876745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2494102690 0x7f24941035f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:04.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.878+0000 7f249949a640 1 --2- 172.21.15.105:0/2415876745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2494100ad0 0x7f2494104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:04.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.878+0000 7f249949a640 1 -- 172.21.15.105:0/2415876745 >> 172.21.15.105:0/2415876745 conn(0x7f24940fc280 msgr2=0x7f249406a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:04.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.878+0000 7f249949a640 1 -- 172.21.15.105:0/2415876745 shutdown_connections 2024-09-16T02:01:04.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:04.878+0000 7f249949a640 1 -- 172.21.15.105:0/2415876745 wait complete. 2024-09-16T02:01:04.923 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:04 smithi137 ceph-mon[34099]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:01:04.923 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:04 smithi137 ceph-mon[34099]: pgmap v291: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-09-16T02:01:05.645 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-16T02:01:05.645 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-16T02:01:05.645 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-16T02:01:05.683 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-16T02:01:05.684 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-16T02:01:05.684 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-16T02:01:05.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:05 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/2415876745' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T02:01:05.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:05 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/2415876745' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T02:01:05.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:05 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/2415876745' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T02:01:05.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:05 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/2415876745' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T02:01:05.930 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:05 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2415876745' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T02:01:05.930 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:05 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2415876745' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T02:01:05.978 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:06.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.478+0000 7f5efd6d7640 1 -- 172.21.15.105:0/207498874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ef8102ef0 msgr2=0x7f5ef810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:06.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.479+0000 7f5efd6d7640 1 --2- 172.21.15.105:0/207498874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ef8102ef0 0x7f5ef810da40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0009ca0 tx=0x7f5ef002a950 comp rx=0 tx=0).stop 2024-09-16T02:01:06.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.479+0000 7f5efd6d7640 1 -- 172.21.15.105:0/207498874 shutdown_connections 2024-09-16T02:01:06.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.479+0000 7f5efd6d7640 1 --2- 172.21.15.105:0/207498874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ef8102570 0x7f5ef81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:06.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.479+0000 7f5efd6d7640 1 --2- 172.21.15.105:0/207498874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ef8102ef0 0x7f5ef810da40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:06.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.479+0000 7f5efd6d7640 1 --2- 172.21.15.105:0/207498874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ef8100e20 0x7f5ef8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:06.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.479+0000 7f5efd6d7640 1 -- 172.21.15.105:0/207498874 >> 172.21.15.105:0/207498874 conn(0x7f5ef80fc7d0 msgr2=0x7f5ef80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:06.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.479+0000 7f5efd6d7640 1 -- 172.21.15.105:0/207498874 shutdown_connections 2024-09-16T02:01:06.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.479+0000 7f5efd6d7640 1 -- 172.21.15.105:0/207498874 wait complete. 2024-09-16T02:01:06.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.479+0000 7f5efd6d7640 1 Processor -- start 2024-09-16T02:01:06.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.480+0000 7f5efd6d7640 1 -- start start 2024-09-16T02:01:06.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.480+0000 7f5efd6d7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ef8100e20 0x7f5ef8199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:06.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.480+0000 7f5efd6d7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ef8102570 0x7f5ef8199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:06.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.480+0000 7f5efd6d7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ef8102ef0 0x7f5ef81a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:06.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.480+0000 7f5efd6d7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef8110e50 con 0x7f5ef8100e20 2024-09-16T02:01:06.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.480+0000 7f5efd6d7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef8110cd0 con 0x7f5ef8102ef0 2024-09-16T02:01:06.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.480+0000 7f5efd6d7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef8110fd0 con 0x7f5ef8102570 2024-09-16T02:01:06.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.480+0000 7f5ef6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ef8100e20 0x7f5ef8199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:06.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.480+0000 7f5ef6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ef8100e20 0x7f5ef8199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51236/0 (socket says 172.21.15.105:51236) 2024-09-16T02:01:06.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.480+0000 7f5ef6ffd640 1 -- 172.21.15.105:0/2121334560 learned_addr learned my addr 172.21.15.105:0/2121334560 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:01:06.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.480+0000 7f5ef67fc640 1 --2- 172.21.15.105:0/2121334560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ef8102570 0x7f5ef8199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:06.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.480+0000 7f5ef77fe640 1 --2- 172.21.15.105:0/2121334560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ef8102ef0 0x7f5ef81a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:06.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.480+0000 7f5ef6ffd640 1 -- 172.21.15.105:0/2121334560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ef8102570 msgr2=0x7f5ef8199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:06.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.480+0000 7f5ef6ffd640 1 --2- 172.21.15.105:0/2121334560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ef8102570 0x7f5ef8199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:06.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.480+0000 7f5ef6ffd640 1 -- 172.21.15.105:0/2121334560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ef8102ef0 msgr2=0x7f5ef81a43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:06.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.480+0000 7f5ef6ffd640 1 --2- 172.21.15.105:0/2121334560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ef8102ef0 0x7f5ef81a43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:06.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.480+0000 7f5ef6ffd640 1 -- 172.21.15.105:0/2121334560 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef81a48e0 con 0x7f5ef8100e20 2024-09-16T02:01:06.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.480+0000 7f5ef67fc640 1 --2- 172.21.15.105:0/2121334560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ef8102570 0x7f5ef8199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:01:06.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.480+0000 7f5ef77fe640 1 --2- 172.21.15.105:0/2121334560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ef8102ef0 0x7f5ef81a43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:01:06.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.480+0000 7f5ef6ffd640 1 --2- 172.21.15.105:0/2121334560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ef8100e20 0x7f5ef8199320 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800d950 tx=0x7f5ee800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:06.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.480+0000 7f5eeffff640 1 -- 172.21.15.105:0/2121334560 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8017070 con 0x7f5ef8100e20 2024-09-16T02:01:06.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.481+0000 7f5eeffff640 1 -- 172.21.15.105:0/2121334560 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ee8009c30 con 0x7f5ef8100e20 2024-09-16T02:01:06.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.481+0000 7f5efd6d7640 1 -- 172.21.15.105:0/2121334560 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ef81a4ab0 con 0x7f5ef8100e20 2024-09-16T02:01:06.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.481+0000 7f5eeffff640 1 -- 172.21.15.105:0/2121334560 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee800f620 con 0x7f5ef8100e20 2024-09-16T02:01:06.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.481+0000 7f5efd6d7640 1 -- 172.21.15.105:0/2121334560 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ef81a4fc0 con 0x7f5ef8100e20 2024-09-16T02:01:06.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.482+0000 7f5efd6d7640 1 -- 172.21.15.105:0/2121334560 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ef8101200 con 0x7f5ef8100e20 2024-09-16T02:01:06.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.483+0000 7f5eeffff640 1 -- 172.21.15.105:0/2121334560 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f5ee80109a0 con 0x7f5ef8100e20 2024-09-16T02:01:06.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.483+0000 7f5eeffff640 1 --2- 172.21.15.105:0/2121334560 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5ee40a32d0 0x7f5ee40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:06.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.484+0000 7f5ef67fc640 1 --2- 172.21.15.105:0/2121334560 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5ee40a32d0 0x7f5ee40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:06.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.484+0000 7f5eeffff640 1 -- 172.21.15.105:0/2121334560 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f5ee80cf5d0 con 0x7f5ef8100e20 2024-09-16T02:01:06.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.484+0000 7f5ef67fc640 1 --2- 172.21.15.105:0/2121334560 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5ee40a32d0 0x7f5ee40a5790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5ef806aac0 tx=0x7f5ee0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:06.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.488+0000 7f5eeffff640 1 -- 172.21.15.105:0/2121334560 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5ee8009de0 con 0x7f5ef8100e20 2024-09-16T02:01:06.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.707+0000 7f5efd6d7640 1 -- 172.21.15.105:0/2121334560 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f5ef8109090 con 0x7f5ef8100e20 2024-09-16T02:01:06.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.712+0000 7f5eeffff640 1 -- 172.21.15.105:0/2121334560 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264056 (secure 0 0 0) 0x7f5ee809c340 con 0x7f5ef8100e20 2024-09-16T02:01:06.716 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:01:06.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.715+0000 7f5efd6d7640 1 -- 172.21.15.105:0/2121334560 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5ee40a32d0 msgr2=0x7f5ee40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:06.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.715+0000 7f5efd6d7640 1 --2- 172.21.15.105:0/2121334560 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5ee40a32d0 0x7f5ee40a5790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5ef806aac0 tx=0x7f5ee0008040 comp rx=0 tx=0).stop 2024-09-16T02:01:06.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.715+0000 7f5efd6d7640 1 -- 172.21.15.105:0/2121334560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ef8100e20 msgr2=0x7f5ef8199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:06.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.715+0000 7f5efd6d7640 1 --2- 172.21.15.105:0/2121334560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ef8100e20 0x7f5ef8199320 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800d950 tx=0x7f5ee800de10 comp rx=0 tx=0).stop 2024-09-16T02:01:06.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.715+0000 7f5efd6d7640 1 -- 172.21.15.105:0/2121334560 shutdown_connections 2024-09-16T02:01:06.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.715+0000 7f5efd6d7640 1 --2- 172.21.15.105:0/2121334560 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5ee40a32d0 0x7f5ee40a5790 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:06.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.715+0000 7f5efd6d7640 1 --2- 172.21.15.105:0/2121334560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ef8102570 0x7f5ef8199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:06.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.715+0000 7f5efd6d7640 1 --2- 172.21.15.105:0/2121334560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ef8102ef0 0x7f5ef81a43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:06.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.715+0000 7f5efd6d7640 1 --2- 172.21.15.105:0/2121334560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ef8100e20 0x7f5ef8199320 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:06.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.715+0000 7f5efd6d7640 1 -- 172.21.15.105:0/2121334560 >> 172.21.15.105:0/2121334560 conn(0x7f5ef80fc7d0 msgr2=0x7f5ef80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:06.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.715+0000 7f5efd6d7640 1 -- 172.21.15.105:0/2121334560 shutdown_connections 2024-09-16T02:01:06.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:06.715+0000 7f5efd6d7640 1 -- 172.21.15.105:0/2121334560 wait complete. 2024-09-16T02:01:06.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:06 smithi190 ceph-mon[34762]: pgmap v292: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-16T02:01:06.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:06 smithi137 ceph-mon[34099]: pgmap v292: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-16T02:01:06.935 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:06 smithi105 ceph-mon[31197]: pgmap v292: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-16T02:01:07.533 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":275626334},{"type":"v1","addr":"172.21.15.105:6801","nonce":275626334}]},"active_addr":"172.21.15.105:6801/275626334","active_change":"2024-09-16T01:53:39.351418+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.105:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.105:0","nonce":3952217751}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.105:0","nonce":1478019961}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.105:0","nonce":2130860808}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.105:0","nonce":2972851893}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.105:0","nonce":2185621474}]}]} 2024-09-16T02:01:07.538 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-16T02:01:07.538 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-16T02:01:07.539 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T02:01:07.788 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:07.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:07 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2121334560' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-16T02:01:07.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:07 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/2121334560' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-16T02:01:07.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:07 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/2121334560' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-16T02:01:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.271+0000 7ffb6edb7640 1 -- 172.21.15.105:0/3890218623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb68100e20 msgr2=0x7ffb68101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.271+0000 7ffb6edb7640 1 --2- 172.21.15.105:0/3890218623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb68100e20 0x7ffb68101200 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009900 tx=0x7ffb6002a480 comp rx=0 tx=0).stop 2024-09-16T02:01:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.271+0000 7ffb6edb7640 1 -- 172.21.15.105:0/3890218623 shutdown_connections 2024-09-16T02:01:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.271+0000 7ffb6edb7640 1 --2- 172.21.15.105:0/3890218623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb68102ef0 0x7ffb6810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.271+0000 7ffb6edb7640 1 --2- 172.21.15.105:0/3890218623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb68102570 0x7ffb681029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.271+0000 7ffb6edb7640 1 --2- 172.21.15.105:0/3890218623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb68100e20 0x7ffb68101200 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.271+0000 7ffb6edb7640 1 -- 172.21.15.105:0/3890218623 >> 172.21.15.105:0/3890218623 conn(0x7ffb680fc7d0 msgr2=0x7ffb680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.271+0000 7ffb6edb7640 1 -- 172.21.15.105:0/3890218623 shutdown_connections 2024-09-16T02:01:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.271+0000 7ffb6edb7640 1 -- 172.21.15.105:0/3890218623 wait complete. 2024-09-16T02:01:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.271+0000 7ffb6edb7640 1 Processor -- start 2024-09-16T02:01:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.271+0000 7ffb6edb7640 1 -- start start 2024-09-16T02:01:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.272+0000 7ffb6edb7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb68100e20 0x7ffb68199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.272+0000 7ffb6edb7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb68102570 0x7ffb68199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.272+0000 7ffb6edb7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb68102ef0 0x7ffb681a4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.272+0000 7ffb6edb7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ffb68110e30 con 0x7ffb68102ef0 2024-09-16T02:01:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.272+0000 7ffb6edb7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ffb68110cb0 con 0x7ffb68100e20 2024-09-16T02:01:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.272+0000 7ffb6edb7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ffb68110fb0 con 0x7ffb68102570 2024-09-16T02:01:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.272+0000 7ffb6d354640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb68102ef0 0x7ffb681a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.272+0000 7ffb6d354640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb68102ef0 0x7ffb681a4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56010/0 (socket says 172.21.15.105:56010) 2024-09-16T02:01:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.272+0000 7ffb6d354640 1 -- 172.21.15.105:0/4125967272 learned_addr learned my addr 172.21.15.105:0/4125967272 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:01:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.272+0000 7ffb6d354640 1 -- 172.21.15.105:0/4125967272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb68102570 msgr2=0x7ffb68199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.272+0000 7ffb6d354640 1 --2- 172.21.15.105:0/4125967272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb68102570 0x7ffb68199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:08.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.272+0000 7ffb6d354640 1 -- 172.21.15.105:0/4125967272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb68100e20 msgr2=0x7ffb68199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:08.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.272+0000 7ffb6d354640 1 --2- 172.21.15.105:0/4125967272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb68100e20 0x7ffb68199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:08.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.272+0000 7ffb6d354640 1 -- 172.21.15.105:0/4125967272 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb681a47d0 con 0x7ffb68102ef0 2024-09-16T02:01:08.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.272+0000 7ffb6d354640 1 --2- 172.21.15.105:0/4125967272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb68102ef0 0x7ffb681a4290 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ffb5000c970 tx=0x7ffb5000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:08.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.272+0000 7ffb65ffb640 1 -- 172.21.15.105:0/4125967272 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb50007bb0 con 0x7ffb68102ef0 2024-09-16T02:01:08.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.272+0000 7ffb65ffb640 1 -- 172.21.15.105:0/4125967272 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb50007d50 con 0x7ffb68102ef0 2024-09-16T02:01:08.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.272+0000 7ffb6edb7640 1 -- 172.21.15.105:0/4125967272 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb681a49a0 con 0x7ffb68102ef0 2024-09-16T02:01:08.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.272+0000 7ffb65ffb640 1 -- 172.21.15.105:0/4125967272 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb50018700 con 0x7ffb68102ef0 2024-09-16T02:01:08.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.273+0000 7ffb6edb7640 1 -- 172.21.15.105:0/4125967272 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb681a4eb0 con 0x7ffb68102ef0 2024-09-16T02:01:08.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.273+0000 7ffb6edb7640 1 -- 172.21.15.105:0/4125967272 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb68101200 con 0x7ffb68102ef0 2024-09-16T02:01:08.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.274+0000 7ffb65ffb640 1 -- 172.21.15.105:0/4125967272 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7ffb5000fa20 con 0x7ffb68102ef0 2024-09-16T02:01:08.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.275+0000 7ffb65ffb640 1 --2- 172.21.15.105:0/4125967272 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ffb540a3350 0x7ffb540a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:08.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.275+0000 7ffb6cb53640 1 --2- 172.21.15.105:0/4125967272 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ffb540a3350 0x7ffb540a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:08.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.275+0000 7ffb65ffb640 1 -- 172.21.15.105:0/4125967272 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7ffb500cf950 con 0x7ffb68102ef0 2024-09-16T02:01:08.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.275+0000 7ffb6cb53640 1 --2- 172.21.15.105:0/4125967272 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ffb540a3350 0x7ffb540a5810 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ffb600096c0 tx=0x7ffb60000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:08.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.279+0000 7ffb65ffb640 1 -- 172.21.15.105:0/4125967272 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffb50014030 con 0x7ffb68102ef0 2024-09-16T02:01:08.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.448+0000 7ffb6edb7640 1 -- 172.21.15.105:0/4125967272 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffb681090c0 con 0x7ffb68102ef0 2024-09-16T02:01:08.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.449+0000 7ffb65ffb640 1 -- 172.21.15.105:0/4125967272 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22327 (secure 0 0 0) 0x7ffb5009c6c0 con 0x7ffb68102ef0 2024-09-16T02:01:08.450 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:01:08.451 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":79,"fsid":"423b7a2a-73ce-11ef-bceb-c7b262605968","created":"2024-09-16T01:52:44.388033+0000","modified":"2024-09-16T02:00:35.112006+0000","last_up_change":"2024-09-16T01:59:42.218034+0000","last_in_change":"2024-09-16T01:59:18.785252+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T01:55:53.680808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T02:00:31.286117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T02:00:33.310045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a60d6c72-da8b-4b02-bb33-672be795e7f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6803","nonce":1226660290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6805","nonce":1226660290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6809","nonce":1226660290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6807","nonce":1226660290}]},"public_addr":"172.21.15.105:6803/1226660290","cluster_addr":"172.21.15.105:6805/1226660290","heartbeat_back_addr":"172.21.15.105:6809/1226660290","heartbeat_front_addr":"172.21.15.105:6807/1226660290","state":["exists","up"]},{"osd":1,"uuid":"b1d9ae84-b9a0-4a7e-b0fe-c402d46809fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6801","nonce":34753956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6803","nonce":34753956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6807","nonce":34753956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6805","nonce":34753956}]},"public_addr":"172.21.15.137:6801/34753956","cluster_addr":"172.21.15.137:6803/34753956","heartbeat_back_addr":"172.21.15.137:6807/34753956","heartbeat_front_addr":"172.21.15.137:6805/34753956","state":["exists","up"]},{"osd":2,"uuid":"137faae6-2e8b-44b5-8976-3d95a6eef1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6801","nonce":1159531918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6803","nonce":1159531918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6807","nonce":1159531918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6805","nonce":1159531918}]},"public_addr":"172.21.15.190:6801/1159531918","cluster_addr":"172.21.15.190:6803/1159531918","heartbeat_back_addr":"172.21.15.190:6807/1159531918","heartbeat_front_addr":"172.21.15.190:6805/1159531918","state":["exists","up"]},{"osd":3,"uuid":"5743c37b-5cd8-42b6-81c8-5cf5afc71e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6811","nonce":604905679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6813","nonce":604905679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6817","nonce":604905679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6815","nonce":604905679}]},"public_addr":"172.21.15.105:6811/604905679","cluster_addr":"172.21.15.105:6813/604905679","heartbeat_back_addr":"172.21.15.105:6817/604905679","heartbeat_front_addr":"172.21.15.105:6815/604905679","state":["exists","up"]},{"osd":4,"uuid":"e439d4a2-b54c-4de9-97f6-3510afe57f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6809","nonce":3758109169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6811","nonce":3758109169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6815","nonce":3758109169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6813","nonce":3758109169}]},"public_addr":"172.21.15.137:6809/3758109169","cluster_addr":"172.21.15.137:6811/3758109169","heartbeat_back_addr":"172.21.15.137:6815/3758109169","heartbeat_front_addr":"172.21.15.137:6813/3758109169","state":["exists","up"]},{"osd":5,"uuid":"d1e2a7d5-99f8-40fa-af6e-0305cc0af677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6809","nonce":2086773754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6811","nonce":2086773754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6815","nonce":2086773754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6813","nonce":2086773754}]},"public_addr":"172.21.15.190:6809/2086773754","cluster_addr":"172.21.15.190:6811/2086773754","heartbeat_back_addr":"172.21.15.190:6815/2086773754","heartbeat_front_addr":"172.21.15.190:6813/2086773754","state":["exists","up"]},{"osd":6,"uuid":"cd418159-a0f6-44c7-9213-c010b132fb56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6819","nonce":4012018725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6821","nonce":4012018725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6825","nonce":4012018725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6823","nonce":4012018725}]},"public_addr":"172.21.15.105:6819/4012018725","cluster_addr":"172.21.15.105:6821/4012018725","heartbeat_back_addr":"172.21.15.105:6825/4012018725","heartbeat_front_addr":"172.21.15.105:6823/4012018725","state":["exists","up"]},{"osd":7,"uuid":"76c15863-801e-4a72-be73-ba9df46c15b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6817","nonce":888678288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6819","nonce":888678288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6823","nonce":888678288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6821","nonce":888678288}]},"public_addr":"172.21.15.137:6817/888678288","cluster_addr":"172.21.15.137:6819/888678288","heartbeat_back_addr":"172.21.15.137:6823/888678288","heartbeat_front_addr":"172.21.15.137:6821/888678288","state":["exists","up"]},{"osd":8,"uuid":"6718597c-6f42-48ee-9493-edb2396fe707","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6817","nonce":2132638341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6819","nonce":2132638341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6823","nonce":2132638341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6821","nonce":2132638341}]},"public_addr":"172.21.15.190:6817/2132638341","cluster_addr":"172.21.15.190:6819/2132638341","heartbeat_back_addr":"172.21.15.190:6823/2132638341","heartbeat_front_addr":"172.21.15.190:6821/2132638341","state":["exists","up"]},{"osd":9,"uuid":"5178edc0-a2e8-491c-821c-6f41ec5aeb2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6827","nonce":3375034979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6829","nonce":3375034979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6833","nonce":3375034979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6831","nonce":3375034979}]},"public_addr":"172.21.15.105:6827/3375034979","cluster_addr":"172.21.15.105:6829/3375034979","heartbeat_back_addr":"172.21.15.105:6833/3375034979","heartbeat_front_addr":"172.21.15.105:6831/3375034979","state":["exists","up"]},{"osd":10,"uuid":"ad145d3e-a6e2-4dd7-bea3-bcd01eb3413e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6825","nonce":318562234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6827","nonce":318562234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6831","nonce":318562234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6829","nonce":318562234}]},"public_addr":"172.21.15.137:6825/318562234","cluster_addr":"172.21.15.137:6827/318562234","heartbeat_back_addr":"172.21.15.137:6831/318562234","heartbeat_front_addr":"172.21.15.137:6829/318562234","state":["exists","up"]},{"osd":11,"uuid":"6d44bced-3a8b-4763-a93e-0ea34acb4cb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6825","nonce":3820460689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6827","nonce":3820460689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6831","nonce":3820460689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6829","nonce":3820460689}]},"public_addr":"172.21.15.190:6825/3820460689","cluster_addr":"172.21.15.190:6827/3820460689","heartbeat_back_addr":"172.21.15.190:6831/3820460689","heartbeat_front_addr":"172.21.15.190:6829/3820460689","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:55:02.759033+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:55:26.829142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:55:51.120200+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:56:17.277870+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:56:42.313701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:07.013416+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:32.477916+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:58.643764+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:23.454908+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:48.869082+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:14.921573+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:40.908993+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:0/2453739657":"2024-09-17T01:53:39.351314+0000","172.21.15.105:6801/1469189484":"2024-09-17T01:53:39.351314+0000","172.21.15.105:0/3466166879":"2024-09-17T01:53:39.351314+0000","172.21.15.105:6801/1268348742":"2024-09-17T01:53:15.522227+0000","172.21.15.105:0/2897248478":"2024-09-17T01:53:39.351314+0000","172.21.15.105:0/3709363371":"2024-09-17T01:53:15.522227+0000","172.21.15.105:6800/1268348742":"2024-09-17T01:53:15.522227+0000","172.21.15.105:6800/1469189484":"2024-09-17T01:53:39.351314+0000","172.21.15.105:0/1198176299":"2024-09-17T01:53:15.522227+0000","172.21.15.105:0/3031917848":"2024-09-17T01:53:15.522227+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T02:01:08.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.452+0000 7ffb6edb7640 1 -- 172.21.15.105:0/4125967272 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ffb540a3350 msgr2=0x7ffb540a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:08.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.452+0000 7ffb6edb7640 1 --2- 172.21.15.105:0/4125967272 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ffb540a3350 0x7ffb540a5810 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ffb600096c0 tx=0x7ffb60000bc0 comp rx=0 tx=0).stop 2024-09-16T02:01:08.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.452+0000 7ffb6edb7640 1 -- 172.21.15.105:0/4125967272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb68102ef0 msgr2=0x7ffb681a4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:08.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.452+0000 7ffb6edb7640 1 --2- 172.21.15.105:0/4125967272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb68102ef0 0x7ffb681a4290 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ffb5000c970 tx=0x7ffb5000ce30 comp rx=0 tx=0).stop 2024-09-16T02:01:08.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.452+0000 7ffb6edb7640 1 -- 172.21.15.105:0/4125967272 shutdown_connections 2024-09-16T02:01:08.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.452+0000 7ffb6edb7640 1 --2- 172.21.15.105:0/4125967272 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ffb540a3350 0x7ffb540a5810 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:08.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.452+0000 7ffb6edb7640 1 --2- 172.21.15.105:0/4125967272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb68102570 0x7ffb68199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:08.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.452+0000 7ffb6edb7640 1 --2- 172.21.15.105:0/4125967272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb68102ef0 0x7ffb681a4290 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:08.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.452+0000 7ffb6edb7640 1 --2- 172.21.15.105:0/4125967272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb68100e20 0x7ffb68199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:08.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.452+0000 7ffb6edb7640 1 -- 172.21.15.105:0/4125967272 >> 172.21.15.105:0/4125967272 conn(0x7ffb680fc7d0 msgr2=0x7ffb680fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:08.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.452+0000 7ffb6edb7640 1 -- 172.21.15.105:0/4125967272 shutdown_connections 2024-09-16T02:01:08.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:08.452+0000 7ffb6edb7640 1 -- 172.21.15.105:0/4125967272 wait complete. 2024-09-16T02:01:08.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:08 smithi105 ceph-mon[31197]: pgmap v293: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-16T02:01:08.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:08 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/4125967272' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:01:08.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:08 smithi190 ceph-mon[34762]: pgmap v293: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-16T02:01:08.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:08 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/4125967272' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:01:08.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:08 smithi137 ceph-mon[34099]: pgmap v293: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-16T02:01:08.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:08 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/4125967272' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:01:09.197 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-16T02:01:09.197 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T02:01:09.445 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:09.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.928+0000 7fa411bd1640 1 -- 172.21.15.105:0/2148345766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa40c106f10 msgr2=0x7fa40c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:09.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.928+0000 7fa411bd1640 1 --2- 172.21.15.105:0/2148345766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa40c106f10 0x7fa40c10ea90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa404009ca0 tx=0x7fa40402a8c0 comp rx=0 tx=0).stop 2024-09-16T02:01:09.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.928+0000 7fa411bd1640 1 -- 172.21.15.105:0/2148345766 shutdown_connections 2024-09-16T02:01:09.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.928+0000 7fa411bd1640 1 --2- 172.21.15.105:0/2148345766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa40c106f10 0x7fa40c10ea90 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:09.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.928+0000 7fa411bd1640 1 --2- 172.21.15.105:0/2148345766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa40c102790 0x7fa40c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:09.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.928+0000 7fa411bd1640 1 --2- 172.21.15.105:0/2148345766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa40c101de0 0x7fa40c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:09.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.928+0000 7fa411bd1640 1 -- 172.21.15.105:0/2148345766 >> 172.21.15.105:0/2148345766 conn(0x7fa40c0fd8c0 msgr2=0x7fa40c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:09.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.928+0000 7fa411bd1640 1 -- 172.21.15.105:0/2148345766 shutdown_connections 2024-09-16T02:01:09.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.928+0000 7fa411bd1640 1 -- 172.21.15.105:0/2148345766 wait complete. 2024-09-16T02:01:09.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.929+0000 7fa411bd1640 1 Processor -- start 2024-09-16T02:01:09.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.929+0000 7fa411bd1640 1 -- start start 2024-09-16T02:01:09.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.929+0000 7fa411bd1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa40c101de0 0x7fa40c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:09.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.929+0000 7fa411bd1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa40c102790 0x7fa40c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:09.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.929+0000 7fa411bd1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa40c106f10 0x7fa40c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:09.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.929+0000 7fa411bd1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c111f10 con 0x7fa40c101de0 2024-09-16T02:01:09.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.929+0000 7fa411bd1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c111d90 con 0x7fa40c106f10 2024-09-16T02:01:09.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.929+0000 7fa411bd1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c112090 con 0x7fa40c102790 2024-09-16T02:01:09.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.929+0000 7fa40b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa40c101de0 0x7fa40c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:09.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.929+0000 7fa40b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa40c101de0 0x7fa40c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56024/0 (socket says 172.21.15.105:56024) 2024-09-16T02:01:09.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.929+0000 7fa40b7fe640 1 -- 172.21.15.105:0/319357100 learned_addr learned my addr 172.21.15.105:0/319357100 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:01:09.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.929+0000 7fa40bfff640 1 --2- 172.21.15.105:0/319357100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa40c106f10 0x7fa40c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:09.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.929+0000 7fa403fff640 1 --2- 172.21.15.105:0/319357100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa40c102790 0x7fa40c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:09.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.929+0000 7fa40b7fe640 1 -- 172.21.15.105:0/319357100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa40c102790 msgr2=0x7fa40c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:09.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.929+0000 7fa40b7fe640 1 --2- 172.21.15.105:0/319357100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa40c102790 0x7fa40c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:09.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.929+0000 7fa40b7fe640 1 -- 172.21.15.105:0/319357100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa40c106f10 msgr2=0x7fa40c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:09.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.929+0000 7fa40b7fe640 1 --2- 172.21.15.105:0/319357100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa40c106f10 0x7fa40c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:09.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.930+0000 7fa40b7fe640 1 -- 172.21.15.105:0/319357100 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa40c1a6910 con 0x7fa40c101de0 2024-09-16T02:01:09.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.930+0000 7fa40bfff640 1 --2- 172.21.15.105:0/319357100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa40c106f10 0x7fa40c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:01:09.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.930+0000 7fa40b7fe640 1 --2- 172.21.15.105:0/319357100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa40c101de0 0x7fa40c0744f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc00b4a0 tx=0x7fa3fc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:09.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.930+0000 7fa4097fa640 1 -- 172.21.15.105:0/319357100 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc014070 con 0x7fa40c101de0 2024-09-16T02:01:09.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.930+0000 7fa4097fa640 1 -- 172.21.15.105:0/319357100 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3fc010ac0 con 0x7fa40c101de0 2024-09-16T02:01:09.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.930+0000 7fa4097fa640 1 -- 172.21.15.105:0/319357100 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc017c30 con 0x7fa40c101de0 2024-09-16T02:01:09.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.930+0000 7fa411bd1640 1 -- 172.21.15.105:0/319357100 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa40c1a6c00 con 0x7fa40c101de0 2024-09-16T02:01:09.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.930+0000 7fa411bd1640 1 -- 172.21.15.105:0/319357100 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa40c1a7140 con 0x7fa40c101de0 2024-09-16T02:01:09.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.931+0000 7fa411bd1640 1 -- 172.21.15.105:0/319357100 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa40c1021c0 con 0x7fa40c101de0 2024-09-16T02:01:09.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.932+0000 7fa4097fa640 1 -- 172.21.15.105:0/319357100 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fa3fc017070 con 0x7fa40c101de0 2024-09-16T02:01:09.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.933+0000 7fa4097fa640 1 --2- 172.21.15.105:0/319357100 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa3f80a32d0 0x7fa3f80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:09.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.933+0000 7fa403fff640 1 --2- 172.21.15.105:0/319357100 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa3f80a32d0 0x7fa3f80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:09.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.933+0000 7fa4097fa640 1 -- 172.21.15.105:0/319357100 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fa3fc0cefa0 con 0x7fa40c101de0 2024-09-16T02:01:09.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.933+0000 7fa403fff640 1 --2- 172.21.15.105:0/319357100 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa3f80a32d0 0x7fa3f80a5790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa40c075f80 tx=0x7fa3f4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:09.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:09.936+0000 7fa4097fa640 1 -- 172.21.15.105:0/319357100 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa3fc09be20 con 0x7fa40c101de0 2024-09-16T02:01:10.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:10.106+0000 7fa411bd1640 1 -- 172.21.15.105:0/319357100 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa40c02d7c0 con 0x7fa40c101de0 2024-09-16T02:01:10.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:10.107+0000 7fa4097fa640 1 -- 172.21.15.105:0/319357100 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22327 (secure 0 0 0) 0x7fa3fc09b720 con 0x7fa40c101de0 2024-09-16T02:01:10.108 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:01:10.109 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":79,"fsid":"423b7a2a-73ce-11ef-bceb-c7b262605968","created":"2024-09-16T01:52:44.388033+0000","modified":"2024-09-16T02:00:35.112006+0000","last_up_change":"2024-09-16T01:59:42.218034+0000","last_in_change":"2024-09-16T01:59:18.785252+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T01:55:53.680808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T02:00:31.286117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T02:00:33.310045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a60d6c72-da8b-4b02-bb33-672be795e7f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6803","nonce":1226660290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6805","nonce":1226660290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6809","nonce":1226660290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6807","nonce":1226660290}]},"public_addr":"172.21.15.105:6803/1226660290","cluster_addr":"172.21.15.105:6805/1226660290","heartbeat_back_addr":"172.21.15.105:6809/1226660290","heartbeat_front_addr":"172.21.15.105:6807/1226660290","state":["exists","up"]},{"osd":1,"uuid":"b1d9ae84-b9a0-4a7e-b0fe-c402d46809fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6801","nonce":34753956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6803","nonce":34753956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6807","nonce":34753956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6805","nonce":34753956}]},"public_addr":"172.21.15.137:6801/34753956","cluster_addr":"172.21.15.137:6803/34753956","heartbeat_back_addr":"172.21.15.137:6807/34753956","heartbeat_front_addr":"172.21.15.137:6805/34753956","state":["exists","up"]},{"osd":2,"uuid":"137faae6-2e8b-44b5-8976-3d95a6eef1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6801","nonce":1159531918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6803","nonce":1159531918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6807","nonce":1159531918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6805","nonce":1159531918}]},"public_addr":"172.21.15.190:6801/1159531918","cluster_addr":"172.21.15.190:6803/1159531918","heartbeat_back_addr":"172.21.15.190:6807/1159531918","heartbeat_front_addr":"172.21.15.190:6805/1159531918","state":["exists","up"]},{"osd":3,"uuid":"5743c37b-5cd8-42b6-81c8-5cf5afc71e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6811","nonce":604905679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6813","nonce":604905679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6817","nonce":604905679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6815","nonce":604905679}]},"public_addr":"172.21.15.105:6811/604905679","cluster_addr":"172.21.15.105:6813/604905679","heartbeat_back_addr":"172.21.15.105:6817/604905679","heartbeat_front_addr":"172.21.15.105:6815/604905679","state":["exists","up"]},{"osd":4,"uuid":"e439d4a2-b54c-4de9-97f6-3510afe57f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6809","nonce":3758109169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6811","nonce":3758109169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6815","nonce":3758109169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6813","nonce":3758109169}]},"public_addr":"172.21.15.137:6809/3758109169","cluster_addr":"172.21.15.137:6811/3758109169","heartbeat_back_addr":"172.21.15.137:6815/3758109169","heartbeat_front_addr":"172.21.15.137:6813/3758109169","state":["exists","up"]},{"osd":5,"uuid":"d1e2a7d5-99f8-40fa-af6e-0305cc0af677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6809","nonce":2086773754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6811","nonce":2086773754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6815","nonce":2086773754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6813","nonce":2086773754}]},"public_addr":"172.21.15.190:6809/2086773754","cluster_addr":"172.21.15.190:6811/2086773754","heartbeat_back_addr":"172.21.15.190:6815/2086773754","heartbeat_front_addr":"172.21.15.190:6813/2086773754","state":["exists","up"]},{"osd":6,"uuid":"cd418159-a0f6-44c7-9213-c010b132fb56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6819","nonce":4012018725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6821","nonce":4012018725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6825","nonce":4012018725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6823","nonce":4012018725}]},"public_addr":"172.21.15.105:6819/4012018725","cluster_addr":"172.21.15.105:6821/4012018725","heartbeat_back_addr":"172.21.15.105:6825/4012018725","heartbeat_front_addr":"172.21.15.105:6823/4012018725","state":["exists","up"]},{"osd":7,"uuid":"76c15863-801e-4a72-be73-ba9df46c15b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6817","nonce":888678288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6819","nonce":888678288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6823","nonce":888678288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6821","nonce":888678288}]},"public_addr":"172.21.15.137:6817/888678288","cluster_addr":"172.21.15.137:6819/888678288","heartbeat_back_addr":"172.21.15.137:6823/888678288","heartbeat_front_addr":"172.21.15.137:6821/888678288","state":["exists","up"]},{"osd":8,"uuid":"6718597c-6f42-48ee-9493-edb2396fe707","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6817","nonce":2132638341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6819","nonce":2132638341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6823","nonce":2132638341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6821","nonce":2132638341}]},"public_addr":"172.21.15.190:6817/2132638341","cluster_addr":"172.21.15.190:6819/2132638341","heartbeat_back_addr":"172.21.15.190:6823/2132638341","heartbeat_front_addr":"172.21.15.190:6821/2132638341","state":["exists","up"]},{"osd":9,"uuid":"5178edc0-a2e8-491c-821c-6f41ec5aeb2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6827","nonce":3375034979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6829","nonce":3375034979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6833","nonce":3375034979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6831","nonce":3375034979}]},"public_addr":"172.21.15.105:6827/3375034979","cluster_addr":"172.21.15.105:6829/3375034979","heartbeat_back_addr":"172.21.15.105:6833/3375034979","heartbeat_front_addr":"172.21.15.105:6831/3375034979","state":["exists","up"]},{"osd":10,"uuid":"ad145d3e-a6e2-4dd7-bea3-bcd01eb3413e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6825","nonce":318562234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6827","nonce":318562234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6831","nonce":318562234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6829","nonce":318562234}]},"public_addr":"172.21.15.137:6825/318562234","cluster_addr":"172.21.15.137:6827/318562234","heartbeat_back_addr":"172.21.15.137:6831/318562234","heartbeat_front_addr":"172.21.15.137:6829/318562234","state":["exists","up"]},{"osd":11,"uuid":"6d44bced-3a8b-4763-a93e-0ea34acb4cb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6825","nonce":3820460689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6827","nonce":3820460689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6831","nonce":3820460689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6829","nonce":3820460689}]},"public_addr":"172.21.15.190:6825/3820460689","cluster_addr":"172.21.15.190:6827/3820460689","heartbeat_back_addr":"172.21.15.190:6831/3820460689","heartbeat_front_addr":"172.21.15.190:6829/3820460689","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:55:02.759033+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:55:26.829142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:55:51.120200+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:56:17.277870+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:56:42.313701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:07.013416+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:32.477916+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:58.643764+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:23.454908+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:48.869082+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:14.921573+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:40.908993+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:0/2453739657":"2024-09-17T01:53:39.351314+0000","172.21.15.105:6801/1469189484":"2024-09-17T01:53:39.351314+0000","172.21.15.105:0/3466166879":"2024-09-17T01:53:39.351314+0000","172.21.15.105:6801/1268348742":"2024-09-17T01:53:15.522227+0000","172.21.15.105:0/2897248478":"2024-09-17T01:53:39.351314+0000","172.21.15.105:0/3709363371":"2024-09-17T01:53:15.522227+0000","172.21.15.105:6800/1268348742":"2024-09-17T01:53:15.522227+0000","172.21.15.105:6800/1469189484":"2024-09-17T01:53:39.351314+0000","172.21.15.105:0/1198176299":"2024-09-17T01:53:15.522227+0000","172.21.15.105:0/3031917848":"2024-09-17T01:53:15.522227+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T02:01:10.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:10.109+0000 7fa411bd1640 1 -- 172.21.15.105:0/319357100 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa3f80a32d0 msgr2=0x7fa3f80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:10.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:10.109+0000 7fa411bd1640 1 --2- 172.21.15.105:0/319357100 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa3f80a32d0 0x7fa3f80a5790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa40c075f80 tx=0x7fa3f4008040 comp rx=0 tx=0).stop 2024-09-16T02:01:10.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:10.109+0000 7fa411bd1640 1 -- 172.21.15.105:0/319357100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa40c101de0 msgr2=0x7fa40c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:10.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:10.109+0000 7fa411bd1640 1 --2- 172.21.15.105:0/319357100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa40c101de0 0x7fa40c0744f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc00b4a0 tx=0x7fa3fc00b960 comp rx=0 tx=0).stop 2024-09-16T02:01:10.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:10.109+0000 7fa411bd1640 1 -- 172.21.15.105:0/319357100 shutdown_connections 2024-09-16T02:01:10.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:10.109+0000 7fa411bd1640 1 --2- 172.21.15.105:0/319357100 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa3f80a32d0 0x7fa3f80a5790 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:10.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:10.109+0000 7fa411bd1640 1 --2- 172.21.15.105:0/319357100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa40c102790 0x7fa40c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:10.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:10.109+0000 7fa411bd1640 1 --2- 172.21.15.105:0/319357100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa40c106f10 0x7fa40c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:10.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:10.110+0000 7fa411bd1640 1 --2- 172.21.15.105:0/319357100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa40c101de0 0x7fa40c0744f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:10.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:10.110+0000 7fa411bd1640 1 -- 172.21.15.105:0/319357100 >> 172.21.15.105:0/319357100 conn(0x7fa40c0fd8c0 msgr2=0x7fa40c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:10.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:10.110+0000 7fa411bd1640 1 -- 172.21.15.105:0/319357100 shutdown_connections 2024-09-16T02:01:10.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:10.110+0000 7fa411bd1640 1 -- 172.21.15.105:0/319357100 wait complete. 2024-09-16T02:01:10.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:10 smithi105 ceph-mon[31197]: pgmap v294: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-09-16T02:01:10.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:10 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/319357100' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:01:10.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:10 smithi190 ceph-mon[34762]: pgmap v294: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-09-16T02:01:10.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:10 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/319357100' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:01:10.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:10 smithi137 ceph-mon[34099]: pgmap v294: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-09-16T02:01:10.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:10 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/319357100' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:01:10.954 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-16T02:01:10.954 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-16T02:01:10.954 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-16T02:01:10.955 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-16T02:01:10.955 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-16T02:01:10.956 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-16T02:01:10.956 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-16T02:01:10.957 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-16T02:01:10.957 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-16T02:01:10.958 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-16T02:01:10.958 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-16T02:01:10.959 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-16T02:01:11.663 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:11.722 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:12.164 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:12.164 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:12.165 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:12.165 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:12.166 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:12.168 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:12.188 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:12.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.392+0000 7f08ca1e6640 1 -- 172.21.15.105:0/2934412571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08c4102570 msgr2=0x7f08c41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:12.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.392+0000 7f08ca1e6640 1 --2- 172.21.15.105:0/2934412571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08c4102570 0x7f08c41029b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f08ac009900 tx=0x7f08ac02a480 comp rx=0 tx=0).stop 2024-09-16T02:01:12.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.392+0000 7f08ca1e6640 1 -- 172.21.15.105:0/2934412571 shutdown_connections 2024-09-16T02:01:12.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.392+0000 7f08ca1e6640 1 --2- 172.21.15.105:0/2934412571 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08c4102ef0 0x7f08c410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:12.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.392+0000 7f08ca1e6640 1 --2- 172.21.15.105:0/2934412571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08c4102570 0x7f08c41029b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:12.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.392+0000 7f08ca1e6640 1 --2- 172.21.15.105:0/2934412571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08c4100e20 0x7f08c4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:12.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.392+0000 7f08ca1e6640 1 -- 172.21.15.105:0/2934412571 >> 172.21.15.105:0/2934412571 conn(0x7f08c40fc7d0 msgr2=0x7f08c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:12.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.392+0000 7f08ca1e6640 1 -- 172.21.15.105:0/2934412571 shutdown_connections 2024-09-16T02:01:12.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.393+0000 7f08ca1e6640 1 -- 172.21.15.105:0/2934412571 wait complete. 2024-09-16T02:01:12.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.393+0000 7f08ca1e6640 1 Processor -- start 2024-09-16T02:01:12.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.393+0000 7f08ca1e6640 1 -- start start 2024-09-16T02:01:12.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.393+0000 7f08ca1e6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08c4100e20 0x7f08c40fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:12.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.393+0000 7f08ca1e6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08c4102570 0x7f08c40fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:12.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.393+0000 7f08ca1e6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08c4102ef0 0x7f08c40fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:12.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.393+0000 7f08ca1e6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f08c4110e30 con 0x7f08c4102ef0 2024-09-16T02:01:12.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.393+0000 7f08ca1e6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f08c4110cb0 con 0x7f08c4100e20 2024-09-16T02:01:12.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.393+0000 7f08ca1e6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f08c4110fb0 con 0x7f08c4102570 2024-09-16T02:01:12.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.393+0000 7f08c3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08c4102ef0 0x7f08c40fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:12.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.393+0000 7f08c3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08c4102ef0 0x7f08c40fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56050/0 (socket says 172.21.15.105:56050) 2024-09-16T02:01:12.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.393+0000 7f08c3fff640 1 -- 172.21.15.105:0/3977037233 learned_addr learned my addr 172.21.15.105:0/3977037233 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:01:12.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.394+0000 7f08bbfff640 1 --2- 172.21.15.105:0/3977037233 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08c4102570 0x7f08c40fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:12.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.394+0000 7f08c37fe640 1 --2- 172.21.15.105:0/3977037233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08c4100e20 0x7f08c40fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:12.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.394+0000 7f08c3fff640 1 -- 172.21.15.105:0/3977037233 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08c4102570 msgr2=0x7f08c40fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:12.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.394+0000 7f08c3fff640 1 --2- 172.21.15.105:0/3977037233 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08c4102570 0x7f08c40fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:12.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.394+0000 7f08c3fff640 1 -- 172.21.15.105:0/3977037233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08c4100e20 msgr2=0x7f08c40fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:12.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.394+0000 7f08c3fff640 1 --2- 172.21.15.105:0/3977037233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08c4100e20 0x7f08c40fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:12.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.394+0000 7f08c3fff640 1 -- 172.21.15.105:0/3977037233 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c41a68e0 con 0x7f08c4102ef0 2024-09-16T02:01:12.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.394+0000 7f08c37fe640 1 --2- 172.21.15.105:0/3977037233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08c4100e20 0x7f08c40fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:01:12.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.394+0000 7f08c3fff640 1 --2- 172.21.15.105:0/3977037233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08c4102ef0 0x7f08c40fd700 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f08bc00eaa0 tx=0x7f08bc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:12.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.394+0000 7f08c17fa640 1 -- 172.21.15.105:0/3977037233 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08bc00cdb0 con 0x7f08c4102ef0 2024-09-16T02:01:12.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.394+0000 7f08c17fa640 1 -- 172.21.15.105:0/3977037233 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08bc005020 con 0x7f08c4102ef0 2024-09-16T02:01:12.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.394+0000 7f08ca1e6640 1 -- 172.21.15.105:0/3977037233 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08c41a6bd0 con 0x7f08c4102ef0 2024-09-16T02:01:12.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.394+0000 7f08c17fa640 1 -- 172.21.15.105:0/3977037233 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08bc0116d0 con 0x7f08c4102ef0 2024-09-16T02:01:12.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.394+0000 7f08ca1e6640 1 -- 172.21.15.105:0/3977037233 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08c41a7160 con 0x7f08c4102ef0 2024-09-16T02:01:12.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.395+0000 7f08ca1e6640 1 -- 172.21.15.105:0/3977037233 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f08c4062900 con 0x7f08c4102ef0 2024-09-16T02:01:12.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.396+0000 7f08c17fa640 1 -- 172.21.15.105:0/3977037233 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f08bc0051d0 con 0x7f08c4102ef0 2024-09-16T02:01:12.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.397+0000 7f08c17fa640 1 --2- 172.21.15.105:0/3977037233 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f08b00a3350 0x7f08b00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:12.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.397+0000 7f08c37fe640 1 --2- 172.21.15.105:0/3977037233 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f08b00a3350 0x7f08b00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:12.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.397+0000 7f08c17fa640 1 -- 172.21.15.105:0/3977037233 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f08bc0cfb60 con 0x7f08c4102ef0 2024-09-16T02:01:12.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.397+0000 7f08c17fa640 1 --2- 172.21.15.105:0/3977037233 >> [v2:172.21.15.190:6800/1159531918,v1:172.21.15.190:6801/1159531918] conn(0x7f08b00acba0 0x7f08b00aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:12.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.397+0000 7f08c17fa640 1 -- 172.21.15.105:0/3977037233 --> [v2:172.21.15.190:6800/1159531918,v1:172.21.15.190:6801/1159531918] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f08b00af670 con 0x7f08b00acba0 2024-09-16T02:01:12.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.397+0000 7f08c17fa640 1 -- 172.21.15.105:0/3977037233 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f08bc0d42a0 con 0x7f08c4102ef0 2024-09-16T02:01:12.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.397+0000 7f08c37fe640 1 --2- 172.21.15.105:0/3977037233 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f08b00a3350 0x7f08b00a5810 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009b40 tx=0x7f08b4005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:12.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.397+0000 7f08bbfff640 1 --2- 172.21.15.105:0/3977037233 >> [v2:172.21.15.190:6800/1159531918,v1:172.21.15.190:6801/1159531918] conn(0x7f08b00acba0 0x7f08b00aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:12.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.398+0000 7f08bbfff640 1 --2- 172.21.15.105:0/3977037233 >> [v2:172.21.15.190:6800/1159531918,v1:172.21.15.190:6801/1159531918] conn(0x7f08b00acba0 0x7f08b00aefc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:12.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.399+0000 7f08c17fa640 1 -- 172.21.15.105:0/3977037233 <== osd.2 v2:172.21.15.190:6800/1159531918 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f08b00af670 con 0x7f08b00acba0 2024-09-16T02:01:12.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.412+0000 7f08ca1e6640 1 -- 172.21.15.105:0/3977037233 --> [v2:172.21.15.190:6800/1159531918,v1:172.21.15.190:6801/1159531918] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f08c4062d30 con 0x7f08b00acba0 2024-09-16T02:01:12.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.412+0000 7f08c17fa640 1 -- 172.21.15.105:0/3977037233 <== osd.2 v2:172.21.15.190:6800/1159531918 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f08c4062d30 con 0x7f08b00acba0 2024-09-16T02:01:12.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.413+0000 7f08ca1e6640 1 -- 172.21.15.105:0/3977037233 >> [v2:172.21.15.190:6800/1159531918,v1:172.21.15.190:6801/1159531918] conn(0x7f08b00acba0 msgr2=0x7f08b00aefc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:12.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.413+0000 7f08ca1e6640 1 --2- 172.21.15.105:0/3977037233 >> [v2:172.21.15.190:6800/1159531918,v1:172.21.15.190:6801/1159531918] conn(0x7f08b00acba0 0x7f08b00aefc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:12.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.413+0000 7f08ca1e6640 1 -- 172.21.15.105:0/3977037233 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f08b00a3350 msgr2=0x7f08b00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:12.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.413+0000 7f08ca1e6640 1 --2- 172.21.15.105:0/3977037233 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f08b00a3350 0x7f08b00a5810 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009b40 tx=0x7f08b4005e50 comp rx=0 tx=0).stop 2024-09-16T02:01:12.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.413+0000 7f08ca1e6640 1 -- 172.21.15.105:0/3977037233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08c4102ef0 msgr2=0x7f08c40fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:12.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.413+0000 7f08ca1e6640 1 --2- 172.21.15.105:0/3977037233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08c4102ef0 0x7f08c40fd700 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f08bc00eaa0 tx=0x7f08bc00ef60 comp rx=0 tx=0).stop 2024-09-16T02:01:12.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.413+0000 7f08c3fff640 1 -- 172.21.15.105:0/3977037233 reap_dead start 2024-09-16T02:01:12.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.413+0000 7f08ca1e6640 1 -- 172.21.15.105:0/3977037233 shutdown_connections 2024-09-16T02:01:12.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.413+0000 7f08ca1e6640 1 -- 172.21.15.105:0/3977037233 >> 172.21.15.105:0/3977037233 conn(0x7f08c40fc7d0 msgr2=0x7f08c406a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:12.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.413+0000 7f08ca1e6640 1 -- 172.21.15.105:0/3977037233 shutdown_connections 2024-09-16T02:01:12.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:12.413+0000 7f08ca1e6640 1 -- 172.21.15.105:0/3977037233 wait complete. 2024-09-16T02:01:12.764 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:12 smithi105 ceph-mon[31197]: pgmap v295: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-16T02:01:12.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:12 smithi190 ceph-mon[34762]: pgmap v295: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-16T02:01:12.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:12 smithi137 ceph-mon[34099]: pgmap v295: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-16T02:01:13.480 INFO:teuthology.orchestra.run.smithi105.stdout:73014444099 2024-09-16T02:01:13.480 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-16T02:01:13.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.683+0000 7f87e2c0d640 1 -- 172.21.15.105:0/2532079193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87dc102570 msgr2=0x7f87dc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:13.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.683+0000 7f87e2c0d640 1 --2- 172.21.15.105:0/2532079193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87dc102570 0x7f87dc1029b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f87c4009900 tx=0x7f87c402a480 comp rx=0 tx=0).stop 2024-09-16T02:01:13.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.683+0000 7f87e2c0d640 1 -- 172.21.15.105:0/2532079193 shutdown_connections 2024-09-16T02:01:13.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.683+0000 7f87e2c0d640 1 --2- 172.21.15.105:0/2532079193 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87dc102ef0 0x7f87dc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:13.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.683+0000 7f87e2c0d640 1 --2- 172.21.15.105:0/2532079193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87dc102570 0x7f87dc1029b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:13.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.683+0000 7f87e2c0d640 1 --2- 172.21.15.105:0/2532079193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87dc100e20 0x7f87dc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:13.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.683+0000 7f87e2c0d640 1 -- 172.21.15.105:0/2532079193 >> 172.21.15.105:0/2532079193 conn(0x7f87dc0fc7d0 msgr2=0x7f87dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:13.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.683+0000 7f87e2c0d640 1 -- 172.21.15.105:0/2532079193 shutdown_connections 2024-09-16T02:01:13.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.683+0000 7f87e2c0d640 1 -- 172.21.15.105:0/2532079193 wait complete. 2024-09-16T02:01:13.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.684+0000 7f87e2c0d640 1 Processor -- start 2024-09-16T02:01:13.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.684+0000 7f87e2c0d640 1 -- start start 2024-09-16T02:01:13.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.684+0000 7f87e2c0d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87dc100e20 0x7f87dc1973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:13.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.684+0000 7f87e2c0d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87dc102570 0x7f87dc197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:13.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.684+0000 7f87e2c0d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87dc102ef0 0x7f87dc19bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:13.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.684+0000 7f87e2c0d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc110ed0 con 0x7f87dc102570 2024-09-16T02:01:13.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.684+0000 7f87e2c0d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc110d50 con 0x7f87dc102ef0 2024-09-16T02:01:13.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.684+0000 7f87e2c0d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc111050 con 0x7f87dc100e20 2024-09-16T02:01:13.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.684+0000 7f87dbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87dc102570 0x7f87dc197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:13.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.684+0000 7f87dbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87dc102570 0x7f87dc197910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56066/0 (socket says 172.21.15.105:56066) 2024-09-16T02:01:13.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.684+0000 7f87dbfff640 1 -- 172.21.15.105:0/2016259689 learned_addr learned my addr 172.21.15.105:0/2016259689 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:01:13.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.684+0000 7f87e09a9640 1 --2- 172.21.15.105:0/2016259689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87dc100e20 0x7f87dc1973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:13.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.684+0000 7f87e11aa640 1 --2- 172.21.15.105:0/2016259689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87dc102ef0 0x7f87dc19bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:13.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.684+0000 7f87dbfff640 1 -- 172.21.15.105:0/2016259689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87dc100e20 msgr2=0x7f87dc1973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:13.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.684+0000 7f87dbfff640 1 --2- 172.21.15.105:0/2016259689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87dc100e20 0x7f87dc1973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:13.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.684+0000 7f87dbfff640 1 -- 172.21.15.105:0/2016259689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87dc102ef0 msgr2=0x7f87dc19bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:13.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.684+0000 7f87dbfff640 1 --2- 172.21.15.105:0/2016259689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87dc102ef0 0x7f87dc19bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:13.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.684+0000 7f87dbfff640 1 -- 172.21.15.105:0/2016259689 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87dc19c380 con 0x7f87dc102570 2024-09-16T02:01:13.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.685+0000 7f87dbfff640 1 --2- 172.21.15.105:0/2016259689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87dc102570 0x7f87dc197910 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f87c4006010 tx=0x7f87c40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:13.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.685+0000 7f87d9ffb640 1 -- 172.21.15.105:0/2016259689 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c403f070 con 0x7f87dc102570 2024-09-16T02:01:13.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.685+0000 7f87d9ffb640 1 -- 172.21.15.105:0/2016259689 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87c4003aa0 con 0x7f87dc102570 2024-09-16T02:01:13.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.685+0000 7f87e2c0d640 1 -- 172.21.15.105:0/2016259689 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87dc19c610 con 0x7f87dc102570 2024-09-16T02:01:13.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.685+0000 7f87d9ffb640 1 -- 172.21.15.105:0/2016259689 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c4035030 con 0x7f87dc102570 2024-09-16T02:01:13.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.685+0000 7f87e2c0d640 1 -- 172.21.15.105:0/2016259689 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87dc069f70 con 0x7f87dc102570 2024-09-16T02:01:13.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.686+0000 7f87e2c0d640 1 -- 172.21.15.105:0/2016259689 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f87dc06a520 con 0x7f87dc102570 2024-09-16T02:01:13.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.687+0000 7f87d9ffb640 1 -- 172.21.15.105:0/2016259689 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f87c4042070 con 0x7f87dc102570 2024-09-16T02:01:13.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.688+0000 7f87d9ffb640 1 --2- 172.21.15.105:0/2016259689 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f87c80a32d0 0x7f87c80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:13.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.688+0000 7f87e09a9640 1 --2- 172.21.15.105:0/2016259689 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f87c80a32d0 0x7f87c80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:13.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.688+0000 7f87d9ffb640 1 -- 172.21.15.105:0/2016259689 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f87c40f08f0 con 0x7f87dc102570 2024-09-16T02:01:13.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.688+0000 7f87d9ffb640 1 --2- 172.21.15.105:0/2016259689 >> [v2:172.21.15.137:6808/3758109169,v1:172.21.15.137:6809/3758109169] conn(0x7f87c80acb20 0x7f87c80aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:13.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.688+0000 7f87d9ffb640 1 -- 172.21.15.105:0/2016259689 --> [v2:172.21.15.137:6808/3758109169,v1:172.21.15.137:6809/3758109169] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f87c4038e90 con 0x7f87c80acb20 2024-09-16T02:01:13.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.688+0000 7f87d9ffb640 1 -- 172.21.15.105:0/2016259689 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f87c40f0dd0 con 0x7f87dc102570 2024-09-16T02:01:13.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.688+0000 7f87e09a9640 1 --2- 172.21.15.105:0/2016259689 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f87c80a32d0 0x7f87c80a5790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f87d4009990 tx=0x7f87d4009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:13.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.688+0000 7f87e11aa640 1 --2- 172.21.15.105:0/2016259689 >> [v2:172.21.15.137:6808/3758109169,v1:172.21.15.137:6809/3758109169] conn(0x7f87c80acb20 0x7f87c80aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:13.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.689+0000 7f87e11aa640 1 --2- 172.21.15.105:0/2016259689 >> [v2:172.21.15.137:6808/3758109169,v1:172.21.15.137:6809/3758109169] conn(0x7f87c80acb20 0x7f87c80aef40 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:13.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.689+0000 7f87d9ffb640 1 -- 172.21.15.105:0/2016259689 <== osd.4 v2:172.21.15.137:6808/3758109169 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f87c4038e90 con 0x7f87c80acb20 2024-09-16T02:01:13.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.704+0000 7f87e2c0d640 1 -- 172.21.15.105:0/2016259689 --> [v2:172.21.15.137:6808/3758109169,v1:172.21.15.137:6809/3758109169] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f87dc06a9a0 con 0x7f87c80acb20 2024-09-16T02:01:13.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.704+0000 7f87d9ffb640 1 -- 172.21.15.105:0/2016259689 <== osd.4 v2:172.21.15.137:6808/3758109169 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f87dc06a9a0 con 0x7f87c80acb20 2024-09-16T02:01:13.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.704+0000 7f87e2c0d640 1 -- 172.21.15.105:0/2016259689 >> [v2:172.21.15.137:6808/3758109169,v1:172.21.15.137:6809/3758109169] conn(0x7f87c80acb20 msgr2=0x7f87c80aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:13.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.705+0000 7f87e2c0d640 1 --2- 172.21.15.105:0/2016259689 >> [v2:172.21.15.137:6808/3758109169,v1:172.21.15.137:6809/3758109169] conn(0x7f87c80acb20 0x7f87c80aef40 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:13.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.705+0000 7f87e2c0d640 1 -- 172.21.15.105:0/2016259689 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f87c80a32d0 msgr2=0x7f87c80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:13.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.705+0000 7f87e2c0d640 1 --2- 172.21.15.105:0/2016259689 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f87c80a32d0 0x7f87c80a5790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f87d4009990 tx=0x7f87d4009110 comp rx=0 tx=0).stop 2024-09-16T02:01:13.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.705+0000 7f87e2c0d640 1 -- 172.21.15.105:0/2016259689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87dc102570 msgr2=0x7f87dc197910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:13.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.705+0000 7f87e2c0d640 1 --2- 172.21.15.105:0/2016259689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87dc102570 0x7f87dc197910 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f87c4006010 tx=0x7f87c40042f0 comp rx=0 tx=0).stop 2024-09-16T02:01:13.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.705+0000 7f87e11aa640 1 -- 172.21.15.105:0/2016259689 reap_dead start 2024-09-16T02:01:13.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.705+0000 7f87e2c0d640 1 -- 172.21.15.105:0/2016259689 shutdown_connections 2024-09-16T02:01:13.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.705+0000 7f87e2c0d640 1 -- 172.21.15.105:0/2016259689 >> 172.21.15.105:0/2016259689 conn(0x7f87dc0fc7d0 msgr2=0x7f87dc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:13.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.705+0000 7f87e2c0d640 1 -- 172.21.15.105:0/2016259689 shutdown_connections 2024-09-16T02:01:13.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:13.705+0000 7f87e2c0d640 1 -- 172.21.15.105:0/2016259689 wait complete. 2024-09-16T02:01:13.719 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:14.766 INFO:teuthology.orchestra.run.smithi105.stdout:128849018936 2024-09-16T02:01:14.766 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-16T02:01:14.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:14 smithi190 ceph-mon[34762]: pgmap v296: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-16T02:01:14.842 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:14 smithi105 ceph-mon[31197]: pgmap v296: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-16T02:01:14.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:14 smithi137 ceph-mon[34099]: pgmap v296: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-16T02:01:15.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.656+0000 7f9d52a93640 1 -- 172.21.15.105:0/622951773 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d4c102570 msgr2=0x7f9d4c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:15.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.656+0000 7f9d52a93640 1 --2- 172.21.15.105:0/622951773 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d4c102570 0x7f9d4c1029b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9d40009900 tx=0x7f9d4002a480 comp rx=0 tx=0).stop 2024-09-16T02:01:15.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.656+0000 7f9d52a93640 1 -- 172.21.15.105:0/622951773 shutdown_connections 2024-09-16T02:01:15.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.656+0000 7f9d52a93640 1 --2- 172.21.15.105:0/622951773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d4c102ef0 0x7f9d4c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:15.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.656+0000 7f9d52a93640 1 --2- 172.21.15.105:0/622951773 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d4c102570 0x7f9d4c1029b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:15.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.656+0000 7f9d52a93640 1 --2- 172.21.15.105:0/622951773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d4c100e20 0x7f9d4c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:15.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.656+0000 7f9d52a93640 1 -- 172.21.15.105:0/622951773 >> 172.21.15.105:0/622951773 conn(0x7f9d4c0fc7d0 msgr2=0x7f9d4c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:15.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.656+0000 7f9d52a93640 1 -- 172.21.15.105:0/622951773 shutdown_connections 2024-09-16T02:01:15.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.656+0000 7f9d52a93640 1 -- 172.21.15.105:0/622951773 wait complete. 2024-09-16T02:01:15.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.656+0000 7f9d52a93640 1 Processor -- start 2024-09-16T02:01:15.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.656+0000 7f9d52a93640 1 -- start start 2024-09-16T02:01:15.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.657+0000 7f9d52a93640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d4c100e20 0x7f9d4c1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:15.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.657+0000 7f9d52a93640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d4c102570 0x7f9d4c198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:15.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.657+0000 7f9d52a93640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d4c102ef0 0x7f9d4c19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:15.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.657+0000 7f9d52a93640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4c111fd0 con 0x7f9d4c100e20 2024-09-16T02:01:15.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.657+0000 7f9d52a93640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4c111e50 con 0x7f9d4c102570 2024-09-16T02:01:15.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.657+0000 7f9d52a93640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4c112150 con 0x7f9d4c102ef0 2024-09-16T02:01:15.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.657+0000 7f9d5082f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d4c100e20 0x7f9d4c1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:15.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.657+0000 7f9d5082f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d4c100e20 0x7f9d4c1984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56088/0 (socket says 172.21.15.105:56088) 2024-09-16T02:01:15.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.657+0000 7f9d5082f640 1 -- 172.21.15.105:0/2993649933 learned_addr learned my addr 172.21.15.105:0/2993649933 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:01:15.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.657+0000 7f9d4bfff640 1 --2- 172.21.15.105:0/2993649933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d4c102570 0x7f9d4c198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:15.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.657+0000 7f9d5082f640 1 -- 172.21.15.105:0/2993649933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d4c102ef0 msgr2=0x7f9d4c19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:15.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.657+0000 7f9d5082f640 1 --2- 172.21.15.105:0/2993649933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d4c102ef0 0x7f9d4c19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:15.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.657+0000 7f9d5082f640 1 -- 172.21.15.105:0/2993649933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d4c102570 msgr2=0x7f9d4c198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:15.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.657+0000 7f9d5082f640 1 --2- 172.21.15.105:0/2993649933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d4c102570 0x7f9d4c198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:15.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.657+0000 7f9d5082f640 1 -- 172.21.15.105:0/2993649933 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d4c19d480 con 0x7f9d4c100e20 2024-09-16T02:01:15.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.657+0000 7f9d5082f640 1 --2- 172.21.15.105:0/2993649933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d4c100e20 0x7f9d4c1984d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c00ba50 tx=0x7f9d3c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:15.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.657+0000 7f9d49ffb640 1 -- 172.21.15.105:0/2993649933 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c019070 con 0x7f9d4c100e20 2024-09-16T02:01:15.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.657+0000 7f9d52a93640 1 -- 172.21.15.105:0/2993649933 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d4c19d770 con 0x7f9d4c100e20 2024-09-16T02:01:15.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.657+0000 7f9d49ffb640 1 -- 172.21.15.105:0/2993649933 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d3c00cbe0 con 0x7f9d4c100e20 2024-09-16T02:01:15.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.657+0000 7f9d49ffb640 1 -- 172.21.15.105:0/2993649933 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c0075d0 con 0x7f9d4c100e20 2024-09-16T02:01:15.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.658+0000 7f9d52a93640 1 -- 172.21.15.105:0/2993649933 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d4c1a5050 con 0x7f9d4c100e20 2024-09-16T02:01:15.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.658+0000 7f9d52a93640 1 -- 172.21.15.105:0/2993649933 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9d4c062980 con 0x7f9d4c100e20 2024-09-16T02:01:15.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.659+0000 7f9d49ffb640 1 -- 172.21.15.105:0/2993649933 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f9d3c01c070 con 0x7f9d4c100e20 2024-09-16T02:01:15.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.660+0000 7f9d49ffb640 1 --2- 172.21.15.105:0/2993649933 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f9d340a32d0 0x7f9d340a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:15.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.660+0000 7f9d4bfff640 1 --2- 172.21.15.105:0/2993649933 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f9d340a32d0 0x7f9d340a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:15.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.660+0000 7f9d49ffb640 1 -- 172.21.15.105:0/2993649933 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f9d3c0cee50 con 0x7f9d4c100e20 2024-09-16T02:01:15.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.660+0000 7f9d49ffb640 1 --2- 172.21.15.105:0/2993649933 >> [v2:172.21.15.137:6816/888678288,v1:172.21.15.137:6817/888678288] conn(0x7f9d340acb20 0x7f9d340aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:15.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.660+0000 7f9d49ffb640 1 -- 172.21.15.105:0/2993649933 --> [v2:172.21.15.137:6816/888678288,v1:172.21.15.137:6817/888678288] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9d3c021e50 con 0x7f9d340acb20 2024-09-16T02:01:15.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.660+0000 7f9d49ffb640 1 -- 172.21.15.105:0/2993649933 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f9d3c0d15a0 con 0x7f9d4c100e20 2024-09-16T02:01:15.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.661+0000 7f9d4bfff640 1 --2- 172.21.15.105:0/2993649933 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f9d340a32d0 0x7f9d340a5790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9d40006010 tx=0x7f9d40000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:15.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.661+0000 7f9d51030640 1 --2- 172.21.15.105:0/2993649933 >> [v2:172.21.15.137:6816/888678288,v1:172.21.15.137:6817/888678288] conn(0x7f9d340acb20 0x7f9d340aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:15.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.661+0000 7f9d51030640 1 --2- 172.21.15.105:0/2993649933 >> [v2:172.21.15.137:6816/888678288,v1:172.21.15.137:6817/888678288] conn(0x7f9d340acb20 0x7f9d340aef40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:15.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.662+0000 7f9d49ffb640 1 -- 172.21.15.105:0/2993649933 <== osd.7 v2:172.21.15.137:6816/888678288 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9d3c021e50 con 0x7f9d340acb20 2024-09-16T02:01:15.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.676+0000 7f9d52a93640 1 -- 172.21.15.105:0/2993649933 --> [v2:172.21.15.137:6816/888678288,v1:172.21.15.137:6817/888678288] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9d4c062d80 con 0x7f9d340acb20 2024-09-16T02:01:15.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.676+0000 7f9d49ffb640 1 -- 172.21.15.105:0/2993649933 <== osd.7 v2:172.21.15.137:6816/888678288 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9d4c062d80 con 0x7f9d340acb20 2024-09-16T02:01:15.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.676+0000 7f9d52a93640 1 -- 172.21.15.105:0/2993649933 >> [v2:172.21.15.137:6816/888678288,v1:172.21.15.137:6817/888678288] conn(0x7f9d340acb20 msgr2=0x7f9d340aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:15.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.676+0000 7f9d52a93640 1 --2- 172.21.15.105:0/2993649933 >> [v2:172.21.15.137:6816/888678288,v1:172.21.15.137:6817/888678288] conn(0x7f9d340acb20 0x7f9d340aef40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:15.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.677+0000 7f9d52a93640 1 -- 172.21.15.105:0/2993649933 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f9d340a32d0 msgr2=0x7f9d340a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:15.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.677+0000 7f9d52a93640 1 --2- 172.21.15.105:0/2993649933 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f9d340a32d0 0x7f9d340a5790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9d40006010 tx=0x7f9d40000bc0 comp rx=0 tx=0).stop 2024-09-16T02:01:15.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.677+0000 7f9d52a93640 1 -- 172.21.15.105:0/2993649933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d4c100e20 msgr2=0x7f9d4c1984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:15.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.677+0000 7f9d52a93640 1 --2- 172.21.15.105:0/2993649933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d4c100e20 0x7f9d4c1984d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c00ba50 tx=0x7f9d3c00bf10 comp rx=0 tx=0).stop 2024-09-16T02:01:15.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.677+0000 7f9d51030640 1 -- 172.21.15.105:0/2993649933 reap_dead start 2024-09-16T02:01:15.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.677+0000 7f9d52a93640 1 -- 172.21.15.105:0/2993649933 shutdown_connections 2024-09-16T02:01:15.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.677+0000 7f9d52a93640 1 -- 172.21.15.105:0/2993649933 >> 172.21.15.105:0/2993649933 conn(0x7f9d4c0fc7d0 msgr2=0x7f9d4c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:15.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.677+0000 7f9d52a93640 1 -- 172.21.15.105:0/2993649933 shutdown_connections 2024-09-16T02:01:15.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:15.677+0000 7f9d52a93640 1 -- 172.21.15.105:0/2993649933 wait complete. 2024-09-16T02:01:16.709 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:16 smithi105 ceph-mon[31197]: pgmap v297: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-16T02:01:16.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.708+0000 7f066d742640 1 -- 172.21.15.105:0/1297938016 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f066810c170 msgr2=0x7f066810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:16.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.708+0000 7f066d742640 1 --2- 172.21.15.105:0/1297938016 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f066810c170 0x7f066810e610 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0660009ca0 tx=0x7f066002a950 comp rx=0 tx=0).stop 2024-09-16T02:01:16.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.708+0000 7f066d742640 1 -- 172.21.15.105:0/1297938016 shutdown_connections 2024-09-16T02:01:16.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.708+0000 7f066d742640 1 --2- 172.21.15.105:0/1297938016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0668103660 0x7f066810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:16.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.708+0000 7f066d742640 1 --2- 172.21.15.105:0/1297938016 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f066810c170 0x7f066810e610 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:16.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.708+0000 7f066d742640 1 --2- 172.21.15.105:0/1297938016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0668102d40 0x7f0668103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:16.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.708+0000 7f066d742640 1 -- 172.21.15.105:0/1297938016 >> 172.21.15.105:0/1297938016 conn(0x7f06680fc7a0 msgr2=0x7f06680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:16.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.708+0000 7f066d742640 1 -- 172.21.15.105:0/1297938016 shutdown_connections 2024-09-16T02:01:16.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.708+0000 7f066d742640 1 -- 172.21.15.105:0/1297938016 wait complete. 2024-09-16T02:01:16.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.709+0000 7f066d742640 1 Processor -- start 2024-09-16T02:01:16.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.709+0000 7f066d742640 1 -- start start 2024-09-16T02:01:16.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.709+0000 7f066d742640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0668102d40 0x7f06681982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:16.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.709+0000 7f066d742640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0668103660 0x7f0668198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:16.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.709+0000 7f066d742640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f066810c170 0x7f066819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:16.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.709+0000 7f066d742640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0668111e20 con 0x7f0668102d40 2024-09-16T02:01:16.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.709+0000 7f066d742640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0668111ca0 con 0x7f066810c170 2024-09-16T02:01:16.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.709+0000 7f066d742640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0668111fa0 con 0x7f0668103660 2024-09-16T02:01:16.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.709+0000 7f0666ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0668102d40 0x7f06681982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:16.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.709+0000 7f0666ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0668102d40 0x7f06681982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56138/0 (socket says 172.21.15.105:56138) 2024-09-16T02:01:16.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.709+0000 7f0666ffd640 1 -- 172.21.15.105:0/553302495 learned_addr learned my addr 172.21.15.105:0/553302495 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:01:16.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.709+0000 7f065e7fc640 1 --2- 172.21.15.105:0/553302495 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0668103660 0x7f0668198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:16.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.709+0000 7f06677fe640 1 --2- 172.21.15.105:0/553302495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f066810c170 0x7f066819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:16.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.710+0000 7f0666ffd640 1 -- 172.21.15.105:0/553302495 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0668103660 msgr2=0x7f0668198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:16.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.710+0000 7f0666ffd640 1 --2- 172.21.15.105:0/553302495 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0668103660 0x7f0668198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:16.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.710+0000 7f0666ffd640 1 -- 172.21.15.105:0/553302495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f066810c170 msgr2=0x7f066819cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:16.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.710+0000 7f0666ffd640 1 --2- 172.21.15.105:0/553302495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f066810c170 0x7f066819cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:16.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.710+0000 7f0666ffd640 1 -- 172.21.15.105:0/553302495 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f066819d290 con 0x7f0668102d40 2024-09-16T02:01:16.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.710+0000 7f065e7fc640 1 --2- 172.21.15.105:0/553302495 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0668103660 0x7f0668198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:01:16.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.710+0000 7f06677fe640 1 --2- 172.21.15.105:0/553302495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f066810c170 0x7f066819cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:01:16.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.710+0000 7f0666ffd640 1 --2- 172.21.15.105:0/553302495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0668102d40 0x7f06681982e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f065800c890 tx=0x7f065800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:16.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.710+0000 7f0664ff9640 1 -- 172.21.15.105:0/553302495 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0658004220 con 0x7f0668102d40 2024-09-16T02:01:16.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.710+0000 7f066d742640 1 -- 172.21.15.105:0/553302495 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f066819d580 con 0x7f0668102d40 2024-09-16T02:01:16.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.710+0000 7f0664ff9640 1 -- 172.21.15.105:0/553302495 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06580043c0 con 0x7f0668102d40 2024-09-16T02:01:16.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.710+0000 7f0664ff9640 1 -- 172.21.15.105:0/553302495 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0658004a00 con 0x7f0668102d40 2024-09-16T02:01:16.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.710+0000 7f066d742640 1 -- 172.21.15.105:0/553302495 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06681a4e60 con 0x7f0668102d40 2024-09-16T02:01:16.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.711+0000 7f066d742640 1 -- 172.21.15.105:0/553302495 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0668062940 con 0x7f0668102d40 2024-09-16T02:01:16.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.712+0000 7f0664ff9640 1 -- 172.21.15.105:0/553302495 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f0658004ba0 con 0x7f0668102d40 2024-09-16T02:01:16.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.713+0000 7f0664ff9640 1 --2- 172.21.15.105:0/553302495 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f06540a32d0 0x7f06540a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:16.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.713+0000 7f065e7fc640 1 --2- 172.21.15.105:0/553302495 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f06540a32d0 0x7f06540a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:16.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.713+0000 7f0664ff9640 1 -- 172.21.15.105:0/553302495 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f06580cfad0 con 0x7f0668102d40 2024-09-16T02:01:16.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.713+0000 7f0664ff9640 1 --2- 172.21.15.105:0/553302495 >> [v2:172.21.15.190:6816/2132638341,v1:172.21.15.190:6817/2132638341] conn(0x7f06540acb20 0x7f06540aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:16.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.713+0000 7f0664ff9640 1 -- 172.21.15.105:0/553302495 --> [v2:172.21.15.190:6816/2132638341,v1:172.21.15.190:6817/2132638341] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0658023ef0 con 0x7f06540acb20 2024-09-16T02:01:16.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.713+0000 7f0664ff9640 1 -- 172.21.15.105:0/553302495 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f06580cffb0 con 0x7f0668102d40 2024-09-16T02:01:16.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.713+0000 7f065e7fc640 1 --2- 172.21.15.105:0/553302495 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f06540a32d0 0x7f06540a5790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0650005fd0 tx=0x7f0650005f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:16.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.715+0000 7f06677fe640 1 --2- 172.21.15.105:0/553302495 >> [v2:172.21.15.190:6816/2132638341,v1:172.21.15.190:6817/2132638341] conn(0x7f06540acb20 0x7f06540aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:16.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.716+0000 7f06677fe640 1 --2- 172.21.15.105:0/553302495 >> [v2:172.21.15.190:6816/2132638341,v1:172.21.15.190:6817/2132638341] conn(0x7f06540acb20 0x7f06540aef40 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:16.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.717+0000 7f0664ff9640 1 -- 172.21.15.105:0/553302495 <== osd.8 v2:172.21.15.190:6816/2132638341 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0658023ef0 con 0x7f06540acb20 2024-09-16T02:01:16.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.730+0000 7f066d742640 1 -- 172.21.15.105:0/553302495 --> [v2:172.21.15.190:6816/2132638341,v1:172.21.15.190:6817/2132638341] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0668062dc0 con 0x7f06540acb20 2024-09-16T02:01:16.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.731+0000 7f0664ff9640 1 -- 172.21.15.105:0/553302495 <== osd.8 v2:172.21.15.190:6816/2132638341 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0668062dc0 con 0x7f06540acb20 2024-09-16T02:01:16.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.731+0000 7f066d742640 1 -- 172.21.15.105:0/553302495 >> [v2:172.21.15.190:6816/2132638341,v1:172.21.15.190:6817/2132638341] conn(0x7f06540acb20 msgr2=0x7f06540aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:16.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.731+0000 7f066d742640 1 --2- 172.21.15.105:0/553302495 >> [v2:172.21.15.190:6816/2132638341,v1:172.21.15.190:6817/2132638341] conn(0x7f06540acb20 0x7f06540aef40 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:16.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.731+0000 7f066d742640 1 -- 172.21.15.105:0/553302495 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f06540a32d0 msgr2=0x7f06540a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:16.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.731+0000 7f066d742640 1 --2- 172.21.15.105:0/553302495 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f06540a32d0 0x7f06540a5790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0650005fd0 tx=0x7f0650005f60 comp rx=0 tx=0).stop 2024-09-16T02:01:16.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.731+0000 7f066d742640 1 -- 172.21.15.105:0/553302495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0668102d40 msgr2=0x7f06681982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:16.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.731+0000 7f066d742640 1 --2- 172.21.15.105:0/553302495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0668102d40 0x7f06681982e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f065800c890 tx=0x7f065800cd50 comp rx=0 tx=0).stop 2024-09-16T02:01:16.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.731+0000 7f06677fe640 1 -- 172.21.15.105:0/553302495 reap_dead start 2024-09-16T02:01:16.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.731+0000 7f066d742640 1 -- 172.21.15.105:0/553302495 shutdown_connections 2024-09-16T02:01:16.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.731+0000 7f066d742640 1 -- 172.21.15.105:0/553302495 >> 172.21.15.105:0/553302495 conn(0x7f06680fc7a0 msgr2=0x7f06680fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:16.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.731+0000 7f066d742640 1 -- 172.21.15.105:0/553302495 shutdown_connections 2024-09-16T02:01:16.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.731+0000 7f066d742640 1 -- 172.21.15.105:0/553302495 wait complete. 2024-09-16T02:01:16.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:16 smithi190 ceph-mon[34762]: pgmap v297: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-16T02:01:16.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:16 smithi137 ceph-mon[34099]: pgmap v297: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-16T02:01:17.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.184+0000 7f35bfa98640 1 -- 172.21.15.105:0/2427435540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35b8101420 msgr2=0x7f35b810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:17.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.184+0000 7f35bfa98640 1 --2- 172.21.15.105:0/2427435540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35b8101420 0x7f35b810d950 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f35b0009cd0 tx=0x7f35b002a8e0 comp rx=0 tx=0).stop 2024-09-16T02:01:17.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.184+0000 7f35bfa98640 1 -- 172.21.15.105:0/2427435540 shutdown_connections 2024-09-16T02:01:17.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.184+0000 7f35bfa98640 1 --2- 172.21.15.105:0/2427435540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35b8101420 0x7f35b810d950 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:17.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.184+0000 7f35bfa98640 1 --2- 172.21.15.105:0/2427435540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35b8100a80 0x7f35b8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:17.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.184+0000 7f35bfa98640 1 --2- 172.21.15.105:0/2427435540 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35b8106a80 0x7f35b8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:17.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.184+0000 7f35bfa98640 1 -- 172.21.15.105:0/2427435540 >> 172.21.15.105:0/2427435540 conn(0x7f35b80fc7a0 msgr2=0x7f35b80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:17.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.184+0000 7f35bfa98640 1 -- 172.21.15.105:0/2427435540 shutdown_connections 2024-09-16T02:01:17.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.184+0000 7f35bfa98640 1 -- 172.21.15.105:0/2427435540 wait complete. 2024-09-16T02:01:17.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.184+0000 7f35bfa98640 1 Processor -- start 2024-09-16T02:01:17.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.184+0000 7f35bfa98640 1 -- start start 2024-09-16T02:01:17.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.185+0000 7f35bfa98640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35b8100a80 0x7f35b8199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:17.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.185+0000 7f35bfa98640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35b8101420 0x7f35b81996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:17.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.185+0000 7f35bfa98640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35b8106a80 0x7f35b81a4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:17.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.185+0000 7f35bfa98640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f35b8110d50 con 0x7f35b8100a80 2024-09-16T02:01:17.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.185+0000 7f35bfa98640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f35b8110bd0 con 0x7f35b8106a80 2024-09-16T02:01:17.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.185+0000 7f35bfa98640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f35b8110ed0 con 0x7f35b8101420 2024-09-16T02:01:17.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.185+0000 7f35bd834640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35b8100a80 0x7f35b8199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:17.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.185+0000 7f35bd834640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35b8100a80 0x7f35b8199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56160/0 (socket says 172.21.15.105:56160) 2024-09-16T02:01:17.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.185+0000 7f35bd834640 1 -- 172.21.15.105:0/2238877333 learned_addr learned my addr 172.21.15.105:0/2238877333 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:01:17.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.185+0000 7f35bd834640 1 -- 172.21.15.105:0/2238877333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35b8101420 msgr2=0x7f35b81996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:17.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.185+0000 7f35bd834640 1 --2- 172.21.15.105:0/2238877333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35b8101420 0x7f35b81996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:17.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.185+0000 7f35bd834640 1 -- 172.21.15.105:0/2238877333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35b8106a80 msgr2=0x7f35b81a4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:17.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.185+0000 7f35bd834640 1 --2- 172.21.15.105:0/2238877333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35b8106a80 0x7f35b81a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:17.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.185+0000 7f35bd834640 1 -- 172.21.15.105:0/2238877333 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35b81a4740 con 0x7f35b8100a80 2024-09-16T02:01:17.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.185+0000 7f35bd834640 1 --2- 172.21.15.105:0/2238877333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35b8100a80 0x7f35b8199180 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f35b400b4a0 tx=0x7f35b400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:17.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.185+0000 7f35aeffd640 1 -- 172.21.15.105:0/2238877333 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b4014070 con 0x7f35b8100a80 2024-09-16T02:01:17.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.185+0000 7f35aeffd640 1 -- 172.21.15.105:0/2238877333 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35b4010ac0 con 0x7f35b8100a80 2024-09-16T02:01:17.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.185+0000 7f35bfa98640 1 -- 172.21.15.105:0/2238877333 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35b81a4910 con 0x7f35b8100a80 2024-09-16T02:01:17.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.185+0000 7f35aeffd640 1 -- 172.21.15.105:0/2238877333 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b4017c30 con 0x7f35b8100a80 2024-09-16T02:01:17.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.185+0000 7f35bfa98640 1 -- 172.21.15.105:0/2238877333 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35b81a4e20 con 0x7f35b8100a80 2024-09-16T02:01:17.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.186+0000 7f35bfa98640 1 -- 172.21.15.105:0/2238877333 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f35b80fd900 con 0x7f35b8100a80 2024-09-16T02:01:17.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.188+0000 7f35aeffd640 1 -- 172.21.15.105:0/2238877333 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f35b4017070 con 0x7f35b8100a80 2024-09-16T02:01:17.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.188+0000 7f35aeffd640 1 --2- 172.21.15.105:0/2238877333 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f35a40a3350 0x7f35a40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:17.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.188+0000 7f35bd033640 1 --2- 172.21.15.105:0/2238877333 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f35a40a3350 0x7f35a40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:17.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.188+0000 7f35aeffd640 1 -- 172.21.15.105:0/2238877333 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f35b40cefe0 con 0x7f35b8100a80 2024-09-16T02:01:17.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.189+0000 7f35aeffd640 1 --2- 172.21.15.105:0/2238877333 >> [v2:172.21.15.137:6800/34753956,v1:172.21.15.137:6801/34753956] conn(0x7f35a40acba0 0x7f35a40aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:17.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.189+0000 7f35aeffd640 1 -- 172.21.15.105:0/2238877333 --> [v2:172.21.15.137:6800/34753956,v1:172.21.15.137:6801/34753956] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f35b400dec0 con 0x7f35a40acba0 2024-09-16T02:01:17.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.189+0000 7f35aeffd640 1 -- 172.21.15.105:0/2238877333 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f35b40cf4c0 con 0x7f35b8100a80 2024-09-16T02:01:17.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.189+0000 7f35bd033640 1 --2- 172.21.15.105:0/2238877333 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f35a40a3350 0x7f35a40a5810 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f35b806b2e0 tx=0x7f35a8005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:17.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.189+0000 7f35be035640 1 --2- 172.21.15.105:0/2238877333 >> [v2:172.21.15.137:6800/34753956,v1:172.21.15.137:6801/34753956] conn(0x7f35a40acba0 0x7f35a40aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:17.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.190+0000 7f35be035640 1 --2- 172.21.15.105:0/2238877333 >> [v2:172.21.15.137:6800/34753956,v1:172.21.15.137:6801/34753956] conn(0x7f35a40acba0 0x7f35a40aefc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:17.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.191+0000 7f35aeffd640 1 -- 172.21.15.105:0/2238877333 <== osd.1 v2:172.21.15.137:6800/34753956 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f35b400dec0 con 0x7f35a40acba0 2024-09-16T02:01:17.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.204+0000 7f35bfa98640 1 -- 172.21.15.105:0/2238877333 --> [v2:172.21.15.137:6800/34753956,v1:172.21.15.137:6801/34753956] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f35b81021c0 con 0x7f35a40acba0 2024-09-16T02:01:17.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.205+0000 7f35aeffd640 1 -- 172.21.15.105:0/2238877333 <== osd.1 v2:172.21.15.137:6800/34753956 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f35b81021c0 con 0x7f35a40acba0 2024-09-16T02:01:17.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.205+0000 7f35bfa98640 1 -- 172.21.15.105:0/2238877333 >> [v2:172.21.15.137:6800/34753956,v1:172.21.15.137:6801/34753956] conn(0x7f35a40acba0 msgr2=0x7f35a40aefc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:17.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.205+0000 7f35bfa98640 1 --2- 172.21.15.105:0/2238877333 >> [v2:172.21.15.137:6800/34753956,v1:172.21.15.137:6801/34753956] conn(0x7f35a40acba0 0x7f35a40aefc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:17.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.205+0000 7f35bfa98640 1 -- 172.21.15.105:0/2238877333 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f35a40a3350 msgr2=0x7f35a40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:17.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.205+0000 7f35bfa98640 1 --2- 172.21.15.105:0/2238877333 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f35a40a3350 0x7f35a40a5810 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f35b806b2e0 tx=0x7f35a8005c70 comp rx=0 tx=0).stop 2024-09-16T02:01:17.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.205+0000 7f35bfa98640 1 -- 172.21.15.105:0/2238877333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35b8100a80 msgr2=0x7f35b8199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:17.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.205+0000 7f35bfa98640 1 --2- 172.21.15.105:0/2238877333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35b8100a80 0x7f35b8199180 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f35b400b4a0 tx=0x7f35b400b960 comp rx=0 tx=0).stop 2024-09-16T02:01:17.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.205+0000 7f35be035640 1 -- 172.21.15.105:0/2238877333 reap_dead start 2024-09-16T02:01:17.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.206+0000 7f35bfa98640 1 -- 172.21.15.105:0/2238877333 shutdown_connections 2024-09-16T02:01:17.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.206+0000 7f35bfa98640 1 -- 172.21.15.105:0/2238877333 >> 172.21.15.105:0/2238877333 conn(0x7f35b80fc7a0 msgr2=0x7f35b80fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:17.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.206+0000 7f35bfa98640 1 -- 172.21.15.105:0/2238877333 shutdown_connections 2024-09-16T02:01:17.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:17.206+0000 7f35bfa98640 1 -- 172.21.15.105:0/2238877333 wait complete. 2024-09-16T02:01:18.019 INFO:teuthology.orchestra.run.smithi105.stdout:55834574921 2024-09-16T02:01:18.019 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-16T02:01:18.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.606+0000 7f6d739a6640 1 -- 172.21.15.105:0/507440447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d6c1027a0 msgr2=0x7f6d6c1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:18.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.606+0000 7f6d739a6640 1 --2- 172.21.15.105:0/507440447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d6c1027a0 0x7f6d6c1067f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6d68009900 tx=0x7f6d6802a410 comp rx=0 tx=0).stop 2024-09-16T02:01:18.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.607+0000 7f6d739a6640 1 -- 172.21.15.105:0/507440447 shutdown_connections 2024-09-16T02:01:18.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.607+0000 7f6d739a6640 1 --2- 172.21.15.105:0/507440447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d6c106f20 0x7f6d6c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:18.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.607+0000 7f6d739a6640 1 --2- 172.21.15.105:0/507440447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d6c1027a0 0x7f6d6c1067f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:18.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.607+0000 7f6d739a6640 1 --2- 172.21.15.105:0/507440447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d6c101df0 0x7f6d6c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:18.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.607+0000 7f6d739a6640 1 -- 172.21.15.105:0/507440447 >> 172.21.15.105:0/507440447 conn(0x7f6d6c0fd8f0 msgr2=0x7f6d6c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:18.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.607+0000 7f6d739a6640 1 -- 172.21.15.105:0/507440447 shutdown_connections 2024-09-16T02:01:18.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.607+0000 7f6d739a6640 1 -- 172.21.15.105:0/507440447 wait complete. 2024-09-16T02:01:18.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.607+0000 7f6d739a6640 1 Processor -- start 2024-09-16T02:01:18.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.607+0000 7f6d739a6640 1 -- start start 2024-09-16T02:01:18.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.607+0000 7f6d739a6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d6c101df0 0x7f6d6c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:18.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.607+0000 7f6d739a6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d6c1027a0 0x7f6d6c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:18.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.608+0000 7f6d739a6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d6c106f20 0x7f6d6c19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:18.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.608+0000 7f6d739a6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6c111e50 con 0x7f6d6c106f20 2024-09-16T02:01:18.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.608+0000 7f6d739a6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6c111cd0 con 0x7f6d6c1027a0 2024-09-16T02:01:18.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.608+0000 7f6d739a6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6c111fd0 con 0x7f6d6c101df0 2024-09-16T02:01:18.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.608+0000 7f6d71f43640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d6c106f20 0x7f6d6c19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:18.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.608+0000 7f6d71f43640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d6c106f20 0x7f6d6c19cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56118/0 (socket says 172.21.15.105:56118) 2024-09-16T02:01:18.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.608+0000 7f6d71f43640 1 -- 172.21.15.105:0/4067706545 learned_addr learned my addr 172.21.15.105:0/4067706545 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:01:18.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.608+0000 7f6d71f43640 1 -- 172.21.15.105:0/4067706545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d6c101df0 msgr2=0x7f6d6c1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:18.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.608+0000 7f6d71742640 1 --2- 172.21.15.105:0/4067706545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d6c101df0 0x7f6d6c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:18.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.608+0000 7f6d71f43640 1 --2- 172.21.15.105:0/4067706545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d6c101df0 0x7f6d6c1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:18.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.608+0000 7f6d70f41640 1 --2- 172.21.15.105:0/4067706545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d6c1027a0 0x7f6d6c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:18.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.608+0000 7f6d71f43640 1 -- 172.21.15.105:0/4067706545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d6c1027a0 msgr2=0x7f6d6c1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:18.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.608+0000 7f6d71f43640 1 --2- 172.21.15.105:0/4067706545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d6c1027a0 0x7f6d6c1988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:18.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.608+0000 7f6d71f43640 1 -- 172.21.15.105:0/4067706545 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d6c19d350 con 0x7f6d6c106f20 2024-09-16T02:01:18.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.608+0000 7f6d71742640 1 --2- 172.21.15.105:0/4067706545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d6c101df0 0x7f6d6c1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:01:18.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.608+0000 7f6d71f43640 1 --2- 172.21.15.105:0/4067706545 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d6c106f20 0x7f6d6c19cc70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6d6400cce0 tx=0x7f6d6400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:18.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.608+0000 7f6d627fc640 1 -- 172.21.15.105:0/4067706545 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6400f450 con 0x7f6d6c106f20 2024-09-16T02:01:18.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.608+0000 7f6d739a6640 1 -- 172.21.15.105:0/4067706545 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d6c19d640 con 0x7f6d6c106f20 2024-09-16T02:01:18.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.608+0000 7f6d627fc640 1 -- 172.21.15.105:0/4067706545 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d6400faa0 con 0x7f6d6c106f20 2024-09-16T02:01:18.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.608+0000 7f6d627fc640 1 -- 172.21.15.105:0/4067706545 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d64018b70 con 0x7f6d6c106f20 2024-09-16T02:01:18.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.608+0000 7f6d739a6640 1 -- 172.21.15.105:0/4067706545 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d6c1a4f20 con 0x7f6d6c106f20 2024-09-16T02:01:18.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.609+0000 7f6d739a6640 1 -- 172.21.15.105:0/4067706545 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6d6c062980 con 0x7f6d6c106f20 2024-09-16T02:01:18.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.610+0000 7f6d627fc640 1 -- 172.21.15.105:0/4067706545 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f6d64018d10 con 0x7f6d6c106f20 2024-09-16T02:01:18.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.611+0000 7f6d627fc640 1 --2- 172.21.15.105:0/4067706545 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f6d580a32d0 0x7f6d580a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:18.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.611+0000 7f6d71742640 1 --2- 172.21.15.105:0/4067706545 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f6d580a32d0 0x7f6d580a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:18.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.611+0000 7f6d627fc640 1 -- 172.21.15.105:0/4067706545 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f6d640d00a0 con 0x7f6d6c106f20 2024-09-16T02:01:18.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.611+0000 7f6d627fc640 1 --2- 172.21.15.105:0/4067706545 >> [v2:172.21.15.137:6824/318562234,v1:172.21.15.137:6825/318562234] conn(0x7f6d580acb20 0x7f6d580aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:18.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.611+0000 7f6d627fc640 1 -- 172.21.15.105:0/4067706545 --> [v2:172.21.15.137:6824/318562234,v1:172.21.15.137:6825/318562234] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6d6401ee50 con 0x7f6d580acb20 2024-09-16T02:01:18.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.611+0000 7f6d627fc640 1 -- 172.21.15.105:0/4067706545 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f6d640d0580 con 0x7f6d6c106f20 2024-09-16T02:01:18.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.612+0000 7f6d71742640 1 --2- 172.21.15.105:0/4067706545 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f6d580a32d0 0x7f6d580a5790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c006fd0 tx=0x7f6d5c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:18.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.612+0000 7f6d70f41640 1 --2- 172.21.15.105:0/4067706545 >> [v2:172.21.15.137:6824/318562234,v1:172.21.15.137:6825/318562234] conn(0x7f6d580acb20 0x7f6d580aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:18.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.612+0000 7f6d70f41640 1 --2- 172.21.15.105:0/4067706545 >> [v2:172.21.15.137:6824/318562234,v1:172.21.15.137:6825/318562234] conn(0x7f6d580acb20 0x7f6d580aef40 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:18.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.613+0000 7f6d627fc640 1 -- 172.21.15.105:0/4067706545 <== osd.10 v2:172.21.15.137:6824/318562234 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6d6401ee50 con 0x7f6d580acb20 2024-09-16T02:01:18.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.627+0000 7f6d739a6640 1 -- 172.21.15.105:0/4067706545 --> [v2:172.21.15.137:6824/318562234,v1:172.21.15.137:6825/318562234] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6d6c062e00 con 0x7f6d580acb20 2024-09-16T02:01:18.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.628+0000 7f6d627fc640 1 -- 172.21.15.105:0/4067706545 <== osd.10 v2:172.21.15.137:6824/318562234 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6d6c062e00 con 0x7f6d580acb20 2024-09-16T02:01:18.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.628+0000 7f6d739a6640 1 -- 172.21.15.105:0/4067706545 >> [v2:172.21.15.137:6824/318562234,v1:172.21.15.137:6825/318562234] conn(0x7f6d580acb20 msgr2=0x7f6d580aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:18.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.628+0000 7f6d739a6640 1 --2- 172.21.15.105:0/4067706545 >> [v2:172.21.15.137:6824/318562234,v1:172.21.15.137:6825/318562234] conn(0x7f6d580acb20 0x7f6d580aef40 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:18.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.628+0000 7f6d739a6640 1 -- 172.21.15.105:0/4067706545 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f6d580a32d0 msgr2=0x7f6d580a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:18.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.628+0000 7f6d739a6640 1 --2- 172.21.15.105:0/4067706545 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f6d580a32d0 0x7f6d580a5790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c006fd0 tx=0x7f6d5c008040 comp rx=0 tx=0).stop 2024-09-16T02:01:18.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.628+0000 7f6d739a6640 1 -- 172.21.15.105:0/4067706545 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d6c106f20 msgr2=0x7f6d6c19cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:18.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.628+0000 7f6d739a6640 1 --2- 172.21.15.105:0/4067706545 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d6c106f20 0x7f6d6c19cc70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6d6400cce0 tx=0x7f6d6400a620 comp rx=0 tx=0).stop 2024-09-16T02:01:18.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.628+0000 7f6d71f43640 1 -- 172.21.15.105:0/4067706545 reap_dead start 2024-09-16T02:01:18.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.628+0000 7f6d739a6640 1 -- 172.21.15.105:0/4067706545 shutdown_connections 2024-09-16T02:01:18.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.628+0000 7f6d739a6640 1 -- 172.21.15.105:0/4067706545 >> 172.21.15.105:0/4067706545 conn(0x7f6d6c0fd8f0 msgr2=0x7f6d6c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:18.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.628+0000 7f6d739a6640 1 -- 172.21.15.105:0/4067706545 shutdown_connections 2024-09-16T02:01:18.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:16.628+0000 7f6d739a6640 1 -- 172.21.15.105:0/4067706545 wait complete. 2024-09-16T02:01:18.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:18 smithi190 ceph-mon[34762]: pgmap v298: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-16T02:01:18.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:18 smithi137 ceph-mon[34099]: pgmap v298: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-16T02:01:19.002 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:18 smithi105 ceph-mon[31197]: pgmap v298: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-16T02:01:19.028 INFO:teuthology.orchestra.run.smithi105.stdout:283467841563 2024-09-16T02:01:19.028 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-16T02:01:19.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.211+0000 7f804b084640 1 -- 172.21.15.105:0/145717187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8044101290 msgr2=0x7f8044107a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:19.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.211+0000 7f804b084640 1 --2- 172.21.15.105:0/145717187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8044101290 0x7f8044107a50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8030009930 tx=0x7f803002a420 comp rx=0 tx=0).stop 2024-09-16T02:01:19.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.211+0000 7f804b084640 1 -- 172.21.15.105:0/145717187 shutdown_connections 2024-09-16T02:01:19.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.211+0000 7f804b084640 1 --2- 172.21.15.105:0/145717187 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8044108160 0x7f804410eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:19.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.211+0000 7f804b084640 1 --2- 172.21.15.105:0/145717187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8044101290 0x7f8044107a50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:19.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.211+0000 7f804b084640 1 --2- 172.21.15.105:0/145717187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80441008e0 0x7f8044100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:19.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.212+0000 7f804b084640 1 -- 172.21.15.105:0/145717187 >> 172.21.15.105:0/145717187 conn(0x7f8044077210 msgr2=0x7f8044077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:19.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.212+0000 7f804b084640 1 -- 172.21.15.105:0/145717187 shutdown_connections 2024-09-16T02:01:19.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.212+0000 7f804b084640 1 -- 172.21.15.105:0/145717187 wait complete. 2024-09-16T02:01:19.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.212+0000 7f804b084640 1 Processor -- start 2024-09-16T02:01:19.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.212+0000 7f804b084640 1 -- start start 2024-09-16T02:01:19.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.213+0000 7f804b084640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80441008e0 0x7f80441984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:19.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.213+0000 7f804b084640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8044101290 0x7f80441989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:19.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.213+0000 7f804b084640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8044108160 0x7f804419cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:19.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.213+0000 7f804b084640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8044111ea0 con 0x7f8044101290 2024-09-16T02:01:19.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.213+0000 7f804b084640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8044111d20 con 0x7f80441008e0 2024-09-16T02:01:19.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.213+0000 7f804b084640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8044112020 con 0x7f8044108160 2024-09-16T02:01:19.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.213+0000 7f8043fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8044101290 0x7f80441989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:19.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.213+0000 7f8043fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8044101290 0x7f80441989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42972/0 (socket says 172.21.15.105:42972) 2024-09-16T02:01:19.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.213+0000 7f8043fff640 1 -- 172.21.15.105:0/195903970 learned_addr learned my addr 172.21.15.105:0/195903970 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:01:19.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.213+0000 7f8048e20640 1 --2- 172.21.15.105:0/195903970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80441008e0 0x7f80441984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:19.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.213+0000 7f8049621640 1 --2- 172.21.15.105:0/195903970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8044108160 0x7f804419cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:19.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.213+0000 7f8043fff640 1 -- 172.21.15.105:0/195903970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8044108160 msgr2=0x7f804419cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:19.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.213+0000 7f8043fff640 1 --2- 172.21.15.105:0/195903970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8044108160 0x7f804419cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:19.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.213+0000 7f8043fff640 1 -- 172.21.15.105:0/195903970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80441008e0 msgr2=0x7f80441984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:19.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.213+0000 7f8043fff640 1 --2- 172.21.15.105:0/195903970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80441008e0 0x7f80441984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:19.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.213+0000 7f8043fff640 1 -- 172.21.15.105:0/195903970 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f804419d450 con 0x7f8044101290 2024-09-16T02:01:19.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.214+0000 7f8043fff640 1 --2- 172.21.15.105:0/195903970 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8044101290 0x7f80441989e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8030031440 tx=0x7f8030004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:19.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.214+0000 7f8041ffb640 1 -- 172.21.15.105:0/195903970 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803003f030 con 0x7f8044101290 2024-09-16T02:01:19.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.214+0000 7f804b084640 1 -- 172.21.15.105:0/195903970 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f804419d6e0 con 0x7f8044101290 2024-09-16T02:01:19.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.214+0000 7f8041ffb640 1 -- 172.21.15.105:0/195903970 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8030003aa0 con 0x7f8044101290 2024-09-16T02:01:19.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.214+0000 7f8041ffb640 1 -- 172.21.15.105:0/195903970 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8030035070 con 0x7f8044101290 2024-09-16T02:01:19.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.214+0000 7f804b084640 1 -- 172.21.15.105:0/195903970 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8044101790 con 0x7f8044101290 2024-09-16T02:01:19.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.215+0000 7f804b084640 1 -- 172.21.15.105:0/195903970 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8044062980 con 0x7f8044101290 2024-09-16T02:01:19.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.216+0000 7f8041ffb640 1 -- 172.21.15.105:0/195903970 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f8030042030 con 0x7f8044101290 2024-09-16T02:01:19.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.217+0000 7f8041ffb640 1 --2- 172.21.15.105:0/195903970 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f80340a3350 0x7f80340a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:19.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.217+0000 7f8048e20640 1 --2- 172.21.15.105:0/195903970 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f80340a3350 0x7f80340a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:19.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.217+0000 7f8041ffb640 1 -- 172.21.15.105:0/195903970 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f80300f0590 con 0x7f8044101290 2024-09-16T02:01:19.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.217+0000 7f8041ffb640 1 --2- 172.21.15.105:0/195903970 >> [v2:172.21.15.105:6802/1226660290,v1:172.21.15.105:6803/1226660290] conn(0x7f80340aca70 0x7f80340aee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:19.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.217+0000 7f8041ffb640 1 -- 172.21.15.105:0/195903970 --> [v2:172.21.15.105:6802/1226660290,v1:172.21.15.105:6803/1226660290] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f80340af540 con 0x7f80340aca70 2024-09-16T02:01:19.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.217+0000 7f8041ffb640 1 -- 172.21.15.105:0/195903970 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f80300f0a70 con 0x7f8044101290 2024-09-16T02:01:19.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.217+0000 7f8048e20640 1 --2- 172.21.15.105:0/195903970 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f80340a3350 0x7f80340a5810 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f803800bbe0 tx=0x7f803800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:19.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.217+0000 7f8049621640 1 --2- 172.21.15.105:0/195903970 >> [v2:172.21.15.105:6802/1226660290,v1:172.21.15.105:6803/1226660290] conn(0x7f80340aca70 0x7f80340aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:19.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.218+0000 7f8049621640 1 --2- 172.21.15.105:0/195903970 >> [v2:172.21.15.105:6802/1226660290,v1:172.21.15.105:6803/1226660290] conn(0x7f80340aca70 0x7f80340aee90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:19.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.218+0000 7f8041ffb640 1 -- 172.21.15.105:0/195903970 <== osd.0 v2:172.21.15.105:6802/1226660290 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f80340af540 con 0x7f80340aca70 2024-09-16T02:01:19.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.232+0000 7f804b084640 1 -- 172.21.15.105:0/195903970 --> [v2:172.21.15.105:6802/1226660290,v1:172.21.15.105:6803/1226660290] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8044062d80 con 0x7f80340aca70 2024-09-16T02:01:19.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.232+0000 7f8041ffb640 1 -- 172.21.15.105:0/195903970 <== osd.0 v2:172.21.15.105:6802/1226660290 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8044062d80 con 0x7f80340aca70 2024-09-16T02:01:19.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.232+0000 7f804b084640 1 -- 172.21.15.105:0/195903970 >> [v2:172.21.15.105:6802/1226660290,v1:172.21.15.105:6803/1226660290] conn(0x7f80340aca70 msgr2=0x7f80340aee90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:19.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.232+0000 7f804b084640 1 --2- 172.21.15.105:0/195903970 >> [v2:172.21.15.105:6802/1226660290,v1:172.21.15.105:6803/1226660290] conn(0x7f80340aca70 0x7f80340aee90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:19.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.232+0000 7f804b084640 1 -- 172.21.15.105:0/195903970 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f80340a3350 msgr2=0x7f80340a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:19.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.232+0000 7f804b084640 1 --2- 172.21.15.105:0/195903970 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f80340a3350 0x7f80340a5810 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f803800bbe0 tx=0x7f803800b390 comp rx=0 tx=0).stop 2024-09-16T02:01:19.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.233+0000 7f804b084640 1 -- 172.21.15.105:0/195903970 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8044101290 msgr2=0x7f80441989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:19.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.233+0000 7f804b084640 1 --2- 172.21.15.105:0/195903970 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8044101290 0x7f80441989e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8030031440 tx=0x7f8030004270 comp rx=0 tx=0).stop 2024-09-16T02:01:19.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.233+0000 7f8049621640 1 -- 172.21.15.105:0/195903970 reap_dead start 2024-09-16T02:01:19.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.233+0000 7f804b084640 1 -- 172.21.15.105:0/195903970 shutdown_connections 2024-09-16T02:01:19.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.233+0000 7f804b084640 1 -- 172.21.15.105:0/195903970 >> 172.21.15.105:0/195903970 conn(0x7f8044077210 msgr2=0x7f8044105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:19.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.233+0000 7f804b084640 1 -- 172.21.15.105:0/195903970 shutdown_connections 2024-09-16T02:01:19.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:19.233+0000 7f804b084640 1 -- 172.21.15.105:0/195903970 wait complete. 2024-09-16T02:01:20.458 INFO:teuthology.orchestra.run.smithi105.stdout:201863462954 2024-09-16T02:01:20.458 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-16T02:01:20.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.623+0000 7fb7f21dc640 1 -- 172.21.15.105:0/2564280893 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7ec102570 msgr2=0x7fb7ec1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:20.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.623+0000 7fb7f21dc640 1 --2- 172.21.15.105:0/2564280893 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7ec102570 0x7fb7ec1029b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4009900 tx=0x7fb7d402a480 comp rx=0 tx=0).stop 2024-09-16T02:01:20.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.623+0000 7fb7f21dc640 1 -- 172.21.15.105:0/2564280893 shutdown_connections 2024-09-16T02:01:20.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.623+0000 7fb7f21dc640 1 --2- 172.21.15.105:0/2564280893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7ec102ef0 0x7fb7ec10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:20.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.623+0000 7fb7f21dc640 1 --2- 172.21.15.105:0/2564280893 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7ec102570 0x7fb7ec1029b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:20.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.623+0000 7fb7f21dc640 1 --2- 172.21.15.105:0/2564280893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7ec100e20 0x7fb7ec101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:20.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.623+0000 7fb7f21dc640 1 -- 172.21.15.105:0/2564280893 >> 172.21.15.105:0/2564280893 conn(0x7fb7ec0fc7d0 msgr2=0x7fb7ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:20.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.623+0000 7fb7f21dc640 1 -- 172.21.15.105:0/2564280893 shutdown_connections 2024-09-16T02:01:20.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.623+0000 7fb7f21dc640 1 -- 172.21.15.105:0/2564280893 wait complete. 2024-09-16T02:01:20.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.623+0000 7fb7f21dc640 1 Processor -- start 2024-09-16T02:01:20.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.623+0000 7fb7f21dc640 1 -- start start 2024-09-16T02:01:20.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.623+0000 7fb7f21dc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7ec100e20 0x7fb7ec1972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:20.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.623+0000 7fb7f21dc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7ec102570 0x7fb7ec197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:20.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.623+0000 7fb7f21dc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7ec102ef0 0x7fb7ec19bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:20.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.623+0000 7fb7f21dc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec110e50 con 0x7fb7ec102ef0 2024-09-16T02:01:20.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.623+0000 7fb7f21dc640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec110cd0 con 0x7fb7ec102570 2024-09-16T02:01:20.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.623+0000 7fb7f21dc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec110fd0 con 0x7fb7ec100e20 2024-09-16T02:01:20.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.624+0000 7fb7ebfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7ec102ef0 0x7fb7ec19bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:20.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.624+0000 7fb7ebfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7ec102ef0 0x7fb7ec19bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43008/0 (socket says 172.21.15.105:43008) 2024-09-16T02:01:20.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.624+0000 7fb7ebfff640 1 -- 172.21.15.105:0/1252089574 learned_addr learned my addr 172.21.15.105:0/1252089574 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:01:20.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.624+0000 7fb7eb7fe640 1 --2- 172.21.15.105:0/1252089574 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7ec100e20 0x7fb7ec1972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:20.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.624+0000 7fb7ebfff640 1 -- 172.21.15.105:0/1252089574 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7ec100e20 msgr2=0x7fb7ec1972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:20.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.624+0000 7fb7ebfff640 1 --2- 172.21.15.105:0/1252089574 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7ec100e20 0x7fb7ec1972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:20.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.624+0000 7fb7ebfff640 1 -- 172.21.15.105:0/1252089574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7ec102570 msgr2=0x7fb7ec197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:20.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.624+0000 7fb7ebfff640 1 --2- 172.21.15.105:0/1252089574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7ec102570 0x7fb7ec197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:20.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.624+0000 7fb7ebfff640 1 -- 172.21.15.105:0/1252089574 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7ec19c280 con 0x7fb7ec102ef0 2024-09-16T02:01:20.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.624+0000 7fb7ebfff640 1 --2- 172.21.15.105:0/1252089574 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7ec102ef0 0x7fb7ec19bba0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb7e400b970 tx=0x7fb7e400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:20.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.624+0000 7fb7e97fa640 1 -- 172.21.15.105:0/1252089574 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4019070 con 0x7fb7ec102ef0 2024-09-16T02:01:20.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.624+0000 7fb7e97fa640 1 -- 172.21.15.105:0/1252089574 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7e4014440 con 0x7fb7ec102ef0 2024-09-16T02:01:20.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.624+0000 7fb7f21dc640 1 -- 172.21.15.105:0/1252089574 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7ec19c570 con 0x7fb7ec102ef0 2024-09-16T02:01:20.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.624+0000 7fb7e97fa640 1 -- 172.21.15.105:0/1252089574 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e40075d0 con 0x7fb7ec102ef0 2024-09-16T02:01:20.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.624+0000 7fb7f21dc640 1 -- 172.21.15.105:0/1252089574 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7ec1a3e50 con 0x7fb7ec102ef0 2024-09-16T02:01:20.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.625+0000 7fb7f21dc640 1 -- 172.21.15.105:0/1252089574 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb7ec062980 con 0x7fb7ec102ef0 2024-09-16T02:01:20.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.626+0000 7fb7e97fa640 1 -- 172.21.15.105:0/1252089574 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fb7e401c070 con 0x7fb7ec102ef0 2024-09-16T02:01:20.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.627+0000 7fb7e97fa640 1 --2- 172.21.15.105:0/1252089574 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb7d80a32d0 0x7fb7d80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:20.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.627+0000 7fb7eb7fe640 1 --2- 172.21.15.105:0/1252089574 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb7d80a32d0 0x7fb7d80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:20.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.627+0000 7fb7e97fa640 1 -- 172.21.15.105:0/1252089574 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fb7e40cf630 con 0x7fb7ec102ef0 2024-09-16T02:01:20.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.627+0000 7fb7e97fa640 1 --2- 172.21.15.105:0/1252089574 >> [v2:172.21.15.190:6824/3820460689,v1:172.21.15.190:6825/3820460689] conn(0x7fb7d80acb20 0x7fb7d80aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:20.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.627+0000 7fb7e97fa640 1 -- 172.21.15.105:0/1252089574 --> [v2:172.21.15.190:6824/3820460689,v1:172.21.15.190:6825/3820460689] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb7e40047d0 con 0x7fb7d80acb20 2024-09-16T02:01:20.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.627+0000 7fb7e97fa640 1 -- 172.21.15.105:0/1252089574 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fb7e40cfb10 con 0x7fb7ec102ef0 2024-09-16T02:01:20.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.628+0000 7fb7eb7fe640 1 --2- 172.21.15.105:0/1252089574 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb7d80a32d0 0x7fb7d80a5790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc005e10 tx=0x7fb7dc005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:20.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.628+0000 7fb7e3fff640 1 --2- 172.21.15.105:0/1252089574 >> [v2:172.21.15.190:6824/3820460689,v1:172.21.15.190:6825/3820460689] conn(0x7fb7d80acb20 0x7fb7d80aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:20.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.628+0000 7fb7e3fff640 1 --2- 172.21.15.105:0/1252089574 >> [v2:172.21.15.190:6824/3820460689,v1:172.21.15.190:6825/3820460689] conn(0x7fb7d80acb20 0x7fb7d80aef40 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:20.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.629+0000 7fb7e97fa640 1 -- 172.21.15.105:0/1252089574 <== osd.11 v2:172.21.15.190:6824/3820460689 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb7e40047d0 con 0x7fb7d80acb20 2024-09-16T02:01:20.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.652+0000 7fb7f21dc640 1 -- 172.21.15.105:0/1252089574 --> [v2:172.21.15.190:6824/3820460689,v1:172.21.15.190:6825/3820460689] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb7ec062e00 con 0x7fb7d80acb20 2024-09-16T02:01:20.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.652+0000 7fb7e97fa640 1 -- 172.21.15.105:0/1252089574 <== osd.11 v2:172.21.15.190:6824/3820460689 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb7ec062e00 con 0x7fb7d80acb20 2024-09-16T02:01:20.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.653+0000 7fb7f21dc640 1 -- 172.21.15.105:0/1252089574 >> [v2:172.21.15.190:6824/3820460689,v1:172.21.15.190:6825/3820460689] conn(0x7fb7d80acb20 msgr2=0x7fb7d80aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:20.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.653+0000 7fb7f21dc640 1 --2- 172.21.15.105:0/1252089574 >> [v2:172.21.15.190:6824/3820460689,v1:172.21.15.190:6825/3820460689] conn(0x7fb7d80acb20 0x7fb7d80aef40 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:20.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.653+0000 7fb7f21dc640 1 -- 172.21.15.105:0/1252089574 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb7d80a32d0 msgr2=0x7fb7d80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:20.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.653+0000 7fb7f21dc640 1 --2- 172.21.15.105:0/1252089574 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb7d80a32d0 0x7fb7d80a5790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc005e10 tx=0x7fb7dc005da0 comp rx=0 tx=0).stop 2024-09-16T02:01:20.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.653+0000 7fb7f21dc640 1 -- 172.21.15.105:0/1252089574 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7ec102ef0 msgr2=0x7fb7ec19bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:20.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.653+0000 7fb7f21dc640 1 --2- 172.21.15.105:0/1252089574 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7ec102ef0 0x7fb7ec19bba0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb7e400b970 tx=0x7fb7e400be30 comp rx=0 tx=0).stop 2024-09-16T02:01:20.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.653+0000 7fb7ebfff640 1 -- 172.21.15.105:0/1252089574 reap_dead start 2024-09-16T02:01:20.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.653+0000 7fb7f21dc640 1 -- 172.21.15.105:0/1252089574 shutdown_connections 2024-09-16T02:01:20.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.653+0000 7fb7f21dc640 1 -- 172.21.15.105:0/1252089574 >> 172.21.15.105:0/1252089574 conn(0x7fb7ec0fc7d0 msgr2=0x7fb7ec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:20.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.653+0000 7fb7f21dc640 1 -- 172.21.15.105:0/1252089574 shutdown_connections 2024-09-16T02:01:20.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.653+0000 7fb7f21dc640 1 -- 172.21.15.105:0/1252089574 wait complete. 2024-09-16T02:01:20.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.201+0000 7f64401b0640 1 -- 172.21.15.105:0/2520821255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6438106f10 msgr2=0x7f643810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:20.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.201+0000 7f64401b0640 1 --2- 172.21.15.105:0/2520821255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6438106f10 0x7f643810ea90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6430009ca0 tx=0x7f643002a8c0 comp rx=0 tx=0).stop 2024-09-16T02:01:20.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.201+0000 7f64401b0640 1 -- 172.21.15.105:0/2520821255 shutdown_connections 2024-09-16T02:01:20.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.201+0000 7f64401b0640 1 --2- 172.21.15.105:0/2520821255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6438106f10 0x7f643810ea90 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:20.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.201+0000 7f64401b0640 1 --2- 172.21.15.105:0/2520821255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6438102790 0x7f64381067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:20.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.201+0000 7f64401b0640 1 --2- 172.21.15.105:0/2520821255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6438101de0 0x7f64381021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:20.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.201+0000 7f64401b0640 1 -- 172.21.15.105:0/2520821255 >> 172.21.15.105:0/2520821255 conn(0x7f64380fd8c0 msgr2=0x7f64380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:20.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.201+0000 7f64401b0640 1 -- 172.21.15.105:0/2520821255 shutdown_connections 2024-09-16T02:01:20.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.201+0000 7f64401b0640 1 -- 172.21.15.105:0/2520821255 wait complete. 2024-09-16T02:01:20.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.202+0000 7f64401b0640 1 Processor -- start 2024-09-16T02:01:20.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.202+0000 7f64401b0640 1 -- start start 2024-09-16T02:01:20.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.202+0000 7f64401b0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6438101de0 0x7f64381983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:20.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.202+0000 7f64401b0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6438102790 0x7f64381988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:20.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.202+0000 7f64401b0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6438106f10 0x7f643819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:20.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.202+0000 7f64401b0640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6438111e60 con 0x7f6438102790 2024-09-16T02:01:20.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.202+0000 7f64401b0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6438111ce0 con 0x7f6438101de0 2024-09-16T02:01:20.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.202+0000 7f64401b0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6438111fe0 con 0x7f6438106f10 2024-09-16T02:01:20.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.202+0000 7f643d74b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6438102790 0x7f64381988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:20.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.202+0000 7f643d74b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6438102790 0x7f64381988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42982/0 (socket says 172.21.15.105:42982) 2024-09-16T02:01:20.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.202+0000 7f643d74b640 1 -- 172.21.15.105:0/2247837895 learned_addr learned my addr 172.21.15.105:0/2247837895 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:01:20.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.202+0000 7f643e74d640 1 --2- 172.21.15.105:0/2247837895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6438106f10 0x7f643819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:20.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.202+0000 7f643df4c640 1 --2- 172.21.15.105:0/2247837895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6438101de0 0x7f64381983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:20.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.202+0000 7f643d74b640 1 -- 172.21.15.105:0/2247837895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6438106f10 msgr2=0x7f643819cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:20.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.202+0000 7f643d74b640 1 --2- 172.21.15.105:0/2247837895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6438106f10 0x7f643819cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:20.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.202+0000 7f643d74b640 1 -- 172.21.15.105:0/2247837895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6438101de0 msgr2=0x7f64381983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:20.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.202+0000 7f643d74b640 1 --2- 172.21.15.105:0/2247837895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6438101de0 0x7f64381983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:20.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.202+0000 7f643d74b640 1 -- 172.21.15.105:0/2247837895 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f643819d350 con 0x7f6438102790 2024-09-16T02:01:20.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.203+0000 7f643d74b640 1 --2- 172.21.15.105:0/2247837895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6438102790 0x7f64381988e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f642800b4a0 tx=0x7f642800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:20.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.203+0000 7f642effd640 1 -- 172.21.15.105:0/2247837895 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6428014070 con 0x7f6438102790 2024-09-16T02:01:20.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.203+0000 7f64401b0640 1 -- 172.21.15.105:0/2247837895 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f643819d640 con 0x7f6438102790 2024-09-16T02:01:20.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.203+0000 7f642effd640 1 -- 172.21.15.105:0/2247837895 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6428010ac0 con 0x7f6438102790 2024-09-16T02:01:20.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.203+0000 7f642effd640 1 -- 172.21.15.105:0/2247837895 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6428017c30 con 0x7f6438102790 2024-09-16T02:01:20.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.203+0000 7f64401b0640 1 -- 172.21.15.105:0/2247837895 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64381a4f20 con 0x7f6438102790 2024-09-16T02:01:20.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.204+0000 7f64401b0640 1 -- 172.21.15.105:0/2247837895 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6438062940 con 0x7f6438102790 2024-09-16T02:01:20.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.205+0000 7f642effd640 1 -- 172.21.15.105:0/2247837895 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f6428017070 con 0x7f6438102790 2024-09-16T02:01:20.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.206+0000 7f642effd640 1 --2- 172.21.15.105:0/2247837895 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f64240a32d0 0x7f64240a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:20.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.206+0000 7f643df4c640 1 --2- 172.21.15.105:0/2247837895 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f64240a32d0 0x7f64240a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:20.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.206+0000 7f642effd640 1 -- 172.21.15.105:0/2247837895 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f64280cefd0 con 0x7f6438102790 2024-09-16T02:01:20.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.206+0000 7f642effd640 1 --2- 172.21.15.105:0/2247837895 >> [v2:172.21.15.105:6818/4012018725,v1:172.21.15.105:6819/4012018725] conn(0x7f64240acb20 0x7f64240aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:20.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.206+0000 7f642effd640 1 -- 172.21.15.105:0/2247837895 --> [v2:172.21.15.105:6818/4012018725,v1:172.21.15.105:6819/4012018725] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f642800ee50 con 0x7f64240acb20 2024-09-16T02:01:20.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.206+0000 7f642effd640 1 -- 172.21.15.105:0/2247837895 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f64280cf4b0 con 0x7f6438102790 2024-09-16T02:01:20.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.206+0000 7f643e74d640 1 --2- 172.21.15.105:0/2247837895 >> [v2:172.21.15.105:6818/4012018725,v1:172.21.15.105:6819/4012018725] conn(0x7f64240acb20 0x7f64240aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:20.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.206+0000 7f643df4c640 1 --2- 172.21.15.105:0/2247837895 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f64240a32d0 0x7f64240a5790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6434005fd0 tx=0x7f6434005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:20.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.206+0000 7f643e74d640 1 --2- 172.21.15.105:0/2247837895 >> [v2:172.21.15.105:6818/4012018725,v1:172.21.15.105:6819/4012018725] conn(0x7f64240acb20 0x7f64240aef40 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:20.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.207+0000 7f642effd640 1 -- 172.21.15.105:0/2247837895 <== osd.6 v2:172.21.15.105:6818/4012018725 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f642800ee50 con 0x7f64240acb20 2024-09-16T02:01:20.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.221+0000 7f64401b0640 1 -- 172.21.15.105:0/2247837895 --> [v2:172.21.15.105:6818/4012018725,v1:172.21.15.105:6819/4012018725] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6438062d40 con 0x7f64240acb20 2024-09-16T02:01:20.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.221+0000 7f642effd640 1 -- 172.21.15.105:0/2247837895 <== osd.6 v2:172.21.15.105:6818/4012018725 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6438062d40 con 0x7f64240acb20 2024-09-16T02:01:20.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.221+0000 7f64401b0640 1 -- 172.21.15.105:0/2247837895 >> [v2:172.21.15.105:6818/4012018725,v1:172.21.15.105:6819/4012018725] conn(0x7f64240acb20 msgr2=0x7f64240aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:20.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.221+0000 7f64401b0640 1 --2- 172.21.15.105:0/2247837895 >> [v2:172.21.15.105:6818/4012018725,v1:172.21.15.105:6819/4012018725] conn(0x7f64240acb20 0x7f64240aef40 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:20.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.221+0000 7f64401b0640 1 -- 172.21.15.105:0/2247837895 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f64240a32d0 msgr2=0x7f64240a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:20.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.221+0000 7f64401b0640 1 --2- 172.21.15.105:0/2247837895 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f64240a32d0 0x7f64240a5790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6434005fd0 tx=0x7f6434005850 comp rx=0 tx=0).stop 2024-09-16T02:01:20.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.221+0000 7f64401b0640 1 -- 172.21.15.105:0/2247837895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6438102790 msgr2=0x7f64381988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:20.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.221+0000 7f64401b0640 1 --2- 172.21.15.105:0/2247837895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6438102790 0x7f64381988e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f642800b4a0 tx=0x7f642800b960 comp rx=0 tx=0).stop 2024-09-16T02:01:20.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.221+0000 7f643e74d640 1 -- 172.21.15.105:0/2247837895 reap_dead start 2024-09-16T02:01:20.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.222+0000 7f64401b0640 1 -- 172.21.15.105:0/2247837895 shutdown_connections 2024-09-16T02:01:20.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.222+0000 7f64401b0640 1 -- 172.21.15.105:0/2247837895 >> 172.21.15.105:0/2247837895 conn(0x7f64380fd8c0 msgr2=0x7f64380feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:20.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.222+0000 7f64401b0640 1 -- 172.21.15.105:0/2247837895 shutdown_connections 2024-09-16T02:01:20.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:20.222+0000 7f64401b0640 1 -- 172.21.15.105:0/2247837895 wait complete. 2024-09-16T02:01:20.758 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:20 smithi105 ceph-mon[31197]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-16T02:01:20.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:20 smithi190 ceph-mon[34762]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-16T02:01:20.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:20 smithi137 ceph-mon[34099]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-16T02:01:21.438 INFO:teuthology.orchestra.run.smithi105.stdout:180388626481 2024-09-16T02:01:21.438 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-16T02:01:21.834 INFO:teuthology.orchestra.run.smithi105.stdout:231928234021 2024-09-16T02:01:21.835 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-16T02:01:22.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.080+0000 7f8920b4e640 1 -- 172.21.15.105:0/3692281958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f891c102570 msgr2=0x7f891c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:22.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.080+0000 7f8920b4e640 1 --2- 172.21.15.105:0/3692281958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f891c102570 0x7f891c1029b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8904009900 tx=0x7f890402a480 comp rx=0 tx=0).stop 2024-09-16T02:01:22.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.080+0000 7f8920b4e640 1 -- 172.21.15.105:0/3692281958 shutdown_connections 2024-09-16T02:01:22.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.080+0000 7f8920b4e640 1 --2- 172.21.15.105:0/3692281958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f891c102ef0 0x7f891c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:22.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.080+0000 7f8920b4e640 1 --2- 172.21.15.105:0/3692281958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f891c102570 0x7f891c1029b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:22.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.080+0000 7f8920b4e640 1 --2- 172.21.15.105:0/3692281958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f891c100e20 0x7f891c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:22.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.080+0000 7f8920b4e640 1 -- 172.21.15.105:0/3692281958 >> 172.21.15.105:0/3692281958 conn(0x7f891c0fc7d0 msgr2=0x7f891c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:22.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.080+0000 7f8920b4e640 1 -- 172.21.15.105:0/3692281958 shutdown_connections 2024-09-16T02:01:22.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.080+0000 7f8920b4e640 1 -- 172.21.15.105:0/3692281958 wait complete. 2024-09-16T02:01:22.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.081+0000 7f8920b4e640 1 Processor -- start 2024-09-16T02:01:22.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.081+0000 7f8920b4e640 1 -- start start 2024-09-16T02:01:22.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.081+0000 7f8920b4e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f891c100e20 0x7f891c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:22.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.081+0000 7f8920b4e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f891c102570 0x7f891c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:22.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.081+0000 7f8920b4e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f891c102ef0 0x7f891c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:22.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.081+0000 7f8920b4e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f891c110e50 con 0x7f891c102ef0 2024-09-16T02:01:22.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.081+0000 7f8920b4e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f891c110cd0 con 0x7f891c102570 2024-09-16T02:01:22.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.081+0000 7f8920b4e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f891c110fd0 con 0x7f891c100e20 2024-09-16T02:01:22.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.081+0000 7f891ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f891c102ef0 0x7f891c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:22.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.082+0000 7f891ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f891c102ef0 0x7f891c19cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43028/0 (socket says 172.21.15.105:43028) 2024-09-16T02:01:22.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.082+0000 7f891ad9d640 1 -- 172.21.15.105:0/2524191607 learned_addr learned my addr 172.21.15.105:0/2524191607 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:01:22.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.082+0000 7f8913fff640 1 --2- 172.21.15.105:0/2524191607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f891c102570 0x7f891c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:22.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.082+0000 7f891a59c640 1 --2- 172.21.15.105:0/2524191607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f891c100e20 0x7f891c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:22.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.082+0000 7f891ad9d640 1 -- 172.21.15.105:0/2524191607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f891c100e20 msgr2=0x7f891c198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:22.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.082+0000 7f891ad9d640 1 --2- 172.21.15.105:0/2524191607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f891c100e20 0x7f891c198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:22.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.082+0000 7f891ad9d640 1 -- 172.21.15.105:0/2524191607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f891c102570 msgr2=0x7f891c1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:22.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.082+0000 7f891ad9d640 1 --2- 172.21.15.105:0/2524191607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f891c102570 0x7f891c1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:22.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.082+0000 7f891ad9d640 1 -- 172.21.15.105:0/2524191607 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f891c19d420 con 0x7f891c102ef0 2024-09-16T02:01:22.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.082+0000 7f891ad9d640 1 --2- 172.21.15.105:0/2524191607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f891c102ef0 0x7f891c19cd40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f891400b970 tx=0x7f891400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:22.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.082+0000 7f89137fe640 1 -- 172.21.15.105:0/2524191607 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8914019070 con 0x7f891c102ef0 2024-09-16T02:01:22.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.082+0000 7f8920b4e640 1 -- 172.21.15.105:0/2524191607 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f891c19d710 con 0x7f891c102ef0 2024-09-16T02:01:22.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.082+0000 7f89137fe640 1 -- 172.21.15.105:0/2524191607 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8914014440 con 0x7f891c102ef0 2024-09-16T02:01:22.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.082+0000 7f89137fe640 1 -- 172.21.15.105:0/2524191607 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89140075d0 con 0x7f891c102ef0 2024-09-16T02:01:22.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.082+0000 7f8920b4e640 1 -- 172.21.15.105:0/2524191607 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f891c1a4ff0 con 0x7f891c102ef0 2024-09-16T02:01:22.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.083+0000 7f8920b4e640 1 -- 172.21.15.105:0/2524191607 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f891c062980 con 0x7f891c102ef0 2024-09-16T02:01:22.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.084+0000 7f89137fe640 1 -- 172.21.15.105:0/2524191607 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f891401c070 con 0x7f891c102ef0 2024-09-16T02:01:22.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.085+0000 7f89137fe640 1 --2- 172.21.15.105:0/2524191607 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f89080a32d0 0x7f89080a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:22.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.085+0000 7f891a59c640 1 --2- 172.21.15.105:0/2524191607 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f89080a32d0 0x7f89080a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:22.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.085+0000 7f89137fe640 1 -- 172.21.15.105:0/2524191607 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f89140cf570 con 0x7f891c102ef0 2024-09-16T02:01:22.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.085+0000 7f89137fe640 1 --2- 172.21.15.105:0/2524191607 >> [v2:172.21.15.105:6826/3375034979,v1:172.21.15.105:6827/3375034979] conn(0x7f89080acb20 0x7f89080aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:22.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.085+0000 7f89137fe640 1 -- 172.21.15.105:0/2524191607 --> [v2:172.21.15.105:6826/3375034979,v1:172.21.15.105:6827/3375034979] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f89140047d0 con 0x7f89080acb20 2024-09-16T02:01:22.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.085+0000 7f89137fe640 1 -- 172.21.15.105:0/2524191607 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f89140cfa50 con 0x7f891c102ef0 2024-09-16T02:01:22.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.085+0000 7f8913fff640 1 --2- 172.21.15.105:0/2524191607 >> [v2:172.21.15.105:6826/3375034979,v1:172.21.15.105:6827/3375034979] conn(0x7f89080acb20 0x7f89080aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:22.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.085+0000 7f891a59c640 1 --2- 172.21.15.105:0/2524191607 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f89080a32d0 0x7f89080a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f890c005fd0 tx=0x7f890c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:22.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.086+0000 7f8913fff640 1 --2- 172.21.15.105:0/2524191607 >> [v2:172.21.15.105:6826/3375034979,v1:172.21.15.105:6827/3375034979] conn(0x7f89080acb20 0x7f89080aef40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:22.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.086+0000 7f89137fe640 1 -- 172.21.15.105:0/2524191607 <== osd.9 v2:172.21.15.105:6826/3375034979 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f89140047d0 con 0x7f89080acb20 2024-09-16T02:01:22.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.100+0000 7f8920b4e640 1 -- 172.21.15.105:0/2524191607 --> [v2:172.21.15.105:6826/3375034979,v1:172.21.15.105:6827/3375034979] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f891c062e00 con 0x7f89080acb20 2024-09-16T02:01:22.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.100+0000 7f89137fe640 1 -- 172.21.15.105:0/2524191607 <== osd.9 v2:172.21.15.105:6826/3375034979 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f891c062e00 con 0x7f89080acb20 2024-09-16T02:01:22.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.100+0000 7f8920b4e640 1 -- 172.21.15.105:0/2524191607 >> [v2:172.21.15.105:6826/3375034979,v1:172.21.15.105:6827/3375034979] conn(0x7f89080acb20 msgr2=0x7f89080aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:22.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.100+0000 7f8920b4e640 1 --2- 172.21.15.105:0/2524191607 >> [v2:172.21.15.105:6826/3375034979,v1:172.21.15.105:6827/3375034979] conn(0x7f89080acb20 0x7f89080aef40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:22.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.101+0000 7f8920b4e640 1 -- 172.21.15.105:0/2524191607 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f89080a32d0 msgr2=0x7f89080a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:22.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.101+0000 7f8920b4e640 1 --2- 172.21.15.105:0/2524191607 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f89080a32d0 0x7f89080a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f890c005fd0 tx=0x7f890c005850 comp rx=0 tx=0).stop 2024-09-16T02:01:22.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.101+0000 7f8920b4e640 1 -- 172.21.15.105:0/2524191607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f891c102ef0 msgr2=0x7f891c19cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:22.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.101+0000 7f8920b4e640 1 --2- 172.21.15.105:0/2524191607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f891c102ef0 0x7f891c19cd40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f891400b970 tx=0x7f891400be30 comp rx=0 tx=0).stop 2024-09-16T02:01:22.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.101+0000 7f891ad9d640 1 -- 172.21.15.105:0/2524191607 reap_dead start 2024-09-16T02:01:22.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.101+0000 7f8920b4e640 1 -- 172.21.15.105:0/2524191607 shutdown_connections 2024-09-16T02:01:22.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.101+0000 7f8920b4e640 1 -- 172.21.15.105:0/2524191607 >> 172.21.15.105:0/2524191607 conn(0x7f891c0fc7d0 msgr2=0x7f891c06a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:22.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.101+0000 7f8920b4e640 1 -- 172.21.15.105:0/2524191607 shutdown_connections 2024-09-16T02:01:22.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:22.101+0000 7f8920b4e640 1 -- 172.21.15.105:0/2524191607 wait complete. 2024-09-16T02:01:22.335 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:22.336 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:22.776 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:22 smithi105 ceph-mon[31197]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-09-16T02:01:22.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:22 smithi190 ceph-mon[34762]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-09-16T02:01:22.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:22 smithi137 ceph-mon[34099]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-09-16T02:01:23.120 INFO:teuthology.orchestra.run.smithi105.stdout:261993005089 2024-09-16T02:01:23.121 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-16T02:01:23.727 INFO:teuthology.orchestra.run.smithi105.stdout:313532612631 2024-09-16T02:01:23.727 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-16T02:01:23.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:23 smithi105 ceph-mon[31197]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-16T02:01:23.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:23 smithi137 ceph-mon[34099]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-16T02:01:24.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:23 smithi190 ceph-mon[34762]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-16T02:01:24.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.172+0000 7f661bc0e640 1 -- 172.21.15.105:0/4232759065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6614102790 msgr2=0x7f66141067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.172+0000 7f661bc0e640 1 --2- 172.21.15.105:0/4232759065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6614102790 0x7f66141067e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6604009900 tx=0x7f660402a410 comp rx=0 tx=0).stop 2024-09-16T02:01:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.172+0000 7f661bc0e640 1 -- 172.21.15.105:0/4232759065 shutdown_connections 2024-09-16T02:01:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.172+0000 7f661bc0e640 1 --2- 172.21.15.105:0/4232759065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6614106f10 0x7f661410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.172+0000 7f661bc0e640 1 --2- 172.21.15.105:0/4232759065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6614102790 0x7f66141067e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.172+0000 7f661bc0e640 1 --2- 172.21.15.105:0/4232759065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6614101de0 0x7f66141021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.172+0000 7f661bc0e640 1 -- 172.21.15.105:0/4232759065 >> 172.21.15.105:0/4232759065 conn(0x7f66140fd8c0 msgr2=0x7f66140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.172+0000 7f661bc0e640 1 -- 172.21.15.105:0/4232759065 shutdown_connections 2024-09-16T02:01:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.172+0000 7f661bc0e640 1 -- 172.21.15.105:0/4232759065 wait complete. 2024-09-16T02:01:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.173+0000 7f661bc0e640 1 Processor -- start 2024-09-16T02:01:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.173+0000 7f661bc0e640 1 -- start start 2024-09-16T02:01:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.173+0000 7f661bc0e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6614101de0 0x7f6614198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.173+0000 7f661bc0e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6614102790 0x7f6614198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.173+0000 7f661bc0e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6614106f10 0x7f661419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.173+0000 7f661bc0e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6614111e60 con 0x7f6614101de0 2024-09-16T02:01:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.173+0000 7f661bc0e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6614111ce0 con 0x7f6614106f10 2024-09-16T02:01:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.173+0000 7f661bc0e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6614111fe0 con 0x7f6614102790 2024-09-16T02:01:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.173+0000 7f66199aa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6614101de0 0x7f6614198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.173+0000 7f66199aa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6614101de0 0x7f6614198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43044/0 (socket says 172.21.15.105:43044) 2024-09-16T02:01:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.173+0000 7f66199aa640 1 -- 172.21.15.105:0/1753327262 learned_addr learned my addr 172.21.15.105:0/1753327262 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:01:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.173+0000 7f661a1ab640 1 --2- 172.21.15.105:0/1753327262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6614106f10 0x7f661419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.173+0000 7f66199aa640 1 -- 172.21.15.105:0/1753327262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6614102790 msgr2=0x7f6614198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.173+0000 7f66199aa640 1 --2- 172.21.15.105:0/1753327262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6614102790 0x7f6614198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.173+0000 7f66199aa640 1 -- 172.21.15.105:0/1753327262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6614106f10 msgr2=0x7f661419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.173+0000 7f66199aa640 1 --2- 172.21.15.105:0/1753327262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6614106f10 0x7f661419ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.173+0000 7f66199aa640 1 -- 172.21.15.105:0/1753327262 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f661419d3b0 con 0x7f6614101de0 2024-09-16T02:01:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.173+0000 7f661a1ab640 1 --2- 172.21.15.105:0/1753327262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6614106f10 0x7f661419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:01:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.173+0000 7f66199aa640 1 --2- 172.21.15.105:0/1753327262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6614101de0 0x7f6614198400 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f661000b4a0 tx=0x7f661000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.173+0000 7f660affd640 1 -- 172.21.15.105:0/1753327262 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6610014070 con 0x7f6614101de0 2024-09-16T02:01:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.174+0000 7f661bc0e640 1 -- 172.21.15.105:0/1753327262 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f661419d6a0 con 0x7f6614101de0 2024-09-16T02:01:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.174+0000 7f660affd640 1 -- 172.21.15.105:0/1753327262 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6610010ac0 con 0x7f6614101de0 2024-09-16T02:01:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.174+0000 7f660affd640 1 -- 172.21.15.105:0/1753327262 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6610017c30 con 0x7f6614101de0 2024-09-16T02:01:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.174+0000 7f661bc0e640 1 -- 172.21.15.105:0/1753327262 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66141a4f80 con 0x7f6614101de0 2024-09-16T02:01:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.174+0000 7f661bc0e640 1 -- 172.21.15.105:0/1753327262 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6614062940 con 0x7f6614101de0 2024-09-16T02:01:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.176+0000 7f660affd640 1 -- 172.21.15.105:0/1753327262 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f6610017070 con 0x7f6614101de0 2024-09-16T02:01:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.176+0000 7f660affd640 1 --2- 172.21.15.105:0/1753327262 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f66000a32d0 0x7f66000a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.176+0000 7f66191a9640 1 --2- 172.21.15.105:0/1753327262 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f66000a32d0 0x7f66000a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.176+0000 7f660affd640 1 -- 172.21.15.105:0/1753327262 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f66100cf020 con 0x7f6614101de0 2024-09-16T02:01:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.177+0000 7f660affd640 1 --2- 172.21.15.105:0/1753327262 >> [v2:172.21.15.105:6810/604905679,v1:172.21.15.105:6811/604905679] conn(0x7f66000acb20 0x7f66000aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.177+0000 7f660affd640 1 -- 172.21.15.105:0/1753327262 --> [v2:172.21.15.105:6810/604905679,v1:172.21.15.105:6811/604905679] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f661000ee50 con 0x7f66000acb20 2024-09-16T02:01:24.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.177+0000 7f660affd640 1 -- 172.21.15.105:0/1753327262 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f66100cf500 con 0x7f6614101de0 2024-09-16T02:01:24.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.177+0000 7f66191a9640 1 --2- 172.21.15.105:0/1753327262 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f66000a32d0 0x7f66000a5790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f66040096c0 tx=0x7f6604031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:24.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.177+0000 7f661a1ab640 1 --2- 172.21.15.105:0/1753327262 >> [v2:172.21.15.105:6810/604905679,v1:172.21.15.105:6811/604905679] conn(0x7f66000acb20 0x7f66000aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:24.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.177+0000 7f661a1ab640 1 --2- 172.21.15.105:0/1753327262 >> [v2:172.21.15.105:6810/604905679,v1:172.21.15.105:6811/604905679] conn(0x7f66000acb20 0x7f66000aef40 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:24.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.178+0000 7f660affd640 1 -- 172.21.15.105:0/1753327262 <== osd.3 v2:172.21.15.105:6810/604905679 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f661000ee50 con 0x7f66000acb20 2024-09-16T02:01:24.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.191+0000 7f661bc0e640 1 -- 172.21.15.105:0/1753327262 --> [v2:172.21.15.105:6810/604905679,v1:172.21.15.105:6811/604905679] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6614062d40 con 0x7f66000acb20 2024-09-16T02:01:24.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.191+0000 7f660affd640 1 -- 172.21.15.105:0/1753327262 <== osd.3 v2:172.21.15.105:6810/604905679 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6614062d40 con 0x7f66000acb20 2024-09-16T02:01:24.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.192+0000 7f6608ff9640 1 -- 172.21.15.105:0/1753327262 >> [v2:172.21.15.105:6810/604905679,v1:172.21.15.105:6811/604905679] conn(0x7f66000acb20 msgr2=0x7f66000aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:24.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.192+0000 7f6608ff9640 1 --2- 172.21.15.105:0/1753327262 >> [v2:172.21.15.105:6810/604905679,v1:172.21.15.105:6811/604905679] conn(0x7f66000acb20 0x7f66000aef40 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:24.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.192+0000 7f6608ff9640 1 -- 172.21.15.105:0/1753327262 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f66000a32d0 msgr2=0x7f66000a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:24.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.192+0000 7f6608ff9640 1 --2- 172.21.15.105:0/1753327262 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f66000a32d0 0x7f66000a5790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f66040096c0 tx=0x7f6604031400 comp rx=0 tx=0).stop 2024-09-16T02:01:24.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.192+0000 7f6608ff9640 1 -- 172.21.15.105:0/1753327262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6614101de0 msgr2=0x7f6614198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:24.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.192+0000 7f6608ff9640 1 --2- 172.21.15.105:0/1753327262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6614101de0 0x7f6614198400 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f661000b4a0 tx=0x7f661000b960 comp rx=0 tx=0).stop 2024-09-16T02:01:24.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.192+0000 7f661a1ab640 1 -- 172.21.15.105:0/1753327262 reap_dead start 2024-09-16T02:01:24.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.192+0000 7f6608ff9640 1 -- 172.21.15.105:0/1753327262 shutdown_connections 2024-09-16T02:01:24.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.192+0000 7f6608ff9640 1 -- 172.21.15.105:0/1753327262 >> 172.21.15.105:0/1753327262 conn(0x7f66140fd8c0 msgr2=0x7f66140fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:24.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.193+0000 7f6608ff9640 1 -- 172.21.15.105:0/1753327262 shutdown_connections 2024-09-16T02:01:24.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.193+0000 7f6608ff9640 1 -- 172.21.15.105:0/1753327262 wait complete. 2024-09-16T02:01:24.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.609+0000 7fc420bbf640 1 -- 172.21.15.105:0/3405347740 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc41c100a80 msgr2=0x7fc41c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:24.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.609+0000 7fc420bbf640 1 --2- 172.21.15.105:0/3405347740 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc41c100a80 0x7fc41c100ee0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc410009930 tx=0x7fc41002a420 comp rx=0 tx=0).stop 2024-09-16T02:01:24.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.609+0000 7fc420bbf640 1 -- 172.21.15.105:0/3405347740 shutdown_connections 2024-09-16T02:01:24.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.609+0000 7fc420bbf640 1 --2- 172.21.15.105:0/3405347740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc41c101420 0x7fc41c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:24.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.609+0000 7fc420bbf640 1 --2- 172.21.15.105:0/3405347740 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc41c100a80 0x7fc41c100ee0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:24.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.609+0000 7fc420bbf640 1 --2- 172.21.15.105:0/3405347740 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc41c106a80 0x7fc41c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:24.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.609+0000 7fc420bbf640 1 -- 172.21.15.105:0/3405347740 >> 172.21.15.105:0/3405347740 conn(0x7fc41c0fc7a0 msgr2=0x7fc41c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:24.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.609+0000 7fc420bbf640 1 -- 172.21.15.105:0/3405347740 shutdown_connections 2024-09-16T02:01:24.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.609+0000 7fc420bbf640 1 -- 172.21.15.105:0/3405347740 wait complete. 2024-09-16T02:01:24.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.610+0000 7fc420bbf640 1 Processor -- start 2024-09-16T02:01:24.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.610+0000 7fc420bbf640 1 -- start start 2024-09-16T02:01:24.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.610+0000 7fc420bbf640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc41c100a80 0x7fc41c19a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:24.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.610+0000 7fc420bbf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc41c101420 0x7fc41c19aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:24.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.610+0000 7fc420bbf640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc41c106a80 0x7fc41c19ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:24.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.610+0000 7fc420bbf640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc41c110d30 con 0x7fc41c101420 2024-09-16T02:01:24.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.610+0000 7fc420bbf640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc41c110bb0 con 0x7fc41c100a80 2024-09-16T02:01:24.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.610+0000 7fc420bbf640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc41c110eb0 con 0x7fc41c106a80 2024-09-16T02:01:24.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.610+0000 7fc419d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc41c101420 0x7fc41c19aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:24.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.610+0000 7fc419d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc41c101420 0x7fc41c19aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43056/0 (socket says 172.21.15.105:43056) 2024-09-16T02:01:24.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.610+0000 7fc419d9b640 1 -- 172.21.15.105:0/2103655625 learned_addr learned my addr 172.21.15.105:0/2103655625 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:01:24.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.610+0000 7fc419d9b640 1 -- 172.21.15.105:0/2103655625 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc41c106a80 msgr2=0x7fc41c19ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:24.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.610+0000 7fc419d9b640 1 --2- 172.21.15.105:0/2103655625 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc41c106a80 0x7fc41c19ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:24.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.610+0000 7fc419d9b640 1 -- 172.21.15.105:0/2103655625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc41c100a80 msgr2=0x7fc41c19a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:24.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.610+0000 7fc419d9b640 1 --2- 172.21.15.105:0/2103655625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc41c100a80 0x7fc41c19a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:24.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.610+0000 7fc419d9b640 1 -- 172.21.15.105:0/2103655625 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc41c19f530 con 0x7fc41c101420 2024-09-16T02:01:24.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.611+0000 7fc419d9b640 1 --2- 172.21.15.105:0/2103655625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc41c101420 0x7fc41c19aac0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc410004770 tx=0x7fc4100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:24.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.611+0000 7fc40b7fe640 1 -- 172.21.15.105:0/2103655625 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41003f070 con 0x7fc41c101420 2024-09-16T02:01:24.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.611+0000 7fc420bbf640 1 -- 172.21.15.105:0/2103655625 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc41c19f7c0 con 0x7fc41c101420 2024-09-16T02:01:24.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.611+0000 7fc40b7fe640 1 -- 172.21.15.105:0/2103655625 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc41003b460 con 0x7fc41c101420 2024-09-16T02:01:24.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.611+0000 7fc40b7fe640 1 -- 172.21.15.105:0/2103655625 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc410035070 con 0x7fc41c101420 2024-09-16T02:01:24.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.611+0000 7fc420bbf640 1 -- 172.21.15.105:0/2103655625 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc41c1a7100 con 0x7fc41c101420 2024-09-16T02:01:24.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.612+0000 7fc420bbf640 1 -- 172.21.15.105:0/2103655625 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc41c06b1f0 con 0x7fc41c101420 2024-09-16T02:01:24.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.613+0000 7fc40b7fe640 1 -- 172.21.15.105:0/2103655625 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fc41003b610 con 0x7fc41c101420 2024-09-16T02:01:24.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.614+0000 7fc40b7fe640 1 --2- 172.21.15.105:0/2103655625 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc4040a32d0 0x7fc4040a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:24.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.614+0000 7fc41a59c640 1 --2- 172.21.15.105:0/2103655625 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc4040a32d0 0x7fc4040a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:24.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.614+0000 7fc40b7fe640 1 -- 172.21.15.105:0/2103655625 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fc4100f0400 con 0x7fc41c101420 2024-09-16T02:01:24.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.614+0000 7fc40b7fe640 1 --2- 172.21.15.105:0/2103655625 >> [v2:172.21.15.190:6808/2086773754,v1:172.21.15.190:6809/2086773754] conn(0x7fc4040acb20 0x7fc4040aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:24.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.614+0000 7fc40b7fe640 1 -- 172.21.15.105:0/2103655625 --> [v2:172.21.15.190:6808/2086773754,v1:172.21.15.190:6809/2086773754] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc410001d30 con 0x7fc4040acb20 2024-09-16T02:01:24.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.614+0000 7fc40b7fe640 1 -- 172.21.15.105:0/2103655625 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fc4100f08e0 con 0x7fc41c101420 2024-09-16T02:01:24.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.614+0000 7fc41a59c640 1 --2- 172.21.15.105:0/2103655625 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc4040a32d0 0x7fc4040a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc40c005d80 tx=0x7fc40c005c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:24.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.614+0000 7fc41ad9d640 1 --2- 172.21.15.105:0/2103655625 >> [v2:172.21.15.190:6808/2086773754,v1:172.21.15.190:6809/2086773754] conn(0x7fc4040acb20 0x7fc4040aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:24.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.615+0000 7fc41ad9d640 1 --2- 172.21.15.105:0/2103655625 >> [v2:172.21.15.190:6808/2086773754,v1:172.21.15.190:6809/2086773754] conn(0x7fc4040acb20 0x7fc4040aef40 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:24.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.616+0000 7fc40b7fe640 1 -- 172.21.15.105:0/2103655625 <== osd.5 v2:172.21.15.190:6808/2086773754 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc410001d30 con 0x7fc4040acb20 2024-09-16T02:01:24.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.630+0000 7fc420bbf640 1 -- 172.21.15.105:0/2103655625 --> [v2:172.21.15.190:6808/2086773754,v1:172.21.15.190:6809/2086773754] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc41c1021c0 con 0x7fc4040acb20 2024-09-16T02:01:24.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.630+0000 7fc40b7fe640 1 -- 172.21.15.105:0/2103655625 <== osd.5 v2:172.21.15.190:6808/2086773754 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc41c1021c0 con 0x7fc4040acb20 2024-09-16T02:01:24.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.631+0000 7fc420bbf640 1 -- 172.21.15.105:0/2103655625 >> [v2:172.21.15.190:6808/2086773754,v1:172.21.15.190:6809/2086773754] conn(0x7fc4040acb20 msgr2=0x7fc4040aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:24.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.631+0000 7fc420bbf640 1 --2- 172.21.15.105:0/2103655625 >> [v2:172.21.15.190:6808/2086773754,v1:172.21.15.190:6809/2086773754] conn(0x7fc4040acb20 0x7fc4040aef40 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:24.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.631+0000 7fc420bbf640 1 -- 172.21.15.105:0/2103655625 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc4040a32d0 msgr2=0x7fc4040a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:24.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.631+0000 7fc420bbf640 1 --2- 172.21.15.105:0/2103655625 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc4040a32d0 0x7fc4040a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc40c005d80 tx=0x7fc40c005c90 comp rx=0 tx=0).stop 2024-09-16T02:01:24.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.631+0000 7fc420bbf640 1 -- 172.21.15.105:0/2103655625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc41c101420 msgr2=0x7fc41c19aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:24.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.631+0000 7fc420bbf640 1 --2- 172.21.15.105:0/2103655625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc41c101420 0x7fc41c19aac0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc410004770 tx=0x7fc4100042f0 comp rx=0 tx=0).stop 2024-09-16T02:01:24.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.631+0000 7fc41ad9d640 1 -- 172.21.15.105:0/2103655625 reap_dead start 2024-09-16T02:01:24.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.631+0000 7fc420bbf640 1 -- 172.21.15.105:0/2103655625 shutdown_connections 2024-09-16T02:01:24.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.631+0000 7fc420bbf640 1 -- 172.21.15.105:0/2103655625 >> 172.21.15.105:0/2103655625 conn(0x7fc41c0fc7a0 msgr2=0x7fc41c06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:24.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.631+0000 7fc420bbf640 1 -- 172.21.15.105:0/2103655625 shutdown_connections 2024-09-16T02:01:24.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:24.631+0000 7fc420bbf640 1 -- 172.21.15.105:0/2103655625 wait complete. 2024-09-16T02:01:25.346 INFO:teuthology.orchestra.run.smithi105.stdout:150323855414 2024-09-16T02:01:25.346 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-16T02:01:25.842 INFO:teuthology.orchestra.run.smithi105.stdout:98784247872 2024-09-16T02:01:25.842 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-16T02:01:26.260 INFO:teuthology.orchestra.run.smithi105.stdout:34359738445 2024-09-16T02:01:26.261 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-16T02:01:26.473 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:26.483 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:26.486 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:26.488 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:26.493 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:26.494 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:26.495 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:26.498 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:26.498 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:26.498 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:26.504 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:26.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:26 smithi105 ceph-mon[31197]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-16T02:01:26.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:26 smithi190 ceph-mon[34762]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-16T02:01:26.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:26 smithi137 ceph-mon[34099]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-16T02:01:27.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.624+0000 7fc7c3206640 1 -- 172.21.15.105:0/2823279881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7bc101420 msgr2=0x7fc7bc10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.624+0000 7fc7c3206640 1 --2- 172.21.15.105:0/2823279881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7bc101420 0x7fc7bc10d950 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4009ca0 tx=0x7fc7b402a8c0 comp rx=0 tx=0).stop 2024-09-16T02:01:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.624+0000 7fc7c3206640 1 -- 172.21.15.105:0/2823279881 shutdown_connections 2024-09-16T02:01:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.624+0000 7fc7c3206640 1 --2- 172.21.15.105:0/2823279881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7bc101420 0x7fc7bc10d950 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.624+0000 7fc7c3206640 1 --2- 172.21.15.105:0/2823279881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7bc100a80 0x7fc7bc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.624+0000 7fc7c3206640 1 --2- 172.21.15.105:0/2823279881 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7bc106a80 0x7fc7bc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.624+0000 7fc7c3206640 1 -- 172.21.15.105:0/2823279881 >> 172.21.15.105:0/2823279881 conn(0x7fc7bc0fc7a0 msgr2=0x7fc7bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.624+0000 7fc7c3206640 1 -- 172.21.15.105:0/2823279881 shutdown_connections 2024-09-16T02:01:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.624+0000 7fc7c3206640 1 -- 172.21.15.105:0/2823279881 wait complete. 2024-09-16T02:01:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.624+0000 7fc7c3206640 1 Processor -- start 2024-09-16T02:01:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.624+0000 7fc7c3206640 1 -- start start 2024-09-16T02:01:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.625+0000 7fc7c3206640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7bc100a80 0x7fc7bc1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:27.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.625+0000 7fc7c3206640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7bc101420 0x7fc7bc198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:27.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.625+0000 7fc7c3206640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7bc106a80 0x7fc7bc19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:27.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.625+0000 7fc7c3206640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc7bc111e50 con 0x7fc7bc100a80 2024-09-16T02:01:27.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.625+0000 7fc7c3206640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc7bc111cd0 con 0x7fc7bc106a80 2024-09-16T02:01:27.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.625+0000 7fc7c3206640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc7bc111fd0 con 0x7fc7bc101420 2024-09-16T02:01:27.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.625+0000 7fc7c0fa2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7bc100a80 0x7fc7bc1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:27.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.625+0000 7fc7c0fa2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7bc100a80 0x7fc7bc1983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43086/0 (socket says 172.21.15.105:43086) 2024-09-16T02:01:27.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.625+0000 7fc7c0fa2640 1 -- 172.21.15.105:0/1756859648 learned_addr learned my addr 172.21.15.105:0/1756859648 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:01:27.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.625+0000 7fc7c17a3640 1 --2- 172.21.15.105:0/1756859648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7bc106a80 0x7fc7bc19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:27.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.625+0000 7fc7c0fa2640 1 -- 172.21.15.105:0/1756859648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7bc101420 msgr2=0x7fc7bc198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:27.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.625+0000 7fc7bbfff640 1 --2- 172.21.15.105:0/1756859648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7bc101420 0x7fc7bc198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:27.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.625+0000 7fc7c0fa2640 1 --2- 172.21.15.105:0/1756859648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7bc101420 0x7fc7bc198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:27.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.625+0000 7fc7c0fa2640 1 -- 172.21.15.105:0/1756859648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7bc106a80 msgr2=0x7fc7bc19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:27.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.625+0000 7fc7c0fa2640 1 --2- 172.21.15.105:0/1756859648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7bc106a80 0x7fc7bc19cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:27.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.625+0000 7fc7c0fa2640 1 -- 172.21.15.105:0/1756859648 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7bc19d380 con 0x7fc7bc100a80 2024-09-16T02:01:27.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.625+0000 7fc7c17a3640 1 --2- 172.21.15.105:0/1756859648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7bc106a80 0x7fc7bc19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:01:27.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.625+0000 7fc7c0fa2640 1 --2- 172.21.15.105:0/1756859648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7bc100a80 0x7fc7bc1983d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac00b4a0 tx=0x7fc7ac00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:27.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.625+0000 7fc7b9ffb640 1 -- 172.21.15.105:0/1756859648 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac014070 con 0x7fc7bc100a80 2024-09-16T02:01:27.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.625+0000 7fc7b9ffb640 1 -- 172.21.15.105:0/1756859648 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc7ac010ac0 con 0x7fc7bc100a80 2024-09-16T02:01:27.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.625+0000 7fc7c3206640 1 -- 172.21.15.105:0/1756859648 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7bc19d670 con 0x7fc7bc100a80 2024-09-16T02:01:27.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.626+0000 7fc7b9ffb640 1 -- 172.21.15.105:0/1756859648 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac017c30 con 0x7fc7bc100a80 2024-09-16T02:01:27.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.626+0000 7fc7c3206640 1 -- 172.21.15.105:0/1756859648 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7bc1a4f50 con 0x7fc7bc100a80 2024-09-16T02:01:27.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.627+0000 7fc7c3206640 1 -- 172.21.15.105:0/1756859648 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7bc100ee0 con 0x7fc7bc100a80 2024-09-16T02:01:27.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.628+0000 7fc7b9ffb640 1 -- 172.21.15.105:0/1756859648 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fc7ac017070 con 0x7fc7bc100a80 2024-09-16T02:01:27.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.628+0000 7fc7b9ffb640 1 --2- 172.21.15.105:0/1756859648 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc7a40a32d0 0x7fc7a40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:27.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.628+0000 7fc7bbfff640 1 --2- 172.21.15.105:0/1756859648 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc7a40a32d0 0x7fc7a40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:27.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.629+0000 7fc7b9ffb640 1 -- 172.21.15.105:0/1756859648 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fc7ac0cef60 con 0x7fc7bc100a80 2024-09-16T02:01:27.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.629+0000 7fc7bbfff640 1 --2- 172.21.15.105:0/1756859648 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc7a40a32d0 0x7fc7a40a5790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc1060c0 tx=0x7fc7b0005c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:27.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.633+0000 7fc7b9ffb640 1 -- 172.21.15.105:0/1756859648 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc7ac09bde0 con 0x7fc7bc100a80 2024-09-16T02:01:27.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.808+0000 7fc7c3206640 1 -- 172.21.15.105:0/1756859648 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fc7bc108a80 con 0x7fc7bc100a80 2024-09-16T02:01:27.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.809+0000 7fc7b9ffb640 1 -- 172.21.15.105:0/1756859648 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc7ac09b6e0 con 0x7fc7bc100a80 2024-09-16T02:01:27.811 INFO:teuthology.orchestra.run.smithi105.stdout:180388626482 2024-09-16T02:01:27.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.812+0000 7fc7c3206640 1 -- 172.21.15.105:0/1756859648 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc7a40a32d0 msgr2=0x7fc7a40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:27.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.812+0000 7fc7c3206640 1 --2- 172.21.15.105:0/1756859648 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc7a40a32d0 0x7fc7a40a5790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc1060c0 tx=0x7fc7b0005c60 comp rx=0 tx=0).stop 2024-09-16T02:01:27.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.812+0000 7fc7c3206640 1 -- 172.21.15.105:0/1756859648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7bc100a80 msgr2=0x7fc7bc1983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:27.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.812+0000 7fc7c3206640 1 --2- 172.21.15.105:0/1756859648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7bc100a80 0x7fc7bc1983d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac00b4a0 tx=0x7fc7ac00b960 comp rx=0 tx=0).stop 2024-09-16T02:01:27.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.812+0000 7fc7c3206640 1 -- 172.21.15.105:0/1756859648 shutdown_connections 2024-09-16T02:01:27.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.812+0000 7fc7c3206640 1 --2- 172.21.15.105:0/1756859648 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc7a40a32d0 0x7fc7a40a5790 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:27.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.812+0000 7fc7c3206640 1 --2- 172.21.15.105:0/1756859648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7bc101420 0x7fc7bc198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:27.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.812+0000 7fc7c3206640 1 --2- 172.21.15.105:0/1756859648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7bc106a80 0x7fc7bc19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:27.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.812+0000 7fc7c3206640 1 --2- 172.21.15.105:0/1756859648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7bc100a80 0x7fc7bc1983d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:27.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.812+0000 7fc7c3206640 1 -- 172.21.15.105:0/1756859648 >> 172.21.15.105:0/1756859648 conn(0x7fc7bc0fc7a0 msgr2=0x7fc7bc0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:27.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.812+0000 7fc7c3206640 1 -- 172.21.15.105:0/1756859648 shutdown_connections 2024-09-16T02:01:27.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.812+0000 7fc7c3206640 1 -- 172.21.15.105:0/1756859648 wait complete. 2024-09-16T02:01:27.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.840+0000 7f4bf9c28640 1 -- 172.21.15.105:0/4233879604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bf4100100 msgr2=0x7f4bf4100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:27.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.840+0000 7f4bf9c28640 1 --2- 172.21.15.105:0/4233879604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bf4100100 0x7f4bf4100560 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc009900 tx=0x7f4bdc02a410 comp rx=0 tx=0).stop 2024-09-16T02:01:27.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.840+0000 7f4bf9c28640 1 -- 172.21.15.105:0/4233879604 shutdown_connections 2024-09-16T02:01:27.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.840+0000 7f4bf9c28640 1 --2- 172.21.15.105:0/4233879604 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bf4100aa0 0x7f4bf410ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:27.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.840+0000 7f4bf9c28640 1 --2- 172.21.15.105:0/4233879604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bf4100100 0x7f4bf4100560 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:27.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.840+0000 7f4bf9c28640 1 --2- 172.21.15.105:0/4233879604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4bf4106b80 0x7f4bf4106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:27.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.840+0000 7f4bf9c28640 1 -- 172.21.15.105:0/4233879604 >> 172.21.15.105:0/4233879604 conn(0x7f4bf4077230 msgr2=0x7f4bf40fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:27.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.840+0000 7f4bf9c28640 1 -- 172.21.15.105:0/4233879604 shutdown_connections 2024-09-16T02:01:27.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.840+0000 7f4bf9c28640 1 -- 172.21.15.105:0/4233879604 wait complete. 2024-09-16T02:01:27.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.841+0000 7f4bf9c28640 1 Processor -- start 2024-09-16T02:01:27.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.841+0000 7f4bf9c28640 1 -- start start 2024-09-16T02:01:27.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.841+0000 7f4bf9c28640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bf4100100 0x7f4bf41983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:27.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.841+0000 7f4bf9c28640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bf4100aa0 0x7f4bf4198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:27.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.841+0000 7f4bf9c28640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4bf4106b80 0x7f4bf419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:27.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.841+0000 7f4bf9c28640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf4111f00 con 0x7f4bf4100aa0 2024-09-16T02:01:27.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.841+0000 7f4bf9c28640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf4111d80 con 0x7f4bf4106b80 2024-09-16T02:01:27.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.841+0000 7f4bf9c28640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf4112080 con 0x7f4bf4100100 2024-09-16T02:01:27.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.841+0000 7f4bebfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bf4100aa0 0x7f4bf4198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:27.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.841+0000 7f4bebfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bf4100aa0 0x7f4bf4198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43116/0 (socket says 172.21.15.105:43116) 2024-09-16T02:01:27.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.841+0000 7f4bebfff640 1 -- 172.21.15.105:0/2251816414 learned_addr learned my addr 172.21.15.105:0/2251816414 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:01:27.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.841+0000 7f4bf37fe640 1 --2- 172.21.15.105:0/2251816414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bf4100100 0x7f4bf41983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:27.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.841+0000 7f4bf3fff640 1 --2- 172.21.15.105:0/2251816414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4bf4106b80 0x7f4bf419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:27.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.842+0000 7f4bebfff640 1 -- 172.21.15.105:0/2251816414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bf4100100 msgr2=0x7f4bf41983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:27.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.842+0000 7f4bebfff640 1 --2- 172.21.15.105:0/2251816414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bf4100100 0x7f4bf41983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:27.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.842+0000 7f4bebfff640 1 -- 172.21.15.105:0/2251816414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4bf4106b80 msgr2=0x7f4bf419ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:27.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.842+0000 7f4bebfff640 1 --2- 172.21.15.105:0/2251816414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4bf4106b80 0x7f4bf419ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:27.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.842+0000 7f4bebfff640 1 -- 172.21.15.105:0/2251816414 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bf419d390 con 0x7f4bf4100aa0 2024-09-16T02:01:27.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.842+0000 7f4bebfff640 1 --2- 172.21.15.105:0/2251816414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bf4100aa0 0x7f4bf4198920 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc005940 tx=0x7f4bdc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:27.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.842+0000 7f4bf17fa640 1 -- 172.21.15.105:0/2251816414 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc03f030 con 0x7f4bf4100aa0 2024-09-16T02:01:27.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.842+0000 7f4bf9c28640 1 -- 172.21.15.105:0/2251816414 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bf419d620 con 0x7f4bf4100aa0 2024-09-16T02:01:27.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.842+0000 7f4bf17fa640 1 -- 172.21.15.105:0/2251816414 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4bdc003cb0 con 0x7f4bf4100aa0 2024-09-16T02:01:27.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.842+0000 7f4bf17fa640 1 -- 172.21.15.105:0/2251816414 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc035070 con 0x7f4bf4100aa0 2024-09-16T02:01:27.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.842+0000 7f4bf9c28640 1 -- 172.21.15.105:0/2251816414 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bf41a4f60 con 0x7f4bf4100aa0 2024-09-16T02:01:27.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.843+0000 7f4bf9c28640 1 -- 172.21.15.105:0/2251816414 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bf4100560 con 0x7f4bf4100aa0 2024-09-16T02:01:27.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.844+0000 7f4bf17fa640 1 -- 172.21.15.105:0/2251816414 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f4bdc042030 con 0x7f4bf4100aa0 2024-09-16T02:01:27.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.845+0000 7f4bf17fa640 1 --2- 172.21.15.105:0/2251816414 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f4be00a32d0 0x7f4be00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:27.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.845+0000 7f4bf37fe640 1 --2- 172.21.15.105:0/2251816414 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f4be00a32d0 0x7f4be00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:27.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.845+0000 7f4bf17fa640 1 -- 172.21.15.105:0/2251816414 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f4bdc0f0290 con 0x7f4bf4100aa0 2024-09-16T02:01:27.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.845+0000 7f4bf37fe640 1 --2- 172.21.15.105:0/2251816414 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f4be00a32d0 0x7f4be00a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4be4009b50 tx=0x7f4be4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:27.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:27.848+0000 7f4bf17fa640 1 -- 172.21.15.105:0/2251816414 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4bdc0bc9b0 con 0x7f4bf4100aa0 2024-09-16T02:01:28.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.016+0000 7f4bf9c28640 1 -- 172.21.15.105:0/2251816414 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f4bf410a010 con 0x7f4bf4100aa0 2024-09-16T02:01:28.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.017+0000 7f4bf17fa640 1 -- 172.21.15.105:0/2251816414 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4bdc0c0cb0 con 0x7f4bf4100aa0 2024-09-16T02:01:28.019 INFO:teuthology.orchestra.run.smithi105.stdout:261993005090 2024-09-16T02:01:28.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.019+0000 7f4bf9c28640 1 -- 172.21.15.105:0/2251816414 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f4be00a32d0 msgr2=0x7f4be00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:28.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.019+0000 7f4bf9c28640 1 --2- 172.21.15.105:0/2251816414 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f4be00a32d0 0x7f4be00a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4be4009b50 tx=0x7f4be4005f90 comp rx=0 tx=0).stop 2024-09-16T02:01:28.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.019+0000 7f4bf9c28640 1 -- 172.21.15.105:0/2251816414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bf4100aa0 msgr2=0x7f4bf4198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:28.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.019+0000 7f4bf9c28640 1 --2- 172.21.15.105:0/2251816414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bf4100aa0 0x7f4bf4198920 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc005940 tx=0x7f4bdc0042f0 comp rx=0 tx=0).stop 2024-09-16T02:01:28.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.019+0000 7f4bf9c28640 1 -- 172.21.15.105:0/2251816414 shutdown_connections 2024-09-16T02:01:28.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.019+0000 7f4bf9c28640 1 --2- 172.21.15.105:0/2251816414 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f4be00a32d0 0x7f4be00a5790 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:28.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.019+0000 7f4bf9c28640 1 --2- 172.21.15.105:0/2251816414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4bf4106b80 0x7f4bf419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:28.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.019+0000 7f4bf9c28640 1 --2- 172.21.15.105:0/2251816414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bf4100aa0 0x7f4bf4198920 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:28.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.019+0000 7f4bf9c28640 1 --2- 172.21.15.105:0/2251816414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bf4100100 0x7f4bf41983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:28.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.019+0000 7f4bf9c28640 1 -- 172.21.15.105:0/2251816414 >> 172.21.15.105:0/2251816414 conn(0x7f4bf4077230 msgr2=0x7f4bf40fc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:28.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.019+0000 7f4bf9c28640 1 -- 172.21.15.105:0/2251816414 shutdown_connections 2024-09-16T02:01:28.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.019+0000 7f4bf9c28640 1 -- 172.21.15.105:0/2251816414 wait complete. 2024-09-16T02:01:28.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.379+0000 7f0c70ed8640 1 -- 172.21.15.105:0/1350533303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c6c074e30 msgr2=0x7f0c6c10df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:28.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.379+0000 7f0c70ed8640 1 --2- 172.21.15.105:0/1350533303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c6c074e30 0x7f0c6c10df10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0c54009900 tx=0x7f0c5402a480 comp rx=0 tx=0).stop 2024-09-16T02:01:28.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.379+0000 7f0c70ed8640 1 -- 172.21.15.105:0/1350533303 shutdown_connections 2024-09-16T02:01:28.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.379+0000 7f0c70ed8640 1 --2- 172.21.15.105:0/1350533303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c6c10e450 0x7f0c6c110840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:28.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.379+0000 7f0c70ed8640 1 --2- 172.21.15.105:0/1350533303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c6c074e30 0x7f0c6c10df10 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:28.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.379+0000 7f0c70ed8640 1 --2- 172.21.15.105:0/1350533303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c6c074510 0x7f0c6c0748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:28.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.379+0000 7f0c70ed8640 1 -- 172.21.15.105:0/1350533303 >> 172.21.15.105:0/1350533303 conn(0x7f0c6c0fd8f0 msgr2=0x7f0c6c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:28.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.379+0000 7f0c70ed8640 1 -- 172.21.15.105:0/1350533303 shutdown_connections 2024-09-16T02:01:28.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.380+0000 7f0c70ed8640 1 -- 172.21.15.105:0/1350533303 wait complete. 2024-09-16T02:01:28.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.380+0000 7f0c70ed8640 1 Processor -- start 2024-09-16T02:01:28.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.380+0000 7f0c70ed8640 1 -- start start 2024-09-16T02:01:28.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.380+0000 7f0c70ed8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c6c074510 0x7f0c6c19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:28.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.380+0000 7f0c70ed8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c6c074e30 0x7f0c6c19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:28.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.380+0000 7f0c70ed8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c6c10e450 0x7f0c6c19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:28.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.380+0000 7f0c70ed8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c114020 con 0x7f0c6c074e30 2024-09-16T02:01:28.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.380+0000 7f0c70ed8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c113ea0 con 0x7f0c6c10e450 2024-09-16T02:01:28.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.380+0000 7f0c70ed8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c1141a0 con 0x7f0c6c074510 2024-09-16T02:01:28.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.381+0000 7f0c69d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c6c074e30 0x7f0c6c19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:28.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.381+0000 7f0c69d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c6c074e30 0x7f0c6c19ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36430/0 (socket says 172.21.15.105:36430) 2024-09-16T02:01:28.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.381+0000 7f0c69d9b640 1 -- 172.21.15.105:0/2131128875 learned_addr learned my addr 172.21.15.105:0/2131128875 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:01:28.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.381+0000 7f0c6a59c640 1 --2- 172.21.15.105:0/2131128875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c6c074510 0x7f0c6c19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:28.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.381+0000 7f0c69d9b640 1 -- 172.21.15.105:0/2131128875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c6c074510 msgr2=0x7f0c6c19a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:28.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.381+0000 7f0c69d9b640 1 --2- 172.21.15.105:0/2131128875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c6c074510 0x7f0c6c19a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:28.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.381+0000 7f0c69d9b640 1 -- 172.21.15.105:0/2131128875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c6c10e450 msgr2=0x7f0c6c19eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:28.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.381+0000 7f0c69d9b640 1 --2- 172.21.15.105:0/2131128875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c6c10e450 0x7f0c6c19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:28.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.381+0000 7f0c69d9b640 1 -- 172.21.15.105:0/2131128875 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c6c19f5d0 con 0x7f0c6c074e30 2024-09-16T02:01:28.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.381+0000 7f0c69d9b640 1 --2- 172.21.15.105:0/2131128875 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c6c074e30 0x7f0c6c19ab60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0c54006010 tx=0x7f0c54004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:28.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.381+0000 7f0c637fe640 1 -- 172.21.15.105:0/2131128875 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5403f030 con 0x7f0c6c074e30 2024-09-16T02:01:28.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.381+0000 7f0c70ed8640 1 -- 172.21.15.105:0/2131128875 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c6c19f860 con 0x7f0c6c074e30 2024-09-16T02:01:28.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.381+0000 7f0c637fe640 1 -- 172.21.15.105:0/2131128875 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c540035f0 con 0x7f0c6c074e30 2024-09-16T02:01:28.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.381+0000 7f0c637fe640 1 -- 172.21.15.105:0/2131128875 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c54035070 con 0x7f0c6c074e30 2024-09-16T02:01:28.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.381+0000 7f0c70ed8640 1 -- 172.21.15.105:0/2131128875 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c6c1a71a0 con 0x7f0c6c074e30 2024-09-16T02:01:28.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.382+0000 7f0c70ed8640 1 -- 172.21.15.105:0/2131128875 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c6c0748f0 con 0x7f0c6c074e30 2024-09-16T02:01:28.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.383+0000 7f0c637fe640 1 -- 172.21.15.105:0/2131128875 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f0c54003b10 con 0x7f0c6c074e30 2024-09-16T02:01:28.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.384+0000 7f0c637fe640 1 --2- 172.21.15.105:0/2131128875 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f0c580a32d0 0x7f0c580a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:28.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.384+0000 7f0c6a59c640 1 --2- 172.21.15.105:0/2131128875 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f0c580a32d0 0x7f0c580a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:28.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.384+0000 7f0c637fe640 1 -- 172.21.15.105:0/2131128875 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f0c540f07b0 con 0x7f0c6c074e30 2024-09-16T02:01:28.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.385+0000 7f0c6a59c640 1 --2- 172.21.15.105:0/2131128875 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f0c580a32d0 0x7f0c580a5790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c005c40 tx=0x7f0c5c00a390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:28.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.387+0000 7f0c637fe640 1 -- 172.21.15.105:0/2131128875 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0c540bd4a0 con 0x7f0c6c074e30 2024-09-16T02:01:28.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.561+0000 7f0c70ed8640 1 -- 172.21.15.105:0/2131128875 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f0c6c109040 con 0x7f0c6c074e30 2024-09-16T02:01:28.564 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:28 smithi105 ceph-mon[31197]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T02:01:28.564 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:28 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/1756859648' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-16T02:01:28.564 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:28 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2251816414' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-16T02:01:28.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.562+0000 7f0c637fe640 1 -- 172.21.15.105:0/2131128875 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0c540bcda0 con 0x7f0c6c074e30 2024-09-16T02:01:28.564 INFO:teuthology.orchestra.run.smithi105.stdout:128849018939 2024-09-16T02:01:28.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.564+0000 7f0c70ed8640 1 -- 172.21.15.105:0/2131128875 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f0c580a32d0 msgr2=0x7f0c580a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:28.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.564+0000 7f0c70ed8640 1 --2- 172.21.15.105:0/2131128875 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f0c580a32d0 0x7f0c580a5790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c005c40 tx=0x7f0c5c00a390 comp rx=0 tx=0).stop 2024-09-16T02:01:28.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.564+0000 7f0c70ed8640 1 -- 172.21.15.105:0/2131128875 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c6c074e30 msgr2=0x7f0c6c19ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:28.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.564+0000 7f0c70ed8640 1 --2- 172.21.15.105:0/2131128875 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c6c074e30 0x7f0c6c19ab60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0c54006010 tx=0x7f0c54004270 comp rx=0 tx=0).stop 2024-09-16T02:01:28.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.564+0000 7f0c70ed8640 1 -- 172.21.15.105:0/2131128875 shutdown_connections 2024-09-16T02:01:28.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.564+0000 7f0c70ed8640 1 --2- 172.21.15.105:0/2131128875 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f0c580a32d0 0x7f0c580a5790 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:28.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.564+0000 7f0c70ed8640 1 --2- 172.21.15.105:0/2131128875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c6c10e450 0x7f0c6c19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:28.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.564+0000 7f0c70ed8640 1 --2- 172.21.15.105:0/2131128875 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c6c074e30 0x7f0c6c19ab60 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:28.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.564+0000 7f0c70ed8640 1 --2- 172.21.15.105:0/2131128875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c6c074510 0x7f0c6c19a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:28.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.564+0000 7f0c70ed8640 1 -- 172.21.15.105:0/2131128875 >> 172.21.15.105:0/2131128875 conn(0x7f0c6c0fd8f0 msgr2=0x7f0c6c0ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:28.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.564+0000 7f0c70ed8640 1 -- 172.21.15.105:0/2131128875 shutdown_connections 2024-09-16T02:01:28.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:28.564+0000 7f0c70ed8640 1 -- 172.21.15.105:0/2131128875 wait complete. 2024-09-16T02:01:28.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:28 smithi190 ceph-mon[34762]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T02:01:28.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:28 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/1756859648' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-16T02:01:28.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:28 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/2251816414' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-16T02:01:28.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:28 smithi137 ceph-mon[34099]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T02:01:28.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:28 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/1756859648' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-16T02:01:28.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:28 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/2251816414' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-16T02:01:29.603 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:29 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2131128875' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-16T02:01:29.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.600+0000 7f35efc78640 1 -- 172.21.15.105:0/1432534209 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35e8102790 msgr2=0x7f35e81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:29.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.600+0000 7f35efc78640 1 --2- 172.21.15.105:0/1432534209 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35e8102790 0x7f35e81067e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f35e4009900 tx=0x7f35e402a410 comp rx=0 tx=0).stop 2024-09-16T02:01:29.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.601+0000 7f35efc78640 1 -- 172.21.15.105:0/1432534209 shutdown_connections 2024-09-16T02:01:29.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.601+0000 7f35efc78640 1 --2- 172.21.15.105:0/1432534209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35e8106f10 0x7f35e810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:29.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.601+0000 7f35efc78640 1 --2- 172.21.15.105:0/1432534209 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35e8102790 0x7f35e81067e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:29.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.601+0000 7f35efc78640 1 --2- 172.21.15.105:0/1432534209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35e8101de0 0x7f35e81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:29.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.601+0000 7f35efc78640 1 -- 172.21.15.105:0/1432534209 >> 172.21.15.105:0/1432534209 conn(0x7f35e80fd8c0 msgr2=0x7f35e80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:29.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.601+0000 7f35efc78640 1 -- 172.21.15.105:0/1432534209 shutdown_connections 2024-09-16T02:01:29.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.601+0000 7f35efc78640 1 -- 172.21.15.105:0/1432534209 wait complete. 2024-09-16T02:01:29.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.601+0000 7f35efc78640 1 Processor -- start 2024-09-16T02:01:29.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.601+0000 7f35efc78640 1 -- start start 2024-09-16T02:01:29.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.602+0000 7f35efc78640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35e8101de0 0x7f35e8198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:29.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.602+0000 7f35efc78640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35e8102790 0x7f35e8198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:29.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.602+0000 7f35efc78640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35e8106f10 0x7f35e819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:29.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.602+0000 7f35efc78640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f35e8111e60 con 0x7f35e8102790 2024-09-16T02:01:29.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.602+0000 7f35efc78640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f35e8111ce0 con 0x7f35e8101de0 2024-09-16T02:01:29.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.602+0000 7f35efc78640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f35e8111fe0 con 0x7f35e8106f10 2024-09-16T02:01:29.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.602+0000 7f35ed213640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35e8102790 0x7f35e8198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:29.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.602+0000 7f35ed213640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35e8102790 0x7f35e8198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36452/0 (socket says 172.21.15.105:36452) 2024-09-16T02:01:29.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.602+0000 7f35ed213640 1 -- 172.21.15.105:0/217718251 learned_addr learned my addr 172.21.15.105:0/217718251 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:01:29.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.602+0000 7f35ed213640 1 -- 172.21.15.105:0/217718251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35e8106f10 msgr2=0x7f35e819ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:29.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.602+0000 7f35ed213640 1 --2- 172.21.15.105:0/217718251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35e8106f10 0x7f35e819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:29.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.602+0000 7f35ed213640 1 -- 172.21.15.105:0/217718251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35e8101de0 msgr2=0x7f35e8198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:29.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.602+0000 7f35ed213640 1 --2- 172.21.15.105:0/217718251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35e8101de0 0x7f35e8198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:29.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.602+0000 7f35ed213640 1 -- 172.21.15.105:0/217718251 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e819d3b0 con 0x7f35e8102790 2024-09-16T02:01:29.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.602+0000 7f35ed213640 1 --2- 172.21.15.105:0/217718251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35e8102790 0x7f35e8198940 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f35e4005940 tx=0x7f35e4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:29.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.602+0000 7f35daffd640 1 -- 172.21.15.105:0/217718251 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e403f030 con 0x7f35e8102790 2024-09-16T02:01:29.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.602+0000 7f35efc78640 1 -- 172.21.15.105:0/217718251 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e819d640 con 0x7f35e8102790 2024-09-16T02:01:29.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.602+0000 7f35daffd640 1 -- 172.21.15.105:0/217718251 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35e4003aa0 con 0x7f35e8102790 2024-09-16T02:01:29.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.602+0000 7f35efc78640 1 -- 172.21.15.105:0/217718251 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e81a4f80 con 0x7f35e8102790 2024-09-16T02:01:29.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.602+0000 7f35daffd640 1 -- 172.21.15.105:0/217718251 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e4035070 con 0x7f35e8102790 2024-09-16T02:01:29.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.603+0000 7f35efc78640 1 -- 172.21.15.105:0/217718251 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35e81021c0 con 0x7f35e8102790 2024-09-16T02:01:29.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.605+0000 7f35daffd640 1 -- 172.21.15.105:0/217718251 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f35e4042030 con 0x7f35e8102790 2024-09-16T02:01:29.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.605+0000 7f35daffd640 1 --2- 172.21.15.105:0/217718251 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f35d40a32d0 0x7f35d40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:29.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.606+0000 7f35eda14640 1 --2- 172.21.15.105:0/217718251 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f35d40a32d0 0x7f35d40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:29.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.606+0000 7f35daffd640 1 -- 172.21.15.105:0/217718251 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f35e40f04b0 con 0x7f35e8102790 2024-09-16T02:01:29.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.606+0000 7f35eda14640 1 --2- 172.21.15.105:0/217718251 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f35d40a32d0 0x7f35d40a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f35dc009710 tx=0x7f35dc006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:29.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.609+0000 7f35daffd640 1 -- 172.21.15.105:0/217718251 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f35e40bd120 con 0x7f35e8102790 2024-09-16T02:01:29.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.784+0000 7f35efc78640 1 -- 172.21.15.105:0/217718251 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f35e810a150 con 0x7f35e8102790 2024-09-16T02:01:29.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.784+0000 7f35daffd640 1 -- 172.21.15.105:0/217718251 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f35e40bca20 con 0x7f35e8102790 2024-09-16T02:01:29.787 INFO:teuthology.orchestra.run.smithi105.stdout:201863462956 2024-09-16T02:01:29.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.786+0000 7f35efc78640 1 -- 172.21.15.105:0/217718251 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f35d40a32d0 msgr2=0x7f35d40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:29.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.786+0000 7f35efc78640 1 --2- 172.21.15.105:0/217718251 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f35d40a32d0 0x7f35d40a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f35dc009710 tx=0x7f35dc006c90 comp rx=0 tx=0).stop 2024-09-16T02:01:29.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.786+0000 7f35efc78640 1 -- 172.21.15.105:0/217718251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35e8102790 msgr2=0x7f35e8198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:29.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.786+0000 7f35efc78640 1 --2- 172.21.15.105:0/217718251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35e8102790 0x7f35e8198940 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f35e4005940 tx=0x7f35e4004270 comp rx=0 tx=0).stop 2024-09-16T02:01:29.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.786+0000 7f35efc78640 1 -- 172.21.15.105:0/217718251 shutdown_connections 2024-09-16T02:01:29.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.786+0000 7f35efc78640 1 --2- 172.21.15.105:0/217718251 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f35d40a32d0 0x7f35d40a5790 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:29.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.786+0000 7f35efc78640 1 --2- 172.21.15.105:0/217718251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35e8106f10 0x7f35e819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:29.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.786+0000 7f35efc78640 1 --2- 172.21.15.105:0/217718251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35e8102790 0x7f35e8198940 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:29.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.786+0000 7f35efc78640 1 --2- 172.21.15.105:0/217718251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35e8101de0 0x7f35e8198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:29.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.786+0000 7f35efc78640 1 -- 172.21.15.105:0/217718251 >> 172.21.15.105:0/217718251 conn(0x7f35e80fd8c0 msgr2=0x7f35e80fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:29.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.787+0000 7f35efc78640 1 -- 172.21.15.105:0/217718251 shutdown_connections 2024-09-16T02:01:29.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:29.787+0000 7f35efc78640 1 -- 172.21.15.105:0/217718251 wait complete. 2024-09-16T02:01:29.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:29 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/2131128875' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-16T02:01:29.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:29 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/2131128875' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-16T02:01:30.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.071+0000 7fcd32d2b640 1 -- 172.21.15.105:0/2476879836 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd2c106a80 msgr2=0x7fcd2c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:30.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.071+0000 7fcd32d2b640 1 --2- 172.21.15.105:0/2476879836 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd2c106a80 0x7fcd2c106e60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcd20009930 tx=0x7fcd2002a420 comp rx=0 tx=0).stop 2024-09-16T02:01:30.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.071+0000 7fcd32d2b640 1 -- 172.21.15.105:0/2476879836 shutdown_connections 2024-09-16T02:01:30.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.071+0000 7fcd32d2b640 1 --2- 172.21.15.105:0/2476879836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd2c100a80 0x7fcd2c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:30.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.071+0000 7fcd32d2b640 1 --2- 172.21.15.105:0/2476879836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd2c101420 0x7fcd2c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:30.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.071+0000 7fcd32d2b640 1 --2- 172.21.15.105:0/2476879836 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd2c106a80 0x7fcd2c106e60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:30.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.071+0000 7fcd32d2b640 1 -- 172.21.15.105:0/2476879836 >> 172.21.15.105:0/2476879836 conn(0x7fcd2c0fc7a0 msgr2=0x7fcd2c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:30.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.071+0000 7fcd32d2b640 1 -- 172.21.15.105:0/2476879836 shutdown_connections 2024-09-16T02:01:30.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.071+0000 7fcd32d2b640 1 -- 172.21.15.105:0/2476879836 wait complete. 2024-09-16T02:01:30.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.072+0000 7fcd32d2b640 1 Processor -- start 2024-09-16T02:01:30.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.072+0000 7fcd32d2b640 1 -- start start 2024-09-16T02:01:30.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.072+0000 7fcd32d2b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd2c100a80 0x7fcd2c199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:30.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.072+0000 7fcd32d2b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd2c101420 0x7fcd2c1998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:30.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.072+0000 7fcd32d2b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd2c106a80 0x7fcd2c1a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:30.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.072+0000 7fcd32d2b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2c110dd0 con 0x7fcd2c101420 2024-09-16T02:01:30.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.072+0000 7fcd32d2b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2c110c50 con 0x7fcd2c100a80 2024-09-16T02:01:30.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.072+0000 7fcd32d2b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2c110f50 con 0x7fcd2c106a80 2024-09-16T02:01:30.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.072+0000 7fcd2bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd2c101420 0x7fcd2c1998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:30.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.072+0000 7fcd2bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd2c101420 0x7fcd2c1998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36474/0 (socket says 172.21.15.105:36474) 2024-09-16T02:01:30.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.072+0000 7fcd2bfff640 1 -- 172.21.15.105:0/493757611 learned_addr learned my addr 172.21.15.105:0/493757611 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:01:30.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.072+0000 7fcd30ac7640 1 --2- 172.21.15.105:0/493757611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd2c100a80 0x7fcd2c199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:30.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.072+0000 7fcd312c8640 1 --2- 172.21.15.105:0/493757611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd2c106a80 0x7fcd2c1a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:30.526 INFO:teuthology.orchestra.run.smithi105.stdout:313532612633 2024-09-16T02:01:30.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.072+0000 7fcd2bfff640 1 -- 172.21.15.105:0/493757611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd2c106a80 msgr2=0x7fcd2c1a43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:30.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.072+0000 7fcd2bfff640 1 --2- 172.21.15.105:0/493757611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd2c106a80 0x7fcd2c1a43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:30.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.072+0000 7fcd2bfff640 1 -- 172.21.15.105:0/493757611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd2c100a80 msgr2=0x7fcd2c199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:30.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.072+0000 7fcd2bfff640 1 --2- 172.21.15.105:0/493757611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd2c100a80 0x7fcd2c199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:30.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.072+0000 7fcd2bfff640 1 -- 172.21.15.105:0/493757611 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd2c1a4920 con 0x7fcd2c101420 2024-09-16T02:01:30.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.072+0000 7fcd2bfff640 1 --2- 172.21.15.105:0/493757611 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd2c101420 0x7fcd2c1998a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcd1800b4a0 tx=0x7fcd1800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:30.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.073+0000 7fcd29ffb640 1 -- 172.21.15.105:0/493757611 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd18014070 con 0x7fcd2c101420 2024-09-16T02:01:30.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.073+0000 7fcd29ffb640 1 -- 172.21.15.105:0/493757611 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd18010ac0 con 0x7fcd2c101420 2024-09-16T02:01:30.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.073+0000 7fcd32d2b640 1 -- 172.21.15.105:0/493757611 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd2c1a4af0 con 0x7fcd2c101420 2024-09-16T02:01:30.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.073+0000 7fcd29ffb640 1 -- 172.21.15.105:0/493757611 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd18017c30 con 0x7fcd2c101420 2024-09-16T02:01:30.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.073+0000 7fcd32d2b640 1 -- 172.21.15.105:0/493757611 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd2c1a5000 con 0x7fcd2c101420 2024-09-16T02:01:30.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.074+0000 7fcd32d2b640 1 -- 172.21.15.105:0/493757611 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd2c100f60 con 0x7fcd2c101420 2024-09-16T02:01:30.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.075+0000 7fcd29ffb640 1 -- 172.21.15.105:0/493757611 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fcd18017070 con 0x7fcd2c101420 2024-09-16T02:01:30.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.075+0000 7fcd29ffb640 1 --2- 172.21.15.105:0/493757611 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fcd1c0a32d0 0x7fcd1c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:30.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.076+0000 7fcd30ac7640 1 --2- 172.21.15.105:0/493757611 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fcd1c0a32d0 0x7fcd1c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:30.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.076+0000 7fcd29ffb640 1 -- 172.21.15.105:0/493757611 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fcd180ceeb0 con 0x7fcd2c101420 2024-09-16T02:01:30.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.076+0000 7fcd30ac7640 1 --2- 172.21.15.105:0/493757611 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fcd1c0a32d0 0x7fcd1c0a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcd20006010 tx=0x7fcd20000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:30.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.079+0000 7fcd29ffb640 1 -- 172.21.15.105:0/493757611 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcd1809bd30 con 0x7fcd2c101420 2024-09-16T02:01:30.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.253+0000 7fcd32d2b640 1 -- 172.21.15.105:0/493757611 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fcd2c108ff0 con 0x7fcd2c101420 2024-09-16T02:01:30.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.253+0000 7fcd29ffb640 1 -- 172.21.15.105:0/493757611 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fcd1809b630 con 0x7fcd2c101420 2024-09-16T02:01:30.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.256+0000 7fcd32d2b640 1 -- 172.21.15.105:0/493757611 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fcd1c0a32d0 msgr2=0x7fcd1c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:30.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.256+0000 7fcd32d2b640 1 --2- 172.21.15.105:0/493757611 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fcd1c0a32d0 0x7fcd1c0a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcd20006010 tx=0x7fcd20000bc0 comp rx=0 tx=0).stop 2024-09-16T02:01:30.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.256+0000 7fcd32d2b640 1 -- 172.21.15.105:0/493757611 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd2c101420 msgr2=0x7fcd2c1998a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:30.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.256+0000 7fcd32d2b640 1 --2- 172.21.15.105:0/493757611 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd2c101420 0x7fcd2c1998a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcd1800b4a0 tx=0x7fcd1800b960 comp rx=0 tx=0).stop 2024-09-16T02:01:30.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.256+0000 7fcd32d2b640 1 -- 172.21.15.105:0/493757611 shutdown_connections 2024-09-16T02:01:30.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.256+0000 7fcd32d2b640 1 --2- 172.21.15.105:0/493757611 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fcd1c0a32d0 0x7fcd1c0a5790 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:30.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.256+0000 7fcd32d2b640 1 --2- 172.21.15.105:0/493757611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd2c106a80 0x7fcd2c1a43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:30.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.256+0000 7fcd32d2b640 1 --2- 172.21.15.105:0/493757611 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd2c101420 0x7fcd2c1998a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:30.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.256+0000 7fcd32d2b640 1 --2- 172.21.15.105:0/493757611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd2c100a80 0x7fcd2c199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:30.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.256+0000 7fcd32d2b640 1 -- 172.21.15.105:0/493757611 >> 172.21.15.105:0/493757611 conn(0x7fcd2c0fc7a0 msgr2=0x7fcd2c0fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:30.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.256+0000 7fcd32d2b640 1 -- 172.21.15.105:0/493757611 shutdown_connections 2024-09-16T02:01:30.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.256+0000 7fcd32d2b640 1 -- 172.21.15.105:0/493757611 wait complete. 2024-09-16T02:01:30.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:30 smithi105 ceph-mon[31197]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-16T02:01:30.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:30 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/217718251' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-16T02:01:30.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:30 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/493757611' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-16T02:01:30.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:30 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:01:30.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:30 smithi190 ceph-mon[34762]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-16T02:01:30.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:30 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/217718251' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-16T02:01:30.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:30 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/493757611' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-16T02:01:30.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:30 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:01:30.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:30 smithi137 ceph-mon[34099]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-16T02:01:30.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:30 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/217718251' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-16T02:01:30.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:30 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/493757611' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-16T02:01:30.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:30 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:01:30.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.921+0000 7fcf3b21b640 1 -- 172.21.15.105:0/1637665579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf34103ff0 msgr2=0x7fcf341043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:30.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.921+0000 7fcf3b21b640 1 --2- 172.21.15.105:0/1637665579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf34103ff0 0x7fcf341043d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcf28009900 tx=0x7fcf2802a410 comp rx=0 tx=0).stop 2024-09-16T02:01:30.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.921+0000 7fcf3b21b640 1 -- 172.21.15.105:0/1637665579 shutdown_connections 2024-09-16T02:01:30.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.921+0000 7fcf3b21b640 1 --2- 172.21.15.105:0/1637665579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf341049a0 0x7fcf341089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:30.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.921+0000 7fcf3b21b640 1 --2- 172.21.15.105:0/1637665579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf34109120 0x7fcf34110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:30.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.921+0000 7fcf3b21b640 1 --2- 172.21.15.105:0/1637665579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf34103ff0 0x7fcf341043d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:30.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.921+0000 7fcf3b21b640 1 -- 172.21.15.105:0/1637665579 >> 172.21.15.105:0/1637665579 conn(0x7fcf340ffa90 msgr2=0x7fcf34101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:30.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.921+0000 7fcf3b21b640 1 -- 172.21.15.105:0/1637665579 shutdown_connections 2024-09-16T02:01:30.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.921+0000 7fcf3b21b640 1 -- 172.21.15.105:0/1637665579 wait complete. 2024-09-16T02:01:30.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.921+0000 7fcf3b21b640 1 Processor -- start 2024-09-16T02:01:30.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.921+0000 7fcf3b21b640 1 -- start start 2024-09-16T02:01:30.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.922+0000 7fcf3b21b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf34103ff0 0x7fcf3419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:30.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.922+0000 7fcf3b21b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf341049a0 0x7fcf3419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:30.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.922+0000 7fcf3b21b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf34109120 0x7fcf3419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:30.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.922+0000 7fcf3b21b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcf34114050 con 0x7fcf341049a0 2024-09-16T02:01:30.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.922+0000 7fcf3b21b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcf34113ed0 con 0x7fcf34103ff0 2024-09-16T02:01:30.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.922+0000 7fcf3b21b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcf341141d0 con 0x7fcf34109120 2024-09-16T02:01:30.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.922+0000 7fcf33fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf341049a0 0x7fcf3419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:30.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.922+0000 7fcf33fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf341049a0 0x7fcf3419abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36488/0 (socket says 172.21.15.105:36488) 2024-09-16T02:01:30.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.922+0000 7fcf33fff640 1 -- 172.21.15.105:0/3575548209 learned_addr learned my addr 172.21.15.105:0/3575548209 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:01:30.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.922+0000 7fcf38fb7640 1 --2- 172.21.15.105:0/3575548209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf34103ff0 0x7fcf3419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:30.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.922+0000 7fcf397b8640 1 --2- 172.21.15.105:0/3575548209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf34109120 0x7fcf3419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:30.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.922+0000 7fcf33fff640 1 -- 172.21.15.105:0/3575548209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf34109120 msgr2=0x7fcf3419ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:30.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.922+0000 7fcf33fff640 1 --2- 172.21.15.105:0/3575548209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf34109120 0x7fcf3419ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:30.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.922+0000 7fcf33fff640 1 -- 172.21.15.105:0/3575548209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf34103ff0 msgr2=0x7fcf3419a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:30.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.922+0000 7fcf33fff640 1 --2- 172.21.15.105:0/3575548209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf34103ff0 0x7fcf3419a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:30.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.922+0000 7fcf33fff640 1 -- 172.21.15.105:0/3575548209 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf3419f620 con 0x7fcf341049a0 2024-09-16T02:01:30.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.922+0000 7fcf397b8640 1 --2- 172.21.15.105:0/3575548209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf34109120 0x7fcf3419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:01:30.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.922+0000 7fcf33fff640 1 --2- 172.21.15.105:0/3575548209 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf341049a0 0x7fcf3419abb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcf2000b4a0 tx=0x7fcf2000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:30.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.922+0000 7fcf31ffb640 1 -- 172.21.15.105:0/3575548209 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf20014070 con 0x7fcf341049a0 2024-09-16T02:01:30.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.922+0000 7fcf31ffb640 1 -- 172.21.15.105:0/3575548209 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf20010ac0 con 0x7fcf341049a0 2024-09-16T02:01:30.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.922+0000 7fcf3b21b640 1 -- 172.21.15.105:0/3575548209 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf3419f910 con 0x7fcf341049a0 2024-09-16T02:01:30.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.922+0000 7fcf31ffb640 1 -- 172.21.15.105:0/3575548209 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf20017c30 con 0x7fcf341049a0 2024-09-16T02:01:30.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.923+0000 7fcf3b21b640 1 -- 172.21.15.105:0/3575548209 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf341a71f0 con 0x7fcf341049a0 2024-09-16T02:01:30.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.924+0000 7fcf3b21b640 1 -- 172.21.15.105:0/3575548209 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf341043d0 con 0x7fcf341049a0 2024-09-16T02:01:30.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.924+0000 7fcf31ffb640 1 -- 172.21.15.105:0/3575548209 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fcf20017070 con 0x7fcf341049a0 2024-09-16T02:01:30.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.925+0000 7fcf31ffb640 1 --2- 172.21.15.105:0/3575548209 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fcf240a3350 0x7fcf240a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:30.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.925+0000 7fcf38fb7640 1 --2- 172.21.15.105:0/3575548209 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fcf240a3350 0x7fcf240a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:30.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.925+0000 7fcf31ffb640 1 -- 172.21.15.105:0/3575548209 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fcf200ceb30 con 0x7fcf341049a0 2024-09-16T02:01:30.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.926+0000 7fcf38fb7640 1 --2- 172.21.15.105:0/3575548209 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fcf240a3350 0x7fcf240a5810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcf28031440 tx=0x7fcf28005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:30.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:30.929+0000 7fcf31ffb640 1 -- 172.21.15.105:0/3575548209 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcf2009b9b0 con 0x7fcf341049a0 2024-09-16T02:01:31.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:31.102+0000 7fcf3b21b640 1 -- 172.21.15.105:0/3575548209 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fcf3410c300 con 0x7fcf341049a0 2024-09-16T02:01:31.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:31.103+0000 7fcf31ffb640 1 -- 172.21.15.105:0/3575548209 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcf20095e20 con 0x7fcf341049a0 2024-09-16T02:01:31.105 INFO:teuthology.orchestra.run.smithi105.stdout:73014444103 2024-09-16T02:01:31.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:31.104+0000 7fcf3b21b640 1 -- 172.21.15.105:0/3575548209 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fcf240a3350 msgr2=0x7fcf240a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:31.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:31.104+0000 7fcf3b21b640 1 --2- 172.21.15.105:0/3575548209 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fcf240a3350 0x7fcf240a5810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcf28031440 tx=0x7fcf28005b50 comp rx=0 tx=0).stop 2024-09-16T02:01:31.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:31.104+0000 7fcf3b21b640 1 -- 172.21.15.105:0/3575548209 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf341049a0 msgr2=0x7fcf3419abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:31.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:31.104+0000 7fcf3b21b640 1 --2- 172.21.15.105:0/3575548209 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf341049a0 0x7fcf3419abb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcf2000b4a0 tx=0x7fcf2000b960 comp rx=0 tx=0).stop 2024-09-16T02:01:31.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:31.105+0000 7fcf3b21b640 1 -- 172.21.15.105:0/3575548209 shutdown_connections 2024-09-16T02:01:31.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:31.105+0000 7fcf3b21b640 1 --2- 172.21.15.105:0/3575548209 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fcf240a3350 0x7fcf240a5810 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:31.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:31.105+0000 7fcf3b21b640 1 --2- 172.21.15.105:0/3575548209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf34109120 0x7fcf3419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:31.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:31.105+0000 7fcf3b21b640 1 --2- 172.21.15.105:0/3575548209 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf341049a0 0x7fcf3419abb0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:31.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:31.105+0000 7fcf3b21b640 1 --2- 172.21.15.105:0/3575548209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf34103ff0 0x7fcf3419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:31.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:31.105+0000 7fcf3b21b640 1 -- 172.21.15.105:0/3575548209 >> 172.21.15.105:0/3575548209 conn(0x7fcf340ffa90 msgr2=0x7fcf34100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:31.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:31.105+0000 7fcf3b21b640 1 -- 172.21.15.105:0/3575548209 shutdown_connections 2024-09-16T02:01:31.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:31.105+0000 7fcf3b21b640 1 -- 172.21.15.105:0/3575548209 wait complete. 2024-09-16T02:01:31.400 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018936 got 128849018939 for osd.4 2024-09-16T02:01:31.400 DEBUG:teuthology.parallel:result is None 2024-09-16T02:01:31.641 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:31 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/3575548209' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-16T02:01:31.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:31 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/3575548209' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-16T02:01:31.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:31 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/3575548209' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-16T02:01:32.612 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612631 got 313532612633 for osd.11 2024-09-16T02:01:32.612 DEBUG:teuthology.parallel:result is None 2024-09-16T02:01:32.789 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:32 smithi105 ceph-mon[31197]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-16T02:01:32.789 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:32 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:01:32.789 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:32 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:01:32.789 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:32 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:01:32.789 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:32 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:01:32.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:32 smithi190 ceph-mon[34762]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-16T02:01:32.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:32 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:01:32.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:32 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:01:32.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:32 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:01:32.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:32 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:01:32.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.867+0000 7fea0b246640 1 -- 172.21.15.105:0/778771029 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea04102d30 msgr2=0x7fea04103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:32.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.867+0000 7fea0b246640 1 --2- 172.21.15.105:0/778771029 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea04102d30 0x7fea04103110 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8009930 tx=0x7fe9f802a420 comp rx=0 tx=0).stop 2024-09-16T02:01:32.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.867+0000 7fea0b246640 1 -- 172.21.15.105:0/778771029 shutdown_connections 2024-09-16T02:01:32.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.867+0000 7fea0b246640 1 --2- 172.21.15.105:0/778771029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea041082a0 0x7fea0410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:32.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.867+0000 7fea0b246640 1 --2- 172.21.15.105:0/778771029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea041036e0 0x7fea04107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:32.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.867+0000 7fea0b246640 1 --2- 172.21.15.105:0/778771029 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea04102d30 0x7fea04103110 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:32.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.867+0000 7fea0b246640 1 -- 172.21.15.105:0/778771029 >> 172.21.15.105:0/778771029 conn(0x7fea040fc7d0 msgr2=0x7fea040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:32.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.867+0000 7fea0b246640 1 -- 172.21.15.105:0/778771029 shutdown_connections 2024-09-16T02:01:32.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.867+0000 7fea0b246640 1 -- 172.21.15.105:0/778771029 wait complete. 2024-09-16T02:01:32.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.868+0000 7fea0b246640 1 Processor -- start 2024-09-16T02:01:32.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.868+0000 7fea0b246640 1 -- start start 2024-09-16T02:01:32.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.868+0000 7fea0b246640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea04102d30 0x7fea041984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:32.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.868+0000 7fea0b246640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea041036e0 0x7fea04198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:32.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.868+0000 7fea0b246640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea041082a0 0x7fea0419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:32.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.868+0000 7fea0b246640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fea04111f50 con 0x7fea04102d30 2024-09-16T02:01:32.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.868+0000 7fea0b246640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fea04111dd0 con 0x7fea041082a0 2024-09-16T02:01:32.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.868+0000 7fea0b246640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fea041120d0 con 0x7fea041036e0 2024-09-16T02:01:32.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.868+0000 7fea08fe2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea04102d30 0x7fea041984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:32.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.868+0000 7fea08fe2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea04102d30 0x7fea041984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36524/0 (socket says 172.21.15.105:36524) 2024-09-16T02:01:32.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.868+0000 7fea08fe2640 1 -- 172.21.15.105:0/3661949377 learned_addr learned my addr 172.21.15.105:0/3661949377 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:01:32.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.868+0000 7fea03fff640 1 --2- 172.21.15.105:0/3661949377 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea041036e0 0x7fea04198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:32.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.869+0000 7fea097e3640 1 --2- 172.21.15.105:0/3661949377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea041082a0 0x7fea0419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:32.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.869+0000 7fea08fe2640 1 -- 172.21.15.105:0/3661949377 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea041036e0 msgr2=0x7fea04198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:32.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.869+0000 7fea08fe2640 1 --2- 172.21.15.105:0/3661949377 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea041036e0 0x7fea04198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:32.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.869+0000 7fea08fe2640 1 -- 172.21.15.105:0/3661949377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea041082a0 msgr2=0x7fea0419cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:32.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.869+0000 7fea08fe2640 1 --2- 172.21.15.105:0/3661949377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea041082a0 0x7fea0419cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:32.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.869+0000 7fea08fe2640 1 -- 172.21.15.105:0/3661949377 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea0419d480 con 0x7fea04102d30 2024-09-16T02:01:32.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.869+0000 7fea097e3640 1 --2- 172.21.15.105:0/3661949377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea041082a0 0x7fea0419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:01:32.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.869+0000 7fea08fe2640 1 --2- 172.21.15.105:0/3661949377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea04102d30 0x7fea041984d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8005940 tx=0x7fe9f80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:32.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.869+0000 7fea01ffb640 1 -- 172.21.15.105:0/3661949377 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f803f030 con 0x7fea04102d30 2024-09-16T02:01:32.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.869+0000 7fea0b246640 1 -- 172.21.15.105:0/3661949377 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea0419d710 con 0x7fea04102d30 2024-09-16T02:01:32.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.869+0000 7fea01ffb640 1 -- 172.21.15.105:0/3661949377 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe9f80318a0 con 0x7fea04102d30 2024-09-16T02:01:32.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.869+0000 7fea01ffb640 1 -- 172.21.15.105:0/3661949377 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8035070 con 0x7fea04102d30 2024-09-16T02:01:32.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.869+0000 7fea0b246640 1 -- 172.21.15.105:0/3661949377 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea041a5050 con 0x7fea04102d30 2024-09-16T02:01:32.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.870+0000 7fea0b246640 1 -- 172.21.15.105:0/3661949377 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea04103110 con 0x7fea04102d30 2024-09-16T02:01:32.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.872+0000 7fea01ffb640 1 -- 172.21.15.105:0/3661949377 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fe9f8042030 con 0x7fea04102d30 2024-09-16T02:01:32.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.873+0000 7fea01ffb640 1 --2- 172.21.15.105:0/3661949377 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe9f40a32d0 0x7fe9f40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:32.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.873+0000 7fea03fff640 1 --2- 172.21.15.105:0/3661949377 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe9f40a32d0 0x7fe9f40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:32.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.873+0000 7fea01ffb640 1 -- 172.21.15.105:0/3661949377 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fe9f80f03d0 con 0x7fea04102d30 2024-09-16T02:01:32.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.873+0000 7fea03fff640 1 --2- 172.21.15.105:0/3661949377 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe9f40a32d0 0x7fe9f40a5790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fea040feeb0 tx=0x7fe9f0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:32.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.875+0000 7fea01ffb640 1 -- 172.21.15.105:0/3661949377 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe9f80bd140 con 0x7fea04102d30 2024-09-16T02:01:32.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:32 smithi137 ceph-mon[34099]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-16T02:01:32.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:32 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:01:32.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:32 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:01:32.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:32 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:01:32.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:32 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:01:33.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.053+0000 7fea0b246640 1 -- 172.21.15.105:0/3661949377 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fea0410a2b0 con 0x7fea04102d30 2024-09-16T02:01:33.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.053+0000 7fea01ffb640 1 -- 172.21.15.105:0/3661949377 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe9f80bca40 con 0x7fea04102d30 2024-09-16T02:01:33.055 INFO:teuthology.orchestra.run.smithi105.stdout:231928234024 2024-09-16T02:01:33.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.055+0000 7fea0b246640 1 -- 172.21.15.105:0/3661949377 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe9f40a32d0 msgr2=0x7fe9f40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:33.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.055+0000 7fea0b246640 1 --2- 172.21.15.105:0/3661949377 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe9f40a32d0 0x7fe9f40a5790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fea040feeb0 tx=0x7fe9f0005f90 comp rx=0 tx=0).stop 2024-09-16T02:01:33.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.055+0000 7fea0b246640 1 -- 172.21.15.105:0/3661949377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea04102d30 msgr2=0x7fea041984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:33.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.055+0000 7fea0b246640 1 --2- 172.21.15.105:0/3661949377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea04102d30 0x7fea041984d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8005940 tx=0x7fe9f80042f0 comp rx=0 tx=0).stop 2024-09-16T02:01:33.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.055+0000 7fea0b246640 1 -- 172.21.15.105:0/3661949377 shutdown_connections 2024-09-16T02:01:33.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.055+0000 7fea0b246640 1 --2- 172.21.15.105:0/3661949377 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe9f40a32d0 0x7fe9f40a5790 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:33.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.055+0000 7fea0b246640 1 --2- 172.21.15.105:0/3661949377 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea041036e0 0x7fea04198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:33.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.055+0000 7fea0b246640 1 --2- 172.21.15.105:0/3661949377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea041082a0 0x7fea0419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:33.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.055+0000 7fea0b246640 1 --2- 172.21.15.105:0/3661949377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea04102d30 0x7fea041984d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:33.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.055+0000 7fea0b246640 1 -- 172.21.15.105:0/3661949377 >> 172.21.15.105:0/3661949377 conn(0x7fea040fc7d0 msgr2=0x7fea04105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:33.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.056+0000 7fea0b246640 1 -- 172.21.15.105:0/3661949377 shutdown_connections 2024-09-16T02:01:33.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.056+0000 7fea0b246640 1 -- 172.21.15.105:0/3661949377 wait complete. 2024-09-16T02:01:33.367 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626481 got 180388626482 for osd.6 2024-09-16T02:01:33.367 DEBUG:teuthology.parallel:result is None 2024-09-16T02:01:33.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.554+0000 7f156c9fa640 1 -- 172.21.15.105:0/620787281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1568102ef0 msgr2=0x7f156810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:33.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.554+0000 7f156c9fa640 1 --2- 172.21.15.105:0/620787281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1568102ef0 0x7f156810da40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1560009ca0 tx=0x7f156002a950 comp rx=0 tx=0).stop 2024-09-16T02:01:33.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.554+0000 7f156c9fa640 1 -- 172.21.15.105:0/620787281 shutdown_connections 2024-09-16T02:01:33.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.554+0000 7f156c9fa640 1 --2- 172.21.15.105:0/620787281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1568102ef0 0x7f156810da40 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:33.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.554+0000 7f156c9fa640 1 --2- 172.21.15.105:0/620787281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1568102570 0x7f15681029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:33.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.554+0000 7f156c9fa640 1 --2- 172.21.15.105:0/620787281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1568100e20 0x7f1568101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:33.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.554+0000 7f156c9fa640 1 -- 172.21.15.105:0/620787281 >> 172.21.15.105:0/620787281 conn(0x7f15680fc7d0 msgr2=0x7f15680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:33.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.554+0000 7f156c9fa640 1 -- 172.21.15.105:0/620787281 shutdown_connections 2024-09-16T02:01:33.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.554+0000 7f156c9fa640 1 -- 172.21.15.105:0/620787281 wait complete. 2024-09-16T02:01:33.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.555+0000 7f156c9fa640 1 Processor -- start 2024-09-16T02:01:33.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.555+0000 7f156c9fa640 1 -- start start 2024-09-16T02:01:33.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.555+0000 7f156c9fa640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1568100e20 0x7f15681972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:33.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.555+0000 7f156c9fa640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1568102570 0x7f1568197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.555+0000 7f156c9fa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1568102ef0 0x7f156819bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.555+0000 7f156c9fa640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1568110e50 con 0x7f1568102ef0 2024-09-16T02:01:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.555+0000 7f156c9fa640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1568110cd0 con 0x7f1568102570 2024-09-16T02:01:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.555+0000 7f156c9fa640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1568110fd0 con 0x7f1568100e20 2024-09-16T02:01:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.555+0000 7f1566d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1568102ef0 0x7f156819bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.555+0000 7f1566d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1568102ef0 0x7f156819bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36554/0 (socket says 172.21.15.105:36554) 2024-09-16T02:01:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.555+0000 7f1566d9d640 1 -- 172.21.15.105:0/446177667 learned_addr learned my addr 172.21.15.105:0/446177667 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:01:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.555+0000 7f155ffff640 1 --2- 172.21.15.105:0/446177667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1568102570 0x7f1568197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.555+0000 7f156659c640 1 --2- 172.21.15.105:0/446177667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1568100e20 0x7f15681972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.555+0000 7f1566d9d640 1 -- 172.21.15.105:0/446177667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1568100e20 msgr2=0x7f15681972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.555+0000 7f1566d9d640 1 --2- 172.21.15.105:0/446177667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1568100e20 0x7f15681972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.555+0000 7f1566d9d640 1 -- 172.21.15.105:0/446177667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1568102570 msgr2=0x7f1568197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.555+0000 7f1566d9d640 1 --2- 172.21.15.105:0/446177667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1568102570 0x7f1568197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.555+0000 7f1566d9d640 1 -- 172.21.15.105:0/446177667 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f156819c2a0 con 0x7f1568102ef0 2024-09-16T02:01:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.555+0000 7f156659c640 1 --2- 172.21.15.105:0/446177667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1568100e20 0x7f15681972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:01:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.555+0000 7f155ffff640 1 --2- 172.21.15.105:0/446177667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1568102570 0x7f1568197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:01:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.556+0000 7f1566d9d640 1 --2- 172.21.15.105:0/446177667 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1568102ef0 0x7f156819bbc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f156002ae70 tx=0x7f156002f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.556+0000 7f155f7fe640 1 -- 172.21.15.105:0/446177667 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1560034040 con 0x7f1568102ef0 2024-09-16T02:01:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.556+0000 7f155f7fe640 1 -- 172.21.15.105:0/446177667 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15600308f0 con 0x7f1568102ef0 2024-09-16T02:01:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.556+0000 7f156c9fa640 1 -- 172.21.15.105:0/446177667 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f156819c530 con 0x7f1568102ef0 2024-09-16T02:01:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.556+0000 7f155f7fe640 1 -- 172.21.15.105:0/446177667 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1560004780 con 0x7f1568102ef0 2024-09-16T02:01:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.556+0000 7f156c9fa640 1 -- 172.21.15.105:0/446177667 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15681a3e70 con 0x7f1568102ef0 2024-09-16T02:01:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.557+0000 7f156c9fa640 1 -- 172.21.15.105:0/446177667 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1568101200 con 0x7f1568102ef0 2024-09-16T02:01:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.558+0000 7f155f7fe640 1 -- 172.21.15.105:0/446177667 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f1560037020 con 0x7f1568102ef0 2024-09-16T02:01:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.559+0000 7f155f7fe640 1 --2- 172.21.15.105:0/446177667 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f15540a3350 0x7f15540a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.559+0000 7f156659c640 1 --2- 172.21.15.105:0/446177667 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f15540a3350 0x7f15540a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.559+0000 7f155f7fe640 1 -- 172.21.15.105:0/446177667 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f15600f0920 con 0x7f1568102ef0 2024-09-16T02:01:33.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.559+0000 7f156659c640 1 --2- 172.21.15.105:0/446177667 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f15540a3350 0x7f15540a5810 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1558006fd0 tx=0x7f1558008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:33.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.562+0000 7f155f7fe640 1 -- 172.21.15.105:0/446177667 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f15600bd7a0 con 0x7f1568102ef0 2024-09-16T02:01:33.596 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:33 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/3085428589' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-16T02:01:33.597 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:33 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/3661949377' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-16T02:01:33.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.732+0000 7f156c9fa640 1 -- 172.21.15.105:0/446177667 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f1568109090 con 0x7f1568102ef0 2024-09-16T02:01:33.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.732+0000 7f155f7fe640 1 -- 172.21.15.105:0/446177667 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f15600bd0a0 con 0x7f1568102ef0 2024-09-16T02:01:33.735 INFO:teuthology.orchestra.run.smithi105.stdout:98784247874 2024-09-16T02:01:33.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.734+0000 7f156c9fa640 1 -- 172.21.15.105:0/446177667 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f15540a3350 msgr2=0x7f15540a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:33.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.734+0000 7f156c9fa640 1 --2- 172.21.15.105:0/446177667 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f15540a3350 0x7f15540a5810 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1558006fd0 tx=0x7f1558008040 comp rx=0 tx=0).stop 2024-09-16T02:01:33.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.734+0000 7f156c9fa640 1 -- 172.21.15.105:0/446177667 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1568102ef0 msgr2=0x7f156819bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:33.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.734+0000 7f156c9fa640 1 --2- 172.21.15.105:0/446177667 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1568102ef0 0x7f156819bbc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f156002ae70 tx=0x7f156002f5d0 comp rx=0 tx=0).stop 2024-09-16T02:01:33.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.734+0000 7f156c9fa640 1 -- 172.21.15.105:0/446177667 shutdown_connections 2024-09-16T02:01:33.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.734+0000 7f156c9fa640 1 --2- 172.21.15.105:0/446177667 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f15540a3350 0x7f15540a5810 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:33.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.734+0000 7f156c9fa640 1 --2- 172.21.15.105:0/446177667 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1568102ef0 0x7f156819bbc0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:33.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.734+0000 7f156c9fa640 1 --2- 172.21.15.105:0/446177667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1568102570 0x7f1568197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:33.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.734+0000 7f156c9fa640 1 --2- 172.21.15.105:0/446177667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1568100e20 0x7f15681972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:33.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.734+0000 7f156c9fa640 1 -- 172.21.15.105:0/446177667 >> 172.21.15.105:0/446177667 conn(0x7f15680fc7d0 msgr2=0x7f15680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:33.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.734+0000 7f156c9fa640 1 -- 172.21.15.105:0/446177667 shutdown_connections 2024-09-16T02:01:33.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:33.734+0000 7f156c9fa640 1 -- 172.21.15.105:0/446177667 wait complete. 2024-09-16T02:01:33.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:33 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/3085428589' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-16T02:01:33.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:33 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/3661949377' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-16T02:01:33.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:33 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/3085428589' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-16T02:01:33.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:33 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/3661949377' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-16T02:01:34.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.764+0000 7fe62343f640 1 -- 172.21.15.105:0/2549377007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe61c100e20 msgr2=0x7fe61c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:34.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.764+0000 7fe62343f640 1 --2- 172.21.15.105:0/2549377007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe61c100e20 0x7fe61c101200 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe618009900 tx=0x7fe61802a480 comp rx=0 tx=0).stop 2024-09-16T02:01:34.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.765+0000 7fe62343f640 1 -- 172.21.15.105:0/2549377007 shutdown_connections 2024-09-16T02:01:34.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.765+0000 7fe62343f640 1 --2- 172.21.15.105:0/2549377007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe61c102570 0x7fe61c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:34.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.765+0000 7fe62343f640 1 --2- 172.21.15.105:0/2549377007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe61c102ef0 0x7fe61c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:34.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.765+0000 7fe62343f640 1 --2- 172.21.15.105:0/2549377007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe61c100e20 0x7fe61c101200 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:34.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.765+0000 7fe62343f640 1 -- 172.21.15.105:0/2549377007 >> 172.21.15.105:0/2549377007 conn(0x7fe61c0fc7d0 msgr2=0x7fe61c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:34.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.765+0000 7fe62343f640 1 -- 172.21.15.105:0/2549377007 shutdown_connections 2024-09-16T02:01:34.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.765+0000 7fe62343f640 1 -- 172.21.15.105:0/2549377007 wait complete. 2024-09-16T02:01:34.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.765+0000 7fe62343f640 1 Processor -- start 2024-09-16T02:01:34.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.765+0000 7fe62343f640 1 -- start start 2024-09-16T02:01:34.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.766+0000 7fe62343f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe61c100e20 0x7fe61c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:34.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.766+0000 7fe62343f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe61c102570 0x7fe61c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:34.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.766+0000 7fe62343f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe61c102ef0 0x7fe61c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:34.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.766+0000 7fe62343f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c110e50 con 0x7fe61c102570 2024-09-16T02:01:34.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.766+0000 7fe62343f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c110cd0 con 0x7fe61c100e20 2024-09-16T02:01:34.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.766+0000 7fe62343f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c110fd0 con 0x7fe61c102ef0 2024-09-16T02:01:34.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.766+0000 7fe6209da640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe61c102570 0x7fe61c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:34.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.766+0000 7fe6209da640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe61c102570 0x7fe61c0fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36512/0 (socket says 172.21.15.105:36512) 2024-09-16T02:01:34.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.766+0000 7fe6209da640 1 -- 172.21.15.105:0/3085428589 learned_addr learned my addr 172.21.15.105:0/3085428589 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:01:34.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.766+0000 7fe6211db640 1 --2- 172.21.15.105:0/3085428589 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe61c100e20 0x7fe61c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:34.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.766+0000 7fe6219dc640 1 --2- 172.21.15.105:0/3085428589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe61c102ef0 0x7fe61c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:34.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.766+0000 7fe6209da640 1 -- 172.21.15.105:0/3085428589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe61c102ef0 msgr2=0x7fe61c0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:34.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.766+0000 7fe6209da640 1 --2- 172.21.15.105:0/3085428589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe61c102ef0 0x7fe61c0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:34.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.766+0000 7fe6209da640 1 -- 172.21.15.105:0/3085428589 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe61c100e20 msgr2=0x7fe61c0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:34.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.766+0000 7fe6209da640 1 --2- 172.21.15.105:0/3085428589 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe61c100e20 0x7fe61c0fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:34.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.766+0000 7fe6209da640 1 -- 172.21.15.105:0/3085428589 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe61c1a68a0 con 0x7fe61c102570 2024-09-16T02:01:34.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.766+0000 7fe6219dc640 1 --2- 172.21.15.105:0/3085428589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe61c102ef0 0x7fe61c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:01:34.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.766+0000 7fe6209da640 1 --2- 172.21.15.105:0/3085428589 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe61c102570 0x7fe61c0fd1c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00b4a0 tx=0x7fe60c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:34.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.766+0000 7fe6127fc640 1 -- 172.21.15.105:0/3085428589 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60c014070 con 0x7fe61c102570 2024-09-16T02:01:34.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.767+0000 7fe6127fc640 1 -- 172.21.15.105:0/3085428589 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe60c010ac0 con 0x7fe61c102570 2024-09-16T02:01:34.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.767+0000 7fe62343f640 1 -- 172.21.15.105:0/3085428589 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe61c1a6b90 con 0x7fe61c102570 2024-09-16T02:01:34.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.767+0000 7fe6127fc640 1 -- 172.21.15.105:0/3085428589 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60c017c30 con 0x7fe61c102570 2024-09-16T02:01:34.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.767+0000 7fe62343f640 1 -- 172.21.15.105:0/3085428589 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe61c1a7080 con 0x7fe61c102570 2024-09-16T02:01:34.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.767+0000 7fe62343f640 1 -- 172.21.15.105:0/3085428589 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe61c101200 con 0x7fe61c102570 2024-09-16T02:01:34.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.769+0000 7fe6127fc640 1 -- 172.21.15.105:0/3085428589 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fe60c017070 con 0x7fe61c102570 2024-09-16T02:01:34.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.769+0000 7fe6127fc640 1 --2- 172.21.15.105:0/3085428589 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe6080a32d0 0x7fe6080a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:34.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.769+0000 7fe6211db640 1 --2- 172.21.15.105:0/3085428589 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe6080a32d0 0x7fe6080a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:34.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.769+0000 7fe6127fc640 1 -- 172.21.15.105:0/3085428589 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fe60c0ceb90 con 0x7fe61c102570 2024-09-16T02:01:34.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.770+0000 7fe6211db640 1 --2- 172.21.15.105:0/3085428589 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe6080a32d0 0x7fe6080a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe618009230 tx=0x7fe6180058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:34.023 INFO:teuthology.orchestra.run.smithi105.stdout:150323855415 2024-09-16T02:01:34.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.773+0000 7fe6127fc640 1 -- 172.21.15.105:0/3085428589 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe60c09ba10 con 0x7fe61c102570 2024-09-16T02:01:34.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.956+0000 7fe62343f640 1 -- 172.21.15.105:0/3085428589 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fe61c0fe1b0 con 0x7fe61c102570 2024-09-16T02:01:34.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.956+0000 7fe6127fc640 1 -- 172.21.15.105:0/3085428589 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe61c0fe1b0 con 0x7fe61c102570 2024-09-16T02:01:34.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.958+0000 7fe62343f640 1 -- 172.21.15.105:0/3085428589 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe6080a32d0 msgr2=0x7fe6080a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:34.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.959+0000 7fe62343f640 1 --2- 172.21.15.105:0/3085428589 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe6080a32d0 0x7fe6080a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe618009230 tx=0x7fe6180058d0 comp rx=0 tx=0).stop 2024-09-16T02:01:34.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.959+0000 7fe62343f640 1 -- 172.21.15.105:0/3085428589 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe61c102570 msgr2=0x7fe61c0fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:34.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.959+0000 7fe62343f640 1 --2- 172.21.15.105:0/3085428589 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe61c102570 0x7fe61c0fd1c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00b4a0 tx=0x7fe60c00b960 comp rx=0 tx=0).stop 2024-09-16T02:01:34.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.959+0000 7fe62343f640 1 -- 172.21.15.105:0/3085428589 shutdown_connections 2024-09-16T02:01:34.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.959+0000 7fe62343f640 1 --2- 172.21.15.105:0/3085428589 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe6080a32d0 0x7fe6080a5790 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:34.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.959+0000 7fe62343f640 1 --2- 172.21.15.105:0/3085428589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe61c102ef0 0x7fe61c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:34.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.959+0000 7fe62343f640 1 --2- 172.21.15.105:0/3085428589 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe61c102570 0x7fe61c0fd1c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:34.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.959+0000 7fe62343f640 1 --2- 172.21.15.105:0/3085428589 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe61c100e20 0x7fe61c0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:34.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.959+0000 7fe62343f640 1 -- 172.21.15.105:0/3085428589 >> 172.21.15.105:0/3085428589 conn(0x7fe61c0fc7d0 msgr2=0x7fe61c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:34.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.959+0000 7fe62343f640 1 -- 172.21.15.105:0/3085428589 shutdown_connections 2024-09-16T02:01:34.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:32.959+0000 7fe62343f640 1 -- 172.21.15.105:0/3085428589 wait complete. 2024-09-16T02:01:34.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:34 smithi105 ceph-mon[31197]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-09-16T02:01:34.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:34 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/446177667' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-16T02:01:34.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:34 smithi190 ceph-mon[34762]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-09-16T02:01:34.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:34 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/446177667' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-16T02:01:34.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:34 smithi137 ceph-mon[34099]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-09-16T02:01:34.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:34 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/446177667' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-16T02:01:34.960 INFO:tasks.cephadm.ceph_manager.ceph:need seq 98784247872 got 98784247874 for osd.3 2024-09-16T02:01:34.960 DEBUG:teuthology.parallel:result is None 2024-09-16T02:01:35.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.134+0000 7f5e4e594640 1 -- 172.21.15.105:0/4112087664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e48102570 msgr2=0x7f5e481029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:35.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.134+0000 7f5e4e594640 1 --2- 172.21.15.105:0/4112087664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e48102570 0x7f5e481029b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c009900 tx=0x7f5e3c02a480 comp rx=0 tx=0).stop 2024-09-16T02:01:35.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.134+0000 7f5e4e594640 1 -- 172.21.15.105:0/4112087664 shutdown_connections 2024-09-16T02:01:35.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.134+0000 7f5e4e594640 1 --2- 172.21.15.105:0/4112087664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e48102ef0 0x7f5e4810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:35.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.134+0000 7f5e4e594640 1 --2- 172.21.15.105:0/4112087664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e48102570 0x7f5e481029b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:35.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.134+0000 7f5e4e594640 1 --2- 172.21.15.105:0/4112087664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e48100e20 0x7f5e48101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:35.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.134+0000 7f5e4e594640 1 -- 172.21.15.105:0/4112087664 >> 172.21.15.105:0/4112087664 conn(0x7f5e480fc7d0 msgr2=0x7f5e480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:35.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.134+0000 7f5e4e594640 1 -- 172.21.15.105:0/4112087664 shutdown_connections 2024-09-16T02:01:35.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.134+0000 7f5e4e594640 1 -- 172.21.15.105:0/4112087664 wait complete. 2024-09-16T02:01:35.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.135+0000 7f5e4e594640 1 Processor -- start 2024-09-16T02:01:35.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.135+0000 7f5e4e594640 1 -- start start 2024-09-16T02:01:35.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.135+0000 7f5e4e594640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e48100e20 0x7f5e48198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:35.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.135+0000 7f5e4e594640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e48102570 0x7f5e48198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:35.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.135+0000 7f5e4e594640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e48102ef0 0x7f5e4819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:35.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.135+0000 7f5e4e594640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5e48111f50 con 0x7f5e48102570 2024-09-16T02:01:35.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.135+0000 7f5e4e594640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5e48111dd0 con 0x7f5e48102ef0 2024-09-16T02:01:35.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.135+0000 7f5e4e594640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5e481120d0 con 0x7f5e48100e20 2024-09-16T02:01:35.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.135+0000 7f5e477fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e48102570 0x7f5e48198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:35.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.135+0000 7f5e477fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e48102570 0x7f5e48198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36570/0 (socket says 172.21.15.105:36570) 2024-09-16T02:01:35.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.135+0000 7f5e477fe640 1 -- 172.21.15.105:0/4164578527 learned_addr learned my addr 172.21.15.105:0/4164578527 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:01:35.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.136+0000 7f5e4cb31640 1 --2- 172.21.15.105:0/4164578527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e48102ef0 0x7f5e4819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:35.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.136+0000 7f5e477fe640 1 -- 172.21.15.105:0/4164578527 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e48100e20 msgr2=0x7f5e48198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:35.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.136+0000 7f5e47fff640 1 --2- 172.21.15.105:0/4164578527 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e48100e20 0x7f5e48198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:35.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.136+0000 7f5e477fe640 1 --2- 172.21.15.105:0/4164578527 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e48100e20 0x7f5e48198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:35.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.136+0000 7f5e477fe640 1 -- 172.21.15.105:0/4164578527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e48102ef0 msgr2=0x7f5e4819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:35.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.136+0000 7f5e477fe640 1 --2- 172.21.15.105:0/4164578527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e48102ef0 0x7f5e4819cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:35.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.136+0000 7f5e477fe640 1 -- 172.21.15.105:0/4164578527 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e4819d3c0 con 0x7f5e48102570 2024-09-16T02:01:35.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.136+0000 7f5e477fe640 1 --2- 172.21.15.105:0/4164578527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e48102570 0x7f5e48198950 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c006010 tx=0x7f5e3c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:35.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.136+0000 7f5e457fa640 1 -- 172.21.15.105:0/4164578527 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c03f030 con 0x7f5e48102570 2024-09-16T02:01:35.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.136+0000 7f5e457fa640 1 -- 172.21.15.105:0/4164578527 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e3c003cb0 con 0x7f5e48102570 2024-09-16T02:01:35.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.136+0000 7f5e4e594640 1 -- 172.21.15.105:0/4164578527 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e4819d650 con 0x7f5e48102570 2024-09-16T02:01:35.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.136+0000 7f5e457fa640 1 -- 172.21.15.105:0/4164578527 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c035070 con 0x7f5e48102570 2024-09-16T02:01:35.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.136+0000 7f5e4e594640 1 -- 172.21.15.105:0/4164578527 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e481a4f90 con 0x7f5e48102570 2024-09-16T02:01:35.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.137+0000 7f5e4e594640 1 -- 172.21.15.105:0/4164578527 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e481021b0 con 0x7f5e48102570 2024-09-16T02:01:35.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.138+0000 7f5e457fa640 1 -- 172.21.15.105:0/4164578527 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f5e3c042030 con 0x7f5e48102570 2024-09-16T02:01:35.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.139+0000 7f5e457fa640 1 --2- 172.21.15.105:0/4164578527 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5e300a32d0 0x7f5e300a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:35.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.139+0000 7f5e47fff640 1 --2- 172.21.15.105:0/4164578527 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5e300a32d0 0x7f5e300a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:35.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.139+0000 7f5e457fa640 1 -- 172.21.15.105:0/4164578527 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f5e3c0efe80 con 0x7f5e48102570 2024-09-16T02:01:35.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.139+0000 7f5e47fff640 1 --2- 172.21.15.105:0/4164578527 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5e300a32d0 0x7f5e300a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5e380058c0 tx=0x7f5e38005c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:35.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.143+0000 7f5e457fa640 1 -- 172.21.15.105:0/4164578527 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5e3c0bcbf0 con 0x7f5e48102570 2024-09-16T02:01:35.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.315+0000 7f5e4e594640 1 -- 172.21.15.105:0/4164578527 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f5e48109090 con 0x7f5e48102570 2024-09-16T02:01:35.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.315+0000 7f5e457fa640 1 -- 172.21.15.105:0/4164578527 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f5e3c0bc4f0 con 0x7f5e48102570 2024-09-16T02:01:35.317 INFO:teuthology.orchestra.run.smithi105.stdout:283467841567 2024-09-16T02:01:35.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.317+0000 7f5e4e594640 1 -- 172.21.15.105:0/4164578527 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5e300a32d0 msgr2=0x7f5e300a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:35.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.317+0000 7f5e4e594640 1 --2- 172.21.15.105:0/4164578527 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5e300a32d0 0x7f5e300a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5e380058c0 tx=0x7f5e38005c60 comp rx=0 tx=0).stop 2024-09-16T02:01:35.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.317+0000 7f5e4e594640 1 -- 172.21.15.105:0/4164578527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e48102570 msgr2=0x7f5e48198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:35.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.317+0000 7f5e4e594640 1 --2- 172.21.15.105:0/4164578527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e48102570 0x7f5e48198950 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c006010 tx=0x7f5e3c0042f0 comp rx=0 tx=0).stop 2024-09-16T02:01:35.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.317+0000 7f5e4e594640 1 -- 172.21.15.105:0/4164578527 shutdown_connections 2024-09-16T02:01:35.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.317+0000 7f5e4e594640 1 --2- 172.21.15.105:0/4164578527 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5e300a32d0 0x7f5e300a5790 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:35.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.318+0000 7f5e4e594640 1 --2- 172.21.15.105:0/4164578527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e48102ef0 0x7f5e4819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:35.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.318+0000 7f5e4e594640 1 --2- 172.21.15.105:0/4164578527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e48102570 0x7f5e48198950 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:35.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.318+0000 7f5e4e594640 1 --2- 172.21.15.105:0/4164578527 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e48100e20 0x7f5e48198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:35.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.318+0000 7f5e4e594640 1 -- 172.21.15.105:0/4164578527 >> 172.21.15.105:0/4164578527 conn(0x7f5e480fc7d0 msgr2=0x7f5e480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:35.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.318+0000 7f5e4e594640 1 -- 172.21.15.105:0/4164578527 shutdown_connections 2024-09-16T02:01:35.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.318+0000 7f5e4e594640 1 -- 172.21.15.105:0/4164578527 wait complete. 2024-09-16T02:01:35.614 INFO:tasks.cephadm.ceph_manager.ceph:need seq 73014444099 got 73014444103 for osd.2 2024-09-16T02:01:35.614 DEBUG:teuthology.parallel:result is None 2024-09-16T02:01:35.808 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:35 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/4164578527' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-16T02:01:35.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.805+0000 7fb04aab2640 1 -- 172.21.15.105:0/2826709555 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb044102ef0 msgr2=0x7fb04410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:35.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.805+0000 7fb04aab2640 1 --2- 172.21.15.105:0/2826709555 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb044102ef0 0x7fb04410da40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb02c009900 tx=0x7fb02c02a480 comp rx=0 tx=0).stop 2024-09-16T02:01:35.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.805+0000 7fb04aab2640 1 -- 172.21.15.105:0/2826709555 shutdown_connections 2024-09-16T02:01:35.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.805+0000 7fb04aab2640 1 --2- 172.21.15.105:0/2826709555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb044102570 0x7fb0441029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:35.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.805+0000 7fb04aab2640 1 --2- 172.21.15.105:0/2826709555 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb044102ef0 0x7fb04410da40 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:35.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.805+0000 7fb04aab2640 1 --2- 172.21.15.105:0/2826709555 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb044100e20 0x7fb044101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:35.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.805+0000 7fb04aab2640 1 -- 172.21.15.105:0/2826709555 >> 172.21.15.105:0/2826709555 conn(0x7fb0440fc7d0 msgr2=0x7fb0440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:35.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.805+0000 7fb04aab2640 1 -- 172.21.15.105:0/2826709555 shutdown_connections 2024-09-16T02:01:35.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.805+0000 7fb04aab2640 1 -- 172.21.15.105:0/2826709555 wait complete. 2024-09-16T02:01:35.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.806+0000 7fb04aab2640 1 Processor -- start 2024-09-16T02:01:35.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.806+0000 7fb04aab2640 1 -- start start 2024-09-16T02:01:35.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.806+0000 7fb04aab2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb044100e20 0x7fb044198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:35.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.806+0000 7fb04aab2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb044102570 0x7fb044198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:35.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.806+0000 7fb04aab2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb044102ef0 0x7fb04419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:35.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.806+0000 7fb04aab2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb044110e30 con 0x7fb044102570 2024-09-16T02:01:35.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.806+0000 7fb04aab2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb044110cb0 con 0x7fb044100e20 2024-09-16T02:01:35.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.806+0000 7fb04aab2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb044110fb0 con 0x7fb044102ef0 2024-09-16T02:01:35.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.807+0000 7fb043fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb044102570 0x7fb044198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:35.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.807+0000 7fb043fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb044102570 0x7fb044198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36580/0 (socket says 172.21.15.105:36580) 2024-09-16T02:01:35.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.807+0000 7fb043fff640 1 -- 172.21.15.105:0/831782241 learned_addr learned my addr 172.21.15.105:0/831782241 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:01:35.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.807+0000 7fb04884e640 1 --2- 172.21.15.105:0/831782241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb044100e20 0x7fb044198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:35.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.807+0000 7fb04904f640 1 --2- 172.21.15.105:0/831782241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb044102ef0 0x7fb04419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:35.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.807+0000 7fb043fff640 1 -- 172.21.15.105:0/831782241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb044102ef0 msgr2=0x7fb04419cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:35.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.807+0000 7fb043fff640 1 --2- 172.21.15.105:0/831782241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb044102ef0 0x7fb04419cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:35.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.807+0000 7fb043fff640 1 -- 172.21.15.105:0/831782241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb044100e20 msgr2=0x7fb044198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:35.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.807+0000 7fb043fff640 1 --2- 172.21.15.105:0/831782241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb044100e20 0x7fb044198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:35.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.807+0000 7fb043fff640 1 -- 172.21.15.105:0/831782241 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb04419d2e0 con 0x7fb044102570 2024-09-16T02:01:35.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.807+0000 7fb043fff640 1 --2- 172.21.15.105:0/831782241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb044102570 0x7fb044198870 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb03400ea80 tx=0x7fb03400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:35.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.807+0000 7fb041ffb640 1 -- 172.21.15.105:0/831782241 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03400cd80 con 0x7fb044102570 2024-09-16T02:01:35.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.807+0000 7fb041ffb640 1 -- 172.21.15.105:0/831782241 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb034005020 con 0x7fb044102570 2024-09-16T02:01:35.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.807+0000 7fb04aab2640 1 -- 172.21.15.105:0/831782241 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb04419d5d0 con 0x7fb044102570 2024-09-16T02:01:35.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.808+0000 7fb041ffb640 1 -- 172.21.15.105:0/831782241 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0340116d0 con 0x7fb044102570 2024-09-16T02:01:35.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.808+0000 7fb04aab2640 1 -- 172.21.15.105:0/831782241 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0441a4eb0 con 0x7fb044102570 2024-09-16T02:01:35.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.809+0000 7fb04aab2640 1 -- 172.21.15.105:0/831782241 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb044101200 con 0x7fb044102570 2024-09-16T02:01:35.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.810+0000 7fb041ffb640 1 -- 172.21.15.105:0/831782241 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fb0340051d0 con 0x7fb044102570 2024-09-16T02:01:35.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.810+0000 7fb041ffb640 1 --2- 172.21.15.105:0/831782241 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb0300a32d0 0x7fb0300a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:35.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.811+0000 7fb04884e640 1 --2- 172.21.15.105:0/831782241 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb0300a32d0 0x7fb0300a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:35.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.811+0000 7fb041ffb640 1 -- 172.21.15.105:0/831782241 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fb0340cfb80 con 0x7fb044102570 2024-09-16T02:01:35.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.811+0000 7fb04884e640 1 --2- 172.21.15.105:0/831782241 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb0300a32d0 0x7fb0300a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00abe0 tx=0x7fb03c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:35.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.816+0000 7fb041ffb640 1 -- 172.21.15.105:0/831782241 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb034016030 con 0x7fb044102570 2024-09-16T02:01:35.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:35 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/4164578527' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-16T02:01:35.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:35 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/4164578527' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-16T02:01:35.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.991+0000 7fb04aab2640 1 -- 172.21.15.105:0/831782241 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fb04410a0a0 con 0x7fb044102570 2024-09-16T02:01:35.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.991+0000 7fb041ffb640 1 -- 172.21.15.105:0/831782241 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb03409c8f0 con 0x7fb044102570 2024-09-16T02:01:35.994 INFO:teuthology.orchestra.run.smithi105.stdout:55834574925 2024-09-16T02:01:35.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.995+0000 7fb04aab2640 1 -- 172.21.15.105:0/831782241 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb0300a32d0 msgr2=0x7fb0300a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:35.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.995+0000 7fb04aab2640 1 --2- 172.21.15.105:0/831782241 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb0300a32d0 0x7fb0300a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00abe0 tx=0x7fb03c00a2d0 comp rx=0 tx=0).stop 2024-09-16T02:01:35.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.995+0000 7fb04aab2640 1 -- 172.21.15.105:0/831782241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb044102570 msgr2=0x7fb044198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:35.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.995+0000 7fb04aab2640 1 --2- 172.21.15.105:0/831782241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb044102570 0x7fb044198870 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb03400ea80 tx=0x7fb03400ee80 comp rx=0 tx=0).stop 2024-09-16T02:01:35.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.995+0000 7fb04aab2640 1 -- 172.21.15.105:0/831782241 shutdown_connections 2024-09-16T02:01:35.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.995+0000 7fb04aab2640 1 --2- 172.21.15.105:0/831782241 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb0300a32d0 0x7fb0300a5790 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:35.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.995+0000 7fb04aab2640 1 --2- 172.21.15.105:0/831782241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb044102ef0 0x7fb04419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:35.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.995+0000 7fb04aab2640 1 --2- 172.21.15.105:0/831782241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb044102570 0x7fb044198870 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:35.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.995+0000 7fb04aab2640 1 --2- 172.21.15.105:0/831782241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb044100e20 0x7fb044198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:35.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.995+0000 7fb04aab2640 1 -- 172.21.15.105:0/831782241 >> 172.21.15.105:0/831782241 conn(0x7fb0440fc7d0 msgr2=0x7fb04406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:35.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.995+0000 7fb04aab2640 1 -- 172.21.15.105:0/831782241 shutdown_connections 2024-09-16T02:01:35.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:35.995+0000 7fb04aab2640 1 -- 172.21.15.105:0/831782241 wait complete. 2024-09-16T02:01:36.508 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234021 got 231928234024 for osd.8 2024-09-16T02:01:36.509 DEBUG:teuthology.parallel:result is None 2024-09-16T02:01:36.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:36 smithi105 ceph-mon[31197]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-16T02:01:36.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:36 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/831782241' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-16T02:01:36.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:36 smithi190 ceph-mon[34762]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-16T02:01:36.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:36 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/831782241' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-16T02:01:36.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:36 smithi137 ceph-mon[34099]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-16T02:01:36.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:36 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/831782241' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-16T02:01:37.215 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462954 got 201863462956 for osd.7 2024-09-16T02:01:37.216 DEBUG:teuthology.parallel:result is None 2024-09-16T02:01:37.901 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574921 got 55834574925 for osd.1 2024-09-16T02:01:37.901 DEBUG:teuthology.parallel:result is None 2024-09-16T02:01:38.342 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855414 got 150323855415 for osd.5 2024-09-16T02:01:38.342 DEBUG:teuthology.parallel:result is None 2024-09-16T02:01:38.790 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:38 smithi105 ceph-mon[31197]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T02:01:38.829 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005089 got 261993005090 for osd.9 2024-09-16T02:01:38.829 DEBUG:teuthology.parallel:result is None 2024-09-16T02:01:38.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:38 smithi190 ceph-mon[34762]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T02:01:38.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:38 smithi137 ceph-mon[34099]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T02:01:39.253 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:39.289 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841563 got 283467841567 for osd.10 2024-09-16T02:01:39.289 DEBUG:teuthology.parallel:result is None 2024-09-16T02:01:39.826 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:39 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-09-16T02:01:39.826 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:39 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-09-16T02:01:39.826 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:39 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-09-16T02:01:39.826 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:39 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-09-16T02:01:39.826 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:39 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 7]}]: dispatch 2024-09-16T02:01:39.826 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:39 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-09-16T02:01:39.826 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:39 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-09-16T02:01:39.826 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:39 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-09-16T02:01:39.826 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:39 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-09-16T02:01:39.826 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:39 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-09-16T02:01:39.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:39 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-09-16T02:01:39.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:39 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-09-16T02:01:39.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:39 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-09-16T02:01:39.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:39 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-09-16T02:01:39.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:39 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 7]}]: dispatch 2024-09-16T02:01:39.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:39 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-09-16T02:01:39.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:39 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-09-16T02:01:39.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:39 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-09-16T02:01:39.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:39 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-09-16T02:01:39.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:39 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-09-16T02:01:39.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:39 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-09-16T02:01:39.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:39 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-09-16T02:01:39.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:39 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-09-16T02:01:39.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:39 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-09-16T02:01:39.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:39 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 7]}]: dispatch 2024-09-16T02:01:39.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:39 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-09-16T02:01:39.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:39 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-09-16T02:01:39.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:39 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-09-16T02:01:39.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:39 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-09-16T02:01:39.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:39 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-09-16T02:01:40.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.172+0000 7f1d9dd06640 1 -- 172.21.15.105:0/77168200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d98100a80 msgr2=0x7f1d98100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:40.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.172+0000 7f1d9dd06640 1 --2- 172.21.15.105:0/77168200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d98100a80 0x7f1d98100ee0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c009930 tx=0x7f1d8c02a420 comp rx=0 tx=0).stop 2024-09-16T02:01:40.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.172+0000 7f1d9dd06640 1 -- 172.21.15.105:0/77168200 shutdown_connections 2024-09-16T02:01:40.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.172+0000 7f1d9dd06640 1 --2- 172.21.15.105:0/77168200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d98101420 0x7f1d9810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:40.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.172+0000 7f1d9dd06640 1 --2- 172.21.15.105:0/77168200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d98100a80 0x7f1d98100ee0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:40.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.172+0000 7f1d9dd06640 1 --2- 172.21.15.105:0/77168200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d98106a80 0x7f1d98106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:40.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.172+0000 7f1d9dd06640 1 -- 172.21.15.105:0/77168200 >> 172.21.15.105:0/77168200 conn(0x7f1d980fc7a0 msgr2=0x7f1d980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:40.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.172+0000 7f1d9dd06640 1 -- 172.21.15.105:0/77168200 shutdown_connections 2024-09-16T02:01:40.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.172+0000 7f1d9dd06640 1 -- 172.21.15.105:0/77168200 wait complete. 2024-09-16T02:01:40.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.173+0000 7f1d9dd06640 1 Processor -- start 2024-09-16T02:01:40.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.173+0000 7f1d9dd06640 1 -- start start 2024-09-16T02:01:40.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.173+0000 7f1d9dd06640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d98100a80 0x7f1d9819a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:40.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.173+0000 7f1d9dd06640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d98101420 0x7f1d9819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:40.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.173+0000 7f1d9dd06640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d98106a80 0x7f1d9819ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:40.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.173+0000 7f1d9dd06640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1d98110d30 con 0x7f1d98101420 2024-09-16T02:01:40.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.173+0000 7f1d9dd06640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1d98110bb0 con 0x7f1d98106a80 2024-09-16T02:01:40.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.173+0000 7f1d9dd06640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1d98110eb0 con 0x7f1d98100a80 2024-09-16T02:01:40.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.173+0000 7f1d96ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d98101420 0x7f1d9819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:40.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.173+0000 7f1d96ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d98101420 0x7f1d9819ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33038/0 (socket says 172.21.15.105:33038) 2024-09-16T02:01:40.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.173+0000 7f1d96ffd640 1 -- 172.21.15.105:0/2390149569 learned_addr learned my addr 172.21.15.105:0/2390149569 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:01:40.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.174+0000 7f1d977fe640 1 --2- 172.21.15.105:0/2390149569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d98100a80 0x7f1d9819a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:40.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.174+0000 7f1d96ffd640 1 -- 172.21.15.105:0/2390149569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d98100a80 msgr2=0x7f1d9819a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:40.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.174+0000 7f1d96ffd640 1 --2- 172.21.15.105:0/2390149569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d98100a80 0x7f1d9819a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:40.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.174+0000 7f1d96ffd640 1 -- 172.21.15.105:0/2390149569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d98106a80 msgr2=0x7f1d9819ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:40.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.174+0000 7f1d96ffd640 1 --2- 172.21.15.105:0/2390149569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d98106a80 0x7f1d9819ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:40.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.174+0000 7f1d96ffd640 1 -- 172.21.15.105:0/2390149569 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d9819f570 con 0x7f1d98101420 2024-09-16T02:01:40.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.174+0000 7f1d96ffd640 1 --2- 172.21.15.105:0/2390149569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d98101420 0x7f1d9819ab00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c004770 tx=0x7f1d8c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:40.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.174+0000 7f1d94ff9640 1 -- 172.21.15.105:0/2390149569 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c03f070 con 0x7f1d98101420 2024-09-16T02:01:40.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.174+0000 7f1d94ff9640 1 -- 172.21.15.105:0/2390149569 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1d8c03b460 con 0x7f1d98101420 2024-09-16T02:01:40.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.174+0000 7f1d9dd06640 1 -- 172.21.15.105:0/2390149569 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d9819f800 con 0x7f1d98101420 2024-09-16T02:01:40.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.174+0000 7f1d94ff9640 1 -- 172.21.15.105:0/2390149569 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c035070 con 0x7f1d98101420 2024-09-16T02:01:40.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.174+0000 7f1d9dd06640 1 -- 172.21.15.105:0/2390149569 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d981a7140 con 0x7f1d98101420 2024-09-16T02:01:40.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.175+0000 7f1d9dd06640 1 -- 172.21.15.105:0/2390149569 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d981021c0 con 0x7f1d98101420 2024-09-16T02:01:40.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.176+0000 7f1d94ff9640 1 -- 172.21.15.105:0/2390149569 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f1d8c03b610 con 0x7f1d98101420 2024-09-16T02:01:40.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.177+0000 7f1d94ff9640 1 --2- 172.21.15.105:0/2390149569 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f1d800a32d0 0x7f1d800a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:40.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.177+0000 7f1d977fe640 1 --2- 172.21.15.105:0/2390149569 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f1d800a32d0 0x7f1d800a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:40.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.177+0000 7f1d94ff9640 1 -- 172.21.15.105:0/2390149569 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f1d8c0f01c0 con 0x7f1d98101420 2024-09-16T02:01:40.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.177+0000 7f1d977fe640 1 --2- 172.21.15.105:0/2390149569 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f1d800a32d0 0x7f1d800a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1d88005fd0 tx=0x7f1d8800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:40.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.180+0000 7f1d94ff9640 1 -- 172.21.15.105:0/2390149569 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1d8c0bcd00 con 0x7f1d98101420 2024-09-16T02:01:40.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.352+0000 7f1d9dd06640 1 -- 172.21.15.105:0/2390149569 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f1d981a73e0 con 0x7f1d98101420 2024-09-16T02:01:40.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.352+0000 7f1d94ff9640 1 -- 172.21.15.105:0/2390149569 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1d8c0bc600 con 0x7f1d98101420 2024-09-16T02:01:40.357 INFO:teuthology.orchestra.run.smithi105.stdout:34359738449 2024-09-16T02:01:40.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.354+0000 7f1d9dd06640 1 -- 172.21.15.105:0/2390149569 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f1d800a32d0 msgr2=0x7f1d800a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:40.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.354+0000 7f1d9dd06640 1 --2- 172.21.15.105:0/2390149569 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f1d800a32d0 0x7f1d800a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1d88005fd0 tx=0x7f1d8800b390 comp rx=0 tx=0).stop 2024-09-16T02:01:40.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.354+0000 7f1d9dd06640 1 -- 172.21.15.105:0/2390149569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d98101420 msgr2=0x7f1d9819ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:40.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.354+0000 7f1d9dd06640 1 --2- 172.21.15.105:0/2390149569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d98101420 0x7f1d9819ab00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c004770 tx=0x7f1d8c0042f0 comp rx=0 tx=0).stop 2024-09-16T02:01:40.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.355+0000 7f1d9dd06640 1 -- 172.21.15.105:0/2390149569 shutdown_connections 2024-09-16T02:01:40.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.355+0000 7f1d9dd06640 1 --2- 172.21.15.105:0/2390149569 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f1d800a32d0 0x7f1d800a5790 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:40.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.355+0000 7f1d9dd06640 1 --2- 172.21.15.105:0/2390149569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d98106a80 0x7f1d9819ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:40.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.355+0000 7f1d9dd06640 1 --2- 172.21.15.105:0/2390149569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d98101420 0x7f1d9819ab00 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:40.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.355+0000 7f1d9dd06640 1 --2- 172.21.15.105:0/2390149569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d98100a80 0x7f1d9819a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:40.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.355+0000 7f1d9dd06640 1 -- 172.21.15.105:0/2390149569 >> 172.21.15.105:0/2390149569 conn(0x7f1d980fc7a0 msgr2=0x7f1d9806a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:40.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.355+0000 7f1d9dd06640 1 -- 172.21.15.105:0/2390149569 shutdown_connections 2024-09-16T02:01:40.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:40.355+0000 7f1d9dd06640 1 -- 172.21.15.105:0/2390149569 wait complete. 2024-09-16T02:01:40.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:40 smithi105 ceph-mon[31197]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-09-16T02:01:40.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:40 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-09-16T02:01:40.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:40 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-09-16T02:01:40.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:40 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-09-16T02:01:40.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:40 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-09-16T02:01:40.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:40 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 7]}]': finished 2024-09-16T02:01:40.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:40 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-09-16T02:01:40.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:40 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-09-16T02:01:40.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:40 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-09-16T02:01:40.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:40 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-09-16T02:01:40.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:40 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-09-16T02:01:40.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:40 smithi105 ceph-mon[31197]: osdmap e80: 12 total, 12 up, 12 in 2024-09-16T02:01:40.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:40 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2390149569' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-16T02:01:40.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:40 smithi190 ceph-mon[34762]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-09-16T02:01:40.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:40 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-09-16T02:01:40.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:40 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-09-16T02:01:40.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:40 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-09-16T02:01:40.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:40 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-09-16T02:01:40.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:40 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 7]}]': finished 2024-09-16T02:01:40.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:40 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-09-16T02:01:40.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:40 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-09-16T02:01:40.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:40 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-09-16T02:01:40.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:40 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-09-16T02:01:40.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:40 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-09-16T02:01:40.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:40 smithi190 ceph-mon[34762]: osdmap e80: 12 total, 12 up, 12 in 2024-09-16T02:01:40.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:40 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/2390149569' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-16T02:01:40.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:40 smithi137 ceph-mon[34099]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-09-16T02:01:40.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:40 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-09-16T02:01:40.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:40 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-09-16T02:01:40.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:40 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-09-16T02:01:40.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:40 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-09-16T02:01:40.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:40 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 7]}]': finished 2024-09-16T02:01:40.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:40 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-09-16T02:01:40.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:40 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-09-16T02:01:40.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:40 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-09-16T02:01:40.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:40 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-09-16T02:01:40.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:40 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-09-16T02:01:40.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:40 smithi137 ceph-mon[34099]: osdmap e80: 12 total, 12 up, 12 in 2024-09-16T02:01:40.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:40 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/2390149569' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-16T02:01:41.222 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738445 got 34359738449 for osd.0 2024-09-16T02:01:41.223 DEBUG:teuthology.parallel:result is None 2024-09-16T02:01:41.223 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-16T02:01:41.223 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T02:01:41.494 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:41.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:41 smithi190 ceph-mon[34762]: osdmap e81: 12 total, 12 up, 12 in 2024-09-16T02:01:41.890 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:41 smithi105 ceph-mon[31197]: osdmap e81: 12 total, 12 up, 12 in 2024-09-16T02:01:41.920 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:41 smithi137 ceph-mon[34099]: osdmap e81: 12 total, 12 up, 12 in 2024-09-16T02:01:42.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.971+0000 7f7b9c2f1640 1 -- 172.21.15.105:0/2502810806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b941027a0 msgr2=0x7f7b941067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:42.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.971+0000 7f7b9c2f1640 1 --2- 172.21.15.105:0/2502810806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b941027a0 0x7f7b941067f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7b84009900 tx=0x7f7b8402a410 comp rx=0 tx=0).stop 2024-09-16T02:01:42.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.971+0000 7f7b9c2f1640 1 -- 172.21.15.105:0/2502810806 shutdown_connections 2024-09-16T02:01:42.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.971+0000 7f7b9c2f1640 1 --2- 172.21.15.105:0/2502810806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b94106f20 0x7f7b9410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:42.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.971+0000 7f7b9c2f1640 1 --2- 172.21.15.105:0/2502810806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b941027a0 0x7f7b941067f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:42.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.971+0000 7f7b9c2f1640 1 --2- 172.21.15.105:0/2502810806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b94101df0 0x7f7b941021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:42.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.971+0000 7f7b9c2f1640 1 -- 172.21.15.105:0/2502810806 >> 172.21.15.105:0/2502810806 conn(0x7f7b940fd8f0 msgr2=0x7f7b940ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:42.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.971+0000 7f7b9c2f1640 1 -- 172.21.15.105:0/2502810806 shutdown_connections 2024-09-16T02:01:42.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.971+0000 7f7b9c2f1640 1 -- 172.21.15.105:0/2502810806 wait complete. 2024-09-16T02:01:42.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.972+0000 7f7b9c2f1640 1 Processor -- start 2024-09-16T02:01:42.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.972+0000 7f7b9c2f1640 1 -- start start 2024-09-16T02:01:42.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.972+0000 7f7b9c2f1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b94101df0 0x7f7b941984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:42.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.972+0000 7f7b9c2f1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b941027a0 0x7f7b94198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:42.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.972+0000 7f7b9c2f1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b94106f20 0x7f7b9419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:42.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.972+0000 7f7b9c2f1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7b94111e50 con 0x7f7b941027a0 2024-09-16T02:01:42.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.972+0000 7f7b9c2f1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7b94111cd0 con 0x7f7b94101df0 2024-09-16T02:01:42.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.972+0000 7f7b9c2f1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7b94111fd0 con 0x7f7b94106f20 2024-09-16T02:01:42.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.972+0000 7f7b9988c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b941027a0 0x7f7b94198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:42.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.972+0000 7f7b9988c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b941027a0 0x7f7b94198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33056/0 (socket says 172.21.15.105:33056) 2024-09-16T02:01:42.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.972+0000 7f7b9988c640 1 -- 172.21.15.105:0/2467850113 learned_addr learned my addr 172.21.15.105:0/2467850113 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:01:42.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.972+0000 7f7b9a08d640 1 --2- 172.21.15.105:0/2467850113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b94101df0 0x7f7b941984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:42.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.972+0000 7f7b9a88e640 1 --2- 172.21.15.105:0/2467850113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b94106f20 0x7f7b9419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:42.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.972+0000 7f7b9988c640 1 -- 172.21.15.105:0/2467850113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b94106f20 msgr2=0x7f7b9419cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:42.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.972+0000 7f7b9988c640 1 --2- 172.21.15.105:0/2467850113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b94106f20 0x7f7b9419cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:42.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.972+0000 7f7b9988c640 1 -- 172.21.15.105:0/2467850113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b94101df0 msgr2=0x7f7b941984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:42.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.972+0000 7f7b9988c640 1 --2- 172.21.15.105:0/2467850113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b94101df0 0x7f7b941984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:42.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.972+0000 7f7b9988c640 1 -- 172.21.15.105:0/2467850113 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b9419d470 con 0x7f7b941027a0 2024-09-16T02:01:42.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.972+0000 7f7b9a88e640 1 --2- 172.21.15.105:0/2467850113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b94106f20 0x7f7b9419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:01:42.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.973+0000 7f7b9988c640 1 --2- 172.21.15.105:0/2467850113 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b941027a0 0x7f7b94198a00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7b84006010 tx=0x7f7b84004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:42.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.973+0000 7f7b8b7fe640 1 -- 172.21.15.105:0/2467850113 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8403f030 con 0x7f7b941027a0 2024-09-16T02:01:42.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.973+0000 7f7b9c2f1640 1 -- 172.21.15.105:0/2467850113 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b9419d700 con 0x7f7b941027a0 2024-09-16T02:01:42.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.973+0000 7f7b8b7fe640 1 -- 172.21.15.105:0/2467850113 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b840035f0 con 0x7f7b941027a0 2024-09-16T02:01:42.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.973+0000 7f7b8b7fe640 1 -- 172.21.15.105:0/2467850113 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b84035070 con 0x7f7b941027a0 2024-09-16T02:01:42.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.973+0000 7f7b9c2f1640 1 -- 172.21.15.105:0/2467850113 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b9419dbb0 con 0x7f7b941027a0 2024-09-16T02:01:42.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.974+0000 7f7b9c2f1640 1 -- 172.21.15.105:0/2467850113 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b941021d0 con 0x7f7b941027a0 2024-09-16T02:01:42.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.975+0000 7f7b8b7fe640 1 -- 172.21.15.105:0/2467850113 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f7b84003b10 con 0x7f7b941027a0 2024-09-16T02:01:42.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.976+0000 7f7b8b7fe640 1 --2- 172.21.15.105:0/2467850113 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7b800a32d0 0x7f7b800a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:42.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.976+0000 7f7b9a08d640 1 --2- 172.21.15.105:0/2467850113 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7b800a32d0 0x7f7b800a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:42.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.976+0000 7f7b8b7fe640 1 -- 172.21.15.105:0/2467850113 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f7b840f0900 con 0x7f7b941027a0 2024-09-16T02:01:42.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.976+0000 7f7b9a08d640 1 --2- 172.21.15.105:0/2467850113 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7b800a32d0 0x7f7b800a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7b9000a810 tx=0x7f7b90008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:42.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:41.979+0000 7f7b8b7fe640 1 -- 172.21.15.105:0/2467850113 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7b840bd4a0 con 0x7f7b941027a0 2024-09-16T02:01:42.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:42.149+0000 7f7b9c2f1640 1 -- 172.21.15.105:0/2467850113 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7b94062630 con 0x7f7b800a32d0 2024-09-16T02:01:42.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:42.157+0000 7f7b8b7fe640 1 -- 172.21.15.105:0/2467850113 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258827 (secure 0 0 0) 0x7f7b94062630 con 0x7f7b800a32d0 2024-09-16T02:01:42.162 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:01:42.163 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-09-16T02:01:42.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:42.160+0000 7f7b9c2f1640 1 -- 172.21.15.105:0/2467850113 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7b800a32d0 msgr2=0x7f7b800a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:42.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:42.160+0000 7f7b9c2f1640 1 --2- 172.21.15.105:0/2467850113 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7b800a32d0 0x7f7b800a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7b9000a810 tx=0x7f7b90008040 comp rx=0 tx=0).stop 2024-09-16T02:01:42.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:42.160+0000 7f7b9c2f1640 1 -- 172.21.15.105:0/2467850113 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b941027a0 msgr2=0x7f7b94198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:42.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:42.160+0000 7f7b9c2f1640 1 --2- 172.21.15.105:0/2467850113 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b941027a0 0x7f7b94198a00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7b84006010 tx=0x7f7b84004270 comp rx=0 tx=0).stop 2024-09-16T02:01:42.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:42.160+0000 7f7b9c2f1640 1 -- 172.21.15.105:0/2467850113 shutdown_connections 2024-09-16T02:01:42.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:42.160+0000 7f7b9c2f1640 1 --2- 172.21.15.105:0/2467850113 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7b800a32d0 0x7f7b800a5790 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:42.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:42.160+0000 7f7b9c2f1640 1 --2- 172.21.15.105:0/2467850113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b94106f20 0x7f7b9419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:42.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:42.160+0000 7f7b9c2f1640 1 --2- 172.21.15.105:0/2467850113 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b941027a0 0x7f7b94198a00 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:42.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:42.160+0000 7f7b9c2f1640 1 --2- 172.21.15.105:0/2467850113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b94101df0 0x7f7b941984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:42.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:42.160+0000 7f7b9c2f1640 1 -- 172.21.15.105:0/2467850113 >> 172.21.15.105:0/2467850113 conn(0x7f7b940fd8f0 msgr2=0x7f7b940fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:42.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:42.160+0000 7f7b9c2f1640 1 -- 172.21.15.105:0/2467850113 shutdown_connections 2024-09-16T02:01:42.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:42.160+0000 7f7b9c2f1640 1 -- 172.21.15.105:0/2467850113 wait complete. 2024-09-16T02:01:42.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:42 smithi190 ceph-mon[34762]: pgmap v312: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-09-16T02:01:42.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:42 smithi190 ceph-mon[34762]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-16T02:01:42.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:42 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:01:42.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:42 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:01:42.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:42 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:01:42.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:42 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:01:42.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:42 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:01:42.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:42 smithi137 ceph-mon[34099]: pgmap v312: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-09-16T02:01:42.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:42 smithi137 ceph-mon[34099]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-16T02:01:42.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:42 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:01:42.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:42 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:01:42.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:42 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:01:42.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:42 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:01:42.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:42 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:01:43.015 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:42 smithi105 ceph-mon[31197]: pgmap v312: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-09-16T02:01:43.015 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:42 smithi105 ceph-mon[31197]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-16T02:01:43.015 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:42 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:01:43.015 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:42 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:01:43.015 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:42 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:01:43.015 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:42 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:01:43.015 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:42 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:01:43.042 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":312,"stamp":"2024-09-16T02:01:41.459965+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656628,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":680,"num_read_kb":457,"num_write":421,"num_write_kb":3070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364176,"kb_used_data":10752,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577648,"statfs":{"total":1151940427776,"available":1151567511552,"internally_reserved":0,"allocated":11010048,"data_stored":6969224,"data_compressed":118948,"data_compressed_allocated":151552,"data_compressed_original":1802240,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001970"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.463390+0000","last_change":"2024-09-16T02:00:34.454121+0000","last_active":"2024-09-16T02:00:35.463390+0000","last_peered":"2024-09-16T02:00:35.463390+0000","last_clean":"2024-09-16T02:00:35.463390+0000","last_became_active":"2024-09-16T02:00:34.453810+0000","last_became_peered":"2024-09-16T02:00:34.453810+0000","last_unstale":"2024-09-16T02:00:35.463390+0000","last_undegraded":"2024-09-16T02:00:35.463390+0000","last_fullsized":"2024-09-16T02:00:35.463390+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:42:38.781928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.563230+0000","last_change":"2024-09-16T02:00:34.462020+0000","last_active":"2024-09-16T02:01:39.563230+0000","last_peered":"2024-09-16T02:01:39.563230+0000","last_clean":"2024-09-16T02:01:39.563230+0000","last_became_active":"2024-09-16T02:00:34.461680+0000","last_became_peered":"2024-09-16T02:00:34.461680+0000","last_unstale":"2024-09-16T02:01:39.563230+0000","last_undegraded":"2024-09-16T02:01:39.563230+0000","last_fullsized":"2024-09-16T02:01:39.563230+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:00:14.493643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.562710+0000","last_change":"2024-09-16T02:00:34.453034+0000","last_active":"2024-09-16T02:01:39.562710+0000","last_peered":"2024-09-16T02:01:39.562710+0000","last_clean":"2024-09-16T02:01:39.562710+0000","last_became_active":"2024-09-16T02:00:34.452719+0000","last_became_peered":"2024-09-16T02:00:34.452719+0000","last_unstale":"2024-09-16T02:01:39.562710+0000","last_undegraded":"2024-09-16T02:01:39.562710+0000","last_fullsized":"2024-09-16T02:01:39.562710+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:48:52.478501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458864+0000","last_change":"2024-09-16T02:00:34.458156+0000","last_active":"2024-09-16T02:00:35.458864+0000","last_peered":"2024-09-16T02:00:35.458864+0000","last_clean":"2024-09-16T02:00:35.458864+0000","last_became_active":"2024-09-16T02:00:34.457831+0000","last_became_peered":"2024-09-16T02:00:34.457831+0000","last_unstale":"2024-09-16T02:00:35.458864+0000","last_undegraded":"2024-09-16T02:00:35.458864+0000","last_fullsized":"2024-09-16T02:00:35.458864+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:35:36.609046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.562428+0000","last_change":"2024-09-16T02:00:34.448638+0000","last_active":"2024-09-16T02:01:39.562428+0000","last_peered":"2024-09-16T02:01:39.562428+0000","last_clean":"2024-09-16T02:01:39.562428+0000","last_became_active":"2024-09-16T02:00:34.447993+0000","last_became_peered":"2024-09-16T02:00:34.447993+0000","last_unstale":"2024-09-16T02:01:39.562428+0000","last_undegraded":"2024-09-16T02:01:39.562428+0000","last_fullsized":"2024-09-16T02:01:39.562428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:33:04.832053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.459310+0000","last_change":"2024-09-16T02:00:34.462651+0000","last_active":"2024-09-16T02:00:35.459310+0000","last_peered":"2024-09-16T02:00:35.459310+0000","last_clean":"2024-09-16T02:00:35.459310+0000","last_became_active":"2024-09-16T02:00:34.461837+0000","last_became_peered":"2024-09-16T02:00:34.461837+0000","last_unstale":"2024-09-16T02:00:35.459310+0000","last_undegraded":"2024-09-16T02:00:35.459310+0000","last_fullsized":"2024-09-16T02:00:35.459310+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:52:13.733591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.562619+0000","last_change":"2024-09-16T02:00:34.455362+0000","last_active":"2024-09-16T02:01:39.562619+0000","last_peered":"2024-09-16T02:01:39.562619+0000","last_clean":"2024-09-16T02:01:39.562619+0000","last_became_active":"2024-09-16T02:00:34.455029+0000","last_became_peered":"2024-09-16T02:00:34.455029+0000","last_unstale":"2024-09-16T02:01:39.562619+0000","last_undegraded":"2024-09-16T02:01:39.562619+0000","last_fullsized":"2024-09-16T02:01:39.562619+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:14:08.316845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.558582+0000","last_change":"2024-09-16T02:00:34.459578+0000","last_active":"2024-09-16T02:01:39.558582+0000","last_peered":"2024-09-16T02:01:39.558582+0000","last_clean":"2024-09-16T02:01:39.558582+0000","last_became_active":"2024-09-16T02:00:34.459288+0000","last_became_peered":"2024-09-16T02:00:34.459288+0000","last_unstale":"2024-09-16T02:01:39.558582+0000","last_undegraded":"2024-09-16T02:01:39.558582+0000","last_fullsized":"2024-09-16T02:01:39.558582+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:52:28.974276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.447603+0000","last_change":"2024-09-16T02:00:34.463011+0000","last_active":"2024-09-16T02:00:35.447603+0000","last_peered":"2024-09-16T02:00:35.447603+0000","last_clean":"2024-09-16T02:00:35.447603+0000","last_became_active":"2024-09-16T02:00:34.462668+0000","last_became_peered":"2024-09-16T02:00:34.462668+0000","last_unstale":"2024-09-16T02:00:35.447603+0000","last_undegraded":"2024-09-16T02:00:35.447603+0000","last_fullsized":"2024-09-16T02:00:35.447603+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:06:31.655869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458464+0000","last_change":"2024-09-16T02:00:34.462241+0000","last_active":"2024-09-16T02:00:35.458464+0000","last_peered":"2024-09-16T02:00:35.458464+0000","last_clean":"2024-09-16T02:00:35.458464+0000","last_became_active":"2024-09-16T02:00:34.461925+0000","last_became_peered":"2024-09-16T02:00:34.461925+0000","last_unstale":"2024-09-16T02:00:35.458464+0000","last_undegraded":"2024-09-16T02:00:35.458464+0000","last_fullsized":"2024-09-16T02:00:35.458464+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:48:02.414471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.561882+0000","last_change":"2024-09-16T02:00:34.466132+0000","last_active":"2024-09-16T02:01:39.561882+0000","last_peered":"2024-09-16T02:01:39.561882+0000","last_clean":"2024-09-16T02:01:39.561882+0000","last_became_active":"2024-09-16T02:00:34.465750+0000","last_became_peered":"2024-09-16T02:00:34.465750+0000","last_unstale":"2024-09-16T02:01:39.561882+0000","last_undegraded":"2024-09-16T02:01:39.561882+0000","last_fullsized":"2024-09-16T02:01:39.561882+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:07:33.072432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458928+0000","last_change":"2024-09-16T02:00:34.461759+0000","last_active":"2024-09-16T02:00:35.458928+0000","last_peered":"2024-09-16T02:00:35.458928+0000","last_clean":"2024-09-16T02:00:35.458928+0000","last_became_active":"2024-09-16T02:00:34.461385+0000","last_became_peered":"2024-09-16T02:00:34.461385+0000","last_unstale":"2024-09-16T02:00:35.458928+0000","last_undegraded":"2024-09-16T02:00:35.458928+0000","last_fullsized":"2024-09-16T02:00:35.458928+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:00:24.179172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.446281+0000","last_change":"2024-09-16T02:00:34.456584+0000","last_active":"2024-09-16T02:00:35.446281+0000","last_peered":"2024-09-16T02:00:35.446281+0000","last_clean":"2024-09-16T02:00:35.446281+0000","last_became_active":"2024-09-16T02:00:34.456257+0000","last_became_peered":"2024-09-16T02:00:34.456257+0000","last_unstale":"2024-09-16T02:00:35.446281+0000","last_undegraded":"2024-09-16T02:00:35.446281+0000","last_fullsized":"2024-09-16T02:00:35.446281+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:26:45.835435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.562760+0000","last_change":"2024-09-16T02:00:34.460680+0000","last_active":"2024-09-16T02:01:39.562760+0000","last_peered":"2024-09-16T02:01:39.562760+0000","last_clean":"2024-09-16T02:01:39.562760+0000","last_became_active":"2024-09-16T02:00:34.460161+0000","last_became_peered":"2024-09-16T02:00:34.460161+0000","last_unstale":"2024-09-16T02:01:39.562760+0000","last_undegraded":"2024-09-16T02:01:39.562760+0000","last_fullsized":"2024-09-16T02:01:39.562760+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:38:19.848420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.560921+0000","last_change":"2024-09-16T02:00:34.454606+0000","last_active":"2024-09-16T02:01:39.560921+0000","last_peered":"2024-09-16T02:01:39.560921+0000","last_clean":"2024-09-16T02:01:39.560921+0000","last_became_active":"2024-09-16T02:00:34.454293+0000","last_became_peered":"2024-09-16T02:00:34.454293+0000","last_unstale":"2024-09-16T02:01:39.560921+0000","last_undegraded":"2024-09-16T02:01:39.560921+0000","last_fullsized":"2024-09-16T02:01:39.560921+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:55:54.466176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.446992+0000","last_change":"2024-09-16T02:00:34.452940+0000","last_active":"2024-09-16T02:00:35.446992+0000","last_peered":"2024-09-16T02:00:35.446992+0000","last_clean":"2024-09-16T02:00:35.446992+0000","last_became_active":"2024-09-16T02:00:34.452560+0000","last_became_peered":"2024-09-16T02:00:34.452560+0000","last_unstale":"2024-09-16T02:00:35.446992+0000","last_undegraded":"2024-09-16T02:00:35.446992+0000","last_fullsized":"2024-09-16T02:00:35.446992+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:42:11.564264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.463540+0000","last_change":"2024-09-16T02:00:34.453457+0000","last_active":"2024-09-16T02:00:35.463540+0000","last_peered":"2024-09-16T02:00:35.463540+0000","last_clean":"2024-09-16T02:00:35.463540+0000","last_became_active":"2024-09-16T02:00:34.453148+0000","last_became_peered":"2024-09-16T02:00:34.453148+0000","last_unstale":"2024-09-16T02:00:35.463540+0000","last_undegraded":"2024-09-16T02:00:35.463540+0000","last_fullsized":"2024-09-16T02:00:35.463540+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:28.456892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.607401+0000","last_change":"2024-09-16T02:00:34.457057+0000","last_active":"2024-09-16T02:01:40.607401+0000","last_peered":"2024-09-16T02:01:40.607401+0000","last_clean":"2024-09-16T02:01:40.607401+0000","last_became_active":"2024-09-16T02:00:34.456751+0000","last_became_peered":"2024-09-16T02:00:34.456751+0000","last_unstale":"2024-09-16T02:01:40.607401+0000","last_undegraded":"2024-09-16T02:01:40.607401+0000","last_fullsized":"2024-09-16T02:01:40.607401+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:32:32.207510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458938+0000","last_change":"2024-09-16T02:00:34.448434+0000","last_active":"2024-09-16T02:00:35.458938+0000","last_peered":"2024-09-16T02:00:35.458938+0000","last_clean":"2024-09-16T02:00:35.458938+0000","last_became_active":"2024-09-16T02:00:34.448056+0000","last_became_peered":"2024-09-16T02:00:34.448056+0000","last_unstale":"2024-09-16T02:00:35.458938+0000","last_undegraded":"2024-09-16T02:00:35.458938+0000","last_fullsized":"2024-09-16T02:00:35.458938+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:15:28.162159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.461643+0000","last_change":"2024-09-16T02:00:34.459499+0000","last_active":"2024-09-16T02:00:35.461643+0000","last_peered":"2024-09-16T02:00:35.461643+0000","last_clean":"2024-09-16T02:00:35.461643+0000","last_became_active":"2024-09-16T02:00:34.459017+0000","last_became_peered":"2024-09-16T02:00:34.459017+0000","last_unstale":"2024-09-16T02:00:35.461643+0000","last_undegraded":"2024-09-16T02:00:35.461643+0000","last_fullsized":"2024-09-16T02:00:35.461643+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:41:36.866100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.446287+0000","last_change":"2024-09-16T02:00:34.448640+0000","last_active":"2024-09-16T02:00:35.446287+0000","last_peered":"2024-09-16T02:00:35.446287+0000","last_clean":"2024-09-16T02:00:35.446287+0000","last_became_active":"2024-09-16T02:00:34.447737+0000","last_became_peered":"2024-09-16T02:00:34.447737+0000","last_unstale":"2024-09-16T02:00:35.446287+0000","last_undegraded":"2024-09-16T02:00:35.446287+0000","last_fullsized":"2024-09-16T02:00:35.446287+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:20.192650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.447240+0000","last_change":"2024-09-16T02:00:34.451718+0000","last_active":"2024-09-16T02:00:35.447240+0000","last_peered":"2024-09-16T02:00:35.447240+0000","last_clean":"2024-09-16T02:00:35.447240+0000","last_became_active":"2024-09-16T02:00:34.451389+0000","last_became_peered":"2024-09-16T02:00:34.451389+0000","last_unstale":"2024-09-16T02:00:35.447240+0000","last_undegraded":"2024-09-16T02:00:35.447240+0000","last_fullsized":"2024-09-16T02:00:35.447240+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:13:19.296178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.558628+0000","last_change":"2024-09-16T02:00:34.458379+0000","last_active":"2024-09-16T02:01:39.558628+0000","last_peered":"2024-09-16T02:01:39.558628+0000","last_clean":"2024-09-16T02:01:39.558628+0000","last_became_active":"2024-09-16T02:00:34.458038+0000","last_became_peered":"2024-09-16T02:00:34.458038+0000","last_unstale":"2024-09-16T02:01:39.558628+0000","last_undegraded":"2024-09-16T02:01:39.558628+0000","last_fullsized":"2024-09-16T02:01:39.558628+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:34.669470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.446886+0000","last_change":"2024-09-16T02:00:34.450927+0000","last_active":"2024-09-16T02:00:35.446886+0000","last_peered":"2024-09-16T02:00:35.446886+0000","last_clean":"2024-09-16T02:00:35.446886+0000","last_became_active":"2024-09-16T02:00:34.447669+0000","last_became_peered":"2024-09-16T02:00:34.447669+0000","last_unstale":"2024-09-16T02:00:35.446886+0000","last_undegraded":"2024-09-16T02:00:35.446886+0000","last_fullsized":"2024-09-16T02:00:35.446886+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:57.861633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.559318+0000","last_change":"2024-09-16T02:00:34.454827+0000","last_active":"2024-09-16T02:01:39.559318+0000","last_peered":"2024-09-16T02:01:39.559318+0000","last_clean":"2024-09-16T02:01:39.559318+0000","last_became_active":"2024-09-16T02:00:34.454549+0000","last_became_peered":"2024-09-16T02:00:34.454549+0000","last_unstale":"2024-09-16T02:01:39.559318+0000","last_undegraded":"2024-09-16T02:01:39.559318+0000","last_fullsized":"2024-09-16T02:01:39.559318+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:53:45.440860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.461087+0000","last_change":"2024-09-16T02:00:34.455121+0000","last_active":"2024-09-16T02:00:35.461087+0000","last_peered":"2024-09-16T02:00:35.461087+0000","last_clean":"2024-09-16T02:00:35.461087+0000","last_became_active":"2024-09-16T02:00:34.454665+0000","last_became_peered":"2024-09-16T02:00:34.454665+0000","last_unstale":"2024-09-16T02:00:35.461087+0000","last_undegraded":"2024-09-16T02:00:35.461087+0000","last_fullsized":"2024-09-16T02:00:35.461087+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:31:42.771230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.462593+0000","last_change":"2024-09-16T02:00:34.458777+0000","last_active":"2024-09-16T02:00:35.462593+0000","last_peered":"2024-09-16T02:00:35.462593+0000","last_clean":"2024-09-16T02:00:35.462593+0000","last_became_active":"2024-09-16T02:00:34.457954+0000","last_became_peered":"2024-09-16T02:00:34.457954+0000","last_unstale":"2024-09-16T02:00:35.462593+0000","last_undegraded":"2024-09-16T02:00:35.462593+0000","last_fullsized":"2024-09-16T02:00:35.462593+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:58:28.032293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.446190+0000","last_change":"2024-09-16T02:00:34.444712+0000","last_active":"2024-09-16T02:00:35.446190+0000","last_peered":"2024-09-16T02:00:35.446190+0000","last_clean":"2024-09-16T02:00:35.446190+0000","last_became_active":"2024-09-16T02:00:34.444421+0000","last_became_peered":"2024-09-16T02:00:34.444421+0000","last_unstale":"2024-09-16T02:00:35.446190+0000","last_undegraded":"2024-09-16T02:00:35.446190+0000","last_fullsized":"2024-09-16T02:00:35.446190+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:36.557558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.563059+0000","last_change":"2024-09-16T02:00:34.454570+0000","last_active":"2024-09-16T02:01:39.563059+0000","last_peered":"2024-09-16T02:01:39.563059+0000","last_clean":"2024-09-16T02:01:39.563059+0000","last_became_active":"2024-09-16T02:00:34.454246+0000","last_became_peered":"2024-09-16T02:00:34.454246+0000","last_unstale":"2024-09-16T02:01:39.563059+0000","last_undegraded":"2024-09-16T02:01:39.563059+0000","last_fullsized":"2024-09-16T02:01:39.563059+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:36:18.634907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458979+0000","last_change":"2024-09-16T02:00:34.449680+0000","last_active":"2024-09-16T02:00:35.458979+0000","last_peered":"2024-09-16T02:00:35.458979+0000","last_clean":"2024-09-16T02:00:35.458979+0000","last_became_active":"2024-09-16T02:00:34.449184+0000","last_became_peered":"2024-09-16T02:00:34.449184+0000","last_unstale":"2024-09-16T02:00:35.458979+0000","last_undegraded":"2024-09-16T02:00:35.458979+0000","last_fullsized":"2024-09-16T02:00:35.458979+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:28:34.867820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458803+0000","last_change":"2024-09-16T02:00:34.455362+0000","last_active":"2024-09-16T02:00:35.458803+0000","last_peered":"2024-09-16T02:00:35.458803+0000","last_clean":"2024-09-16T02:00:35.458803+0000","last_became_active":"2024-09-16T02:00:34.455018+0000","last_became_peered":"2024-09-16T02:00:34.455018+0000","last_unstale":"2024-09-16T02:00:35.458803+0000","last_undegraded":"2024-09-16T02:00:35.458803+0000","last_fullsized":"2024-09-16T02:00:35.458803+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:38.278418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.462886+0000","last_change":"2024-09-16T02:00:34.459426+0000","last_active":"2024-09-16T02:00:35.462886+0000","last_peered":"2024-09-16T02:00:35.462886+0000","last_clean":"2024-09-16T02:00:35.462886+0000","last_became_active":"2024-09-16T02:00:34.459082+0000","last_became_peered":"2024-09-16T02:00:34.459082+0000","last_unstale":"2024-09-16T02:00:35.462886+0000","last_undegraded":"2024-09-16T02:00:35.462886+0000","last_fullsized":"2024-09-16T02:00:35.462886+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:59:42.226065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.562286+0000","last_change":"2024-09-16T02:00:34.450936+0000","last_active":"2024-09-16T02:01:39.562286+0000","last_peered":"2024-09-16T02:01:39.562286+0000","last_clean":"2024-09-16T02:01:39.562286+0000","last_became_active":"2024-09-16T02:00:34.450503+0000","last_became_peered":"2024-09-16T02:00:34.450503+0000","last_unstale":"2024-09-16T02:01:39.562286+0000","last_undegraded":"2024-09-16T02:01:39.562286+0000","last_fullsized":"2024-09-16T02:01:39.562286+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:35:55.806193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'5","reported_seq":77,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:01:37.603159+0000","last_change":"2024-09-16T02:00:32.443193+0000","last_active":"2024-09-16T02:01:37.603159+0000","last_peered":"2024-09-16T02:01:37.603159+0000","last_clean":"2024-09-16T02:01:37.603159+0000","last_became_active":"2024-09-16T02:00:32.442863+0000","last_became_peered":"2024-09-16T02:00:32.442863+0000","last_unstale":"2024-09-16T02:01:37.603159+0000","last_undegraded":"2024-09-16T02:01:37.603159+0000","last_fullsized":"2024-09-16T02:01:37.603159+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:19.734946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.447318+0000","last_change":"2024-09-16T02:00:34.454819+0000","last_active":"2024-09-16T02:00:35.447318+0000","last_peered":"2024-09-16T02:00:35.447318+0000","last_clean":"2024-09-16T02:00:35.447318+0000","last_became_active":"2024-09-16T02:00:34.454492+0000","last_became_peered":"2024-09-16T02:00:34.454492+0000","last_unstale":"2024-09-16T02:00:35.447318+0000","last_undegraded":"2024-09-16T02:00:35.447318+0000","last_fullsized":"2024-09-16T02:00:35.447318+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:11.479330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:45.518640+0000","last_change":"2024-09-16T02:00:32.438666+0000","last_active":"2024-09-16T02:00:45.518640+0000","last_peered":"2024-09-16T02:00:45.518640+0000","last_clean":"2024-09-16T02:00:45.518640+0000","last_became_active":"2024-09-16T02:00:32.438324+0000","last_became_peered":"2024-09-16T02:00:32.438324+0000","last_unstale":"2024-09-16T02:00:45.518640+0000","last_undegraded":"2024-09-16T02:00:45.518640+0000","last_fullsized":"2024-09-16T02:00:45.518640+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:56:28.203977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:44.506262+0000","last_change":"2024-09-16T02:00:32.438530+0000","last_active":"2024-09-16T02:00:44.506262+0000","last_peered":"2024-09-16T02:00:44.506262+0000","last_clean":"2024-09-16T02:00:44.506262+0000","last_became_active":"2024-09-16T02:00:32.438238+0000","last_became_peered":"2024-09-16T02:00:32.438238+0000","last_unstale":"2024-09-16T02:00:44.506262+0000","last_undegraded":"2024-09-16T02:00:44.506262+0000","last_fullsized":"2024-09-16T02:00:44.506262+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:52:32.799701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.558632+0000","last_change":"2024-09-16T02:00:34.456299+0000","last_active":"2024-09-16T02:01:39.558632+0000","last_peered":"2024-09-16T02:01:39.558632+0000","last_clean":"2024-09-16T02:01:39.558632+0000","last_became_active":"2024-09-16T02:00:34.455925+0000","last_became_peered":"2024-09-16T02:00:34.455925+0000","last_unstale":"2024-09-16T02:01:39.558632+0000","last_undegraded":"2024-09-16T02:01:39.558632+0000","last_fullsized":"2024-09-16T02:01:39.558632+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:33:27.162664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:46.527624+0000","last_change":"2024-09-16T02:00:32.442029+0000","last_active":"2024-09-16T02:00:46.527624+0000","last_peered":"2024-09-16T02:00:46.527624+0000","last_clean":"2024-09-16T02:00:46.527624+0000","last_became_active":"2024-09-16T02:00:32.441705+0000","last_became_peered":"2024-09-16T02:00:32.441705+0000","last_unstale":"2024-09-16T02:00:46.527624+0000","last_undegraded":"2024-09-16T02:00:46.527624+0000","last_fullsized":"2024-09-16T02:00:46.527624+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:50:54.725768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.607165+0000","last_change":"2024-09-16T02:00:34.457833+0000","last_active":"2024-09-16T02:01:40.607165+0000","last_peered":"2024-09-16T02:01:40.607165+0000","last_clean":"2024-09-16T02:01:40.607165+0000","last_became_active":"2024-09-16T02:00:34.457453+0000","last_became_peered":"2024-09-16T02:00:34.457453+0000","last_unstale":"2024-09-16T02:01:40.607165+0000","last_undegraded":"2024-09-16T02:01:40.607165+0000","last_fullsized":"2024-09-16T02:01:40.607165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:21.777574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":122,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:01:36.600847+0000","last_change":"2024-09-16T02:00:32.441341+0000","last_active":"2024-09-16T02:01:36.600847+0000","last_peered":"2024-09-16T02:01:36.600847+0000","last_clean":"2024-09-16T02:01:36.600847+0000","last_became_active":"2024-09-16T02:00:32.441019+0000","last_became_peered":"2024-09-16T02:00:32.441019+0000","last_unstale":"2024-09-16T02:01:36.600847+0000","last_undegraded":"2024-09-16T02:01:36.600847+0000","last_fullsized":"2024-09-16T02:01:36.600847+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:10:10.171714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.447426+0000","last_change":"2024-09-16T02:00:34.461746+0000","last_active":"2024-09-16T02:00:35.447426+0000","last_peered":"2024-09-16T02:00:35.447426+0000","last_clean":"2024-09-16T02:00:35.447426+0000","last_became_active":"2024-09-16T02:00:34.461441+0000","last_became_peered":"2024-09-16T02:00:34.461441+0000","last_unstale":"2024-09-16T02:00:35.447426+0000","last_undegraded":"2024-09-16T02:00:35.447426+0000","last_fullsized":"2024-09-16T02:00:35.447426+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:28:14.519159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.561070+0000","last_change":"2024-09-16T02:00:32.443129+0000","last_active":"2024-09-16T02:01:39.561070+0000","last_peered":"2024-09-16T02:01:39.561070+0000","last_clean":"2024-09-16T02:01:39.561070+0000","last_became_active":"2024-09-16T02:00:32.442772+0000","last_became_peered":"2024-09-16T02:00:32.442772+0000","last_unstale":"2024-09-16T02:01:39.561070+0000","last_undegraded":"2024-09-16T02:01:39.561070+0000","last_fullsized":"2024-09-16T02:01:39.561070+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:10:03.964583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.563645+0000","last_change":"2024-09-16T02:00:34.457449+0000","last_active":"2024-09-16T02:01:39.563645+0000","last_peered":"2024-09-16T02:01:39.563645+0000","last_clean":"2024-09-16T02:01:39.563645+0000","last_became_active":"2024-09-16T02:00:34.457110+0000","last_became_peered":"2024-09-16T02:00:34.457110+0000","last_unstale":"2024-09-16T02:01:39.563645+0000","last_undegraded":"2024-09-16T02:01:39.563645+0000","last_fullsized":"2024-09-16T02:01:39.563645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:35.662503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:45.516635+0000","last_change":"2024-09-16T02:00:32.587419+0000","last_active":"2024-09-16T02:00:45.516635+0000","last_peered":"2024-09-16T02:00:45.516635+0000","last_clean":"2024-09-16T02:00:45.516635+0000","last_became_active":"2024-09-16T02:00:32.587037+0000","last_became_peered":"2024-09-16T02:00:32.587037+0000","last_unstale":"2024-09-16T02:00:45.516635+0000","last_undegraded":"2024-09-16T02:00:45.516635+0000","last_fullsized":"2024-09-16T02:00:45.516635+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:11.835505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.447483+0000","last_change":"2024-09-16T02:00:34.456371+0000","last_active":"2024-09-16T02:00:35.447483+0000","last_peered":"2024-09-16T02:00:35.447483+0000","last_clean":"2024-09-16T02:00:35.447483+0000","last_became_active":"2024-09-16T02:00:34.456013+0000","last_became_peered":"2024-09-16T02:00:34.456013+0000","last_unstale":"2024-09-16T02:00:35.447483+0000","last_undegraded":"2024-09-16T02:00:35.447483+0000","last_fullsized":"2024-09-16T02:00:35.447483+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:30:15.847189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'28","reported_seq":154,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:01:36.600863+0000","last_change":"2024-09-16T02:00:32.444922+0000","last_active":"2024-09-16T02:01:36.600863+0000","last_peered":"2024-09-16T02:01:36.600863+0000","last_clean":"2024-09-16T02:01:36.600863+0000","last_became_active":"2024-09-16T02:00:32.444540+0000","last_became_peered":"2024-09-16T02:00:32.444540+0000","last_unstale":"2024-09-16T02:01:36.600863+0000","last_undegraded":"2024-09-16T02:01:36.600863+0000","last_fullsized":"2024-09-16T02:01:36.600863+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:59.183952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59211,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":108,"num_read_kb":111,"num_write":30,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.563273+0000","last_change":"2024-09-16T02:00:34.453366+0000","last_active":"2024-09-16T02:01:39.563273+0000","last_peered":"2024-09-16T02:01:39.563273+0000","last_clean":"2024-09-16T02:01:39.563273+0000","last_became_active":"2024-09-16T02:00:34.453048+0000","last_became_peered":"2024-09-16T02:00:34.453048+0000","last_unstale":"2024-09-16T02:01:39.563273+0000","last_undegraded":"2024-09-16T02:01:39.563273+0000","last_fullsized":"2024-09-16T02:01:39.563273+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:15:52.721893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:47.535399+0000","last_change":"2024-09-16T02:00:32.445516+0000","last_active":"2024-09-16T02:00:47.535399+0000","last_peered":"2024-09-16T02:00:47.535399+0000","last_clean":"2024-09-16T02:00:47.535399+0000","last_became_active":"2024-09-16T02:00:32.445092+0000","last_became_peered":"2024-09-16T02:00:32.445092+0000","last_unstale":"2024-09-16T02:00:47.535399+0000","last_undegraded":"2024-09-16T02:00:47.535399+0000","last_fullsized":"2024-09-16T02:00:47.535399+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:54.274692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.446077+0000","last_change":"2024-09-16T02:00:34.448450+0000","last_active":"2024-09-16T02:00:35.446077+0000","last_peered":"2024-09-16T02:00:35.446077+0000","last_clean":"2024-09-16T02:00:35.446077+0000","last_became_active":"2024-09-16T02:00:34.448116+0000","last_became_peered":"2024-09-16T02:00:34.448116+0000","last_unstale":"2024-09-16T02:00:35.446077+0000","last_undegraded":"2024-09-16T02:00:35.446077+0000","last_fullsized":"2024-09-16T02:00:35.446077+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:12:34.834798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:46.527344+0000","last_change":"2024-09-16T02:00:32.445427+0000","last_active":"2024-09-16T02:00:46.527344+0000","last_peered":"2024-09-16T02:00:46.527344+0000","last_clean":"2024-09-16T02:00:46.527344+0000","last_became_active":"2024-09-16T02:00:32.445058+0000","last_became_peered":"2024-09-16T02:00:32.445058+0000","last_unstale":"2024-09-16T02:00:46.527344+0000","last_undegraded":"2024-09-16T02:00:46.527344+0000","last_fullsized":"2024-09-16T02:00:46.527344+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:35:34.701331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.561778+0000","last_change":"2024-09-16T02:00:34.458479+0000","last_active":"2024-09-16T02:01:39.561778+0000","last_peered":"2024-09-16T02:01:39.561778+0000","last_clean":"2024-09-16T02:01:39.561778+0000","last_became_active":"2024-09-16T02:00:34.458135+0000","last_became_peered":"2024-09-16T02:00:34.458135+0000","last_unstale":"2024-09-16T02:01:39.561778+0000","last_undegraded":"2024-09-16T02:01:39.561778+0000","last_fullsized":"2024-09-16T02:01:39.561778+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:37:40.188861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'6","reported_seq":79,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:01:37.603159+0000","last_change":"2024-09-16T02:00:32.440319+0000","last_active":"2024-09-16T02:01:37.603159+0000","last_peered":"2024-09-16T02:01:37.603159+0000","last_clean":"2024-09-16T02:01:37.603159+0000","last_became_active":"2024-09-16T02:00:32.440005+0000","last_became_peered":"2024-09-16T02:00:32.440005+0000","last_unstale":"2024-09-16T02:01:37.603159+0000","last_undegraded":"2024-09-16T02:01:37.603159+0000","last_fullsized":"2024-09-16T02:01:37.603159+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:07:33.876577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.561168+0000","last_change":"2024-09-16T02:00:34.460468+0000","last_active":"2024-09-16T02:01:39.561168+0000","last_peered":"2024-09-16T02:01:39.561168+0000","last_clean":"2024-09-16T02:01:39.561168+0000","last_became_active":"2024-09-16T02:00:34.459190+0000","last_became_peered":"2024-09-16T02:00:34.459190+0000","last_unstale":"2024-09-16T02:01:39.561168+0000","last_undegraded":"2024-09-16T02:01:39.561168+0000","last_fullsized":"2024-09-16T02:01:39.561168+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:46:43.654991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.562089+0000","last_change":"2024-09-16T02:00:32.587700+0000","last_active":"2024-09-16T02:01:39.562089+0000","last_peered":"2024-09-16T02:01:39.562089+0000","last_clean":"2024-09-16T02:01:39.562089+0000","last_became_active":"2024-09-16T02:00:32.586964+0000","last_became_peered":"2024-09-16T02:00:32.586964+0000","last_unstale":"2024-09-16T02:01:39.562089+0000","last_undegraded":"2024-09-16T02:01:39.562089+0000","last_fullsized":"2024-09-16T02:01:39.562089+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:41.309918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.446915+0000","last_change":"2024-09-16T02:00:34.455652+0000","last_active":"2024-09-16T02:00:35.446915+0000","last_peered":"2024-09-16T02:00:35.446915+0000","last_clean":"2024-09-16T02:00:35.446915+0000","last_became_active":"2024-09-16T02:00:34.455306+0000","last_became_peered":"2024-09-16T02:00:34.455306+0000","last_unstale":"2024-09-16T02:00:35.446915+0000","last_undegraded":"2024-09-16T02:00:35.446915+0000","last_fullsized":"2024-09-16T02:00:35.446915+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:44:09.505096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.563741+0000","last_change":"2024-09-16T02:00:32.441443+0000","last_active":"2024-09-16T02:01:39.563741+0000","last_peered":"2024-09-16T02:01:39.563741+0000","last_clean":"2024-09-16T02:01:39.563741+0000","last_became_active":"2024-09-16T02:00:32.441073+0000","last_became_peered":"2024-09-16T02:00:32.441073+0000","last_unstale":"2024-09-16T02:01:39.563741+0000","last_undegraded":"2024-09-16T02:01:39.563741+0000","last_fullsized":"2024-09-16T02:01:39.563741+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:16.865970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.462234+0000","last_change":"2024-09-16T02:00:34.458108+0000","last_active":"2024-09-16T02:00:35.462234+0000","last_peered":"2024-09-16T02:00:35.462234+0000","last_clean":"2024-09-16T02:00:35.462234+0000","last_became_active":"2024-09-16T02:00:34.457766+0000","last_became_peered":"2024-09-16T02:00:34.457766+0000","last_unstale":"2024-09-16T02:00:35.462234+0000","last_undegraded":"2024-09-16T02:00:35.462234+0000","last_fullsized":"2024-09-16T02:00:35.462234+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:21:54.410438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:46.526168+0000","last_change":"2024-09-16T02:00:32.441728+0000","last_active":"2024-09-16T02:00:46.526168+0000","last_peered":"2024-09-16T02:00:46.526168+0000","last_clean":"2024-09-16T02:00:46.526168+0000","last_became_active":"2024-09-16T02:00:32.441388+0000","last_became_peered":"2024-09-16T02:00:32.441388+0000","last_unstale":"2024-09-16T02:00:46.526168+0000","last_undegraded":"2024-09-16T02:00:46.526168+0000","last_fullsized":"2024-09-16T02:00:46.526168+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:18:24.412876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.562332+0000","last_change":"2024-09-16T02:00:34.461889+0000","last_active":"2024-09-16T02:01:39.562332+0000","last_peered":"2024-09-16T02:01:39.562332+0000","last_clean":"2024-09-16T02:01:39.562332+0000","last_became_active":"2024-09-16T02:00:34.461583+0000","last_became_peered":"2024-09-16T02:00:34.461583+0000","last_unstale":"2024-09-16T02:01:39.562332+0000","last_undegraded":"2024-09-16T02:01:39.562332+0000","last_fullsized":"2024-09-16T02:01:39.562332+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:32.598422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:46.526605+0000","last_change":"2024-09-16T02:00:32.445201+0000","last_active":"2024-09-16T02:00:46.526605+0000","last_peered":"2024-09-16T02:00:46.526605+0000","last_clean":"2024-09-16T02:00:46.526605+0000","last_became_active":"2024-09-16T02:00:32.444722+0000","last_became_peered":"2024-09-16T02:00:32.444722+0000","last_unstale":"2024-09-16T02:00:46.526605+0000","last_undegraded":"2024-09-16T02:00:46.526605+0000","last_fullsized":"2024-09-16T02:00:46.526605+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:41:11.657028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'1","reported_seq":15,"reported_epoch":79,"state":"peering","last_fresh":"2024-09-16T02:00:54.509925+0000","last_change":"2024-09-16T02:01:39.562036+0000","last_active":"2024-09-16T02:01:39.562036+0000","last_peered":"2024-09-16T02:00:54.509925+0000","last_clean":"2024-09-16T02:00:54.509925+0000","last_became_active":"2024-09-16T02:00:34.455718+0000","last_became_peered":"2024-09-16T02:00:34.455718+0000","last_unstale":"2024-09-16T02:00:54.509925+0000","last_undegraded":"2024-09-16T02:00:54.509925+0000","last_fullsized":"2024-09-16T02:00:54.509925+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5,9],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'5","reported_seq":77,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:01:37.603025+0000","last_change":"2024-09-16T02:00:32.860059+0000","last_active":"2024-09-16T02:01:37.603025+0000","last_peered":"2024-09-16T02:01:37.603025+0000","last_clean":"2024-09-16T02:01:37.603025+0000","last_became_active":"2024-09-16T02:00:32.859617+0000","last_became_peered":"2024-09-16T02:00:32.859617+0000","last_unstale":"2024-09-16T02:01:37.603025+0000","last_undegraded":"2024-09-16T02:01:37.603025+0000","last_fullsized":"2024-09-16T02:01:37.603025+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:41:50.248194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.561990+0000","last_change":"2024-09-16T02:00:34.454344+0000","last_active":"2024-09-16T02:01:39.561990+0000","last_peered":"2024-09-16T02:01:39.561990+0000","last_clean":"2024-09-16T02:01:39.561990+0000","last_became_active":"2024-09-16T02:00:34.454027+0000","last_became_peered":"2024-09-16T02:00:34.454027+0000","last_unstale":"2024-09-16T02:01:39.561990+0000","last_undegraded":"2024-09-16T02:01:39.561990+0000","last_fullsized":"2024-09-16T02:01:39.561990+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:33.020365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":332,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.562523+0000","last_change":"2024-09-16T01:59:19.798159+0000","last_active":"2024-09-16T02:01:39.562523+0000","last_peered":"2024-09-16T02:01:39.562523+0000","last_clean":"2024-09-16T02:01:39.562523+0000","last_became_active":"2024-09-16T01:59:19.797688+0000","last_became_peered":"2024-09-16T01:59:19.797688+0000","last_unstale":"2024-09-16T02:01:39.562523+0000","last_undegraded":"2024-09-16T02:01:39.562523+0000","last_fullsized":"2024-09-16T02:01:39.562523+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:55:54.312750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:55:54.312750+0000","last_clean_scrub_stamp":"2024-09-16T01:55:54.312750+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:00:36.860158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:47.534936+0000","last_change":"2024-09-16T02:00:32.441925+0000","last_active":"2024-09-16T02:00:47.534936+0000","last_peered":"2024-09-16T02:00:47.534936+0000","last_clean":"2024-09-16T02:00:47.534936+0000","last_became_active":"2024-09-16T02:00:32.441565+0000","last_became_peered":"2024-09-16T02:00:32.441565+0000","last_unstale":"2024-09-16T02:00:47.534936+0000","last_undegraded":"2024-09-16T02:00:47.534936+0000","last_fullsized":"2024-09-16T02:00:47.534936+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:23:09.755034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.558803+0000","last_change":"2024-09-16T02:00:34.452986+0000","last_active":"2024-09-16T02:01:39.558803+0000","last_peered":"2024-09-16T02:01:39.558803+0000","last_clean":"2024-09-16T02:01:39.558803+0000","last_became_active":"2024-09-16T02:00:34.452688+0000","last_became_peered":"2024-09-16T02:00:34.452688+0000","last_unstale":"2024-09-16T02:01:39.558803+0000","last_undegraded":"2024-09-16T02:01:39.558803+0000","last_fullsized":"2024-09-16T02:01:39.558803+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:12.958209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.561340+0000","last_change":"2024-09-16T02:00:32.443964+0000","last_active":"2024-09-16T02:01:39.561340+0000","last_peered":"2024-09-16T02:01:39.561340+0000","last_clean":"2024-09-16T02:01:39.561340+0000","last_became_active":"2024-09-16T02:00:32.443623+0000","last_became_peered":"2024-09-16T02:00:32.443623+0000","last_unstale":"2024-09-16T02:01:39.561340+0000","last_undegraded":"2024-09-16T02:01:39.561340+0000","last_fullsized":"2024-09-16T02:01:39.561340+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:30:34.406024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.446017+0000","last_change":"2024-09-16T02:00:34.449615+0000","last_active":"2024-09-16T02:00:35.446017+0000","last_peered":"2024-09-16T02:00:35.446017+0000","last_clean":"2024-09-16T02:00:35.446017+0000","last_became_active":"2024-09-16T02:00:34.449109+0000","last_became_peered":"2024-09-16T02:00:34.449109+0000","last_unstale":"2024-09-16T02:00:35.446017+0000","last_undegraded":"2024-09-16T02:00:35.446017+0000","last_fullsized":"2024-09-16T02:00:35.446017+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:45.036177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":76,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:01:37.602422+0000","last_change":"2024-09-16T02:00:32.587107+0000","last_active":"2024-09-16T02:01:37.602422+0000","last_peered":"2024-09-16T02:01:37.602422+0000","last_clean":"2024-09-16T02:01:37.602422+0000","last_became_active":"2024-09-16T02:00:32.586777+0000","last_became_peered":"2024-09-16T02:00:32.586777+0000","last_unstale":"2024-09-16T02:01:37.602422+0000","last_undegraded":"2024-09-16T02:01:37.602422+0000","last_fullsized":"2024-09-16T02:01:37.602422+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:03.968987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.447491+0000","last_change":"2024-09-16T02:00:34.453500+0000","last_active":"2024-09-16T02:00:35.447491+0000","last_peered":"2024-09-16T02:00:35.447491+0000","last_clean":"2024-09-16T02:00:35.447491+0000","last_became_active":"2024-09-16T02:00:34.453175+0000","last_became_peered":"2024-09-16T02:00:34.453175+0000","last_unstale":"2024-09-16T02:00:35.447491+0000","last_undegraded":"2024-09-16T02:00:35.447491+0000","last_fullsized":"2024-09-16T02:00:35.447491+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:52.666456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'5","reported_seq":79,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.561707+0000","last_change":"2024-09-16T02:00:32.443201+0000","last_active":"2024-09-16T02:01:39.561707+0000","last_peered":"2024-09-16T02:01:39.561707+0000","last_clean":"2024-09-16T02:01:39.561707+0000","last_became_active":"2024-09-16T02:00:32.442778+0000","last_became_peered":"2024-09-16T02:00:32.442778+0000","last_unstale":"2024-09-16T02:01:39.561707+0000","last_undegraded":"2024-09-16T02:01:39.561707+0000","last_fullsized":"2024-09-16T02:01:39.561707+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:43.933817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458153+0000","last_change":"2024-09-16T02:00:34.463000+0000","last_active":"2024-09-16T02:00:35.458153+0000","last_peered":"2024-09-16T02:00:35.458153+0000","last_clean":"2024-09-16T02:00:35.458153+0000","last_became_active":"2024-09-16T02:00:34.461890+0000","last_became_peered":"2024-09-16T02:00:34.461890+0000","last_unstale":"2024-09-16T02:00:35.458153+0000","last_undegraded":"2024-09-16T02:00:35.458153+0000","last_fullsized":"2024-09-16T02:00:35.458153+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:21:15.161295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.607447+0000","last_change":"2024-09-16T02:00:32.586933+0000","last_active":"2024-09-16T02:01:40.607447+0000","last_peered":"2024-09-16T02:01:40.607447+0000","last_clean":"2024-09-16T02:01:40.607447+0000","last_became_active":"2024-09-16T02:00:32.586565+0000","last_became_peered":"2024-09-16T02:00:32.586565+0000","last_unstale":"2024-09-16T02:01:40.607447+0000","last_undegraded":"2024-09-16T02:01:40.607447+0000","last_fullsized":"2024-09-16T02:01:40.607447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:45:07.130649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:37.582252+0000","last_change":"2024-09-16T02:00:34.460805+0000","last_active":"2024-09-16T02:00:37.582252+0000","last_peered":"2024-09-16T02:00:37.582252+0000","last_clean":"2024-09-16T02:00:37.582252+0000","last_became_active":"2024-09-16T02:00:34.460482+0000","last_became_peered":"2024-09-16T02:00:34.460482+0000","last_unstale":"2024-09-16T02:00:37.582252+0000","last_undegraded":"2024-09-16T02:00:37.582252+0000","last_fullsized":"2024-09-16T02:00:37.582252+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:07:51.565053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.563435+0000","last_change":"2024-09-16T02:00:32.587516+0000","last_active":"2024-09-16T02:01:39.563435+0000","last_peered":"2024-09-16T02:01:39.563435+0000","last_clean":"2024-09-16T02:01:39.563435+0000","last_became_active":"2024-09-16T02:00:32.587172+0000","last_became_peered":"2024-09-16T02:00:32.587172+0000","last_unstale":"2024-09-16T02:01:39.563435+0000","last_undegraded":"2024-09-16T02:01:39.563435+0000","last_fullsized":"2024-09-16T02:01:39.563435+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:52:29.018290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.446340+0000","last_change":"2024-09-16T02:00:34.446021+0000","last_active":"2024-09-16T02:00:35.446340+0000","last_peered":"2024-09-16T02:00:35.446340+0000","last_clean":"2024-09-16T02:00:35.446340+0000","last_became_active":"2024-09-16T02:00:34.445732+0000","last_became_peered":"2024-09-16T02:00:34.445732+0000","last_unstale":"2024-09-16T02:00:35.446340+0000","last_undegraded":"2024-09-16T02:00:35.446340+0000","last_fullsized":"2024-09-16T02:00:35.446340+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:51:17.960792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'14","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.607435+0000","last_change":"2024-09-16T02:00:32.439750+0000","last_active":"2024-09-16T02:01:40.607435+0000","last_peered":"2024-09-16T02:01:40.607435+0000","last_clean":"2024-09-16T02:01:40.607435+0000","last_became_active":"2024-09-16T02:00:32.439438+0000","last_became_peered":"2024-09-16T02:00:32.439438+0000","last_unstale":"2024-09-16T02:01:40.607435+0000","last_undegraded":"2024-09-16T02:01:40.607435+0000","last_fullsized":"2024-09-16T02:01:40.607435+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:50.764733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.559493+0000","last_change":"2024-09-16T02:00:34.453810+0000","last_active":"2024-09-16T02:01:39.559493+0000","last_peered":"2024-09-16T02:01:39.559493+0000","last_clean":"2024-09-16T02:01:39.559493+0000","last_became_active":"2024-09-16T02:00:34.453421+0000","last_became_peered":"2024-09-16T02:00:34.453421+0000","last_unstale":"2024-09-16T02:01:39.559493+0000","last_undegraded":"2024-09-16T02:01:39.559493+0000","last_fullsized":"2024-09-16T02:01:39.559493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:23:50.405749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:46.525899+0000","last_change":"2024-09-16T02:00:32.860565+0000","last_active":"2024-09-16T02:00:46.525899+0000","last_peered":"2024-09-16T02:00:46.525899+0000","last_clean":"2024-09-16T02:00:46.525899+0000","last_became_active":"2024-09-16T02:00:32.860130+0000","last_became_peered":"2024-09-16T02:00:32.860130+0000","last_unstale":"2024-09-16T02:00:46.525899+0000","last_undegraded":"2024-09-16T02:00:46.525899+0000","last_fullsized":"2024-09-16T02:00:46.525899+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:09:55.759883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"peering","last_fresh":"2024-09-16T02:00:35.467174+0000","last_change":"2024-09-16T02:01:39.561205+0000","last_active":"2024-09-16T02:01:39.561205+0000","last_peered":"2024-09-16T02:00:35.467174+0000","last_clean":"2024-09-16T02:00:35.467174+0000","last_became_active":"2024-09-16T02:00:34.453725+0000","last_became_peered":"2024-09-16T02:00:34.453725+0000","last_unstale":"2024-09-16T02:00:35.467174+0000","last_undegraded":"2024-09-16T02:00:35.467174+0000","last_fullsized":"2024-09-16T02:00:35.467174+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,4,6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:43.496005+0000","last_change":"2024-09-16T02:00:32.587115+0000","last_active":"2024-09-16T02:00:43.496005+0000","last_peered":"2024-09-16T02:00:43.496005+0000","last_clean":"2024-09-16T02:00:43.496005+0000","last_became_active":"2024-09-16T02:00:32.586835+0000","last_became_peered":"2024-09-16T02:00:32.586835+0000","last_unstale":"2024-09-16T02:00:43.496005+0000","last_undegraded":"2024-09-16T02:00:43.496005+0000","last_fullsized":"2024-09-16T02:00:43.496005+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:12:58.820005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.562201+0000","last_change":"2024-09-16T02:00:34.457673+0000","last_active":"2024-09-16T02:01:39.562201+0000","last_peered":"2024-09-16T02:01:39.562201+0000","last_clean":"2024-09-16T02:01:39.562201+0000","last_became_active":"2024-09-16T02:00:34.457331+0000","last_became_peered":"2024-09-16T02:00:34.457331+0000","last_unstale":"2024-09-16T02:01:39.562201+0000","last_undegraded":"2024-09-16T02:01:39.562201+0000","last_fullsized":"2024-09-16T02:01:39.562201+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:12:16.100036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.558734+0000","last_change":"2024-09-16T02:00:32.437528+0000","last_active":"2024-09-16T02:01:39.558734+0000","last_peered":"2024-09-16T02:01:39.558734+0000","last_clean":"2024-09-16T02:01:39.558734+0000","last_became_active":"2024-09-16T02:00:32.437080+0000","last_became_peered":"2024-09-16T02:00:32.437080+0000","last_unstale":"2024-09-16T02:01:39.558734+0000","last_undegraded":"2024-09-16T02:01:39.558734+0000","last_fullsized":"2024-09-16T02:01:39.558734+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:59:33.908026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.463059+0000","last_change":"2024-09-16T02:00:34.453504+0000","last_active":"2024-09-16T02:00:35.463059+0000","last_peered":"2024-09-16T02:00:35.463059+0000","last_clean":"2024-09-16T02:00:35.463059+0000","last_became_active":"2024-09-16T02:00:34.453212+0000","last_became_peered":"2024-09-16T02:00:34.453212+0000","last_unstale":"2024-09-16T02:00:35.463059+0000","last_undegraded":"2024-09-16T02:00:35.463059+0000","last_fullsized":"2024-09-16T02:00:35.463059+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:00:56.414548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:45.518572+0000","last_change":"2024-09-16T02:00:32.442282+0000","last_active":"2024-09-16T02:00:45.518572+0000","last_peered":"2024-09-16T02:00:45.518572+0000","last_clean":"2024-09-16T02:00:45.518572+0000","last_became_active":"2024-09-16T02:00:32.441910+0000","last_became_peered":"2024-09-16T02:00:32.441910+0000","last_unstale":"2024-09-16T02:00:45.518572+0000","last_undegraded":"2024-09-16T02:00:45.518572+0000","last_fullsized":"2024-09-16T02:00:45.518572+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:01:46.045021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.559236+0000","last_change":"2024-09-16T02:00:34.460177+0000","last_active":"2024-09-16T02:01:39.559236+0000","last_peered":"2024-09-16T02:01:39.559236+0000","last_clean":"2024-09-16T02:01:39.559236+0000","last_became_active":"2024-09-16T02:00:34.459874+0000","last_became_peered":"2024-09-16T02:00:34.459874+0000","last_unstale":"2024-09-16T02:01:39.559236+0000","last_undegraded":"2024-09-16T02:01:39.559236+0000","last_fullsized":"2024-09-16T02:01:39.559236+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:35.855591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":24,"reported_epoch":79,"state":"peering","last_fresh":"2024-09-16T02:00:47.535124+0000","last_change":"2024-09-16T02:01:39.563085+0000","last_active":"2024-09-16T02:01:39.563085+0000","last_peered":"2024-09-16T02:00:47.535124+0000","last_clean":"2024-09-16T02:00:47.535124+0000","last_became_active":"2024-09-16T02:00:32.443095+0000","last_became_peered":"2024-09-16T02:00:32.443095+0000","last_unstale":"2024-09-16T02:00:47.535124+0000","last_undegraded":"2024-09-16T02:00:47.535124+0000","last_fullsized":"2024-09-16T02:00:47.535124+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4,7],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.561554+0000","last_change":"2024-09-16T02:00:34.452964+0000","last_active":"2024-09-16T02:01:39.561554+0000","last_peered":"2024-09-16T02:01:39.561554+0000","last_clean":"2024-09-16T02:01:39.561554+0000","last_became_active":"2024-09-16T02:00:34.452617+0000","last_became_peered":"2024-09-16T02:00:34.452617+0000","last_unstale":"2024-09-16T02:01:39.561554+0000","last_undegraded":"2024-09-16T02:01:39.561554+0000","last_fullsized":"2024-09-16T02:01:39.561554+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:04.617970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.447223+0000","last_change":"2024-09-16T02:00:34.456585+0000","last_active":"2024-09-16T02:00:35.447223+0000","last_peered":"2024-09-16T02:00:35.447223+0000","last_clean":"2024-09-16T02:00:35.447223+0000","last_became_active":"2024-09-16T02:00:34.456199+0000","last_became_peered":"2024-09-16T02:00:34.456199+0000","last_unstale":"2024-09-16T02:00:35.447223+0000","last_undegraded":"2024-09-16T02:00:35.447223+0000","last_fullsized":"2024-09-16T02:00:35.447223+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:37:54.376785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:46.527336+0000","last_change":"2024-09-16T02:00:32.446324+0000","last_active":"2024-09-16T02:00:46.527336+0000","last_peered":"2024-09-16T02:00:46.527336+0000","last_clean":"2024-09-16T02:00:46.527336+0000","last_became_active":"2024-09-16T02:00:32.441840+0000","last_became_peered":"2024-09-16T02:00:32.441840+0000","last_unstale":"2024-09-16T02:00:46.527336+0000","last_undegraded":"2024-09-16T02:00:46.527336+0000","last_fullsized":"2024-09-16T02:00:46.527336+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:46:08.601738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.447200+0000","last_change":"2024-09-16T02:00:34.460925+0000","last_active":"2024-09-16T02:00:35.447200+0000","last_peered":"2024-09-16T02:00:35.447200+0000","last_clean":"2024-09-16T02:00:35.447200+0000","last_became_active":"2024-09-16T02:00:34.460581+0000","last_became_peered":"2024-09-16T02:00:34.460581+0000","last_unstale":"2024-09-16T02:00:35.447200+0000","last_undegraded":"2024-09-16T02:00:35.447200+0000","last_fullsized":"2024-09-16T02:00:35.447200+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:19:44.460695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.563378+0000","last_change":"2024-09-16T02:00:32.443877+0000","last_active":"2024-09-16T02:01:39.563378+0000","last_peered":"2024-09-16T02:01:39.563378+0000","last_clean":"2024-09-16T02:01:39.563378+0000","last_became_active":"2024-09-16T02:00:32.443416+0000","last_became_peered":"2024-09-16T02:00:32.443416+0000","last_unstale":"2024-09-16T02:01:39.563378+0000","last_undegraded":"2024-09-16T02:01:39.563378+0000","last_fullsized":"2024-09-16T02:01:39.563378+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:36:57.367232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.563554+0000","last_change":"2024-09-16T02:00:34.453048+0000","last_active":"2024-09-16T02:01:39.563554+0000","last_peered":"2024-09-16T02:01:39.563554+0000","last_clean":"2024-09-16T02:01:39.563554+0000","last_became_active":"2024-09-16T02:00:34.452752+0000","last_became_peered":"2024-09-16T02:00:34.452752+0000","last_unstale":"2024-09-16T02:01:39.563554+0000","last_undegraded":"2024-09-16T02:01:39.563554+0000","last_fullsized":"2024-09-16T02:01:39.563554+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:55.970195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.559416+0000","last_change":"2024-09-16T02:00:32.586974+0000","last_active":"2024-09-16T02:01:39.559416+0000","last_peered":"2024-09-16T02:01:39.559416+0000","last_clean":"2024-09-16T02:01:39.559416+0000","last_became_active":"2024-09-16T02:00:32.586644+0000","last_became_peered":"2024-09-16T02:00:32.586644+0000","last_unstale":"2024-09-16T02:01:39.559416+0000","last_undegraded":"2024-09-16T02:01:39.559416+0000","last_fullsized":"2024-09-16T02:01:39.559416+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:52.913287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.558953+0000","last_change":"2024-09-16T02:00:34.447548+0000","last_active":"2024-09-16T02:01:39.558953+0000","last_peered":"2024-09-16T02:01:39.558953+0000","last_clean":"2024-09-16T02:01:39.558953+0000","last_became_active":"2024-09-16T02:00:34.447207+0000","last_became_peered":"2024-09-16T02:00:34.447207+0000","last_unstale":"2024-09-16T02:01:39.558953+0000","last_undegraded":"2024-09-16T02:01:39.558953+0000","last_fullsized":"2024-09-16T02:01:39.558953+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:22:02.494343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":90,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.607561+0000","last_change":"2024-09-16T02:00:32.441129+0000","last_active":"2024-09-16T02:01:40.607561+0000","last_peered":"2024-09-16T02:01:40.607561+0000","last_clean":"2024-09-16T02:01:40.607561+0000","last_became_active":"2024-09-16T02:00:32.440843+0000","last_became_peered":"2024-09-16T02:00:32.440843+0000","last_unstale":"2024-09-16T02:01:40.607561+0000","last_undegraded":"2024-09-16T02:01:40.607561+0000","last_fullsized":"2024-09-16T02:01:40.607561+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:17.721740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66026,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":554,"num_read_kb":348,"num_write":238,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":198078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281576,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612635,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":864256,"data_stored":414884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841568,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1019904,"data_stored":1065517,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005093,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":901120,"data_stored":423573,"data_compressed":2878,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234026,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":897024,"data_stored":423951,"data_compressed":2878,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462959,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":950272,"data_stored":1016471,"data_compressed":38690,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626485,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":872448,"data_stored":414531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855417,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":1006997,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018942,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":414785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247876,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":414356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444105,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":474875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":933888,"data_stored":474655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":905216,"data_stored":424629,"data_compressed":2878,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T02:01:43.047 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T02:01:43.289 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:43.695 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:43 smithi105 ceph-mon[31197]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:01:43.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.772+0000 7f15651ae640 1 -- 172.21.15.105:0/306917950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1560102570 msgr2=0x7f15601029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:43.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.772+0000 7f15651ae640 1 --2- 172.21.15.105:0/306917950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1560102570 0x7f15601029b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1548009900 tx=0x7f154802a480 comp rx=0 tx=0).stop 2024-09-16T02:01:43.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.773+0000 7f15651ae640 1 -- 172.21.15.105:0/306917950 shutdown_connections 2024-09-16T02:01:43.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.773+0000 7f15651ae640 1 --2- 172.21.15.105:0/306917950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1560102ef0 0x7f156010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:43.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.773+0000 7f15651ae640 1 --2- 172.21.15.105:0/306917950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1560102570 0x7f15601029b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:43.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.773+0000 7f15651ae640 1 --2- 172.21.15.105:0/306917950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1560100e20 0x7f1560101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:43.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.773+0000 7f15651ae640 1 -- 172.21.15.105:0/306917950 >> 172.21.15.105:0/306917950 conn(0x7f15600fc7d0 msgr2=0x7f15600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:43.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.773+0000 7f15651ae640 1 -- 172.21.15.105:0/306917950 shutdown_connections 2024-09-16T02:01:43.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.773+0000 7f15651ae640 1 -- 172.21.15.105:0/306917950 wait complete. 2024-09-16T02:01:43.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.773+0000 7f15651ae640 1 Processor -- start 2024-09-16T02:01:43.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.773+0000 7f15651ae640 1 -- start start 2024-09-16T02:01:43.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.774+0000 7f15651ae640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1560100e20 0x7f1560198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:43.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.774+0000 7f15651ae640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1560102570 0x7f1560198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:43.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.774+0000 7f15651ae640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1560102ef0 0x7f156019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:43.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.774+0000 7f15651ae640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1560110ed0 con 0x7f1560100e20 2024-09-16T02:01:43.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.774+0000 7f15651ae640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1560110d50 con 0x7f1560102ef0 2024-09-16T02:01:43.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.774+0000 7f15651ae640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1560111050 con 0x7f1560102570 2024-09-16T02:01:43.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.774+0000 7f155ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1560100e20 0x7f1560198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:43.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.774+0000 7f155ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1560100e20 0x7f1560198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33066/0 (socket says 172.21.15.105:33066) 2024-09-16T02:01:43.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.774+0000 7f155ed9d640 1 -- 172.21.15.105:0/1448090985 learned_addr learned my addr 172.21.15.105:0/1448090985 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:01:43.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.774+0000 7f155ed9d640 1 -- 172.21.15.105:0/1448090985 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1560102570 msgr2=0x7f1560198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:43.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.774+0000 7f155ed9d640 1 --2- 172.21.15.105:0/1448090985 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1560102570 0x7f1560198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:43.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.774+0000 7f155ed9d640 1 -- 172.21.15.105:0/1448090985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1560102ef0 msgr2=0x7f156019cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:43.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.774+0000 7f155ed9d640 1 --2- 172.21.15.105:0/1448090985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1560102ef0 0x7f156019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:43.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.774+0000 7f155ed9d640 1 -- 172.21.15.105:0/1448090985 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f156019d4b0 con 0x7f1560100e20 2024-09-16T02:01:43.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.774+0000 7f155ed9d640 1 --2- 172.21.15.105:0/1448090985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1560100e20 0x7f1560198500 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f155000b4a0 tx=0x7f155000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:43.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.774+0000 7f155cd99640 1 -- 172.21.15.105:0/1448090985 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1550014070 con 0x7f1560100e20 2024-09-16T02:01:43.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.774+0000 7f155cd99640 1 -- 172.21.15.105:0/1448090985 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1550010ac0 con 0x7f1560100e20 2024-09-16T02:01:43.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.774+0000 7f15651ae640 1 -- 172.21.15.105:0/1448090985 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f156019d7a0 con 0x7f1560100e20 2024-09-16T02:01:43.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.774+0000 7f155cd99640 1 -- 172.21.15.105:0/1448090985 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1550017c30 con 0x7f1560100e20 2024-09-16T02:01:43.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.774+0000 7f15651ae640 1 -- 172.21.15.105:0/1448090985 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15601a5080 con 0x7f1560100e20 2024-09-16T02:01:43.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.775+0000 7f15651ae640 1 -- 172.21.15.105:0/1448090985 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1560101200 con 0x7f1560100e20 2024-09-16T02:01:43.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.776+0000 7f155cd99640 1 -- 172.21.15.105:0/1448090985 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f1550017070 con 0x7f1560100e20 2024-09-16T02:01:43.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.777+0000 7f155cd99640 1 --2- 172.21.15.105:0/1448090985 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f154c0a32d0 0x7f154c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:43.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.777+0000 7f1557fff640 1 --2- 172.21.15.105:0/1448090985 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f154c0a32d0 0x7f154c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:43.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.777+0000 7f155cd99640 1 -- 172.21.15.105:0/1448090985 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f15500cf130 con 0x7f1560100e20 2024-09-16T02:01:43.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.778+0000 7f1557fff640 1 --2- 172.21.15.105:0/1448090985 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f154c0a32d0 0x7f154c0a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f154802a9a0 tx=0x7f1548000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:43.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.781+0000 7f155cd99640 1 -- 172.21.15.105:0/1448090985 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f155009bde0 con 0x7f1560100e20 2024-09-16T02:01:43.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:43 smithi190 ceph-mon[34762]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:01:43.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:43 smithi137 ceph-mon[34099]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:01:43.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.944+0000 7f15651ae640 1 -- 172.21.15.105:0/1448090985 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1560062630 con 0x7f154c0a32d0 2024-09-16T02:01:43.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.950+0000 7f155cd99640 1 -- 172.21.15.105:0/1448090985 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258849 (secure 0 0 0) 0x7f1560062630 con 0x7f154c0a32d0 2024-09-16T02:01:43.954 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:01:43.955 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-09-16T02:01:43.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.953+0000 7f15651ae640 1 -- 172.21.15.105:0/1448090985 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f154c0a32d0 msgr2=0x7f154c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:43.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.953+0000 7f15651ae640 1 --2- 172.21.15.105:0/1448090985 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f154c0a32d0 0x7f154c0a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f154802a9a0 tx=0x7f1548000bc0 comp rx=0 tx=0).stop 2024-09-16T02:01:43.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.953+0000 7f15651ae640 1 -- 172.21.15.105:0/1448090985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1560100e20 msgr2=0x7f1560198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:43.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.953+0000 7f15651ae640 1 --2- 172.21.15.105:0/1448090985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1560100e20 0x7f1560198500 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f155000b4a0 tx=0x7f155000b960 comp rx=0 tx=0).stop 2024-09-16T02:01:43.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.953+0000 7f15651ae640 1 -- 172.21.15.105:0/1448090985 shutdown_connections 2024-09-16T02:01:43.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.953+0000 7f15651ae640 1 --2- 172.21.15.105:0/1448090985 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f154c0a32d0 0x7f154c0a5790 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:43.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.953+0000 7f15651ae640 1 --2- 172.21.15.105:0/1448090985 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1560102570 0x7f1560198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:43.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.953+0000 7f15651ae640 1 --2- 172.21.15.105:0/1448090985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1560102ef0 0x7f156019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:43.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.953+0000 7f15651ae640 1 --2- 172.21.15.105:0/1448090985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1560100e20 0x7f1560198500 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:43.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.953+0000 7f15651ae640 1 -- 172.21.15.105:0/1448090985 >> 172.21.15.105:0/1448090985 conn(0x7f15600fc7d0 msgr2=0x7f156006a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:43.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.953+0000 7f15651ae640 1 -- 172.21.15.105:0/1448090985 shutdown_connections 2024-09-16T02:01:43.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:43.953+0000 7f15651ae640 1 -- 172.21.15.105:0/1448090985 wait complete. 2024-09-16T02:01:44.651 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:44 smithi105 ceph-mon[31197]: pgmap v313: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s; 72 B/s, 0 objects/s recovering 2024-09-16T02:01:44.681 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":313,"stamp":"2024-09-16T02:01:43.460453+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656628,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":699,"num_read_kb":471,"num_write":421,"num_write_kb":3070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":292,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364236,"kb_used_data":10804,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577588,"statfs":{"total":1151940427776,"available":1151567450112,"internally_reserved":0,"allocated":11063296,"data_stored":7004374,"data_compressed":130474,"data_compressed_allocated":167936,"data_compressed_original":1835008,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":44,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001905"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560827+0000","last_change":"2024-09-16T02:00:34.454121+0000","last_active":"2024-09-16T02:01:40.560827+0000","last_peered":"2024-09-16T02:01:40.560827+0000","last_clean":"2024-09-16T02:01:40.560827+0000","last_became_active":"2024-09-16T02:00:34.453810+0000","last_became_peered":"2024-09-16T02:00:34.453810+0000","last_unstale":"2024-09-16T02:01:40.560827+0000","last_undegraded":"2024-09-16T02:01:40.560827+0000","last_fullsized":"2024-09-16T02:01:40.560827+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:42:38.781928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.563230+0000","last_change":"2024-09-16T02:00:34.462020+0000","last_active":"2024-09-16T02:01:39.563230+0000","last_peered":"2024-09-16T02:01:39.563230+0000","last_clean":"2024-09-16T02:01:39.563230+0000","last_became_active":"2024-09-16T02:00:34.461680+0000","last_became_peered":"2024-09-16T02:00:34.461680+0000","last_unstale":"2024-09-16T02:01:39.563230+0000","last_undegraded":"2024-09-16T02:01:39.563230+0000","last_fullsized":"2024-09-16T02:01:39.563230+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:00:14.493643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.562710+0000","last_change":"2024-09-16T02:00:34.453034+0000","last_active":"2024-09-16T02:01:39.562710+0000","last_peered":"2024-09-16T02:01:39.562710+0000","last_clean":"2024-09-16T02:01:39.562710+0000","last_became_active":"2024-09-16T02:00:34.452719+0000","last_became_peered":"2024-09-16T02:00:34.452719+0000","last_unstale":"2024-09-16T02:01:39.562710+0000","last_undegraded":"2024-09-16T02:01:39.562710+0000","last_fullsized":"2024-09-16T02:01:39.562710+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:48:52.478501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458864+0000","last_change":"2024-09-16T02:00:34.458156+0000","last_active":"2024-09-16T02:00:35.458864+0000","last_peered":"2024-09-16T02:00:35.458864+0000","last_clean":"2024-09-16T02:00:35.458864+0000","last_became_active":"2024-09-16T02:00:34.457831+0000","last_became_peered":"2024-09-16T02:00:34.457831+0000","last_unstale":"2024-09-16T02:00:35.458864+0000","last_undegraded":"2024-09-16T02:00:35.458864+0000","last_fullsized":"2024-09-16T02:00:35.458864+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:35:36.609046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.562428+0000","last_change":"2024-09-16T02:00:34.448638+0000","last_active":"2024-09-16T02:01:39.562428+0000","last_peered":"2024-09-16T02:01:39.562428+0000","last_clean":"2024-09-16T02:01:39.562428+0000","last_became_active":"2024-09-16T02:00:34.447993+0000","last_became_peered":"2024-09-16T02:00:34.447993+0000","last_unstale":"2024-09-16T02:01:39.562428+0000","last_undegraded":"2024-09-16T02:01:39.562428+0000","last_fullsized":"2024-09-16T02:01:39.562428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:33:04.832053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560794+0000","last_change":"2024-09-16T02:00:34.462651+0000","last_active":"2024-09-16T02:01:40.560794+0000","last_peered":"2024-09-16T02:01:40.560794+0000","last_clean":"2024-09-16T02:01:40.560794+0000","last_became_active":"2024-09-16T02:00:34.461837+0000","last_became_peered":"2024-09-16T02:00:34.461837+0000","last_unstale":"2024-09-16T02:01:40.560794+0000","last_undegraded":"2024-09-16T02:01:40.560794+0000","last_fullsized":"2024-09-16T02:01:40.560794+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:52:13.733591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.562619+0000","last_change":"2024-09-16T02:00:34.455362+0000","last_active":"2024-09-16T02:01:39.562619+0000","last_peered":"2024-09-16T02:01:39.562619+0000","last_clean":"2024-09-16T02:01:39.562619+0000","last_became_active":"2024-09-16T02:00:34.455029+0000","last_became_peered":"2024-09-16T02:00:34.455029+0000","last_unstale":"2024-09-16T02:01:39.562619+0000","last_undegraded":"2024-09-16T02:01:39.562619+0000","last_fullsized":"2024-09-16T02:01:39.562619+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:14:08.316845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.558582+0000","last_change":"2024-09-16T02:00:34.459578+0000","last_active":"2024-09-16T02:01:39.558582+0000","last_peered":"2024-09-16T02:01:39.558582+0000","last_clean":"2024-09-16T02:01:39.558582+0000","last_became_active":"2024-09-16T02:00:34.459288+0000","last_became_peered":"2024-09-16T02:00:34.459288+0000","last_unstale":"2024-09-16T02:01:39.558582+0000","last_undegraded":"2024-09-16T02:01:39.558582+0000","last_fullsized":"2024-09-16T02:01:39.558582+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:52:28.974276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.447603+0000","last_change":"2024-09-16T02:00:34.463011+0000","last_active":"2024-09-16T02:00:35.447603+0000","last_peered":"2024-09-16T02:00:35.447603+0000","last_clean":"2024-09-16T02:00:35.447603+0000","last_became_active":"2024-09-16T02:00:34.462668+0000","last_became_peered":"2024-09-16T02:00:34.462668+0000","last_unstale":"2024-09-16T02:00:35.447603+0000","last_undegraded":"2024-09-16T02:00:35.447603+0000","last_fullsized":"2024-09-16T02:00:35.447603+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:06:31.655869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560622+0000","last_change":"2024-09-16T02:00:34.462241+0000","last_active":"2024-09-16T02:01:40.560622+0000","last_peered":"2024-09-16T02:01:40.560622+0000","last_clean":"2024-09-16T02:01:40.560622+0000","last_became_active":"2024-09-16T02:00:34.461925+0000","last_became_peered":"2024-09-16T02:00:34.461925+0000","last_unstale":"2024-09-16T02:01:40.560622+0000","last_undegraded":"2024-09-16T02:01:40.560622+0000","last_fullsized":"2024-09-16T02:01:40.560622+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:48:02.414471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.561882+0000","last_change":"2024-09-16T02:00:34.466132+0000","last_active":"2024-09-16T02:01:39.561882+0000","last_peered":"2024-09-16T02:01:39.561882+0000","last_clean":"2024-09-16T02:01:39.561882+0000","last_became_active":"2024-09-16T02:00:34.465750+0000","last_became_peered":"2024-09-16T02:00:34.465750+0000","last_unstale":"2024-09-16T02:01:39.561882+0000","last_undegraded":"2024-09-16T02:01:39.561882+0000","last_fullsized":"2024-09-16T02:01:39.561882+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:07:33.072432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458928+0000","last_change":"2024-09-16T02:00:34.461759+0000","last_active":"2024-09-16T02:00:35.458928+0000","last_peered":"2024-09-16T02:00:35.458928+0000","last_clean":"2024-09-16T02:00:35.458928+0000","last_became_active":"2024-09-16T02:00:34.461385+0000","last_became_peered":"2024-09-16T02:00:34.461385+0000","last_unstale":"2024-09-16T02:00:35.458928+0000","last_undegraded":"2024-09-16T02:00:35.458928+0000","last_fullsized":"2024-09-16T02:00:35.458928+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:00:24.179172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.446281+0000","last_change":"2024-09-16T02:00:34.456584+0000","last_active":"2024-09-16T02:00:35.446281+0000","last_peered":"2024-09-16T02:00:35.446281+0000","last_clean":"2024-09-16T02:00:35.446281+0000","last_became_active":"2024-09-16T02:00:34.456257+0000","last_became_peered":"2024-09-16T02:00:34.456257+0000","last_unstale":"2024-09-16T02:00:35.446281+0000","last_undegraded":"2024-09-16T02:00:35.446281+0000","last_fullsized":"2024-09-16T02:00:35.446281+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:26:45.835435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.562760+0000","last_change":"2024-09-16T02:00:34.460680+0000","last_active":"2024-09-16T02:01:39.562760+0000","last_peered":"2024-09-16T02:01:39.562760+0000","last_clean":"2024-09-16T02:01:39.562760+0000","last_became_active":"2024-09-16T02:00:34.460161+0000","last_became_peered":"2024-09-16T02:00:34.460161+0000","last_unstale":"2024-09-16T02:01:39.562760+0000","last_undegraded":"2024-09-16T02:01:39.562760+0000","last_fullsized":"2024-09-16T02:01:39.562760+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:38:19.848420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.560921+0000","last_change":"2024-09-16T02:00:34.454606+0000","last_active":"2024-09-16T02:01:39.560921+0000","last_peered":"2024-09-16T02:01:39.560921+0000","last_clean":"2024-09-16T02:01:39.560921+0000","last_became_active":"2024-09-16T02:00:34.454293+0000","last_became_peered":"2024-09-16T02:00:34.454293+0000","last_unstale":"2024-09-16T02:01:39.560921+0000","last_undegraded":"2024-09-16T02:01:39.560921+0000","last_fullsized":"2024-09-16T02:01:39.560921+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:55:54.466176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.446992+0000","last_change":"2024-09-16T02:00:34.452940+0000","last_active":"2024-09-16T02:00:35.446992+0000","last_peered":"2024-09-16T02:00:35.446992+0000","last_clean":"2024-09-16T02:00:35.446992+0000","last_became_active":"2024-09-16T02:00:34.452560+0000","last_became_peered":"2024-09-16T02:00:34.452560+0000","last_unstale":"2024-09-16T02:00:35.446992+0000","last_undegraded":"2024-09-16T02:00:35.446992+0000","last_fullsized":"2024-09-16T02:00:35.446992+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:42:11.564264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560911+0000","last_change":"2024-09-16T02:00:34.453457+0000","last_active":"2024-09-16T02:01:40.560911+0000","last_peered":"2024-09-16T02:01:40.560911+0000","last_clean":"2024-09-16T02:01:40.560911+0000","last_became_active":"2024-09-16T02:00:34.453148+0000","last_became_peered":"2024-09-16T02:00:34.453148+0000","last_unstale":"2024-09-16T02:01:40.560911+0000","last_undegraded":"2024-09-16T02:01:40.560911+0000","last_fullsized":"2024-09-16T02:01:40.560911+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:28.456892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.607401+0000","last_change":"2024-09-16T02:00:34.457057+0000","last_active":"2024-09-16T02:01:40.607401+0000","last_peered":"2024-09-16T02:01:40.607401+0000","last_clean":"2024-09-16T02:01:40.607401+0000","last_became_active":"2024-09-16T02:00:34.456751+0000","last_became_peered":"2024-09-16T02:00:34.456751+0000","last_unstale":"2024-09-16T02:01:40.607401+0000","last_undegraded":"2024-09-16T02:01:40.607401+0000","last_fullsized":"2024-09-16T02:01:40.607401+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:32:32.207510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458938+0000","last_change":"2024-09-16T02:00:34.448434+0000","last_active":"2024-09-16T02:00:35.458938+0000","last_peered":"2024-09-16T02:00:35.458938+0000","last_clean":"2024-09-16T02:00:35.458938+0000","last_became_active":"2024-09-16T02:00:34.448056+0000","last_became_peered":"2024-09-16T02:00:34.448056+0000","last_unstale":"2024-09-16T02:00:35.458938+0000","last_undegraded":"2024-09-16T02:00:35.458938+0000","last_fullsized":"2024-09-16T02:00:35.458938+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:15:28.162159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560368+0000","last_change":"2024-09-16T02:00:34.459499+0000","last_active":"2024-09-16T02:01:40.560368+0000","last_peered":"2024-09-16T02:01:40.560368+0000","last_clean":"2024-09-16T02:01:40.560368+0000","last_became_active":"2024-09-16T02:00:34.459017+0000","last_became_peered":"2024-09-16T02:00:34.459017+0000","last_unstale":"2024-09-16T02:01:40.560368+0000","last_undegraded":"2024-09-16T02:01:40.560368+0000","last_fullsized":"2024-09-16T02:01:40.560368+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:41:36.866100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.446287+0000","last_change":"2024-09-16T02:00:34.448640+0000","last_active":"2024-09-16T02:00:35.446287+0000","last_peered":"2024-09-16T02:00:35.446287+0000","last_clean":"2024-09-16T02:00:35.446287+0000","last_became_active":"2024-09-16T02:00:34.447737+0000","last_became_peered":"2024-09-16T02:00:34.447737+0000","last_unstale":"2024-09-16T02:00:35.446287+0000","last_undegraded":"2024-09-16T02:00:35.446287+0000","last_fullsized":"2024-09-16T02:00:35.446287+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:20.192650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.447240+0000","last_change":"2024-09-16T02:00:34.451718+0000","last_active":"2024-09-16T02:00:35.447240+0000","last_peered":"2024-09-16T02:00:35.447240+0000","last_clean":"2024-09-16T02:00:35.447240+0000","last_became_active":"2024-09-16T02:00:34.451389+0000","last_became_peered":"2024-09-16T02:00:34.451389+0000","last_unstale":"2024-09-16T02:00:35.447240+0000","last_undegraded":"2024-09-16T02:00:35.447240+0000","last_fullsized":"2024-09-16T02:00:35.447240+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:13:19.296178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.558628+0000","last_change":"2024-09-16T02:00:34.458379+0000","last_active":"2024-09-16T02:01:39.558628+0000","last_peered":"2024-09-16T02:01:39.558628+0000","last_clean":"2024-09-16T02:01:39.558628+0000","last_became_active":"2024-09-16T02:00:34.458038+0000","last_became_peered":"2024-09-16T02:00:34.458038+0000","last_unstale":"2024-09-16T02:01:39.558628+0000","last_undegraded":"2024-09-16T02:01:39.558628+0000","last_fullsized":"2024-09-16T02:01:39.558628+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:34.669470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.446886+0000","last_change":"2024-09-16T02:00:34.450927+0000","last_active":"2024-09-16T02:00:35.446886+0000","last_peered":"2024-09-16T02:00:35.446886+0000","last_clean":"2024-09-16T02:00:35.446886+0000","last_became_active":"2024-09-16T02:00:34.447669+0000","last_became_peered":"2024-09-16T02:00:34.447669+0000","last_unstale":"2024-09-16T02:00:35.446886+0000","last_undegraded":"2024-09-16T02:00:35.446886+0000","last_fullsized":"2024-09-16T02:00:35.446886+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:57.861633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.559318+0000","last_change":"2024-09-16T02:00:34.454827+0000","last_active":"2024-09-16T02:01:39.559318+0000","last_peered":"2024-09-16T02:01:39.559318+0000","last_clean":"2024-09-16T02:01:39.559318+0000","last_became_active":"2024-09-16T02:00:34.454549+0000","last_became_peered":"2024-09-16T02:00:34.454549+0000","last_unstale":"2024-09-16T02:01:39.559318+0000","last_undegraded":"2024-09-16T02:01:39.559318+0000","last_fullsized":"2024-09-16T02:01:39.559318+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:53:45.440860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.461087+0000","last_change":"2024-09-16T02:00:34.455121+0000","last_active":"2024-09-16T02:00:35.461087+0000","last_peered":"2024-09-16T02:00:35.461087+0000","last_clean":"2024-09-16T02:00:35.461087+0000","last_became_active":"2024-09-16T02:00:34.454665+0000","last_became_peered":"2024-09-16T02:00:34.454665+0000","last_unstale":"2024-09-16T02:00:35.461087+0000","last_undegraded":"2024-09-16T02:00:35.461087+0000","last_fullsized":"2024-09-16T02:00:35.461087+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:31:42.771230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560656+0000","last_change":"2024-09-16T02:00:34.458777+0000","last_active":"2024-09-16T02:01:40.560656+0000","last_peered":"2024-09-16T02:01:40.560656+0000","last_clean":"2024-09-16T02:01:40.560656+0000","last_became_active":"2024-09-16T02:00:34.457954+0000","last_became_peered":"2024-09-16T02:00:34.457954+0000","last_unstale":"2024-09-16T02:01:40.560656+0000","last_undegraded":"2024-09-16T02:01:40.560656+0000","last_fullsized":"2024-09-16T02:01:40.560656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:58:28.032293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.446190+0000","last_change":"2024-09-16T02:00:34.444712+0000","last_active":"2024-09-16T02:00:35.446190+0000","last_peered":"2024-09-16T02:00:35.446190+0000","last_clean":"2024-09-16T02:00:35.446190+0000","last_became_active":"2024-09-16T02:00:34.444421+0000","last_became_peered":"2024-09-16T02:00:34.444421+0000","last_unstale":"2024-09-16T02:00:35.446190+0000","last_undegraded":"2024-09-16T02:00:35.446190+0000","last_fullsized":"2024-09-16T02:00:35.446190+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:36.557558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.563059+0000","last_change":"2024-09-16T02:00:34.454570+0000","last_active":"2024-09-16T02:01:39.563059+0000","last_peered":"2024-09-16T02:01:39.563059+0000","last_clean":"2024-09-16T02:01:39.563059+0000","last_became_active":"2024-09-16T02:00:34.454246+0000","last_became_peered":"2024-09-16T02:00:34.454246+0000","last_unstale":"2024-09-16T02:01:39.563059+0000","last_undegraded":"2024-09-16T02:01:39.563059+0000","last_fullsized":"2024-09-16T02:01:39.563059+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:36:18.634907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458979+0000","last_change":"2024-09-16T02:00:34.449680+0000","last_active":"2024-09-16T02:00:35.458979+0000","last_peered":"2024-09-16T02:00:35.458979+0000","last_clean":"2024-09-16T02:00:35.458979+0000","last_became_active":"2024-09-16T02:00:34.449184+0000","last_became_peered":"2024-09-16T02:00:34.449184+0000","last_unstale":"2024-09-16T02:00:35.458979+0000","last_undegraded":"2024-09-16T02:00:35.458979+0000","last_fullsized":"2024-09-16T02:00:35.458979+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:28:34.867820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.458803+0000","last_change":"2024-09-16T02:00:34.455362+0000","last_active":"2024-09-16T02:00:35.458803+0000","last_peered":"2024-09-16T02:00:35.458803+0000","last_clean":"2024-09-16T02:00:35.458803+0000","last_became_active":"2024-09-16T02:00:34.455018+0000","last_became_peered":"2024-09-16T02:00:34.455018+0000","last_unstale":"2024-09-16T02:00:35.458803+0000","last_undegraded":"2024-09-16T02:00:35.458803+0000","last_fullsized":"2024-09-16T02:00:35.458803+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:38.278418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560207+0000","last_change":"2024-09-16T02:00:34.459426+0000","last_active":"2024-09-16T02:01:40.560207+0000","last_peered":"2024-09-16T02:01:40.560207+0000","last_clean":"2024-09-16T02:01:40.560207+0000","last_became_active":"2024-09-16T02:00:34.459082+0000","last_became_peered":"2024-09-16T02:00:34.459082+0000","last_unstale":"2024-09-16T02:01:40.560207+0000","last_undegraded":"2024-09-16T02:01:40.560207+0000","last_fullsized":"2024-09-16T02:01:40.560207+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:59:42.226065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.562286+0000","last_change":"2024-09-16T02:00:34.450936+0000","last_active":"2024-09-16T02:01:39.562286+0000","last_peered":"2024-09-16T02:01:39.562286+0000","last_clean":"2024-09-16T02:01:39.562286+0000","last_became_active":"2024-09-16T02:00:34.450503+0000","last_became_peered":"2024-09-16T02:00:34.450503+0000","last_unstale":"2024-09-16T02:01:39.562286+0000","last_undegraded":"2024-09-16T02:01:39.562286+0000","last_fullsized":"2024-09-16T02:01:39.562286+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:35:55.806193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'5","reported_seq":77,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:01:37.603159+0000","last_change":"2024-09-16T02:00:32.443193+0000","last_active":"2024-09-16T02:01:37.603159+0000","last_peered":"2024-09-16T02:01:37.603159+0000","last_clean":"2024-09-16T02:01:37.603159+0000","last_became_active":"2024-09-16T02:00:32.442863+0000","last_became_peered":"2024-09-16T02:00:32.442863+0000","last_unstale":"2024-09-16T02:01:37.603159+0000","last_undegraded":"2024-09-16T02:01:37.603159+0000","last_fullsized":"2024-09-16T02:01:37.603159+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:19.734946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.447318+0000","last_change":"2024-09-16T02:00:34.454819+0000","last_active":"2024-09-16T02:00:35.447318+0000","last_peered":"2024-09-16T02:00:35.447318+0000","last_clean":"2024-09-16T02:00:35.447318+0000","last_became_active":"2024-09-16T02:00:34.454492+0000","last_became_peered":"2024-09-16T02:00:34.454492+0000","last_unstale":"2024-09-16T02:00:35.447318+0000","last_undegraded":"2024-09-16T02:00:35.447318+0000","last_fullsized":"2024-09-16T02:00:35.447318+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:11.479330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.572282+0000","last_change":"2024-09-16T02:01:40.572282+0000","last_active":"2024-09-16T02:01:40.572282+0000","last_peered":"2024-09-16T02:01:40.572282+0000","last_clean":"2024-09-16T02:01:40.572282+0000","last_became_active":"2024-09-16T02:01:40.564577+0000","last_became_peered":"2024-09-16T02:01:40.564577+0000","last_unstale":"2024-09-16T02:01:40.572282+0000","last_undegraded":"2024-09-16T02:01:40.572282+0000","last_fullsized":"2024-09-16T02:01:40.572282+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:19:40.834173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:44.506262+0000","last_change":"2024-09-16T02:00:32.438530+0000","last_active":"2024-09-16T02:00:44.506262+0000","last_peered":"2024-09-16T02:00:44.506262+0000","last_clean":"2024-09-16T02:00:44.506262+0000","last_became_active":"2024-09-16T02:00:32.438238+0000","last_became_peered":"2024-09-16T02:00:32.438238+0000","last_unstale":"2024-09-16T02:00:44.506262+0000","last_undegraded":"2024-09-16T02:00:44.506262+0000","last_fullsized":"2024-09-16T02:00:44.506262+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:52:32.799701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.558632+0000","last_change":"2024-09-16T02:00:34.456299+0000","last_active":"2024-09-16T02:01:39.558632+0000","last_peered":"2024-09-16T02:01:39.558632+0000","last_clean":"2024-09-16T02:01:39.558632+0000","last_became_active":"2024-09-16T02:00:34.455925+0000","last_became_peered":"2024-09-16T02:00:34.455925+0000","last_unstale":"2024-09-16T02:01:39.558632+0000","last_undegraded":"2024-09-16T02:01:39.558632+0000","last_fullsized":"2024-09-16T02:01:39.558632+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:33:27.162664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:46.527624+0000","last_change":"2024-09-16T02:00:32.442029+0000","last_active":"2024-09-16T02:00:46.527624+0000","last_peered":"2024-09-16T02:00:46.527624+0000","last_clean":"2024-09-16T02:00:46.527624+0000","last_became_active":"2024-09-16T02:00:32.441705+0000","last_became_peered":"2024-09-16T02:00:32.441705+0000","last_unstale":"2024-09-16T02:00:46.527624+0000","last_undegraded":"2024-09-16T02:00:46.527624+0000","last_fullsized":"2024-09-16T02:00:46.527624+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:50:54.725768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.607165+0000","last_change":"2024-09-16T02:00:34.457833+0000","last_active":"2024-09-16T02:01:40.607165+0000","last_peered":"2024-09-16T02:01:40.607165+0000","last_clean":"2024-09-16T02:01:40.607165+0000","last_became_active":"2024-09-16T02:00:34.457453+0000","last_became_peered":"2024-09-16T02:00:34.457453+0000","last_unstale":"2024-09-16T02:01:40.607165+0000","last_undegraded":"2024-09-16T02:01:40.607165+0000","last_fullsized":"2024-09-16T02:01:40.607165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:21.777574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":136,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:41.607968+0000","last_change":"2024-09-16T02:00:32.441341+0000","last_active":"2024-09-16T02:01:41.607968+0000","last_peered":"2024-09-16T02:01:41.607968+0000","last_clean":"2024-09-16T02:01:41.607968+0000","last_became_active":"2024-09-16T02:00:32.441019+0000","last_became_peered":"2024-09-16T02:00:32.441019+0000","last_unstale":"2024-09-16T02:01:41.607968+0000","last_undegraded":"2024-09-16T02:01:41.607968+0000","last_fullsized":"2024-09-16T02:01:41.607968+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:10:10.171714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":110,"num_read_kb":110,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.447426+0000","last_change":"2024-09-16T02:00:34.461746+0000","last_active":"2024-09-16T02:00:35.447426+0000","last_peered":"2024-09-16T02:00:35.447426+0000","last_clean":"2024-09-16T02:00:35.447426+0000","last_became_active":"2024-09-16T02:00:34.461441+0000","last_became_peered":"2024-09-16T02:00:34.461441+0000","last_unstale":"2024-09-16T02:00:35.447426+0000","last_undegraded":"2024-09-16T02:00:35.447426+0000","last_fullsized":"2024-09-16T02:00:35.447426+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:28:14.519159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.561070+0000","last_change":"2024-09-16T02:00:32.443129+0000","last_active":"2024-09-16T02:01:39.561070+0000","last_peered":"2024-09-16T02:01:39.561070+0000","last_clean":"2024-09-16T02:01:39.561070+0000","last_became_active":"2024-09-16T02:00:32.442772+0000","last_became_peered":"2024-09-16T02:00:32.442772+0000","last_unstale":"2024-09-16T02:01:39.561070+0000","last_undegraded":"2024-09-16T02:01:39.561070+0000","last_fullsized":"2024-09-16T02:01:39.561070+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:10:03.964583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.563645+0000","last_change":"2024-09-16T02:00:34.457449+0000","last_active":"2024-09-16T02:01:39.563645+0000","last_peered":"2024-09-16T02:01:39.563645+0000","last_clean":"2024-09-16T02:01:39.563645+0000","last_became_active":"2024-09-16T02:00:34.457110+0000","last_became_peered":"2024-09-16T02:00:34.457110+0000","last_unstale":"2024-09-16T02:01:39.563645+0000","last_undegraded":"2024-09-16T02:01:39.563645+0000","last_fullsized":"2024-09-16T02:01:39.563645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:35.662503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:45.516635+0000","last_change":"2024-09-16T02:00:32.587419+0000","last_active":"2024-09-16T02:00:45.516635+0000","last_peered":"2024-09-16T02:00:45.516635+0000","last_clean":"2024-09-16T02:00:45.516635+0000","last_became_active":"2024-09-16T02:00:32.587037+0000","last_became_peered":"2024-09-16T02:00:32.587037+0000","last_unstale":"2024-09-16T02:00:45.516635+0000","last_undegraded":"2024-09-16T02:00:45.516635+0000","last_fullsized":"2024-09-16T02:00:45.516635+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:11.835505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.447483+0000","last_change":"2024-09-16T02:00:34.456371+0000","last_active":"2024-09-16T02:00:35.447483+0000","last_peered":"2024-09-16T02:00:35.447483+0000","last_clean":"2024-09-16T02:00:35.447483+0000","last_became_active":"2024-09-16T02:00:34.456013+0000","last_became_peered":"2024-09-16T02:00:34.456013+0000","last_unstale":"2024-09-16T02:00:35.447483+0000","last_undegraded":"2024-09-16T02:00:35.447483+0000","last_fullsized":"2024-09-16T02:00:35.447483+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:30:15.847189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'28","reported_seq":168,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:41.607971+0000","last_change":"2024-09-16T02:00:32.444922+0000","last_active":"2024-09-16T02:01:41.607971+0000","last_peered":"2024-09-16T02:01:41.607971+0000","last_clean":"2024-09-16T02:01:41.607971+0000","last_became_active":"2024-09-16T02:00:32.444540+0000","last_became_peered":"2024-09-16T02:00:32.444540+0000","last_unstale":"2024-09-16T02:01:41.607971+0000","last_undegraded":"2024-09-16T02:01:41.607971+0000","last_fullsized":"2024-09-16T02:01:41.607971+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:59.183952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59211,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":118,"num_read_kb":116,"num_write":30,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.563273+0000","last_change":"2024-09-16T02:00:34.453366+0000","last_active":"2024-09-16T02:01:39.563273+0000","last_peered":"2024-09-16T02:01:39.563273+0000","last_clean":"2024-09-16T02:01:39.563273+0000","last_became_active":"2024-09-16T02:00:34.453048+0000","last_became_peered":"2024-09-16T02:00:34.453048+0000","last_unstale":"2024-09-16T02:01:39.563273+0000","last_undegraded":"2024-09-16T02:01:39.563273+0000","last_fullsized":"2024-09-16T02:01:39.563273+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:15:52.721893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:47.535399+0000","last_change":"2024-09-16T02:00:32.445516+0000","last_active":"2024-09-16T02:00:47.535399+0000","last_peered":"2024-09-16T02:00:47.535399+0000","last_clean":"2024-09-16T02:00:47.535399+0000","last_became_active":"2024-09-16T02:00:32.445092+0000","last_became_peered":"2024-09-16T02:00:32.445092+0000","last_unstale":"2024-09-16T02:00:47.535399+0000","last_undegraded":"2024-09-16T02:00:47.535399+0000","last_fullsized":"2024-09-16T02:00:47.535399+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:54.274692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.446077+0000","last_change":"2024-09-16T02:00:34.448450+0000","last_active":"2024-09-16T02:00:35.446077+0000","last_peered":"2024-09-16T02:00:35.446077+0000","last_clean":"2024-09-16T02:00:35.446077+0000","last_became_active":"2024-09-16T02:00:34.448116+0000","last_became_peered":"2024-09-16T02:00:34.448116+0000","last_unstale":"2024-09-16T02:00:35.446077+0000","last_undegraded":"2024-09-16T02:00:35.446077+0000","last_fullsized":"2024-09-16T02:00:35.446077+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:12:34.834798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:46.527344+0000","last_change":"2024-09-16T02:00:32.445427+0000","last_active":"2024-09-16T02:00:46.527344+0000","last_peered":"2024-09-16T02:00:46.527344+0000","last_clean":"2024-09-16T02:00:46.527344+0000","last_became_active":"2024-09-16T02:00:32.445058+0000","last_became_peered":"2024-09-16T02:00:32.445058+0000","last_unstale":"2024-09-16T02:00:46.527344+0000","last_undegraded":"2024-09-16T02:00:46.527344+0000","last_fullsized":"2024-09-16T02:00:46.527344+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:35:34.701331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.561778+0000","last_change":"2024-09-16T02:00:34.458479+0000","last_active":"2024-09-16T02:01:39.561778+0000","last_peered":"2024-09-16T02:01:39.561778+0000","last_clean":"2024-09-16T02:01:39.561778+0000","last_became_active":"2024-09-16T02:00:34.458135+0000","last_became_peered":"2024-09-16T02:00:34.458135+0000","last_unstale":"2024-09-16T02:01:39.561778+0000","last_undegraded":"2024-09-16T02:01:39.561778+0000","last_fullsized":"2024-09-16T02:01:39.561778+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:37:40.188861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'6","reported_seq":79,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:01:37.603159+0000","last_change":"2024-09-16T02:00:32.440319+0000","last_active":"2024-09-16T02:01:37.603159+0000","last_peered":"2024-09-16T02:01:37.603159+0000","last_clean":"2024-09-16T02:01:37.603159+0000","last_became_active":"2024-09-16T02:00:32.440005+0000","last_became_peered":"2024-09-16T02:00:32.440005+0000","last_unstale":"2024-09-16T02:01:37.603159+0000","last_undegraded":"2024-09-16T02:01:37.603159+0000","last_fullsized":"2024-09-16T02:01:37.603159+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:07:33.876577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.561168+0000","last_change":"2024-09-16T02:00:34.460468+0000","last_active":"2024-09-16T02:01:39.561168+0000","last_peered":"2024-09-16T02:01:39.561168+0000","last_clean":"2024-09-16T02:01:39.561168+0000","last_became_active":"2024-09-16T02:00:34.459190+0000","last_became_peered":"2024-09-16T02:00:34.459190+0000","last_unstale":"2024-09-16T02:01:39.561168+0000","last_undegraded":"2024-09-16T02:01:39.561168+0000","last_fullsized":"2024-09-16T02:01:39.561168+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:46:43.654991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.562089+0000","last_change":"2024-09-16T02:00:32.587700+0000","last_active":"2024-09-16T02:01:39.562089+0000","last_peered":"2024-09-16T02:01:39.562089+0000","last_clean":"2024-09-16T02:01:39.562089+0000","last_became_active":"2024-09-16T02:00:32.586964+0000","last_became_peered":"2024-09-16T02:00:32.586964+0000","last_unstale":"2024-09-16T02:01:39.562089+0000","last_undegraded":"2024-09-16T02:01:39.562089+0000","last_fullsized":"2024-09-16T02:01:39.562089+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:41.309918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.446915+0000","last_change":"2024-09-16T02:00:34.455652+0000","last_active":"2024-09-16T02:00:35.446915+0000","last_peered":"2024-09-16T02:00:35.446915+0000","last_clean":"2024-09-16T02:00:35.446915+0000","last_became_active":"2024-09-16T02:00:34.455306+0000","last_became_peered":"2024-09-16T02:00:34.455306+0000","last_unstale":"2024-09-16T02:00:35.446915+0000","last_undegraded":"2024-09-16T02:00:35.446915+0000","last_fullsized":"2024-09-16T02:00:35.446915+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:44:09.505096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.563741+0000","last_change":"2024-09-16T02:00:32.441443+0000","last_active":"2024-09-16T02:01:39.563741+0000","last_peered":"2024-09-16T02:01:39.563741+0000","last_clean":"2024-09-16T02:01:39.563741+0000","last_became_active":"2024-09-16T02:00:32.441073+0000","last_became_peered":"2024-09-16T02:00:32.441073+0000","last_unstale":"2024-09-16T02:01:39.563741+0000","last_undegraded":"2024-09-16T02:01:39.563741+0000","last_fullsized":"2024-09-16T02:01:39.563741+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:16.865970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.611317+0000","last_change":"2024-09-16T02:01:40.611317+0000","last_active":"2024-09-16T02:01:40.611317+0000","last_peered":"2024-09-16T02:01:40.611317+0000","last_clean":"2024-09-16T02:01:40.611317+0000","last_became_active":"2024-09-16T02:01:40.610837+0000","last_became_peered":"2024-09-16T02:01:40.610837+0000","last_unstale":"2024-09-16T02:01:40.611317+0000","last_undegraded":"2024-09-16T02:01:40.611317+0000","last_fullsized":"2024-09-16T02:01:40.611317+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:19:55.376738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:46.526168+0000","last_change":"2024-09-16T02:00:32.441728+0000","last_active":"2024-09-16T02:00:46.526168+0000","last_peered":"2024-09-16T02:00:46.526168+0000","last_clean":"2024-09-16T02:00:46.526168+0000","last_became_active":"2024-09-16T02:00:32.441388+0000","last_became_peered":"2024-09-16T02:00:32.441388+0000","last_unstale":"2024-09-16T02:00:46.526168+0000","last_undegraded":"2024-09-16T02:00:46.526168+0000","last_fullsized":"2024-09-16T02:00:46.526168+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:18:24.412876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.562332+0000","last_change":"2024-09-16T02:00:34.461889+0000","last_active":"2024-09-16T02:01:39.562332+0000","last_peered":"2024-09-16T02:01:39.562332+0000","last_clean":"2024-09-16T02:01:39.562332+0000","last_became_active":"2024-09-16T02:00:34.461583+0000","last_became_peered":"2024-09-16T02:00:34.461583+0000","last_unstale":"2024-09-16T02:01:39.562332+0000","last_undegraded":"2024-09-16T02:01:39.562332+0000","last_fullsized":"2024-09-16T02:01:39.562332+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:32.598422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:46.526605+0000","last_change":"2024-09-16T02:00:32.445201+0000","last_active":"2024-09-16T02:00:46.526605+0000","last_peered":"2024-09-16T02:00:46.526605+0000","last_clean":"2024-09-16T02:00:46.526605+0000","last_became_active":"2024-09-16T02:00:32.444722+0000","last_became_peered":"2024-09-16T02:00:32.444722+0000","last_unstale":"2024-09-16T02:00:46.526605+0000","last_undegraded":"2024-09-16T02:00:46.526605+0000","last_fullsized":"2024-09-16T02:00:46.526605+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:41:11.657028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'1","reported_seq":15,"reported_epoch":79,"state":"peering","last_fresh":"2024-09-16T02:00:54.509925+0000","last_change":"2024-09-16T02:01:39.562036+0000","last_active":"2024-09-16T02:01:39.562036+0000","last_peered":"2024-09-16T02:00:54.509925+0000","last_clean":"2024-09-16T02:00:54.509925+0000","last_became_active":"2024-09-16T02:00:34.455718+0000","last_became_peered":"2024-09-16T02:00:34.455718+0000","last_unstale":"2024-09-16T02:00:54.509925+0000","last_undegraded":"2024-09-16T02:00:54.509925+0000","last_fullsized":"2024-09-16T02:00:54.509925+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5,9],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'5","reported_seq":77,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:01:37.603025+0000","last_change":"2024-09-16T02:00:32.860059+0000","last_active":"2024-09-16T02:01:37.603025+0000","last_peered":"2024-09-16T02:01:37.603025+0000","last_clean":"2024-09-16T02:01:37.603025+0000","last_became_active":"2024-09-16T02:00:32.859617+0000","last_became_peered":"2024-09-16T02:00:32.859617+0000","last_unstale":"2024-09-16T02:01:37.603025+0000","last_undegraded":"2024-09-16T02:01:37.603025+0000","last_fullsized":"2024-09-16T02:01:37.603025+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:41:50.248194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.561990+0000","last_change":"2024-09-16T02:00:34.454344+0000","last_active":"2024-09-16T02:01:39.561990+0000","last_peered":"2024-09-16T02:01:39.561990+0000","last_clean":"2024-09-16T02:01:39.561990+0000","last_became_active":"2024-09-16T02:00:34.454027+0000","last_became_peered":"2024-09-16T02:00:34.454027+0000","last_unstale":"2024-09-16T02:01:39.561990+0000","last_undegraded":"2024-09-16T02:01:39.561990+0000","last_fullsized":"2024-09-16T02:01:39.561990+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:33.020365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":332,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.562523+0000","last_change":"2024-09-16T01:59:19.798159+0000","last_active":"2024-09-16T02:01:39.562523+0000","last_peered":"2024-09-16T02:01:39.562523+0000","last_clean":"2024-09-16T02:01:39.562523+0000","last_became_active":"2024-09-16T01:59:19.797688+0000","last_became_peered":"2024-09-16T01:59:19.797688+0000","last_unstale":"2024-09-16T02:01:39.562523+0000","last_undegraded":"2024-09-16T02:01:39.562523+0000","last_fullsized":"2024-09-16T02:01:39.562523+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:55:54.312750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:55:54.312750+0000","last_clean_scrub_stamp":"2024-09-16T01:55:54.312750+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:00:36.860158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:47.534936+0000","last_change":"2024-09-16T02:00:32.441925+0000","last_active":"2024-09-16T02:00:47.534936+0000","last_peered":"2024-09-16T02:00:47.534936+0000","last_clean":"2024-09-16T02:00:47.534936+0000","last_became_active":"2024-09-16T02:00:32.441565+0000","last_became_peered":"2024-09-16T02:00:32.441565+0000","last_unstale":"2024-09-16T02:00:47.534936+0000","last_undegraded":"2024-09-16T02:00:47.534936+0000","last_fullsized":"2024-09-16T02:00:47.534936+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:23:09.755034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.558803+0000","last_change":"2024-09-16T02:00:34.452986+0000","last_active":"2024-09-16T02:01:39.558803+0000","last_peered":"2024-09-16T02:01:39.558803+0000","last_clean":"2024-09-16T02:01:39.558803+0000","last_became_active":"2024-09-16T02:00:34.452688+0000","last_became_peered":"2024-09-16T02:00:34.452688+0000","last_unstale":"2024-09-16T02:01:39.558803+0000","last_undegraded":"2024-09-16T02:01:39.558803+0000","last_fullsized":"2024-09-16T02:01:39.558803+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:12.958209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.561340+0000","last_change":"2024-09-16T02:00:32.443964+0000","last_active":"2024-09-16T02:01:39.561340+0000","last_peered":"2024-09-16T02:01:39.561340+0000","last_clean":"2024-09-16T02:01:39.561340+0000","last_became_active":"2024-09-16T02:00:32.443623+0000","last_became_peered":"2024-09-16T02:00:32.443623+0000","last_unstale":"2024-09-16T02:01:39.561340+0000","last_undegraded":"2024-09-16T02:01:39.561340+0000","last_fullsized":"2024-09-16T02:01:39.561340+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:30:34.406024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.446017+0000","last_change":"2024-09-16T02:00:34.449615+0000","last_active":"2024-09-16T02:00:35.446017+0000","last_peered":"2024-09-16T02:00:35.446017+0000","last_clean":"2024-09-16T02:00:35.446017+0000","last_became_active":"2024-09-16T02:00:34.449109+0000","last_became_peered":"2024-09-16T02:00:34.449109+0000","last_unstale":"2024-09-16T02:00:35.446017+0000","last_undegraded":"2024-09-16T02:00:35.446017+0000","last_fullsized":"2024-09-16T02:00:35.446017+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:45.036177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":76,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:01:37.602422+0000","last_change":"2024-09-16T02:00:32.587107+0000","last_active":"2024-09-16T02:01:37.602422+0000","last_peered":"2024-09-16T02:01:37.602422+0000","last_clean":"2024-09-16T02:01:37.602422+0000","last_became_active":"2024-09-16T02:00:32.586777+0000","last_became_peered":"2024-09-16T02:00:32.586777+0000","last_unstale":"2024-09-16T02:01:37.602422+0000","last_undegraded":"2024-09-16T02:01:37.602422+0000","last_fullsized":"2024-09-16T02:01:37.602422+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:03.968987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.447491+0000","last_change":"2024-09-16T02:00:34.453500+0000","last_active":"2024-09-16T02:00:35.447491+0000","last_peered":"2024-09-16T02:00:35.447491+0000","last_clean":"2024-09-16T02:00:35.447491+0000","last_became_active":"2024-09-16T02:00:34.453175+0000","last_became_peered":"2024-09-16T02:00:34.453175+0000","last_unstale":"2024-09-16T02:00:35.447491+0000","last_undegraded":"2024-09-16T02:00:35.447491+0000","last_fullsized":"2024-09-16T02:00:35.447491+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:52.666456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'5","reported_seq":79,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.561707+0000","last_change":"2024-09-16T02:00:32.443201+0000","last_active":"2024-09-16T02:01:39.561707+0000","last_peered":"2024-09-16T02:01:39.561707+0000","last_clean":"2024-09-16T02:01:39.561707+0000","last_became_active":"2024-09-16T02:00:32.442778+0000","last_became_peered":"2024-09-16T02:00:32.442778+0000","last_unstale":"2024-09-16T02:01:39.561707+0000","last_undegraded":"2024-09-16T02:01:39.561707+0000","last_fullsized":"2024-09-16T02:01:39.561707+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:43.933817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560410+0000","last_change":"2024-09-16T02:00:34.463000+0000","last_active":"2024-09-16T02:01:40.560410+0000","last_peered":"2024-09-16T02:01:40.560410+0000","last_clean":"2024-09-16T02:01:40.560410+0000","last_became_active":"2024-09-16T02:00:34.461890+0000","last_became_peered":"2024-09-16T02:00:34.461890+0000","last_unstale":"2024-09-16T02:01:40.560410+0000","last_undegraded":"2024-09-16T02:01:40.560410+0000","last_fullsized":"2024-09-16T02:01:40.560410+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:21:15.161295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.607447+0000","last_change":"2024-09-16T02:00:32.586933+0000","last_active":"2024-09-16T02:01:40.607447+0000","last_peered":"2024-09-16T02:01:40.607447+0000","last_clean":"2024-09-16T02:01:40.607447+0000","last_became_active":"2024-09-16T02:00:32.586565+0000","last_became_peered":"2024-09-16T02:00:32.586565+0000","last_unstale":"2024-09-16T02:01:40.607447+0000","last_undegraded":"2024-09-16T02:01:40.607447+0000","last_fullsized":"2024-09-16T02:01:40.607447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:45:07.130649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560456+0000","last_change":"2024-09-16T02:00:34.460805+0000","last_active":"2024-09-16T02:01:40.560456+0000","last_peered":"2024-09-16T02:01:40.560456+0000","last_clean":"2024-09-16T02:01:40.560456+0000","last_became_active":"2024-09-16T02:00:34.460482+0000","last_became_peered":"2024-09-16T02:00:34.460482+0000","last_unstale":"2024-09-16T02:01:40.560456+0000","last_undegraded":"2024-09-16T02:01:40.560456+0000","last_fullsized":"2024-09-16T02:01:40.560456+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:07:51.565053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.563435+0000","last_change":"2024-09-16T02:00:32.587516+0000","last_active":"2024-09-16T02:01:39.563435+0000","last_peered":"2024-09-16T02:01:39.563435+0000","last_clean":"2024-09-16T02:01:39.563435+0000","last_became_active":"2024-09-16T02:00:32.587172+0000","last_became_peered":"2024-09-16T02:00:32.587172+0000","last_unstale":"2024-09-16T02:01:39.563435+0000","last_undegraded":"2024-09-16T02:01:39.563435+0000","last_fullsized":"2024-09-16T02:01:39.563435+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:52:29.018290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.446340+0000","last_change":"2024-09-16T02:00:34.446021+0000","last_active":"2024-09-16T02:00:35.446340+0000","last_peered":"2024-09-16T02:00:35.446340+0000","last_clean":"2024-09-16T02:00:35.446340+0000","last_became_active":"2024-09-16T02:00:34.445732+0000","last_became_peered":"2024-09-16T02:00:34.445732+0000","last_unstale":"2024-09-16T02:00:35.446340+0000","last_undegraded":"2024-09-16T02:00:35.446340+0000","last_fullsized":"2024-09-16T02:00:35.446340+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:51:17.960792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'14","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.607435+0000","last_change":"2024-09-16T02:00:32.439750+0000","last_active":"2024-09-16T02:01:40.607435+0000","last_peered":"2024-09-16T02:01:40.607435+0000","last_clean":"2024-09-16T02:01:40.607435+0000","last_became_active":"2024-09-16T02:00:32.439438+0000","last_became_peered":"2024-09-16T02:00:32.439438+0000","last_unstale":"2024-09-16T02:01:40.607435+0000","last_undegraded":"2024-09-16T02:01:40.607435+0000","last_fullsized":"2024-09-16T02:01:40.607435+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:50.764733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.559493+0000","last_change":"2024-09-16T02:00:34.453810+0000","last_active":"2024-09-16T02:01:39.559493+0000","last_peered":"2024-09-16T02:01:39.559493+0000","last_clean":"2024-09-16T02:01:39.559493+0000","last_became_active":"2024-09-16T02:00:34.453421+0000","last_became_peered":"2024-09-16T02:00:34.453421+0000","last_unstale":"2024-09-16T02:01:39.559493+0000","last_undegraded":"2024-09-16T02:01:39.559493+0000","last_fullsized":"2024-09-16T02:01:39.559493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:23:50.405749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:46.525899+0000","last_change":"2024-09-16T02:00:32.860565+0000","last_active":"2024-09-16T02:00:46.525899+0000","last_peered":"2024-09-16T02:00:46.525899+0000","last_clean":"2024-09-16T02:00:46.525899+0000","last_became_active":"2024-09-16T02:00:32.860130+0000","last_became_peered":"2024-09-16T02:00:32.860130+0000","last_unstale":"2024-09-16T02:00:46.525899+0000","last_undegraded":"2024-09-16T02:00:46.525899+0000","last_fullsized":"2024-09-16T02:00:46.525899+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:09:55.759883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"peering","last_fresh":"2024-09-16T02:00:35.467174+0000","last_change":"2024-09-16T02:01:39.561205+0000","last_active":"2024-09-16T02:01:39.561205+0000","last_peered":"2024-09-16T02:00:35.467174+0000","last_clean":"2024-09-16T02:00:35.467174+0000","last_became_active":"2024-09-16T02:00:34.453725+0000","last_became_peered":"2024-09-16T02:00:34.453725+0000","last_unstale":"2024-09-16T02:00:35.467174+0000","last_undegraded":"2024-09-16T02:00:35.467174+0000","last_fullsized":"2024-09-16T02:00:35.467174+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,4,6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:43.496005+0000","last_change":"2024-09-16T02:00:32.587115+0000","last_active":"2024-09-16T02:00:43.496005+0000","last_peered":"2024-09-16T02:00:43.496005+0000","last_clean":"2024-09-16T02:00:43.496005+0000","last_became_active":"2024-09-16T02:00:32.586835+0000","last_became_peered":"2024-09-16T02:00:32.586835+0000","last_unstale":"2024-09-16T02:00:43.496005+0000","last_undegraded":"2024-09-16T02:00:43.496005+0000","last_fullsized":"2024-09-16T02:00:43.496005+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:12:58.820005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.562201+0000","last_change":"2024-09-16T02:00:34.457673+0000","last_active":"2024-09-16T02:01:39.562201+0000","last_peered":"2024-09-16T02:01:39.562201+0000","last_clean":"2024-09-16T02:01:39.562201+0000","last_became_active":"2024-09-16T02:00:34.457331+0000","last_became_peered":"2024-09-16T02:00:34.457331+0000","last_unstale":"2024-09-16T02:01:39.562201+0000","last_undegraded":"2024-09-16T02:01:39.562201+0000","last_fullsized":"2024-09-16T02:01:39.562201+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:12:16.100036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.558734+0000","last_change":"2024-09-16T02:00:32.437528+0000","last_active":"2024-09-16T02:01:39.558734+0000","last_peered":"2024-09-16T02:01:39.558734+0000","last_clean":"2024-09-16T02:01:39.558734+0000","last_became_active":"2024-09-16T02:00:32.437080+0000","last_became_peered":"2024-09-16T02:00:32.437080+0000","last_unstale":"2024-09-16T02:01:39.558734+0000","last_undegraded":"2024-09-16T02:01:39.558734+0000","last_fullsized":"2024-09-16T02:01:39.558734+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:59:33.908026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560289+0000","last_change":"2024-09-16T02:00:34.453504+0000","last_active":"2024-09-16T02:01:40.560289+0000","last_peered":"2024-09-16T02:01:40.560289+0000","last_clean":"2024-09-16T02:01:40.560289+0000","last_became_active":"2024-09-16T02:00:34.453212+0000","last_became_peered":"2024-09-16T02:00:34.453212+0000","last_unstale":"2024-09-16T02:01:40.560289+0000","last_undegraded":"2024-09-16T02:01:40.560289+0000","last_fullsized":"2024-09-16T02:01:40.560289+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:00:56.414548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:45.518572+0000","last_change":"2024-09-16T02:00:32.442282+0000","last_active":"2024-09-16T02:00:45.518572+0000","last_peered":"2024-09-16T02:00:45.518572+0000","last_clean":"2024-09-16T02:00:45.518572+0000","last_became_active":"2024-09-16T02:00:32.441910+0000","last_became_peered":"2024-09-16T02:00:32.441910+0000","last_unstale":"2024-09-16T02:00:45.518572+0000","last_undegraded":"2024-09-16T02:00:45.518572+0000","last_fullsized":"2024-09-16T02:00:45.518572+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:01:46.045021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.559236+0000","last_change":"2024-09-16T02:00:34.460177+0000","last_active":"2024-09-16T02:01:39.559236+0000","last_peered":"2024-09-16T02:01:39.559236+0000","last_clean":"2024-09-16T02:01:39.559236+0000","last_became_active":"2024-09-16T02:00:34.459874+0000","last_became_peered":"2024-09-16T02:00:34.459874+0000","last_unstale":"2024-09-16T02:01:39.559236+0000","last_undegraded":"2024-09-16T02:01:39.559236+0000","last_fullsized":"2024-09-16T02:01:39.559236+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:35.855591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":24,"reported_epoch":79,"state":"peering","last_fresh":"2024-09-16T02:00:47.535124+0000","last_change":"2024-09-16T02:01:39.563085+0000","last_active":"2024-09-16T02:01:39.563085+0000","last_peered":"2024-09-16T02:00:47.535124+0000","last_clean":"2024-09-16T02:00:47.535124+0000","last_became_active":"2024-09-16T02:00:32.443095+0000","last_became_peered":"2024-09-16T02:00:32.443095+0000","last_unstale":"2024-09-16T02:00:47.535124+0000","last_undegraded":"2024-09-16T02:00:47.535124+0000","last_fullsized":"2024-09-16T02:00:47.535124+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4,7],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.561554+0000","last_change":"2024-09-16T02:00:34.452964+0000","last_active":"2024-09-16T02:01:39.561554+0000","last_peered":"2024-09-16T02:01:39.561554+0000","last_clean":"2024-09-16T02:01:39.561554+0000","last_became_active":"2024-09-16T02:00:34.452617+0000","last_became_peered":"2024-09-16T02:00:34.452617+0000","last_unstale":"2024-09-16T02:01:39.561554+0000","last_undegraded":"2024-09-16T02:01:39.561554+0000","last_fullsized":"2024-09-16T02:01:39.561554+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:04.617970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.447223+0000","last_change":"2024-09-16T02:00:34.456585+0000","last_active":"2024-09-16T02:00:35.447223+0000","last_peered":"2024-09-16T02:00:35.447223+0000","last_clean":"2024-09-16T02:00:35.447223+0000","last_became_active":"2024-09-16T02:00:34.456199+0000","last_became_peered":"2024-09-16T02:00:34.456199+0000","last_unstale":"2024-09-16T02:00:35.447223+0000","last_undegraded":"2024-09-16T02:00:35.447223+0000","last_fullsized":"2024-09-16T02:00:35.447223+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:37:54.376785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.579320+0000","last_change":"2024-09-16T02:01:40.578904+0000","last_active":"2024-09-16T02:01:40.579320+0000","last_peered":"2024-09-16T02:01:40.579320+0000","last_clean":"2024-09-16T02:01:40.579320+0000","last_became_active":"2024-09-16T02:01:40.567221+0000","last_became_peered":"2024-09-16T02:01:40.567221+0000","last_unstale":"2024-09-16T02:01:40.579320+0000","last_undegraded":"2024-09-16T02:01:40.579320+0000","last_fullsized":"2024-09-16T02:01:40.579320+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:01.407145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:00:35.447200+0000","last_change":"2024-09-16T02:00:34.460925+0000","last_active":"2024-09-16T02:00:35.447200+0000","last_peered":"2024-09-16T02:00:35.447200+0000","last_clean":"2024-09-16T02:00:35.447200+0000","last_became_active":"2024-09-16T02:00:34.460581+0000","last_became_peered":"2024-09-16T02:00:34.460581+0000","last_unstale":"2024-09-16T02:00:35.447200+0000","last_undegraded":"2024-09-16T02:00:35.447200+0000","last_fullsized":"2024-09-16T02:00:35.447200+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:19:44.460695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.563378+0000","last_change":"2024-09-16T02:00:32.443877+0000","last_active":"2024-09-16T02:01:39.563378+0000","last_peered":"2024-09-16T02:01:39.563378+0000","last_clean":"2024-09-16T02:01:39.563378+0000","last_became_active":"2024-09-16T02:00:32.443416+0000","last_became_peered":"2024-09-16T02:00:32.443416+0000","last_unstale":"2024-09-16T02:01:39.563378+0000","last_undegraded":"2024-09-16T02:01:39.563378+0000","last_fullsized":"2024-09-16T02:01:39.563378+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:36:57.367232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.563554+0000","last_change":"2024-09-16T02:00:34.453048+0000","last_active":"2024-09-16T02:01:39.563554+0000","last_peered":"2024-09-16T02:01:39.563554+0000","last_clean":"2024-09-16T02:01:39.563554+0000","last_became_active":"2024-09-16T02:00:34.452752+0000","last_became_peered":"2024-09-16T02:00:34.452752+0000","last_unstale":"2024-09-16T02:01:39.563554+0000","last_undegraded":"2024-09-16T02:01:39.563554+0000","last_fullsized":"2024-09-16T02:01:39.563554+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:55.970195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.559416+0000","last_change":"2024-09-16T02:00:32.586974+0000","last_active":"2024-09-16T02:01:39.559416+0000","last_peered":"2024-09-16T02:01:39.559416+0000","last_clean":"2024-09-16T02:01:39.559416+0000","last_became_active":"2024-09-16T02:00:32.586644+0000","last_became_peered":"2024-09-16T02:00:32.586644+0000","last_unstale":"2024-09-16T02:01:39.559416+0000","last_undegraded":"2024-09-16T02:01:39.559416+0000","last_fullsized":"2024-09-16T02:01:39.559416+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:52.913287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.558953+0000","last_change":"2024-09-16T02:00:34.447548+0000","last_active":"2024-09-16T02:01:39.558953+0000","last_peered":"2024-09-16T02:01:39.558953+0000","last_clean":"2024-09-16T02:01:39.558953+0000","last_became_active":"2024-09-16T02:00:34.447207+0000","last_became_peered":"2024-09-16T02:00:34.447207+0000","last_unstale":"2024-09-16T02:01:39.558953+0000","last_undegraded":"2024-09-16T02:01:39.558953+0000","last_fullsized":"2024-09-16T02:01:39.558953+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:22:02.494343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":90,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.607561+0000","last_change":"2024-09-16T02:00:32.441129+0000","last_active":"2024-09-16T02:01:40.607561+0000","last_peered":"2024-09-16T02:01:40.607561+0000","last_clean":"2024-09-16T02:01:40.607561+0000","last_became_active":"2024-09-16T02:00:32.440843+0000","last_became_peered":"2024-09-16T02:00:32.440843+0000","last_unstale":"2024-09-16T02:01:40.607561+0000","last_undegraded":"2024-09-16T02:01:40.607561+0000","last_fullsized":"2024-09-16T02:01:40.607561+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:17.721740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66026,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":573,"num_read_kb":362,"num_write":238,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":569344,"data_stored":198660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281576,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612636,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":892928,"data_stored":432750,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841569,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1044480,"data_stored":1082801,"data_compressed":41575,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005093,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":901120,"data_stored":423573,"data_compressed":2878,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234026,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":897024,"data_stored":423951,"data_compressed":2878,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462959,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":950272,"data_stored":1016471,"data_compressed":38690,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626485,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":872448,"data_stored":414531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855417,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":1006997,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018942,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":414785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247876,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":414356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444105,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":474875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":933888,"data_stored":474655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":905216,"data_stored":424629,"data_compressed":2878,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T02:01:44.686 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T02:01:44.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:44 smithi190 ceph-mon[34762]: pgmap v313: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s; 72 B/s, 0 objects/s recovering 2024-09-16T02:01:44.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:44 smithi137 ceph-mon[34099]: pgmap v313: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s; 72 B/s, 0 objects/s recovering 2024-09-16T02:01:44.932 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:45.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.392+0000 7f00b4178640 1 -- 172.21.15.105:0/986042251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00ac10c180 msgr2=0x7f00ac10e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:45.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.392+0000 7f00b4178640 1 --2- 172.21.15.105:0/986042251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00ac10c180 0x7f00ac10e600 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f00a4009ca0 tx=0x7f00a402a950 comp rx=0 tx=0).stop 2024-09-16T02:01:45.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.392+0000 7f00b4178640 1 -- 172.21.15.105:0/986042251 shutdown_connections 2024-09-16T02:01:45.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.392+0000 7f00b4178640 1 --2- 172.21.15.105:0/986042251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00ac10c180 0x7f00ac10e600 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:45.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.392+0000 7f00b4178640 1 --2- 172.21.15.105:0/986042251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00ac1036c0 0x7f00ac10bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:45.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.392+0000 7f00b4178640 1 --2- 172.21.15.105:0/986042251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00ac102da0 0x7f00ac103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:45.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.392+0000 7f00b4178640 1 -- 172.21.15.105:0/986042251 >> 172.21.15.105:0/986042251 conn(0x7f00ac0fc7a0 msgr2=0x7f00ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:45.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.392+0000 7f00b4178640 1 -- 172.21.15.105:0/986042251 shutdown_connections 2024-09-16T02:01:45.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.392+0000 7f00b4178640 1 -- 172.21.15.105:0/986042251 wait complete. 2024-09-16T02:01:45.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.393+0000 7f00b4178640 1 Processor -- start 2024-09-16T02:01:45.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.393+0000 7f00b4178640 1 -- start start 2024-09-16T02:01:45.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.393+0000 7f00b4178640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00ac102da0 0x7f00ac198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:45.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.393+0000 7f00b4178640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00ac1036c0 0x7f00ac1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:45.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.393+0000 7f00b4178640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00ac10c180 0x7f00ac19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:45.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.393+0000 7f00b4178640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f00ac111e50 con 0x7f00ac1036c0 2024-09-16T02:01:45.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.393+0000 7f00b4178640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f00ac111cd0 con 0x7f00ac10c180 2024-09-16T02:01:45.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.393+0000 7f00b4178640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f00ac111fd0 con 0x7f00ac102da0 2024-09-16T02:01:45.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.393+0000 7f00b1713640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00ac1036c0 0x7f00ac1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:45.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.393+0000 7f00b1713640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00ac1036c0 0x7f00ac1989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33084/0 (socket says 172.21.15.105:33084) 2024-09-16T02:01:45.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.393+0000 7f00b1713640 1 -- 172.21.15.105:0/448217640 learned_addr learned my addr 172.21.15.105:0/448217640 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:01:45.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.393+0000 7f00b1f14640 1 --2- 172.21.15.105:0/448217640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00ac102da0 0x7f00ac198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:45.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.393+0000 7f00b2715640 1 --2- 172.21.15.105:0/448217640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00ac10c180 0x7f00ac19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:45.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.393+0000 7f00b1713640 1 -- 172.21.15.105:0/448217640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00ac102da0 msgr2=0x7f00ac198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:45.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.393+0000 7f00b1713640 1 --2- 172.21.15.105:0/448217640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00ac102da0 0x7f00ac198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:45.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.393+0000 7f00b1713640 1 -- 172.21.15.105:0/448217640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00ac10c180 msgr2=0x7f00ac19cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:45.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.393+0000 7f00b1713640 1 --2- 172.21.15.105:0/448217640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00ac10c180 0x7f00ac19cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:45.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.393+0000 7f00b1713640 1 -- 172.21.15.105:0/448217640 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00ac19d430 con 0x7f00ac1036c0 2024-09-16T02:01:45.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.393+0000 7f00b2715640 1 --2- 172.21.15.105:0/448217640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00ac10c180 0x7f00ac19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:01:45.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.394+0000 7f00b1713640 1 --2- 172.21.15.105:0/448217640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00ac1036c0 0x7f00ac1989c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f00a800d950 tx=0x7f00a800dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:45.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.394+0000 7f00a2ffd640 1 -- 172.21.15.105:0/448217640 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a8017070 con 0x7f00ac1036c0 2024-09-16T02:01:45.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.394+0000 7f00b4178640 1 -- 172.21.15.105:0/448217640 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00ac19d720 con 0x7f00ac1036c0 2024-09-16T02:01:45.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.394+0000 7f00a2ffd640 1 -- 172.21.15.105:0/448217640 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00a800de80 con 0x7f00ac1036c0 2024-09-16T02:01:45.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.394+0000 7f00a2ffd640 1 -- 172.21.15.105:0/448217640 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a800f620 con 0x7f00ac1036c0 2024-09-16T02:01:45.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.394+0000 7f00b4178640 1 -- 172.21.15.105:0/448217640 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00ac1a5000 con 0x7f00ac1036c0 2024-09-16T02:01:45.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.395+0000 7f00b4178640 1 -- 172.21.15.105:0/448217640 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00ac103180 con 0x7f00ac1036c0 2024-09-16T02:01:45.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.397+0000 7f00a2ffd640 1 -- 172.21.15.105:0/448217640 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f00a8009c30 con 0x7f00ac1036c0 2024-09-16T02:01:45.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.397+0000 7f00a2ffd640 1 --2- 172.21.15.105:0/448217640 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f00980a32d0 0x7f00980a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:45.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.397+0000 7f00b1f14640 1 --2- 172.21.15.105:0/448217640 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f00980a32d0 0x7f00980a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:45.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.397+0000 7f00a2ffd640 1 -- 172.21.15.105:0/448217640 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f00a80cf6d0 con 0x7f00ac1036c0 2024-09-16T02:01:45.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.398+0000 7f00b1f14640 1 --2- 172.21.15.105:0/448217640 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f00980a32d0 0x7f00980a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f009c009990 tx=0x7f009c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:45.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.401+0000 7f00a2ffd640 1 -- 172.21.15.105:0/448217640 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f00a809c270 con 0x7f00ac1036c0 2024-09-16T02:01:45.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.565+0000 7f00b4178640 1 -- 172.21.15.105:0/448217640 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f00ac107250 con 0x7f00980a32d0 2024-09-16T02:01:45.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.573+0000 7f00a2ffd640 1 -- 172.21.15.105:0/448217640 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258974 (secure 0 0 0) 0x7f00ac107250 con 0x7f00980a32d0 2024-09-16T02:01:45.578 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:01:45.578 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-09-16T02:01:45.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.576+0000 7f00b4178640 1 -- 172.21.15.105:0/448217640 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f00980a32d0 msgr2=0x7f00980a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:45.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.576+0000 7f00b4178640 1 --2- 172.21.15.105:0/448217640 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f00980a32d0 0x7f00980a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f009c009990 tx=0x7f009c009110 comp rx=0 tx=0).stop 2024-09-16T02:01:45.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.576+0000 7f00b4178640 1 -- 172.21.15.105:0/448217640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00ac1036c0 msgr2=0x7f00ac1989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:45.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.576+0000 7f00b4178640 1 --2- 172.21.15.105:0/448217640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00ac1036c0 0x7f00ac1989c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f00a800d950 tx=0x7f00a800dd50 comp rx=0 tx=0).stop 2024-09-16T02:01:45.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.576+0000 7f00b4178640 1 -- 172.21.15.105:0/448217640 shutdown_connections 2024-09-16T02:01:45.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.576+0000 7f00b4178640 1 --2- 172.21.15.105:0/448217640 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f00980a32d0 0x7f00980a5790 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:45.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.576+0000 7f00b4178640 1 --2- 172.21.15.105:0/448217640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00ac10c180 0x7f00ac19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:45.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.576+0000 7f00b4178640 1 --2- 172.21.15.105:0/448217640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00ac1036c0 0x7f00ac1989c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:45.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.576+0000 7f00b4178640 1 --2- 172.21.15.105:0/448217640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00ac102da0 0x7f00ac198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:45.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.576+0000 7f00b4178640 1 -- 172.21.15.105:0/448217640 >> 172.21.15.105:0/448217640 conn(0x7f00ac0fc7a0 msgr2=0x7f00ac0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:45.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.576+0000 7f00b4178640 1 -- 172.21.15.105:0/448217640 shutdown_connections 2024-09-16T02:01:45.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:45.576+0000 7f00b4178640 1 -- 172.21.15.105:0/448217640 wait complete. 2024-09-16T02:01:45.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:45 smithi190 ceph-mon[34762]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:01:45.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:45 smithi137 ceph-mon[34099]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:01:45.969 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:45 smithi105 ceph-mon[31197]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:01:46.348 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-09-16T02:01:45.460876+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656628,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":719,"num_read_kb":476,"num_write":421,"num_write_kb":3070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":287,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368572,"kb_used_data":10952,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124573252,"statfs":{"total":1151940427776,"available":1151563010048,"internally_reserved":0,"allocated":11214848,"data_stored":7115904,"data_compressed":167937,"data_compressed_allocated":221184,"data_compressed_original":1941504,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":89,"num_read_kb":44,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001820"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560827+0000","last_change":"2024-09-16T02:00:34.454121+0000","last_active":"2024-09-16T02:01:40.560827+0000","last_peered":"2024-09-16T02:01:40.560827+0000","last_clean":"2024-09-16T02:01:40.560827+0000","last_became_active":"2024-09-16T02:00:34.453810+0000","last_became_peered":"2024-09-16T02:00:34.453810+0000","last_unstale":"2024-09-16T02:01:40.560827+0000","last_undegraded":"2024-09-16T02:01:40.560827+0000","last_fullsized":"2024-09-16T02:01:40.560827+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:42:38.781928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.563230+0000","last_change":"2024-09-16T02:00:34.462020+0000","last_active":"2024-09-16T02:01:39.563230+0000","last_peered":"2024-09-16T02:01:39.563230+0000","last_clean":"2024-09-16T02:01:39.563230+0000","last_became_active":"2024-09-16T02:00:34.461680+0000","last_became_peered":"2024-09-16T02:00:34.461680+0000","last_unstale":"2024-09-16T02:01:39.563230+0000","last_undegraded":"2024-09-16T02:01:39.563230+0000","last_fullsized":"2024-09-16T02:01:39.563230+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:00:14.493643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.562710+0000","last_change":"2024-09-16T02:00:34.453034+0000","last_active":"2024-09-16T02:01:39.562710+0000","last_peered":"2024-09-16T02:01:39.562710+0000","last_clean":"2024-09-16T02:01:39.562710+0000","last_became_active":"2024-09-16T02:00:34.452719+0000","last_became_peered":"2024-09-16T02:00:34.452719+0000","last_unstale":"2024-09-16T02:01:39.562710+0000","last_undegraded":"2024-09-16T02:01:39.562710+0000","last_fullsized":"2024-09-16T02:01:39.562710+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:48:52.478501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.610231+0000","last_change":"2024-09-16T02:00:34.458156+0000","last_active":"2024-09-16T02:01:40.610231+0000","last_peered":"2024-09-16T02:01:40.610231+0000","last_clean":"2024-09-16T02:01:40.610231+0000","last_became_active":"2024-09-16T02:00:34.457831+0000","last_became_peered":"2024-09-16T02:00:34.457831+0000","last_unstale":"2024-09-16T02:01:40.610231+0000","last_undegraded":"2024-09-16T02:01:40.610231+0000","last_fullsized":"2024-09-16T02:01:40.610231+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:35:36.609046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.562621+0000","last_change":"2024-09-16T02:00:34.448638+0000","last_active":"2024-09-16T02:01:40.562621+0000","last_peered":"2024-09-16T02:01:40.562621+0000","last_clean":"2024-09-16T02:01:40.562621+0000","last_became_active":"2024-09-16T02:00:34.447993+0000","last_became_peered":"2024-09-16T02:00:34.447993+0000","last_unstale":"2024-09-16T02:01:40.562621+0000","last_undegraded":"2024-09-16T02:01:40.562621+0000","last_fullsized":"2024-09-16T02:01:40.562621+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:33:04.832053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560794+0000","last_change":"2024-09-16T02:00:34.462651+0000","last_active":"2024-09-16T02:01:40.560794+0000","last_peered":"2024-09-16T02:01:40.560794+0000","last_clean":"2024-09-16T02:01:40.560794+0000","last_became_active":"2024-09-16T02:00:34.461837+0000","last_became_peered":"2024-09-16T02:00:34.461837+0000","last_unstale":"2024-09-16T02:01:40.560794+0000","last_undegraded":"2024-09-16T02:01:40.560794+0000","last_fullsized":"2024-09-16T02:01:40.560794+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:52:13.733591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.562619+0000","last_change":"2024-09-16T02:00:34.455362+0000","last_active":"2024-09-16T02:01:39.562619+0000","last_peered":"2024-09-16T02:01:39.562619+0000","last_clean":"2024-09-16T02:01:39.562619+0000","last_became_active":"2024-09-16T02:00:34.455029+0000","last_became_peered":"2024-09-16T02:00:34.455029+0000","last_unstale":"2024-09-16T02:01:39.562619+0000","last_undegraded":"2024-09-16T02:01:39.562619+0000","last_fullsized":"2024-09-16T02:01:39.562619+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:14:08.316845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560955+0000","last_change":"2024-09-16T02:00:34.459578+0000","last_active":"2024-09-16T02:01:40.560955+0000","last_peered":"2024-09-16T02:01:40.560955+0000","last_clean":"2024-09-16T02:01:40.560955+0000","last_became_active":"2024-09-16T02:00:34.459288+0000","last_became_peered":"2024-09-16T02:00:34.459288+0000","last_unstale":"2024-09-16T02:01:40.560955+0000","last_undegraded":"2024-09-16T02:01:40.560955+0000","last_fullsized":"2024-09-16T02:01:40.560955+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:52:28.974276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.563476+0000","last_change":"2024-09-16T02:00:34.463011+0000","last_active":"2024-09-16T02:01:40.563476+0000","last_peered":"2024-09-16T02:01:40.563476+0000","last_clean":"2024-09-16T02:01:40.563476+0000","last_became_active":"2024-09-16T02:00:34.462668+0000","last_became_peered":"2024-09-16T02:00:34.462668+0000","last_unstale":"2024-09-16T02:01:40.563476+0000","last_undegraded":"2024-09-16T02:01:40.563476+0000","last_fullsized":"2024-09-16T02:01:40.563476+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:06:31.655869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560622+0000","last_change":"2024-09-16T02:00:34.462241+0000","last_active":"2024-09-16T02:01:40.560622+0000","last_peered":"2024-09-16T02:01:40.560622+0000","last_clean":"2024-09-16T02:01:40.560622+0000","last_became_active":"2024-09-16T02:00:34.461925+0000","last_became_peered":"2024-09-16T02:00:34.461925+0000","last_unstale":"2024-09-16T02:01:40.560622+0000","last_undegraded":"2024-09-16T02:01:40.560622+0000","last_fullsized":"2024-09-16T02:01:40.560622+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:48:02.414471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.561882+0000","last_change":"2024-09-16T02:00:34.466132+0000","last_active":"2024-09-16T02:01:39.561882+0000","last_peered":"2024-09-16T02:01:39.561882+0000","last_clean":"2024-09-16T02:01:39.561882+0000","last_became_active":"2024-09-16T02:00:34.465750+0000","last_became_peered":"2024-09-16T02:00:34.465750+0000","last_unstale":"2024-09-16T02:01:39.561882+0000","last_undegraded":"2024-09-16T02:01:39.561882+0000","last_fullsized":"2024-09-16T02:01:39.561882+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:07:33.072432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.610362+0000","last_change":"2024-09-16T02:00:34.461759+0000","last_active":"2024-09-16T02:01:40.610362+0000","last_peered":"2024-09-16T02:01:40.610362+0000","last_clean":"2024-09-16T02:01:40.610362+0000","last_became_active":"2024-09-16T02:00:34.461385+0000","last_became_peered":"2024-09-16T02:00:34.461385+0000","last_unstale":"2024-09-16T02:01:40.610362+0000","last_undegraded":"2024-09-16T02:01:40.610362+0000","last_fullsized":"2024-09-16T02:01:40.610362+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:00:24.179172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.608361+0000","last_change":"2024-09-16T02:01:40.608361+0000","last_active":"2024-09-16T02:01:40.608361+0000","last_peered":"2024-09-16T02:01:40.608361+0000","last_clean":"2024-09-16T02:01:40.608361+0000","last_became_active":"2024-09-16T02:01:40.607954+0000","last_became_peered":"2024-09-16T02:01:40.607954+0000","last_unstale":"2024-09-16T02:01:40.608361+0000","last_undegraded":"2024-09-16T02:01:40.608361+0000","last_fullsized":"2024-09-16T02:01:40.608361+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:18:31.244969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.562760+0000","last_change":"2024-09-16T02:00:34.460680+0000","last_active":"2024-09-16T02:01:39.562760+0000","last_peered":"2024-09-16T02:01:39.562760+0000","last_clean":"2024-09-16T02:01:39.562760+0000","last_became_active":"2024-09-16T02:00:34.460161+0000","last_became_peered":"2024-09-16T02:00:34.460161+0000","last_unstale":"2024-09-16T02:01:39.562760+0000","last_undegraded":"2024-09-16T02:01:39.562760+0000","last_fullsized":"2024-09-16T02:01:39.562760+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:38:19.848420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.560921+0000","last_change":"2024-09-16T02:00:34.454606+0000","last_active":"2024-09-16T02:01:39.560921+0000","last_peered":"2024-09-16T02:01:39.560921+0000","last_clean":"2024-09-16T02:01:39.560921+0000","last_became_active":"2024-09-16T02:00:34.454293+0000","last_became_peered":"2024-09-16T02:00:34.454293+0000","last_unstale":"2024-09-16T02:01:39.560921+0000","last_undegraded":"2024-09-16T02:01:39.560921+0000","last_fullsized":"2024-09-16T02:01:39.560921+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:55:54.466176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.562971+0000","last_change":"2024-09-16T02:00:34.452940+0000","last_active":"2024-09-16T02:01:40.562971+0000","last_peered":"2024-09-16T02:01:40.562971+0000","last_clean":"2024-09-16T02:01:40.562971+0000","last_became_active":"2024-09-16T02:00:34.452560+0000","last_became_peered":"2024-09-16T02:00:34.452560+0000","last_unstale":"2024-09-16T02:01:40.562971+0000","last_undegraded":"2024-09-16T02:01:40.562971+0000","last_fullsized":"2024-09-16T02:01:40.562971+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:42:11.564264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560911+0000","last_change":"2024-09-16T02:00:34.453457+0000","last_active":"2024-09-16T02:01:40.560911+0000","last_peered":"2024-09-16T02:01:40.560911+0000","last_clean":"2024-09-16T02:01:40.560911+0000","last_became_active":"2024-09-16T02:00:34.453148+0000","last_became_peered":"2024-09-16T02:00:34.453148+0000","last_unstale":"2024-09-16T02:01:40.560911+0000","last_undegraded":"2024-09-16T02:01:40.560911+0000","last_fullsized":"2024-09-16T02:01:40.560911+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:28.456892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.607401+0000","last_change":"2024-09-16T02:00:34.457057+0000","last_active":"2024-09-16T02:01:40.607401+0000","last_peered":"2024-09-16T02:01:40.607401+0000","last_clean":"2024-09-16T02:01:40.607401+0000","last_became_active":"2024-09-16T02:00:34.456751+0000","last_became_peered":"2024-09-16T02:00:34.456751+0000","last_unstale":"2024-09-16T02:01:40.607401+0000","last_undegraded":"2024-09-16T02:01:40.607401+0000","last_fullsized":"2024-09-16T02:01:40.607401+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:32:32.207510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.561837+0000","last_change":"2024-09-16T02:00:34.448434+0000","last_active":"2024-09-16T02:01:40.561837+0000","last_peered":"2024-09-16T02:01:40.561837+0000","last_clean":"2024-09-16T02:01:40.561837+0000","last_became_active":"2024-09-16T02:00:34.448056+0000","last_became_peered":"2024-09-16T02:00:34.448056+0000","last_unstale":"2024-09-16T02:01:40.561837+0000","last_undegraded":"2024-09-16T02:01:40.561837+0000","last_fullsized":"2024-09-16T02:01:40.561837+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:15:28.162159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560368+0000","last_change":"2024-09-16T02:00:34.459499+0000","last_active":"2024-09-16T02:01:40.560368+0000","last_peered":"2024-09-16T02:01:40.560368+0000","last_clean":"2024-09-16T02:01:40.560368+0000","last_became_active":"2024-09-16T02:00:34.459017+0000","last_became_peered":"2024-09-16T02:00:34.459017+0000","last_unstale":"2024-09-16T02:01:40.560368+0000","last_undegraded":"2024-09-16T02:01:40.560368+0000","last_fullsized":"2024-09-16T02:01:40.560368+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:41:36.866100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.559296+0000","last_change":"2024-09-16T02:00:34.448640+0000","last_active":"2024-09-16T02:01:40.559296+0000","last_peered":"2024-09-16T02:01:40.559296+0000","last_clean":"2024-09-16T02:01:40.559296+0000","last_became_active":"2024-09-16T02:00:34.447737+0000","last_became_peered":"2024-09-16T02:00:34.447737+0000","last_unstale":"2024-09-16T02:01:40.559296+0000","last_undegraded":"2024-09-16T02:01:40.559296+0000","last_fullsized":"2024-09-16T02:01:40.559296+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:20.192650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.563552+0000","last_change":"2024-09-16T02:00:34.451718+0000","last_active":"2024-09-16T02:01:40.563552+0000","last_peered":"2024-09-16T02:01:40.563552+0000","last_clean":"2024-09-16T02:01:40.563552+0000","last_became_active":"2024-09-16T02:00:34.451389+0000","last_became_peered":"2024-09-16T02:00:34.451389+0000","last_unstale":"2024-09-16T02:01:40.563552+0000","last_undegraded":"2024-09-16T02:01:40.563552+0000","last_fullsized":"2024-09-16T02:01:40.563552+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:13:19.296178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.561207+0000","last_change":"2024-09-16T02:00:34.458379+0000","last_active":"2024-09-16T02:01:40.561207+0000","last_peered":"2024-09-16T02:01:40.561207+0000","last_clean":"2024-09-16T02:01:40.561207+0000","last_became_active":"2024-09-16T02:00:34.458038+0000","last_became_peered":"2024-09-16T02:00:34.458038+0000","last_unstale":"2024-09-16T02:01:40.561207+0000","last_undegraded":"2024-09-16T02:01:40.561207+0000","last_fullsized":"2024-09-16T02:01:40.561207+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:34.669470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.565143+0000","last_change":"2024-09-16T02:00:34.450927+0000","last_active":"2024-09-16T02:01:40.565143+0000","last_peered":"2024-09-16T02:01:40.565143+0000","last_clean":"2024-09-16T02:01:40.565143+0000","last_became_active":"2024-09-16T02:00:34.447669+0000","last_became_peered":"2024-09-16T02:00:34.447669+0000","last_unstale":"2024-09-16T02:01:40.565143+0000","last_undegraded":"2024-09-16T02:01:40.565143+0000","last_fullsized":"2024-09-16T02:01:40.565143+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:57.861633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.561666+0000","last_change":"2024-09-16T02:00:34.454827+0000","last_active":"2024-09-16T02:01:40.561666+0000","last_peered":"2024-09-16T02:01:40.561666+0000","last_clean":"2024-09-16T02:01:40.561666+0000","last_became_active":"2024-09-16T02:00:34.454549+0000","last_became_peered":"2024-09-16T02:00:34.454549+0000","last_unstale":"2024-09-16T02:01:40.561666+0000","last_undegraded":"2024-09-16T02:01:40.561666+0000","last_fullsized":"2024-09-16T02:01:40.561666+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:53:45.440860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.563093+0000","last_change":"2024-09-16T02:00:34.455121+0000","last_active":"2024-09-16T02:01:40.563093+0000","last_peered":"2024-09-16T02:01:40.563093+0000","last_clean":"2024-09-16T02:01:40.563093+0000","last_became_active":"2024-09-16T02:00:34.454665+0000","last_became_peered":"2024-09-16T02:00:34.454665+0000","last_unstale":"2024-09-16T02:01:40.563093+0000","last_undegraded":"2024-09-16T02:01:40.563093+0000","last_fullsized":"2024-09-16T02:01:40.563093+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:31:42.771230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560656+0000","last_change":"2024-09-16T02:00:34.458777+0000","last_active":"2024-09-16T02:01:40.560656+0000","last_peered":"2024-09-16T02:01:40.560656+0000","last_clean":"2024-09-16T02:01:40.560656+0000","last_became_active":"2024-09-16T02:00:34.457954+0000","last_became_peered":"2024-09-16T02:00:34.457954+0000","last_unstale":"2024-09-16T02:01:40.560656+0000","last_undegraded":"2024-09-16T02:01:40.560656+0000","last_fullsized":"2024-09-16T02:01:40.560656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:58:28.032293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.559354+0000","last_change":"2024-09-16T02:00:34.444712+0000","last_active":"2024-09-16T02:01:40.559354+0000","last_peered":"2024-09-16T02:01:40.559354+0000","last_clean":"2024-09-16T02:01:40.559354+0000","last_became_active":"2024-09-16T02:00:34.444421+0000","last_became_peered":"2024-09-16T02:00:34.444421+0000","last_unstale":"2024-09-16T02:01:40.559354+0000","last_undegraded":"2024-09-16T02:01:40.559354+0000","last_fullsized":"2024-09-16T02:01:40.559354+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:36.557558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.563059+0000","last_change":"2024-09-16T02:00:34.454570+0000","last_active":"2024-09-16T02:01:39.563059+0000","last_peered":"2024-09-16T02:01:39.563059+0000","last_clean":"2024-09-16T02:01:39.563059+0000","last_became_active":"2024-09-16T02:00:34.454246+0000","last_became_peered":"2024-09-16T02:00:34.454246+0000","last_unstale":"2024-09-16T02:01:39.563059+0000","last_undegraded":"2024-09-16T02:01:39.563059+0000","last_fullsized":"2024-09-16T02:01:39.563059+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:36:18.634907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.561677+0000","last_change":"2024-09-16T02:00:34.449680+0000","last_active":"2024-09-16T02:01:40.561677+0000","last_peered":"2024-09-16T02:01:40.561677+0000","last_clean":"2024-09-16T02:01:40.561677+0000","last_became_active":"2024-09-16T02:00:34.449184+0000","last_became_peered":"2024-09-16T02:00:34.449184+0000","last_unstale":"2024-09-16T02:01:40.561677+0000","last_undegraded":"2024-09-16T02:01:40.561677+0000","last_fullsized":"2024-09-16T02:01:40.561677+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:28:34.867820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.562212+0000","last_change":"2024-09-16T02:00:34.455362+0000","last_active":"2024-09-16T02:01:40.562212+0000","last_peered":"2024-09-16T02:01:40.562212+0000","last_clean":"2024-09-16T02:01:40.562212+0000","last_became_active":"2024-09-16T02:00:34.455018+0000","last_became_peered":"2024-09-16T02:00:34.455018+0000","last_unstale":"2024-09-16T02:01:40.562212+0000","last_undegraded":"2024-09-16T02:01:40.562212+0000","last_fullsized":"2024-09-16T02:01:40.562212+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:38.278418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560207+0000","last_change":"2024-09-16T02:00:34.459426+0000","last_active":"2024-09-16T02:01:40.560207+0000","last_peered":"2024-09-16T02:01:40.560207+0000","last_clean":"2024-09-16T02:01:40.560207+0000","last_became_active":"2024-09-16T02:00:34.459082+0000","last_became_peered":"2024-09-16T02:00:34.459082+0000","last_unstale":"2024-09-16T02:01:40.560207+0000","last_undegraded":"2024-09-16T02:01:40.560207+0000","last_fullsized":"2024-09-16T02:01:40.560207+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:59:42.226065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.563365+0000","last_change":"2024-09-16T02:00:34.450936+0000","last_active":"2024-09-16T02:01:40.563365+0000","last_peered":"2024-09-16T02:01:40.563365+0000","last_clean":"2024-09-16T02:01:40.563365+0000","last_became_active":"2024-09-16T02:00:34.450503+0000","last_became_peered":"2024-09-16T02:00:34.450503+0000","last_unstale":"2024-09-16T02:01:40.563365+0000","last_undegraded":"2024-09-16T02:01:40.563365+0000","last_fullsized":"2024-09-16T02:01:40.563365+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:35:55.806193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'5","reported_seq":86,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:42.610457+0000","last_change":"2024-09-16T02:00:32.443193+0000","last_active":"2024-09-16T02:01:42.610457+0000","last_peered":"2024-09-16T02:01:42.610457+0000","last_clean":"2024-09-16T02:01:42.610457+0000","last_became_active":"2024-09-16T02:00:32.442863+0000","last_became_peered":"2024-09-16T02:00:32.442863+0000","last_unstale":"2024-09-16T02:01:42.610457+0000","last_undegraded":"2024-09-16T02:01:42.610457+0000","last_fullsized":"2024-09-16T02:01:42.610457+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:19.734946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":59,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.570518+0000","last_change":"2024-09-16T02:01:40.570518+0000","last_active":"2024-09-16T02:01:40.570518+0000","last_peered":"2024-09-16T02:01:40.570518+0000","last_clean":"2024-09-16T02:01:40.570518+0000","last_became_active":"2024-09-16T02:01:40.570124+0000","last_became_peered":"2024-09-16T02:01:40.570124+0000","last_unstale":"2024-09-16T02:01:40.570518+0000","last_undegraded":"2024-09-16T02:01:40.570518+0000","last_fullsized":"2024-09-16T02:01:40.570518+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:44:21.062742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.572282+0000","last_change":"2024-09-16T02:01:40.572282+0000","last_active":"2024-09-16T02:01:40.572282+0000","last_peered":"2024-09-16T02:01:40.572282+0000","last_clean":"2024-09-16T02:01:40.572282+0000","last_became_active":"2024-09-16T02:01:40.564577+0000","last_became_peered":"2024-09-16T02:01:40.564577+0000","last_unstale":"2024-09-16T02:01:40.572282+0000","last_undegraded":"2024-09-16T02:01:40.572282+0000","last_fullsized":"2024-09-16T02:01:40.572282+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:19:40.834173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.559077+0000","last_change":"2024-09-16T02:00:32.438530+0000","last_active":"2024-09-16T02:01:40.559077+0000","last_peered":"2024-09-16T02:01:40.559077+0000","last_clean":"2024-09-16T02:01:40.559077+0000","last_became_active":"2024-09-16T02:00:32.438238+0000","last_became_peered":"2024-09-16T02:00:32.438238+0000","last_unstale":"2024-09-16T02:01:40.559077+0000","last_undegraded":"2024-09-16T02:01:40.559077+0000","last_fullsized":"2024-09-16T02:01:40.559077+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:52:32.799701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.561050+0000","last_change":"2024-09-16T02:00:34.456299+0000","last_active":"2024-09-16T02:01:40.561050+0000","last_peered":"2024-09-16T02:01:40.561050+0000","last_clean":"2024-09-16T02:01:40.561050+0000","last_became_active":"2024-09-16T02:00:34.455925+0000","last_became_peered":"2024-09-16T02:00:34.455925+0000","last_unstale":"2024-09-16T02:01:40.561050+0000","last_undegraded":"2024-09-16T02:01:40.561050+0000","last_fullsized":"2024-09-16T02:01:40.561050+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:33:27.162664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.565294+0000","last_change":"2024-09-16T02:00:32.442029+0000","last_active":"2024-09-16T02:01:40.565294+0000","last_peered":"2024-09-16T02:01:40.565294+0000","last_clean":"2024-09-16T02:01:40.565294+0000","last_became_active":"2024-09-16T02:00:32.441705+0000","last_became_peered":"2024-09-16T02:00:32.441705+0000","last_unstale":"2024-09-16T02:01:40.565294+0000","last_undegraded":"2024-09-16T02:01:40.565294+0000","last_fullsized":"2024-09-16T02:01:40.565294+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:50:54.725768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.607165+0000","last_change":"2024-09-16T02:00:34.457833+0000","last_active":"2024-09-16T02:01:40.607165+0000","last_peered":"2024-09-16T02:01:40.607165+0000","last_clean":"2024-09-16T02:01:40.607165+0000","last_became_active":"2024-09-16T02:00:34.457453+0000","last_became_peered":"2024-09-16T02:00:34.457453+0000","last_unstale":"2024-09-16T02:01:40.607165+0000","last_undegraded":"2024-09-16T02:01:40.607165+0000","last_fullsized":"2024-09-16T02:01:40.607165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:21.777574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":136,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:41.607968+0000","last_change":"2024-09-16T02:00:32.441341+0000","last_active":"2024-09-16T02:01:41.607968+0000","last_peered":"2024-09-16T02:01:41.607968+0000","last_clean":"2024-09-16T02:01:41.607968+0000","last_became_active":"2024-09-16T02:00:32.441019+0000","last_became_peered":"2024-09-16T02:00:32.441019+0000","last_unstale":"2024-09-16T02:01:41.607968+0000","last_undegraded":"2024-09-16T02:01:41.607968+0000","last_fullsized":"2024-09-16T02:01:41.607968+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:10:10.171714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":110,"num_read_kb":110,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.563417+0000","last_change":"2024-09-16T02:00:34.461746+0000","last_active":"2024-09-16T02:01:40.563417+0000","last_peered":"2024-09-16T02:01:40.563417+0000","last_clean":"2024-09-16T02:01:40.563417+0000","last_became_active":"2024-09-16T02:00:34.461441+0000","last_became_peered":"2024-09-16T02:00:34.461441+0000","last_unstale":"2024-09-16T02:01:40.563417+0000","last_undegraded":"2024-09-16T02:01:40.563417+0000","last_fullsized":"2024-09-16T02:01:40.563417+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:28:14.519159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.561070+0000","last_change":"2024-09-16T02:00:32.443129+0000","last_active":"2024-09-16T02:01:39.561070+0000","last_peered":"2024-09-16T02:01:39.561070+0000","last_clean":"2024-09-16T02:01:39.561070+0000","last_became_active":"2024-09-16T02:00:32.442772+0000","last_became_peered":"2024-09-16T02:00:32.442772+0000","last_unstale":"2024-09-16T02:01:39.561070+0000","last_undegraded":"2024-09-16T02:01:39.561070+0000","last_fullsized":"2024-09-16T02:01:39.561070+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:10:03.964583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.563645+0000","last_change":"2024-09-16T02:00:34.457449+0000","last_active":"2024-09-16T02:01:39.563645+0000","last_peered":"2024-09-16T02:01:39.563645+0000","last_clean":"2024-09-16T02:01:39.563645+0000","last_became_active":"2024-09-16T02:00:34.457110+0000","last_became_peered":"2024-09-16T02:00:34.457110+0000","last_unstale":"2024-09-16T02:01:39.563645+0000","last_undegraded":"2024-09-16T02:01:39.563645+0000","last_fullsized":"2024-09-16T02:01:39.563645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:35.662503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.565563+0000","last_change":"2024-09-16T02:00:32.587419+0000","last_active":"2024-09-16T02:01:40.565563+0000","last_peered":"2024-09-16T02:01:40.565563+0000","last_clean":"2024-09-16T02:01:40.565563+0000","last_became_active":"2024-09-16T02:00:32.587037+0000","last_became_peered":"2024-09-16T02:00:32.587037+0000","last_unstale":"2024-09-16T02:01:40.565563+0000","last_undegraded":"2024-09-16T02:01:40.565563+0000","last_fullsized":"2024-09-16T02:01:40.565563+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:11.835505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.563152+0000","last_change":"2024-09-16T02:00:34.456371+0000","last_active":"2024-09-16T02:01:40.563152+0000","last_peered":"2024-09-16T02:01:40.563152+0000","last_clean":"2024-09-16T02:01:40.563152+0000","last_became_active":"2024-09-16T02:00:34.456013+0000","last_became_peered":"2024-09-16T02:00:34.456013+0000","last_unstale":"2024-09-16T02:01:40.563152+0000","last_undegraded":"2024-09-16T02:01:40.563152+0000","last_fullsized":"2024-09-16T02:01:40.563152+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:30:15.847189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'28","reported_seq":168,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:41.607971+0000","last_change":"2024-09-16T02:00:32.444922+0000","last_active":"2024-09-16T02:01:41.607971+0000","last_peered":"2024-09-16T02:01:41.607971+0000","last_clean":"2024-09-16T02:01:41.607971+0000","last_became_active":"2024-09-16T02:00:32.444540+0000","last_became_peered":"2024-09-16T02:00:32.444540+0000","last_unstale":"2024-09-16T02:01:41.607971+0000","last_undegraded":"2024-09-16T02:01:41.607971+0000","last_fullsized":"2024-09-16T02:01:41.607971+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:59.183952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59211,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":118,"num_read_kb":116,"num_write":30,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564383+0000","last_change":"2024-09-16T02:00:34.453366+0000","last_active":"2024-09-16T02:01:40.564383+0000","last_peered":"2024-09-16T02:01:40.564383+0000","last_clean":"2024-09-16T02:01:40.564383+0000","last_became_active":"2024-09-16T02:00:34.453048+0000","last_became_peered":"2024-09-16T02:00:34.453048+0000","last_unstale":"2024-09-16T02:01:40.564383+0000","last_undegraded":"2024-09-16T02:01:40.564383+0000","last_fullsized":"2024-09-16T02:01:40.564383+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:15:52.721893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.562558+0000","last_change":"2024-09-16T02:00:32.445516+0000","last_active":"2024-09-16T02:01:40.562558+0000","last_peered":"2024-09-16T02:01:40.562558+0000","last_clean":"2024-09-16T02:01:40.562558+0000","last_became_active":"2024-09-16T02:00:32.445092+0000","last_became_peered":"2024-09-16T02:00:32.445092+0000","last_unstale":"2024-09-16T02:01:40.562558+0000","last_undegraded":"2024-09-16T02:01:40.562558+0000","last_fullsized":"2024-09-16T02:01:40.562558+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:54.274692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.559134+0000","last_change":"2024-09-16T02:00:34.448450+0000","last_active":"2024-09-16T02:01:40.559134+0000","last_peered":"2024-09-16T02:01:40.559134+0000","last_clean":"2024-09-16T02:01:40.559134+0000","last_became_active":"2024-09-16T02:00:34.448116+0000","last_became_peered":"2024-09-16T02:00:34.448116+0000","last_unstale":"2024-09-16T02:01:40.559134+0000","last_undegraded":"2024-09-16T02:01:40.559134+0000","last_fullsized":"2024-09-16T02:01:40.559134+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:12:34.834798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.565309+0000","last_change":"2024-09-16T02:00:32.445427+0000","last_active":"2024-09-16T02:01:40.565309+0000","last_peered":"2024-09-16T02:01:40.565309+0000","last_clean":"2024-09-16T02:01:40.565309+0000","last_became_active":"2024-09-16T02:00:32.445058+0000","last_became_peered":"2024-09-16T02:00:32.445058+0000","last_unstale":"2024-09-16T02:01:40.565309+0000","last_undegraded":"2024-09-16T02:01:40.565309+0000","last_fullsized":"2024-09-16T02:01:40.565309+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:35:34.701331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.561778+0000","last_change":"2024-09-16T02:00:34.458479+0000","last_active":"2024-09-16T02:01:39.561778+0000","last_peered":"2024-09-16T02:01:39.561778+0000","last_clean":"2024-09-16T02:01:39.561778+0000","last_became_active":"2024-09-16T02:00:34.458135+0000","last_became_peered":"2024-09-16T02:00:34.458135+0000","last_unstale":"2024-09-16T02:01:39.561778+0000","last_undegraded":"2024-09-16T02:01:39.561778+0000","last_fullsized":"2024-09-16T02:01:39.561778+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:37:40.188861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'6","reported_seq":88,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:42.613273+0000","last_change":"2024-09-16T02:00:32.440319+0000","last_active":"2024-09-16T02:01:42.613273+0000","last_peered":"2024-09-16T02:01:42.613273+0000","last_clean":"2024-09-16T02:01:42.613273+0000","last_became_active":"2024-09-16T02:00:32.440005+0000","last_became_peered":"2024-09-16T02:00:32.440005+0000","last_unstale":"2024-09-16T02:01:42.613273+0000","last_undegraded":"2024-09-16T02:01:42.613273+0000","last_fullsized":"2024-09-16T02:01:42.613273+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:07:33.876577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":61,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.561168+0000","last_change":"2024-09-16T02:00:34.460468+0000","last_active":"2024-09-16T02:01:39.561168+0000","last_peered":"2024-09-16T02:01:39.561168+0000","last_clean":"2024-09-16T02:01:39.561168+0000","last_became_active":"2024-09-16T02:00:34.459190+0000","last_became_peered":"2024-09-16T02:00:34.459190+0000","last_unstale":"2024-09-16T02:01:39.561168+0000","last_undegraded":"2024-09-16T02:01:39.561168+0000","last_fullsized":"2024-09-16T02:01:39.561168+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:46:43.654991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.562649+0000","last_change":"2024-09-16T02:00:32.587700+0000","last_active":"2024-09-16T02:01:40.562649+0000","last_peered":"2024-09-16T02:01:40.562649+0000","last_clean":"2024-09-16T02:01:40.562649+0000","last_became_active":"2024-09-16T02:00:32.586964+0000","last_became_peered":"2024-09-16T02:00:32.586964+0000","last_unstale":"2024-09-16T02:01:40.562649+0000","last_undegraded":"2024-09-16T02:01:40.562649+0000","last_fullsized":"2024-09-16T02:01:40.562649+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:41.309918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.565038+0000","last_change":"2024-09-16T02:00:34.455652+0000","last_active":"2024-09-16T02:01:40.565038+0000","last_peered":"2024-09-16T02:01:40.565038+0000","last_clean":"2024-09-16T02:01:40.565038+0000","last_became_active":"2024-09-16T02:00:34.455306+0000","last_became_peered":"2024-09-16T02:00:34.455306+0000","last_unstale":"2024-09-16T02:01:40.565038+0000","last_undegraded":"2024-09-16T02:01:40.565038+0000","last_fullsized":"2024-09-16T02:01:40.565038+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:44:09.505096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564111+0000","last_change":"2024-09-16T02:00:32.441443+0000","last_active":"2024-09-16T02:01:40.564111+0000","last_peered":"2024-09-16T02:01:40.564111+0000","last_clean":"2024-09-16T02:01:40.564111+0000","last_became_active":"2024-09-16T02:00:32.441073+0000","last_became_peered":"2024-09-16T02:00:32.441073+0000","last_unstale":"2024-09-16T02:01:40.564111+0000","last_undegraded":"2024-09-16T02:01:40.564111+0000","last_fullsized":"2024-09-16T02:01:40.564111+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:16.865970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.611317+0000","last_change":"2024-09-16T02:01:40.611317+0000","last_active":"2024-09-16T02:01:40.611317+0000","last_peered":"2024-09-16T02:01:40.611317+0000","last_clean":"2024-09-16T02:01:40.611317+0000","last_became_active":"2024-09-16T02:01:40.610837+0000","last_became_peered":"2024-09-16T02:01:40.610837+0000","last_unstale":"2024-09-16T02:01:40.611317+0000","last_undegraded":"2024-09-16T02:01:40.611317+0000","last_fullsized":"2024-09-16T02:01:40.611317+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:19:55.376738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.617877+0000","last_change":"2024-09-16T02:01:40.617877+0000","last_active":"2024-09-16T02:01:40.617877+0000","last_peered":"2024-09-16T02:01:40.617877+0000","last_clean":"2024-09-16T02:01:40.617877+0000","last_became_active":"2024-09-16T02:01:40.611597+0000","last_became_peered":"2024-09-16T02:01:40.611597+0000","last_unstale":"2024-09-16T02:01:40.617877+0000","last_undegraded":"2024-09-16T02:01:40.617877+0000","last_fullsized":"2024-09-16T02:01:40.617877+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:53:35.796243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.562332+0000","last_change":"2024-09-16T02:00:34.461889+0000","last_active":"2024-09-16T02:01:39.562332+0000","last_peered":"2024-09-16T02:01:39.562332+0000","last_clean":"2024-09-16T02:01:39.562332+0000","last_became_active":"2024-09-16T02:00:34.461583+0000","last_became_peered":"2024-09-16T02:00:34.461583+0000","last_unstale":"2024-09-16T02:01:39.562332+0000","last_undegraded":"2024-09-16T02:01:39.562332+0000","last_fullsized":"2024-09-16T02:01:39.562332+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:32.598422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.565617+0000","last_change":"2024-09-16T02:00:32.445201+0000","last_active":"2024-09-16T02:01:40.565617+0000","last_peered":"2024-09-16T02:01:40.565617+0000","last_clean":"2024-09-16T02:01:40.565617+0000","last_became_active":"2024-09-16T02:00:32.444722+0000","last_became_peered":"2024-09-16T02:00:32.444722+0000","last_unstale":"2024-09-16T02:01:40.565617+0000","last_undegraded":"2024-09-16T02:01:40.565617+0000","last_fullsized":"2024-09-16T02:01:40.565617+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:41:11.657028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'1","reported_seq":15,"reported_epoch":79,"state":"peering","last_fresh":"2024-09-16T02:00:54.509925+0000","last_change":"2024-09-16T02:01:39.562036+0000","last_active":"2024-09-16T02:01:39.562036+0000","last_peered":"2024-09-16T02:00:54.509925+0000","last_clean":"2024-09-16T02:00:54.509925+0000","last_became_active":"2024-09-16T02:00:34.455718+0000","last_became_peered":"2024-09-16T02:00:34.455718+0000","last_unstale":"2024-09-16T02:00:54.509925+0000","last_undegraded":"2024-09-16T02:00:54.509925+0000","last_fullsized":"2024-09-16T02:00:54.509925+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5,9],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'5","reported_seq":86,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:42.610441+0000","last_change":"2024-09-16T02:00:32.860059+0000","last_active":"2024-09-16T02:01:42.610441+0000","last_peered":"2024-09-16T02:01:42.610441+0000","last_clean":"2024-09-16T02:01:42.610441+0000","last_became_active":"2024-09-16T02:00:32.859617+0000","last_became_peered":"2024-09-16T02:00:32.859617+0000","last_unstale":"2024-09-16T02:01:42.610441+0000","last_undegraded":"2024-09-16T02:01:42.610441+0000","last_fullsized":"2024-09-16T02:01:42.610441+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:41:50.248194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":59,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.561990+0000","last_change":"2024-09-16T02:00:34.454344+0000","last_active":"2024-09-16T02:01:39.561990+0000","last_peered":"2024-09-16T02:01:39.561990+0000","last_clean":"2024-09-16T02:01:39.561990+0000","last_became_active":"2024-09-16T02:00:34.454027+0000","last_became_peered":"2024-09-16T02:00:34.454027+0000","last_unstale":"2024-09-16T02:01:39.561990+0000","last_undegraded":"2024-09-16T02:01:39.561990+0000","last_fullsized":"2024-09-16T02:01:39.561990+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:33.020365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":332,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.562523+0000","last_change":"2024-09-16T01:59:19.798159+0000","last_active":"2024-09-16T02:01:39.562523+0000","last_peered":"2024-09-16T02:01:39.562523+0000","last_clean":"2024-09-16T02:01:39.562523+0000","last_became_active":"2024-09-16T01:59:19.797688+0000","last_became_peered":"2024-09-16T01:59:19.797688+0000","last_unstale":"2024-09-16T02:01:39.562523+0000","last_undegraded":"2024-09-16T02:01:39.562523+0000","last_fullsized":"2024-09-16T02:01:39.562523+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:55:54.312750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:55:54.312750+0000","last_clean_scrub_stamp":"2024-09-16T01:55:54.312750+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:00:36.860158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.610108+0000","last_change":"2024-09-16T02:00:32.441925+0000","last_active":"2024-09-16T02:01:40.610108+0000","last_peered":"2024-09-16T02:01:40.610108+0000","last_clean":"2024-09-16T02:01:40.610108+0000","last_became_active":"2024-09-16T02:00:32.441565+0000","last_became_peered":"2024-09-16T02:00:32.441565+0000","last_unstale":"2024-09-16T02:01:40.610108+0000","last_undegraded":"2024-09-16T02:01:40.610108+0000","last_fullsized":"2024-09-16T02:01:40.610108+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:23:09.755034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.561404+0000","last_change":"2024-09-16T02:00:34.452986+0000","last_active":"2024-09-16T02:01:40.561404+0000","last_peered":"2024-09-16T02:01:40.561404+0000","last_clean":"2024-09-16T02:01:40.561404+0000","last_became_active":"2024-09-16T02:00:34.452688+0000","last_became_peered":"2024-09-16T02:00:34.452688+0000","last_unstale":"2024-09-16T02:01:40.561404+0000","last_undegraded":"2024-09-16T02:01:40.561404+0000","last_fullsized":"2024-09-16T02:01:40.561404+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:12.958209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.561340+0000","last_change":"2024-09-16T02:00:32.443964+0000","last_active":"2024-09-16T02:01:39.561340+0000","last_peered":"2024-09-16T02:01:39.561340+0000","last_clean":"2024-09-16T02:01:39.561340+0000","last_became_active":"2024-09-16T02:00:32.443623+0000","last_became_peered":"2024-09-16T02:00:32.443623+0000","last_unstale":"2024-09-16T02:01:39.561340+0000","last_undegraded":"2024-09-16T02:01:39.561340+0000","last_fullsized":"2024-09-16T02:01:39.561340+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:30:34.406024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.559065+0000","last_change":"2024-09-16T02:00:34.449615+0000","last_active":"2024-09-16T02:01:40.559065+0000","last_peered":"2024-09-16T02:01:40.559065+0000","last_clean":"2024-09-16T02:01:40.559065+0000","last_became_active":"2024-09-16T02:00:34.449109+0000","last_became_peered":"2024-09-16T02:00:34.449109+0000","last_unstale":"2024-09-16T02:01:40.559065+0000","last_undegraded":"2024-09-16T02:01:40.559065+0000","last_fullsized":"2024-09-16T02:01:40.559065+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:45.036177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":85,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:42.612676+0000","last_change":"2024-09-16T02:00:32.587107+0000","last_active":"2024-09-16T02:01:42.612676+0000","last_peered":"2024-09-16T02:01:42.612676+0000","last_clean":"2024-09-16T02:01:42.612676+0000","last_became_active":"2024-09-16T02:00:32.586777+0000","last_became_peered":"2024-09-16T02:00:32.586777+0000","last_unstale":"2024-09-16T02:01:42.612676+0000","last_undegraded":"2024-09-16T02:01:42.612676+0000","last_fullsized":"2024-09-16T02:01:42.612676+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:03.968987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":58,"num_read_kb":58,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.570572+0000","last_change":"2024-09-16T02:01:40.570572+0000","last_active":"2024-09-16T02:01:40.570572+0000","last_peered":"2024-09-16T02:01:40.570572+0000","last_clean":"2024-09-16T02:01:40.570572+0000","last_became_active":"2024-09-16T02:01:40.570176+0000","last_became_peered":"2024-09-16T02:01:40.570176+0000","last_unstale":"2024-09-16T02:01:40.570572+0000","last_undegraded":"2024-09-16T02:01:40.570572+0000","last_fullsized":"2024-09-16T02:01:40.570572+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:24:13.138925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'5","reported_seq":79,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.561707+0000","last_change":"2024-09-16T02:00:32.443201+0000","last_active":"2024-09-16T02:01:39.561707+0000","last_peered":"2024-09-16T02:01:39.561707+0000","last_clean":"2024-09-16T02:01:39.561707+0000","last_became_active":"2024-09-16T02:00:32.442778+0000","last_became_peered":"2024-09-16T02:00:32.442778+0000","last_unstale":"2024-09-16T02:01:39.561707+0000","last_undegraded":"2024-09-16T02:01:39.561707+0000","last_fullsized":"2024-09-16T02:01:39.561707+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:43.933817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560410+0000","last_change":"2024-09-16T02:00:34.463000+0000","last_active":"2024-09-16T02:01:40.560410+0000","last_peered":"2024-09-16T02:01:40.560410+0000","last_clean":"2024-09-16T02:01:40.560410+0000","last_became_active":"2024-09-16T02:00:34.461890+0000","last_became_peered":"2024-09-16T02:00:34.461890+0000","last_unstale":"2024-09-16T02:01:40.560410+0000","last_undegraded":"2024-09-16T02:01:40.560410+0000","last_fullsized":"2024-09-16T02:01:40.560410+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:21:15.161295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.607447+0000","last_change":"2024-09-16T02:00:32.586933+0000","last_active":"2024-09-16T02:01:40.607447+0000","last_peered":"2024-09-16T02:01:40.607447+0000","last_clean":"2024-09-16T02:01:40.607447+0000","last_became_active":"2024-09-16T02:00:32.586565+0000","last_became_peered":"2024-09-16T02:00:32.586565+0000","last_unstale":"2024-09-16T02:01:40.607447+0000","last_undegraded":"2024-09-16T02:01:40.607447+0000","last_fullsized":"2024-09-16T02:01:40.607447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:45:07.130649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560456+0000","last_change":"2024-09-16T02:00:34.460805+0000","last_active":"2024-09-16T02:01:40.560456+0000","last_peered":"2024-09-16T02:01:40.560456+0000","last_clean":"2024-09-16T02:01:40.560456+0000","last_became_active":"2024-09-16T02:00:34.460482+0000","last_became_peered":"2024-09-16T02:00:34.460482+0000","last_unstale":"2024-09-16T02:01:40.560456+0000","last_undegraded":"2024-09-16T02:01:40.560456+0000","last_fullsized":"2024-09-16T02:01:40.560456+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:07:51.565053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564575+0000","last_change":"2024-09-16T02:00:32.587516+0000","last_active":"2024-09-16T02:01:40.564575+0000","last_peered":"2024-09-16T02:01:40.564575+0000","last_clean":"2024-09-16T02:01:40.564575+0000","last_became_active":"2024-09-16T02:00:32.587172+0000","last_became_peered":"2024-09-16T02:00:32.587172+0000","last_unstale":"2024-09-16T02:01:40.564575+0000","last_undegraded":"2024-09-16T02:01:40.564575+0000","last_fullsized":"2024-09-16T02:01:40.564575+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:52:29.018290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.559443+0000","last_change":"2024-09-16T02:00:34.446021+0000","last_active":"2024-09-16T02:01:40.559443+0000","last_peered":"2024-09-16T02:01:40.559443+0000","last_clean":"2024-09-16T02:01:40.559443+0000","last_became_active":"2024-09-16T02:00:34.445732+0000","last_became_peered":"2024-09-16T02:00:34.445732+0000","last_unstale":"2024-09-16T02:01:40.559443+0000","last_undegraded":"2024-09-16T02:01:40.559443+0000","last_fullsized":"2024-09-16T02:01:40.559443+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:51:17.960792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'14","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.607435+0000","last_change":"2024-09-16T02:00:32.439750+0000","last_active":"2024-09-16T02:01:40.607435+0000","last_peered":"2024-09-16T02:01:40.607435+0000","last_clean":"2024-09-16T02:01:40.607435+0000","last_became_active":"2024-09-16T02:00:32.439438+0000","last_became_peered":"2024-09-16T02:00:32.439438+0000","last_unstale":"2024-09-16T02:01:40.607435+0000","last_undegraded":"2024-09-16T02:01:40.607435+0000","last_fullsized":"2024-09-16T02:01:40.607435+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:50.764733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.561712+0000","last_change":"2024-09-16T02:00:34.453810+0000","last_active":"2024-09-16T02:01:40.561712+0000","last_peered":"2024-09-16T02:01:40.561712+0000","last_clean":"2024-09-16T02:01:40.561712+0000","last_became_active":"2024-09-16T02:00:34.453421+0000","last_became_peered":"2024-09-16T02:00:34.453421+0000","last_unstale":"2024-09-16T02:01:40.561712+0000","last_undegraded":"2024-09-16T02:01:40.561712+0000","last_fullsized":"2024-09-16T02:01:40.561712+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:23:50.405749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.610378+0000","last_change":"2024-09-16T02:00:32.860565+0000","last_active":"2024-09-16T02:01:40.610378+0000","last_peered":"2024-09-16T02:01:40.610378+0000","last_clean":"2024-09-16T02:01:40.610378+0000","last_became_active":"2024-09-16T02:00:32.860130+0000","last_became_peered":"2024-09-16T02:00:32.860130+0000","last_unstale":"2024-09-16T02:01:40.610378+0000","last_undegraded":"2024-09-16T02:01:40.610378+0000","last_fullsized":"2024-09-16T02:01:40.610378+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:09:55.759883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"peering","last_fresh":"2024-09-16T02:00:35.467174+0000","last_change":"2024-09-16T02:01:39.561205+0000","last_active":"2024-09-16T02:01:39.561205+0000","last_peered":"2024-09-16T02:00:35.467174+0000","last_clean":"2024-09-16T02:00:35.467174+0000","last_became_active":"2024-09-16T02:00:34.453725+0000","last_became_peered":"2024-09-16T02:00:34.453725+0000","last_unstale":"2024-09-16T02:00:35.467174+0000","last_undegraded":"2024-09-16T02:00:35.467174+0000","last_fullsized":"2024-09-16T02:00:35.467174+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,4,6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.563115+0000","last_change":"2024-09-16T02:00:32.587115+0000","last_active":"2024-09-16T02:01:40.563115+0000","last_peered":"2024-09-16T02:01:40.563115+0000","last_clean":"2024-09-16T02:01:40.563115+0000","last_became_active":"2024-09-16T02:00:32.586835+0000","last_became_peered":"2024-09-16T02:00:32.586835+0000","last_unstale":"2024-09-16T02:01:40.563115+0000","last_undegraded":"2024-09-16T02:01:40.563115+0000","last_fullsized":"2024-09-16T02:01:40.563115+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:12:58.820005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.562587+0000","last_change":"2024-09-16T02:00:34.457673+0000","last_active":"2024-09-16T02:01:40.562587+0000","last_peered":"2024-09-16T02:01:40.562587+0000","last_clean":"2024-09-16T02:01:40.562587+0000","last_became_active":"2024-09-16T02:00:34.457331+0000","last_became_peered":"2024-09-16T02:00:34.457331+0000","last_unstale":"2024-09-16T02:01:40.562587+0000","last_undegraded":"2024-09-16T02:01:40.562587+0000","last_fullsized":"2024-09-16T02:01:40.562587+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:12:16.100036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.561350+0000","last_change":"2024-09-16T02:00:32.437528+0000","last_active":"2024-09-16T02:01:40.561350+0000","last_peered":"2024-09-16T02:01:40.561350+0000","last_clean":"2024-09-16T02:01:40.561350+0000","last_became_active":"2024-09-16T02:00:32.437080+0000","last_became_peered":"2024-09-16T02:00:32.437080+0000","last_unstale":"2024-09-16T02:01:40.561350+0000","last_undegraded":"2024-09-16T02:01:40.561350+0000","last_fullsized":"2024-09-16T02:01:40.561350+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:59:33.908026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560289+0000","last_change":"2024-09-16T02:00:34.453504+0000","last_active":"2024-09-16T02:01:40.560289+0000","last_peered":"2024-09-16T02:01:40.560289+0000","last_clean":"2024-09-16T02:01:40.560289+0000","last_became_active":"2024-09-16T02:00:34.453212+0000","last_became_peered":"2024-09-16T02:00:34.453212+0000","last_unstale":"2024-09-16T02:01:40.560289+0000","last_undegraded":"2024-09-16T02:01:40.560289+0000","last_fullsized":"2024-09-16T02:01:40.560289+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:00:56.414548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.562417+0000","last_change":"2024-09-16T02:00:32.442282+0000","last_active":"2024-09-16T02:01:40.562417+0000","last_peered":"2024-09-16T02:01:40.562417+0000","last_clean":"2024-09-16T02:01:40.562417+0000","last_became_active":"2024-09-16T02:00:32.441910+0000","last_became_peered":"2024-09-16T02:00:32.441910+0000","last_unstale":"2024-09-16T02:01:40.562417+0000","last_undegraded":"2024-09-16T02:01:40.562417+0000","last_fullsized":"2024-09-16T02:01:40.562417+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:01:46.045021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.561467+0000","last_change":"2024-09-16T02:00:34.460177+0000","last_active":"2024-09-16T02:01:40.561467+0000","last_peered":"2024-09-16T02:01:40.561467+0000","last_clean":"2024-09-16T02:01:40.561467+0000","last_became_active":"2024-09-16T02:00:34.459874+0000","last_became_peered":"2024-09-16T02:00:34.459874+0000","last_unstale":"2024-09-16T02:01:40.561467+0000","last_undegraded":"2024-09-16T02:01:40.561467+0000","last_fullsized":"2024-09-16T02:01:40.561467+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:35.855591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.614997+0000","last_change":"2024-09-16T02:01:40.614997+0000","last_active":"2024-09-16T02:01:40.614997+0000","last_peered":"2024-09-16T02:01:40.614997+0000","last_clean":"2024-09-16T02:01:40.614997+0000","last_became_active":"2024-09-16T02:01:40.610889+0000","last_became_peered":"2024-09-16T02:01:40.610889+0000","last_unstale":"2024-09-16T02:01:40.614997+0000","last_undegraded":"2024-09-16T02:01:40.614997+0000","last_fullsized":"2024-09-16T02:01:40.614997+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:39:00.989594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.561554+0000","last_change":"2024-09-16T02:00:34.452964+0000","last_active":"2024-09-16T02:01:39.561554+0000","last_peered":"2024-09-16T02:01:39.561554+0000","last_clean":"2024-09-16T02:01:39.561554+0000","last_became_active":"2024-09-16T02:00:34.452617+0000","last_became_peered":"2024-09-16T02:00:34.452617+0000","last_unstale":"2024-09-16T02:01:39.561554+0000","last_undegraded":"2024-09-16T02:01:39.561554+0000","last_fullsized":"2024-09-16T02:01:39.561554+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:04.617970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.565506+0000","last_change":"2024-09-16T02:00:34.456585+0000","last_active":"2024-09-16T02:01:40.565506+0000","last_peered":"2024-09-16T02:01:40.565506+0000","last_clean":"2024-09-16T02:01:40.565506+0000","last_became_active":"2024-09-16T02:00:34.456199+0000","last_became_peered":"2024-09-16T02:00:34.456199+0000","last_unstale":"2024-09-16T02:01:40.565506+0000","last_undegraded":"2024-09-16T02:01:40.565506+0000","last_fullsized":"2024-09-16T02:01:40.565506+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:37:54.376785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.579320+0000","last_change":"2024-09-16T02:01:40.578904+0000","last_active":"2024-09-16T02:01:40.579320+0000","last_peered":"2024-09-16T02:01:40.579320+0000","last_clean":"2024-09-16T02:01:40.579320+0000","last_became_active":"2024-09-16T02:01:40.567221+0000","last_became_peered":"2024-09-16T02:01:40.567221+0000","last_unstale":"2024-09-16T02:01:40.579320+0000","last_undegraded":"2024-09-16T02:01:40.579320+0000","last_fullsized":"2024-09-16T02:01:40.579320+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:01.407145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.563083+0000","last_change":"2024-09-16T02:00:34.460925+0000","last_active":"2024-09-16T02:01:40.563083+0000","last_peered":"2024-09-16T02:01:40.563083+0000","last_clean":"2024-09-16T02:01:40.563083+0000","last_became_active":"2024-09-16T02:00:34.460581+0000","last_became_peered":"2024-09-16T02:00:34.460581+0000","last_unstale":"2024-09-16T02:01:40.563083+0000","last_undegraded":"2024-09-16T02:01:40.563083+0000","last_fullsized":"2024-09-16T02:01:40.563083+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:19:44.460695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564476+0000","last_change":"2024-09-16T02:00:32.443877+0000","last_active":"2024-09-16T02:01:40.564476+0000","last_peered":"2024-09-16T02:01:40.564476+0000","last_clean":"2024-09-16T02:01:40.564476+0000","last_became_active":"2024-09-16T02:00:32.443416+0000","last_became_peered":"2024-09-16T02:00:32.443416+0000","last_unstale":"2024-09-16T02:01:40.564476+0000","last_undegraded":"2024-09-16T02:01:40.564476+0000","last_fullsized":"2024-09-16T02:01:40.564476+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:36:57.367232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T02:01:39.563554+0000","last_change":"2024-09-16T02:00:34.453048+0000","last_active":"2024-09-16T02:01:39.563554+0000","last_peered":"2024-09-16T02:01:39.563554+0000","last_clean":"2024-09-16T02:01:39.563554+0000","last_became_active":"2024-09-16T02:00:34.452752+0000","last_became_peered":"2024-09-16T02:00:34.452752+0000","last_unstale":"2024-09-16T02:01:39.563554+0000","last_undegraded":"2024-09-16T02:01:39.563554+0000","last_fullsized":"2024-09-16T02:01:39.563554+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:55.970195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.561951+0000","last_change":"2024-09-16T02:00:32.586974+0000","last_active":"2024-09-16T02:01:40.561951+0000","last_peered":"2024-09-16T02:01:40.561951+0000","last_clean":"2024-09-16T02:01:40.561951+0000","last_became_active":"2024-09-16T02:00:32.586644+0000","last_became_peered":"2024-09-16T02:00:32.586644+0000","last_unstale":"2024-09-16T02:01:40.561951+0000","last_undegraded":"2024-09-16T02:01:40.561951+0000","last_fullsized":"2024-09-16T02:01:40.561951+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:52.913287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.561774+0000","last_change":"2024-09-16T02:00:34.447548+0000","last_active":"2024-09-16T02:01:40.561774+0000","last_peered":"2024-09-16T02:01:40.561774+0000","last_clean":"2024-09-16T02:01:40.561774+0000","last_became_active":"2024-09-16T02:00:34.447207+0000","last_became_peered":"2024-09-16T02:00:34.447207+0000","last_unstale":"2024-09-16T02:01:40.561774+0000","last_undegraded":"2024-09-16T02:01:40.561774+0000","last_fullsized":"2024-09-16T02:01:40.561774+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:22:02.494343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":90,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.607561+0000","last_change":"2024-09-16T02:00:32.441129+0000","last_active":"2024-09-16T02:01:40.607561+0000","last_peered":"2024-09-16T02:01:40.607561+0000","last_clean":"2024-09-16T02:01:40.607561+0000","last_became_active":"2024-09-16T02:00:32.440843+0000","last_became_peered":"2024-09-16T02:00:32.440843+0000","last_unstale":"2024-09-16T02:01:40.607561+0000","last_undegraded":"2024-09-16T02:01:40.607561+0000","last_fullsized":"2024-09-16T02:01:40.607561+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:17.721740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66026,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":593,"num_read_kb":367,"num_write":238,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":198078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281576,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612636,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":892928,"data_stored":432750,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841569,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1044480,"data_stored":1082801,"data_compressed":41575,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005093,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":901120,"data_stored":423573,"data_compressed":2878,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234027,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":431894,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462959,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":950272,"data_stored":1016471,"data_compressed":38690,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626486,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":897024,"data_stored":431815,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855418,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":974848,"data_stored":1024281,"data_compressed":41575,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018943,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":897024,"data_stored":432069,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247877,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":909312,"data_stored":431523,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444106,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":978944,"data_stored":492159,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":958464,"data_stored":491939,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":905216,"data_stored":424629,"data_compressed":2878,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T02:01:46.352 INFO:tasks.cephadm.ceph_manager.ceph:PG 3.c is not active+clean 2024-09-16T02:01:46.352 INFO:tasks.cephadm.ceph_manager.ceph:{'pgid': '3.c', 'version': "79'1", 'reported_seq': 15, 'reported_epoch': 79, 'state': 'peering', 'last_fresh': '2024-09-16T02:00:54.509925+0000', 'last_change': '2024-09-16T02:01:39.562036+0000', 'last_active': '2024-09-16T02:01:39.562036+0000', 'last_peered': '2024-09-16T02:00:54.509925+0000', 'last_clean': '2024-09-16T02:00:54.509925+0000', 'last_became_active': '2024-09-16T02:00:34.455718+0000', 'last_became_peered': '2024-09-16T02:00:34.455718+0000', 'last_unstale': '2024-09-16T02:00:54.509925+0000', 'last_undegraded': '2024-09-16T02:00:54.509925+0000', 'last_fullsized': '2024-09-16T02:00:54.509925+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 78, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T02:00:33.428129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T02:00:33.428129+0000', 'last_clean_scrub_stamp': '2024-09-16T02:00:33.428129+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 117, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 3, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1, 'num_write_kb': 1, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 9, 5], 'acting': [7, 9, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3, 5, 9], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T02:01:46.352 INFO:tasks.cephadm.ceph_manager.ceph:PG 3.6 is not active+clean 2024-09-16T02:01:46.352 INFO:tasks.cephadm.ceph_manager.ceph:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 14, 'reported_epoch': 79, 'state': 'peering', 'last_fresh': '2024-09-16T02:00:35.467174+0000', 'last_change': '2024-09-16T02:01:39.561205+0000', 'last_active': '2024-09-16T02:01:39.561205+0000', 'last_peered': '2024-09-16T02:00:35.467174+0000', 'last_clean': '2024-09-16T02:00:35.467174+0000', 'last_became_active': '2024-09-16T02:00:34.453725+0000', 'last_became_peered': '2024-09-16T02:00:34.453725+0000', 'last_unstale': '2024-09-16T02:00:35.467174+0000', 'last_undegraded': '2024-09-16T02:00:35.467174+0000', 'last_fullsized': '2024-09-16T02:00:35.467174+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 78, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T02:00:33.428129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T02:00:33.428129+0000', 'last_clean_scrub_stamp': '2024-09-16T02:00:33.428129+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3, 6], 'acting': [0, 3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3, 4, 6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T02:01:46.353 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph status --format=json 2024-09-16T02:01:46.601 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:46.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:46 smithi190 ceph-mon[34762]: pgmap v314: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s; 72 B/s, 1 objects/s recovering 2024-09-16T02:01:46.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:46 smithi190 ceph-mon[34762]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:01:46.865 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:46 smithi105 ceph-mon[31197]: pgmap v314: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s; 72 B/s, 1 objects/s recovering 2024-09-16T02:01:46.865 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:46 smithi105 ceph-mon[31197]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:01:46.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:46 smithi137 ceph-mon[34099]: pgmap v314: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s; 72 B/s, 1 objects/s recovering 2024-09-16T02:01:46.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:46 smithi137 ceph-mon[34099]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:01:47.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.087+0000 7f1877439640 1 -- 172.21.15.105:0/1090463280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1870103660 msgr2=0x7f187010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:47.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.087+0000 7f1877439640 1 --2- 172.21.15.105:0/1090463280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1870103660 0x7f187010bb50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1864009ca0 tx=0x7f186402a950 comp rx=0 tx=0).stop 2024-09-16T02:01:47.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.087+0000 7f1877439640 1 -- 172.21.15.105:0/1090463280 shutdown_connections 2024-09-16T02:01:47.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.087+0000 7f1877439640 1 --2- 172.21.15.105:0/1090463280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f187010c170 0x7f187010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:47.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.087+0000 7f1877439640 1 --2- 172.21.15.105:0/1090463280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1870103660 0x7f187010bb50 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:47.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.087+0000 7f1877439640 1 --2- 172.21.15.105:0/1090463280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1870102d40 0x7f1870103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:47.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.087+0000 7f1877439640 1 -- 172.21.15.105:0/1090463280 >> 172.21.15.105:0/1090463280 conn(0x7f18700fc7a0 msgr2=0x7f18700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:47.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.087+0000 7f1877439640 1 -- 172.21.15.105:0/1090463280 shutdown_connections 2024-09-16T02:01:47.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.087+0000 7f1877439640 1 -- 172.21.15.105:0/1090463280 wait complete. 2024-09-16T02:01:47.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.087+0000 7f1877439640 1 Processor -- start 2024-09-16T02:01:47.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.087+0000 7f1877439640 1 -- start start 2024-09-16T02:01:47.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.088+0000 7f1877439640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1870102d40 0x7f1870198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:47.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.088+0000 7f1877439640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1870103660 0x7f1870198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:47.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.088+0000 7f1877439640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f187010c170 0x7f187019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:47.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.088+0000 7f1877439640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1870111e60 con 0x7f1870102d40 2024-09-16T02:01:47.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.088+0000 7f1877439640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1870111ce0 con 0x7f1870103660 2024-09-16T02:01:47.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.088+0000 7f1877439640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1870111fe0 con 0x7f187010c170 2024-09-16T02:01:47.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.088+0000 7f18751d5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1870102d40 0x7f1870198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:47.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.088+0000 7f18751d5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1870102d40 0x7f1870198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33102/0 (socket says 172.21.15.105:33102) 2024-09-16T02:01:47.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.088+0000 7f18751d5640 1 -- 172.21.15.105:0/1534579337 learned_addr learned my addr 172.21.15.105:0/1534579337 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:01:47.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.088+0000 7f18749d4640 1 --2- 172.21.15.105:0/1534579337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1870103660 0x7f1870198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:47.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.088+0000 7f18751d5640 1 -- 172.21.15.105:0/1534579337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f187010c170 msgr2=0x7f187019cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:47.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.088+0000 7f18751d5640 1 --2- 172.21.15.105:0/1534579337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f187010c170 0x7f187019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:47.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.088+0000 7f18751d5640 1 -- 172.21.15.105:0/1534579337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1870103660 msgr2=0x7f1870198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:47.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.088+0000 7f18751d5640 1 --2- 172.21.15.105:0/1534579337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1870103660 0x7f1870198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:47.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.088+0000 7f18751d5640 1 -- 172.21.15.105:0/1534579337 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f187019d300 con 0x7f1870102d40 2024-09-16T02:01:47.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.088+0000 7f18751d5640 1 --2- 172.21.15.105:0/1534579337 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1870102d40 0x7f1870198350 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f186800c920 tx=0x7f186800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:47.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.088+0000 7f18627fc640 1 -- 172.21.15.105:0/1534579337 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1868007c70 con 0x7f1870102d40 2024-09-16T02:01:47.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.088+0000 7f18627fc640 1 -- 172.21.15.105:0/1534579337 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1868007e10 con 0x7f1870102d40 2024-09-16T02:01:47.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.088+0000 7f18627fc640 1 -- 172.21.15.105:0/1534579337 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1868018820 con 0x7f1870102d40 2024-09-16T02:01:47.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.088+0000 7f1877439640 1 -- 172.21.15.105:0/1534579337 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f187019d5f0 con 0x7f1870102d40 2024-09-16T02:01:47.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.088+0000 7f1877439640 1 -- 172.21.15.105:0/1534579337 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18701a4ed0 con 0x7f1870102d40 2024-09-16T02:01:47.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.089+0000 7f1877439640 1 -- 172.21.15.105:0/1534579337 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1870103120 con 0x7f1870102d40 2024-09-16T02:01:47.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.091+0000 7f18627fc640 1 -- 172.21.15.105:0/1534579337 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f18680189c0 con 0x7f1870102d40 2024-09-16T02:01:47.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.091+0000 7f18627fc640 1 --2- 172.21.15.105:0/1534579337 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f185c0a3350 0x7f185c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.091+0000 7f18749d4640 1 --2- 172.21.15.105:0/1534579337 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f185c0a3350 0x7f185c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.091+0000 7f18627fc640 1 -- 172.21.15.105:0/1534579337 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f18680cfec0 con 0x7f1870102d40 2024-09-16T02:01:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.092+0000 7f18749d4640 1 --2- 172.21.15.105:0/1534579337 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f185c0a3350 0x7f185c0a5810 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1864009a60 tx=0x7f1864009890 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.094+0000 7f18627fc640 1 -- 172.21.15.105:0/1534579337 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1868014030 con 0x7f1870102d40 2024-09-16T02:01:47.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.327+0000 7f1877439640 1 -- 172.21.15.105:0/1534579337 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f18701996b0 con 0x7f1870102d40 2024-09-16T02:01:47.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.328+0000 7f18627fc640 1 -- 172.21.15.105:0/1534579337 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+3393 (secure 0 0 0) 0x7f186809cb70 con 0x7f1870102d40 2024-09-16T02:01:47.329 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:01:47.329 INFO:teuthology.orchestra.run.smithi105.stdout:{"fsid":"423b7a2a-73ce-11ef-bceb-c7b262605968","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 1 pg peering","count":1},"muted":false}},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":441,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1726451982,"num_in_osds":12,"osd_in_since":1726451958,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":95},{"state_name":"peering","count":2}],"num_pgs":97,"num_pools":3,"num_objects":102,"data_bytes":656628,"bytes_used":377417728,"bytes_avail":1151563010048,"bytes_total":1151940427776,"inactive_pgs_ratio":0.020618556067347527,"recovering_objects_per_sec":1,"recovering_bytes_per_sec":72,"recovering_keys_per_sec":0,"num_objects_recovered":10,"num_bytes_recovered":582,"num_keys_recovered":0,"read_bytes_sec":5630,"read_op_per_sec":11,"write_op_per_sec":0},"fsmap":{"epoch":27,"btime":"2024-09-16T02:00:47:526725+0000","id":1,"up":5,"in":5,"max":5,"by_rank":[{"filesystem_id":1,"rank":4,"name":"g","status":"up:standby-replay","gid":14613},{"filesystem_id":1,"rank":3,"name":"a","status":"up:standby-replay","gid":14625},{"filesystem_id":1,"rank":2,"name":"l","status":"up:standby-replay","gid":24421},{"filesystem_id":1,"rank":1,"name":"k","status":"up:standby-replay","gid":24422},{"filesystem_id":1,"rank":0,"name":"e","status":"up:standby-replay","gid":24443},{"filesystem_id":1,"rank":4,"name":"f","status":"up:active","gid":24445},{"filesystem_id":1,"rank":3,"name":"c","status":"up:active","gid":24457},{"filesystem_id":1,"rank":2,"name":"h","status":"up:active","gid":24461},{"filesystem_id":1,"rank":1,"name":"i","status":"up:active","gid":24469},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24482}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.105:8443/"}},"servicemap":{"epoch":21,"modified":"2024-09-16T02:01:41.460051+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"e":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"f":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"g":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"k":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"l":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-16T02:01:47.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.329+0000 7f1877439640 1 -- 172.21.15.105:0/1534579337 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f185c0a3350 msgr2=0x7f185c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:47.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.329+0000 7f1877439640 1 --2- 172.21.15.105:0/1534579337 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f185c0a3350 0x7f185c0a5810 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1864009a60 tx=0x7f1864009890 comp rx=0 tx=0).stop 2024-09-16T02:01:47.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.330+0000 7f1877439640 1 -- 172.21.15.105:0/1534579337 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1870102d40 msgr2=0x7f1870198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:47.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.330+0000 7f1877439640 1 --2- 172.21.15.105:0/1534579337 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1870102d40 0x7f1870198350 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f186800c920 tx=0x7f186800cde0 comp rx=0 tx=0).stop 2024-09-16T02:01:47.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.330+0000 7f1877439640 1 -- 172.21.15.105:0/1534579337 shutdown_connections 2024-09-16T02:01:47.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.330+0000 7f1877439640 1 --2- 172.21.15.105:0/1534579337 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f185c0a3350 0x7f185c0a5810 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:47.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.330+0000 7f1877439640 1 --2- 172.21.15.105:0/1534579337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f187010c170 0x7f187019cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:47.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.330+0000 7f1877439640 1 --2- 172.21.15.105:0/1534579337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1870103660 0x7f1870198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:47.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.330+0000 7f1877439640 1 --2- 172.21.15.105:0/1534579337 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1870102d40 0x7f1870198350 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:47.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.330+0000 7f1877439640 1 -- 172.21.15.105:0/1534579337 >> 172.21.15.105:0/1534579337 conn(0x7f18700fc7a0 msgr2=0x7f18700fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:47.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.330+0000 7f1877439640 1 -- 172.21.15.105:0/1534579337 shutdown_connections 2024-09-16T02:01:47.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:47.330+0000 7f1877439640 1 -- 172.21.15.105:0/1534579337 wait complete. 2024-09-16T02:01:47.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:47 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/1534579337' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-16T02:01:47.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:47 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/1534579337' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-16T02:01:48.023 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:47 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/1534579337' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-16T02:01:48.052 INFO:tasks.cephadm.ceph_manager.ceph:making progress, resetting timeout 2024-09-16T02:01:48.052 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T02:01:48.297 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:48.703 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:48 smithi105 ceph-mon[31197]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 87 B/s, 1 objects/s recovering 2024-09-16T02:01:48.703 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:48 smithi105 ceph-mon[31197]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-16T02:01:48.703 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:48 smithi105 ceph-mon[31197]: Cluster is now healthy 2024-09-16T02:01:48.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.780+0000 7f398efb0640 1 -- 172.21.15.105:0/2070225597 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3988100a80 msgr2=0x7f3988100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:48.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.780+0000 7f398efb0640 1 --2- 172.21.15.105:0/2070225597 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3988100a80 0x7f3988100ee0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3974009930 tx=0x7f397402a420 comp rx=0 tx=0).stop 2024-09-16T02:01:48.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.780+0000 7f398efb0640 1 -- 172.21.15.105:0/2070225597 shutdown_connections 2024-09-16T02:01:48.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.780+0000 7f398efb0640 1 --2- 172.21.15.105:0/2070225597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3988101420 0x7f398810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:48.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.780+0000 7f398efb0640 1 --2- 172.21.15.105:0/2070225597 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3988100a80 0x7f3988100ee0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:48.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.780+0000 7f398efb0640 1 --2- 172.21.15.105:0/2070225597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3988106a80 0x7f3988106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:48.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.780+0000 7f398efb0640 1 -- 172.21.15.105:0/2070225597 >> 172.21.15.105:0/2070225597 conn(0x7f39880fc7a0 msgr2=0x7f39880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:48.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.780+0000 7f398efb0640 1 -- 172.21.15.105:0/2070225597 shutdown_connections 2024-09-16T02:01:48.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.780+0000 7f398efb0640 1 -- 172.21.15.105:0/2070225597 wait complete. 2024-09-16T02:01:48.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.780+0000 7f398efb0640 1 Processor -- start 2024-09-16T02:01:48.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.781+0000 7f398efb0640 1 -- start start 2024-09-16T02:01:48.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.781+0000 7f398efb0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3988100a80 0x7f398819a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:48.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.781+0000 7f398efb0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3988101420 0x7f398819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:48.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.781+0000 7f398efb0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3988106a80 0x7f398819ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:48.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.781+0000 7f398efb0640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3988110d50 con 0x7f3988101420 2024-09-16T02:01:48.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.781+0000 7f398efb0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3988110bd0 con 0x7f3988100a80 2024-09-16T02:01:48.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.781+0000 7f398efb0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3988110ed0 con 0x7f3988106a80 2024-09-16T02:01:48.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.781+0000 7f3987fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3988101420 0x7f398819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:48.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.781+0000 7f3987fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3988101420 0x7f398819aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54790/0 (socket says 172.21.15.105:54790) 2024-09-16T02:01:48.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.781+0000 7f3987fff640 1 -- 172.21.15.105:0/1149649043 learned_addr learned my addr 172.21.15.105:0/1149649043 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:01:48.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.781+0000 7f3987fff640 1 -- 172.21.15.105:0/1149649043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3988106a80 msgr2=0x7f398819ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:48.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.781+0000 7f3987fff640 1 --2- 172.21.15.105:0/1149649043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3988106a80 0x7f398819ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:48.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.781+0000 7f3987fff640 1 -- 172.21.15.105:0/1149649043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3988100a80 msgr2=0x7f398819a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:48.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.781+0000 7f3987fff640 1 --2- 172.21.15.105:0/1149649043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3988100a80 0x7f398819a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:48.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.781+0000 7f3987fff640 1 -- 172.21.15.105:0/1149649043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f398819f520 con 0x7f3988101420 2024-09-16T02:01:48.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.781+0000 7f3987fff640 1 --2- 172.21.15.105:0/1149649043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3988101420 0x7f398819aab0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3974005940 tx=0x7f39740042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:48.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.781+0000 7f3985ffb640 1 -- 172.21.15.105:0/1149649043 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397403f030 con 0x7f3988101420 2024-09-16T02:01:48.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.781+0000 7f3985ffb640 1 -- 172.21.15.105:0/1149649043 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3974003cb0 con 0x7f3988101420 2024-09-16T02:01:48.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.781+0000 7f398efb0640 1 -- 172.21.15.105:0/1149649043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f398819f7b0 con 0x7f3988101420 2024-09-16T02:01:48.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.781+0000 7f3985ffb640 1 -- 172.21.15.105:0/1149649043 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3974035070 con 0x7f3988101420 2024-09-16T02:01:48.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.782+0000 7f398efb0640 1 -- 172.21.15.105:0/1149649043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39881a70f0 con 0x7f3988101420 2024-09-16T02:01:48.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.783+0000 7f398efb0640 1 -- 172.21.15.105:0/1149649043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3988100ee0 con 0x7f3988101420 2024-09-16T02:01:48.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.784+0000 7f3985ffb640 1 -- 172.21.15.105:0/1149649043 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f3974042030 con 0x7f3988101420 2024-09-16T02:01:48.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.784+0000 7f3985ffb640 1 --2- 172.21.15.105:0/1149649043 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f39780a3350 0x7f39780a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:48.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.785+0000 7f398cd4c640 1 --2- 172.21.15.105:0/1149649043 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f39780a3350 0x7f39780a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:48.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.785+0000 7f3985ffb640 1 -- 172.21.15.105:0/1149649043 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f39740f02a0 con 0x7f3988101420 2024-09-16T02:01:48.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.785+0000 7f398cd4c640 1 --2- 172.21.15.105:0/1149649043 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f39780a3350 0x7f39780a5810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f397c009710 tx=0x7f397c006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:48.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.787+0000 7f3985ffb640 1 -- 172.21.15.105:0/1149649043 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f39740bce40 con 0x7f3988101420 2024-09-16T02:01:48.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:48 smithi137 ceph-mon[34099]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 87 B/s, 1 objects/s recovering 2024-09-16T02:01:48.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:48 smithi137 ceph-mon[34099]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-16T02:01:48.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:48 smithi137 ceph-mon[34099]: Cluster is now healthy 2024-09-16T02:01:48.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.951+0000 7f398efb0640 1 -- 172.21.15.105:0/1149649043 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f398810cd20 con 0x7f39780a3350 2024-09-16T02:01:48.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.957+0000 7f3985ffb640 1 -- 172.21.15.105:0/1149649043 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259054 (secure 0 0 0) 0x7f398810cd20 con 0x7f39780a3350 2024-09-16T02:01:48.960 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:01:48.961 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-09-16T02:01:48.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.960+0000 7f398efb0640 1 -- 172.21.15.105:0/1149649043 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f39780a3350 msgr2=0x7f39780a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:48.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.960+0000 7f398efb0640 1 --2- 172.21.15.105:0/1149649043 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f39780a3350 0x7f39780a5810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f397c009710 tx=0x7f397c006c90 comp rx=0 tx=0).stop 2024-09-16T02:01:48.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.960+0000 7f398efb0640 1 -- 172.21.15.105:0/1149649043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3988101420 msgr2=0x7f398819aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:48.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.960+0000 7f398efb0640 1 --2- 172.21.15.105:0/1149649043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3988101420 0x7f398819aab0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3974005940 tx=0x7f39740042f0 comp rx=0 tx=0).stop 2024-09-16T02:01:48.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.960+0000 7f398efb0640 1 -- 172.21.15.105:0/1149649043 shutdown_connections 2024-09-16T02:01:48.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.960+0000 7f398efb0640 1 --2- 172.21.15.105:0/1149649043 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f39780a3350 0x7f39780a5810 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:48.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.960+0000 7f398efb0640 1 --2- 172.21.15.105:0/1149649043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3988106a80 0x7f398819ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:48.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.960+0000 7f398efb0640 1 --2- 172.21.15.105:0/1149649043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3988101420 0x7f398819aab0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:48.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.960+0000 7f398efb0640 1 --2- 172.21.15.105:0/1149649043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3988100a80 0x7f398819a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:48.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.960+0000 7f398efb0640 1 -- 172.21.15.105:0/1149649043 >> 172.21.15.105:0/1149649043 conn(0x7f39880fc7a0 msgr2=0x7f398806a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:48.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.960+0000 7f398efb0640 1 -- 172.21.15.105:0/1149649043 shutdown_connections 2024-09-16T02:01:48.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:48.960+0000 7f398efb0640 1 -- 172.21.15.105:0/1149649043 wait complete. 2024-09-16T02:01:49.015 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:48 smithi190 ceph-mon[34762]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 87 B/s, 1 objects/s recovering 2024-09-16T02:01:49.015 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:48 smithi190 ceph-mon[34762]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-16T02:01:49.015 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:48 smithi190 ceph-mon[34762]: Cluster is now healthy 2024-09-16T02:01:49.702 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":315,"stamp":"2024-09-16T02:01:47.461364+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656628,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":729,"num_read_kb":481,"num_write":421,"num_write_kb":3070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1181435,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368624,"kb_used_data":10992,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124573200,"statfs":{"total":1151940427776,"available":1151562956800,"internally_reserved":0,"allocated":11255808,"data_stored":7141596,"data_compressed":176592,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":34,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001893"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560827+0000","last_change":"2024-09-16T02:00:34.454121+0000","last_active":"2024-09-16T02:01:40.560827+0000","last_peered":"2024-09-16T02:01:40.560827+0000","last_clean":"2024-09-16T02:01:40.560827+0000","last_became_active":"2024-09-16T02:00:34.453810+0000","last_became_peered":"2024-09-16T02:00:34.453810+0000","last_unstale":"2024-09-16T02:01:40.560827+0000","last_undegraded":"2024-09-16T02:01:40.560827+0000","last_fullsized":"2024-09-16T02:01:40.560827+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:42:38.781928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.562651+0000","last_change":"2024-09-16T02:00:34.462020+0000","last_active":"2024-09-16T02:01:40.562651+0000","last_peered":"2024-09-16T02:01:40.562651+0000","last_clean":"2024-09-16T02:01:40.562651+0000","last_became_active":"2024-09-16T02:00:34.461680+0000","last_became_peered":"2024-09-16T02:00:34.461680+0000","last_unstale":"2024-09-16T02:01:40.562651+0000","last_undegraded":"2024-09-16T02:01:40.562651+0000","last_fullsized":"2024-09-16T02:01:40.562651+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:00:14.493643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.562347+0000","last_change":"2024-09-16T02:00:34.453034+0000","last_active":"2024-09-16T02:01:40.562347+0000","last_peered":"2024-09-16T02:01:40.562347+0000","last_clean":"2024-09-16T02:01:40.562347+0000","last_became_active":"2024-09-16T02:00:34.452719+0000","last_became_peered":"2024-09-16T02:00:34.452719+0000","last_unstale":"2024-09-16T02:01:40.562347+0000","last_undegraded":"2024-09-16T02:01:40.562347+0000","last_fullsized":"2024-09-16T02:01:40.562347+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:48:52.478501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.610231+0000","last_change":"2024-09-16T02:00:34.458156+0000","last_active":"2024-09-16T02:01:40.610231+0000","last_peered":"2024-09-16T02:01:40.610231+0000","last_clean":"2024-09-16T02:01:40.610231+0000","last_became_active":"2024-09-16T02:00:34.457831+0000","last_became_peered":"2024-09-16T02:00:34.457831+0000","last_unstale":"2024-09-16T02:01:40.610231+0000","last_undegraded":"2024-09-16T02:01:40.610231+0000","last_fullsized":"2024-09-16T02:01:40.610231+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:35:36.609046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.562621+0000","last_change":"2024-09-16T02:00:34.448638+0000","last_active":"2024-09-16T02:01:40.562621+0000","last_peered":"2024-09-16T02:01:40.562621+0000","last_clean":"2024-09-16T02:01:40.562621+0000","last_became_active":"2024-09-16T02:00:34.447993+0000","last_became_peered":"2024-09-16T02:00:34.447993+0000","last_unstale":"2024-09-16T02:01:40.562621+0000","last_undegraded":"2024-09-16T02:01:40.562621+0000","last_fullsized":"2024-09-16T02:01:40.562621+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:33:04.832053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560794+0000","last_change":"2024-09-16T02:00:34.462651+0000","last_active":"2024-09-16T02:01:40.560794+0000","last_peered":"2024-09-16T02:01:40.560794+0000","last_clean":"2024-09-16T02:01:40.560794+0000","last_became_active":"2024-09-16T02:00:34.461837+0000","last_became_peered":"2024-09-16T02:00:34.461837+0000","last_unstale":"2024-09-16T02:01:40.560794+0000","last_undegraded":"2024-09-16T02:01:40.560794+0000","last_fullsized":"2024-09-16T02:01:40.560794+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:52:13.733591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564160+0000","last_change":"2024-09-16T02:00:34.455362+0000","last_active":"2024-09-16T02:01:40.564160+0000","last_peered":"2024-09-16T02:01:40.564160+0000","last_clean":"2024-09-16T02:01:40.564160+0000","last_became_active":"2024-09-16T02:00:34.455029+0000","last_became_peered":"2024-09-16T02:00:34.455029+0000","last_unstale":"2024-09-16T02:01:40.564160+0000","last_undegraded":"2024-09-16T02:01:40.564160+0000","last_fullsized":"2024-09-16T02:01:40.564160+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:14:08.316845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560955+0000","last_change":"2024-09-16T02:00:34.459578+0000","last_active":"2024-09-16T02:01:40.560955+0000","last_peered":"2024-09-16T02:01:40.560955+0000","last_clean":"2024-09-16T02:01:40.560955+0000","last_became_active":"2024-09-16T02:00:34.459288+0000","last_became_peered":"2024-09-16T02:00:34.459288+0000","last_unstale":"2024-09-16T02:01:40.560955+0000","last_undegraded":"2024-09-16T02:01:40.560955+0000","last_fullsized":"2024-09-16T02:01:40.560955+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:52:28.974276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.563476+0000","last_change":"2024-09-16T02:00:34.463011+0000","last_active":"2024-09-16T02:01:40.563476+0000","last_peered":"2024-09-16T02:01:40.563476+0000","last_clean":"2024-09-16T02:01:40.563476+0000","last_became_active":"2024-09-16T02:00:34.462668+0000","last_became_peered":"2024-09-16T02:00:34.462668+0000","last_unstale":"2024-09-16T02:01:40.563476+0000","last_undegraded":"2024-09-16T02:01:40.563476+0000","last_fullsized":"2024-09-16T02:01:40.563476+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:06:31.655869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560622+0000","last_change":"2024-09-16T02:00:34.462241+0000","last_active":"2024-09-16T02:01:40.560622+0000","last_peered":"2024-09-16T02:01:40.560622+0000","last_clean":"2024-09-16T02:01:40.560622+0000","last_became_active":"2024-09-16T02:00:34.461925+0000","last_became_peered":"2024-09-16T02:00:34.461925+0000","last_unstale":"2024-09-16T02:01:40.560622+0000","last_undegraded":"2024-09-16T02:01:40.560622+0000","last_fullsized":"2024-09-16T02:01:40.560622+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:48:02.414471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564743+0000","last_change":"2024-09-16T02:00:34.466132+0000","last_active":"2024-09-16T02:01:40.564743+0000","last_peered":"2024-09-16T02:01:40.564743+0000","last_clean":"2024-09-16T02:01:40.564743+0000","last_became_active":"2024-09-16T02:00:34.465750+0000","last_became_peered":"2024-09-16T02:00:34.465750+0000","last_unstale":"2024-09-16T02:01:40.564743+0000","last_undegraded":"2024-09-16T02:01:40.564743+0000","last_fullsized":"2024-09-16T02:01:40.564743+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:07:33.072432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.610362+0000","last_change":"2024-09-16T02:00:34.461759+0000","last_active":"2024-09-16T02:01:40.610362+0000","last_peered":"2024-09-16T02:01:40.610362+0000","last_clean":"2024-09-16T02:01:40.610362+0000","last_became_active":"2024-09-16T02:00:34.461385+0000","last_became_peered":"2024-09-16T02:00:34.461385+0000","last_unstale":"2024-09-16T02:01:40.610362+0000","last_undegraded":"2024-09-16T02:01:40.610362+0000","last_fullsized":"2024-09-16T02:01:40.610362+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:00:24.179172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.608361+0000","last_change":"2024-09-16T02:01:40.608361+0000","last_active":"2024-09-16T02:01:40.608361+0000","last_peered":"2024-09-16T02:01:40.608361+0000","last_clean":"2024-09-16T02:01:40.608361+0000","last_became_active":"2024-09-16T02:01:40.607954+0000","last_became_peered":"2024-09-16T02:01:40.607954+0000","last_unstale":"2024-09-16T02:01:40.608361+0000","last_undegraded":"2024-09-16T02:01:40.608361+0000","last_fullsized":"2024-09-16T02:01:40.608361+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:18:31.244969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.562228+0000","last_change":"2024-09-16T02:00:34.460680+0000","last_active":"2024-09-16T02:01:40.562228+0000","last_peered":"2024-09-16T02:01:40.562228+0000","last_clean":"2024-09-16T02:01:40.562228+0000","last_became_active":"2024-09-16T02:00:34.460161+0000","last_became_peered":"2024-09-16T02:00:34.460161+0000","last_unstale":"2024-09-16T02:01:40.562228+0000","last_undegraded":"2024-09-16T02:01:40.562228+0000","last_fullsized":"2024-09-16T02:01:40.562228+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:38:19.848420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.562137+0000","last_change":"2024-09-16T02:00:34.454606+0000","last_active":"2024-09-16T02:01:40.562137+0000","last_peered":"2024-09-16T02:01:40.562137+0000","last_clean":"2024-09-16T02:01:40.562137+0000","last_became_active":"2024-09-16T02:00:34.454293+0000","last_became_peered":"2024-09-16T02:00:34.454293+0000","last_unstale":"2024-09-16T02:01:40.562137+0000","last_undegraded":"2024-09-16T02:01:40.562137+0000","last_fullsized":"2024-09-16T02:01:40.562137+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:55:54.466176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.562971+0000","last_change":"2024-09-16T02:00:34.452940+0000","last_active":"2024-09-16T02:01:40.562971+0000","last_peered":"2024-09-16T02:01:40.562971+0000","last_clean":"2024-09-16T02:01:40.562971+0000","last_became_active":"2024-09-16T02:00:34.452560+0000","last_became_peered":"2024-09-16T02:00:34.452560+0000","last_unstale":"2024-09-16T02:01:40.562971+0000","last_undegraded":"2024-09-16T02:01:40.562971+0000","last_fullsized":"2024-09-16T02:01:40.562971+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:42:11.564264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560911+0000","last_change":"2024-09-16T02:00:34.453457+0000","last_active":"2024-09-16T02:01:40.560911+0000","last_peered":"2024-09-16T02:01:40.560911+0000","last_clean":"2024-09-16T02:01:40.560911+0000","last_became_active":"2024-09-16T02:00:34.453148+0000","last_became_peered":"2024-09-16T02:00:34.453148+0000","last_unstale":"2024-09-16T02:01:40.560911+0000","last_undegraded":"2024-09-16T02:01:40.560911+0000","last_fullsized":"2024-09-16T02:01:40.560911+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:28.456892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.607401+0000","last_change":"2024-09-16T02:00:34.457057+0000","last_active":"2024-09-16T02:01:40.607401+0000","last_peered":"2024-09-16T02:01:40.607401+0000","last_clean":"2024-09-16T02:01:40.607401+0000","last_became_active":"2024-09-16T02:00:34.456751+0000","last_became_peered":"2024-09-16T02:00:34.456751+0000","last_unstale":"2024-09-16T02:01:40.607401+0000","last_undegraded":"2024-09-16T02:01:40.607401+0000","last_fullsized":"2024-09-16T02:01:40.607401+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:32:32.207510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.561837+0000","last_change":"2024-09-16T02:00:34.448434+0000","last_active":"2024-09-16T02:01:40.561837+0000","last_peered":"2024-09-16T02:01:40.561837+0000","last_clean":"2024-09-16T02:01:40.561837+0000","last_became_active":"2024-09-16T02:00:34.448056+0000","last_became_peered":"2024-09-16T02:00:34.448056+0000","last_unstale":"2024-09-16T02:01:40.561837+0000","last_undegraded":"2024-09-16T02:01:40.561837+0000","last_fullsized":"2024-09-16T02:01:40.561837+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:15:28.162159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560368+0000","last_change":"2024-09-16T02:00:34.459499+0000","last_active":"2024-09-16T02:01:40.560368+0000","last_peered":"2024-09-16T02:01:40.560368+0000","last_clean":"2024-09-16T02:01:40.560368+0000","last_became_active":"2024-09-16T02:00:34.459017+0000","last_became_peered":"2024-09-16T02:00:34.459017+0000","last_unstale":"2024-09-16T02:01:40.560368+0000","last_undegraded":"2024-09-16T02:01:40.560368+0000","last_fullsized":"2024-09-16T02:01:40.560368+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:41:36.866100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.559296+0000","last_change":"2024-09-16T02:00:34.448640+0000","last_active":"2024-09-16T02:01:40.559296+0000","last_peered":"2024-09-16T02:01:40.559296+0000","last_clean":"2024-09-16T02:01:40.559296+0000","last_became_active":"2024-09-16T02:00:34.447737+0000","last_became_peered":"2024-09-16T02:00:34.447737+0000","last_unstale":"2024-09-16T02:01:40.559296+0000","last_undegraded":"2024-09-16T02:01:40.559296+0000","last_fullsized":"2024-09-16T02:01:40.559296+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:20.192650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.563552+0000","last_change":"2024-09-16T02:00:34.451718+0000","last_active":"2024-09-16T02:01:40.563552+0000","last_peered":"2024-09-16T02:01:40.563552+0000","last_clean":"2024-09-16T02:01:40.563552+0000","last_became_active":"2024-09-16T02:00:34.451389+0000","last_became_peered":"2024-09-16T02:00:34.451389+0000","last_unstale":"2024-09-16T02:01:40.563552+0000","last_undegraded":"2024-09-16T02:01:40.563552+0000","last_fullsized":"2024-09-16T02:01:40.563552+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:13:19.296178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.561207+0000","last_change":"2024-09-16T02:00:34.458379+0000","last_active":"2024-09-16T02:01:40.561207+0000","last_peered":"2024-09-16T02:01:40.561207+0000","last_clean":"2024-09-16T02:01:40.561207+0000","last_became_active":"2024-09-16T02:00:34.458038+0000","last_became_peered":"2024-09-16T02:00:34.458038+0000","last_unstale":"2024-09-16T02:01:40.561207+0000","last_undegraded":"2024-09-16T02:01:40.561207+0000","last_fullsized":"2024-09-16T02:01:40.561207+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:34.669470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.565143+0000","last_change":"2024-09-16T02:00:34.450927+0000","last_active":"2024-09-16T02:01:40.565143+0000","last_peered":"2024-09-16T02:01:40.565143+0000","last_clean":"2024-09-16T02:01:40.565143+0000","last_became_active":"2024-09-16T02:00:34.447669+0000","last_became_peered":"2024-09-16T02:00:34.447669+0000","last_unstale":"2024-09-16T02:01:40.565143+0000","last_undegraded":"2024-09-16T02:01:40.565143+0000","last_fullsized":"2024-09-16T02:01:40.565143+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:57.861633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.561666+0000","last_change":"2024-09-16T02:00:34.454827+0000","last_active":"2024-09-16T02:01:40.561666+0000","last_peered":"2024-09-16T02:01:40.561666+0000","last_clean":"2024-09-16T02:01:40.561666+0000","last_became_active":"2024-09-16T02:00:34.454549+0000","last_became_peered":"2024-09-16T02:00:34.454549+0000","last_unstale":"2024-09-16T02:01:40.561666+0000","last_undegraded":"2024-09-16T02:01:40.561666+0000","last_fullsized":"2024-09-16T02:01:40.561666+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:53:45.440860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.563093+0000","last_change":"2024-09-16T02:00:34.455121+0000","last_active":"2024-09-16T02:01:40.563093+0000","last_peered":"2024-09-16T02:01:40.563093+0000","last_clean":"2024-09-16T02:01:40.563093+0000","last_became_active":"2024-09-16T02:00:34.454665+0000","last_became_peered":"2024-09-16T02:00:34.454665+0000","last_unstale":"2024-09-16T02:01:40.563093+0000","last_undegraded":"2024-09-16T02:01:40.563093+0000","last_fullsized":"2024-09-16T02:01:40.563093+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:31:42.771230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560656+0000","last_change":"2024-09-16T02:00:34.458777+0000","last_active":"2024-09-16T02:01:40.560656+0000","last_peered":"2024-09-16T02:01:40.560656+0000","last_clean":"2024-09-16T02:01:40.560656+0000","last_became_active":"2024-09-16T02:00:34.457954+0000","last_became_peered":"2024-09-16T02:00:34.457954+0000","last_unstale":"2024-09-16T02:01:40.560656+0000","last_undegraded":"2024-09-16T02:01:40.560656+0000","last_fullsized":"2024-09-16T02:01:40.560656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:58:28.032293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.559354+0000","last_change":"2024-09-16T02:00:34.444712+0000","last_active":"2024-09-16T02:01:40.559354+0000","last_peered":"2024-09-16T02:01:40.559354+0000","last_clean":"2024-09-16T02:01:40.559354+0000","last_became_active":"2024-09-16T02:00:34.444421+0000","last_became_peered":"2024-09-16T02:00:34.444421+0000","last_unstale":"2024-09-16T02:01:40.559354+0000","last_undegraded":"2024-09-16T02:01:40.559354+0000","last_fullsized":"2024-09-16T02:01:40.559354+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:36.557558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.562379+0000","last_change":"2024-09-16T02:00:34.454570+0000","last_active":"2024-09-16T02:01:40.562379+0000","last_peered":"2024-09-16T02:01:40.562379+0000","last_clean":"2024-09-16T02:01:40.562379+0000","last_became_active":"2024-09-16T02:00:34.454246+0000","last_became_peered":"2024-09-16T02:00:34.454246+0000","last_unstale":"2024-09-16T02:01:40.562379+0000","last_undegraded":"2024-09-16T02:01:40.562379+0000","last_fullsized":"2024-09-16T02:01:40.562379+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:36:18.634907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.561677+0000","last_change":"2024-09-16T02:00:34.449680+0000","last_active":"2024-09-16T02:01:40.561677+0000","last_peered":"2024-09-16T02:01:40.561677+0000","last_clean":"2024-09-16T02:01:40.561677+0000","last_became_active":"2024-09-16T02:00:34.449184+0000","last_became_peered":"2024-09-16T02:00:34.449184+0000","last_unstale":"2024-09-16T02:01:40.561677+0000","last_undegraded":"2024-09-16T02:01:40.561677+0000","last_fullsized":"2024-09-16T02:01:40.561677+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:28:34.867820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.562212+0000","last_change":"2024-09-16T02:00:34.455362+0000","last_active":"2024-09-16T02:01:40.562212+0000","last_peered":"2024-09-16T02:01:40.562212+0000","last_clean":"2024-09-16T02:01:40.562212+0000","last_became_active":"2024-09-16T02:00:34.455018+0000","last_became_peered":"2024-09-16T02:00:34.455018+0000","last_unstale":"2024-09-16T02:01:40.562212+0000","last_undegraded":"2024-09-16T02:01:40.562212+0000","last_fullsized":"2024-09-16T02:01:40.562212+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:38.278418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560207+0000","last_change":"2024-09-16T02:00:34.459426+0000","last_active":"2024-09-16T02:01:40.560207+0000","last_peered":"2024-09-16T02:01:40.560207+0000","last_clean":"2024-09-16T02:01:40.560207+0000","last_became_active":"2024-09-16T02:00:34.459082+0000","last_became_peered":"2024-09-16T02:00:34.459082+0000","last_unstale":"2024-09-16T02:01:40.560207+0000","last_undegraded":"2024-09-16T02:01:40.560207+0000","last_fullsized":"2024-09-16T02:01:40.560207+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:59:42.226065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.563365+0000","last_change":"2024-09-16T02:00:34.450936+0000","last_active":"2024-09-16T02:01:40.563365+0000","last_peered":"2024-09-16T02:01:40.563365+0000","last_clean":"2024-09-16T02:01:40.563365+0000","last_became_active":"2024-09-16T02:00:34.450503+0000","last_became_peered":"2024-09-16T02:00:34.450503+0000","last_unstale":"2024-09-16T02:01:40.563365+0000","last_undegraded":"2024-09-16T02:01:40.563365+0000","last_fullsized":"2024-09-16T02:01:40.563365+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:35:55.806193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'5","reported_seq":86,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:42.610457+0000","last_change":"2024-09-16T02:00:32.443193+0000","last_active":"2024-09-16T02:01:42.610457+0000","last_peered":"2024-09-16T02:01:42.610457+0000","last_clean":"2024-09-16T02:01:42.610457+0000","last_became_active":"2024-09-16T02:00:32.442863+0000","last_became_peered":"2024-09-16T02:00:32.442863+0000","last_unstale":"2024-09-16T02:01:42.610457+0000","last_undegraded":"2024-09-16T02:01:42.610457+0000","last_fullsized":"2024-09-16T02:01:42.610457+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:19.734946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":59,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.570518+0000","last_change":"2024-09-16T02:01:40.570518+0000","last_active":"2024-09-16T02:01:40.570518+0000","last_peered":"2024-09-16T02:01:40.570518+0000","last_clean":"2024-09-16T02:01:40.570518+0000","last_became_active":"2024-09-16T02:01:40.570124+0000","last_became_peered":"2024-09-16T02:01:40.570124+0000","last_unstale":"2024-09-16T02:01:40.570518+0000","last_undegraded":"2024-09-16T02:01:40.570518+0000","last_fullsized":"2024-09-16T02:01:40.570518+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:44:21.062742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.572282+0000","last_change":"2024-09-16T02:01:40.572282+0000","last_active":"2024-09-16T02:01:40.572282+0000","last_peered":"2024-09-16T02:01:40.572282+0000","last_clean":"2024-09-16T02:01:40.572282+0000","last_became_active":"2024-09-16T02:01:40.564577+0000","last_became_peered":"2024-09-16T02:01:40.564577+0000","last_unstale":"2024-09-16T02:01:40.572282+0000","last_undegraded":"2024-09-16T02:01:40.572282+0000","last_fullsized":"2024-09-16T02:01:40.572282+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:19:40.834173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.559077+0000","last_change":"2024-09-16T02:00:32.438530+0000","last_active":"2024-09-16T02:01:40.559077+0000","last_peered":"2024-09-16T02:01:40.559077+0000","last_clean":"2024-09-16T02:01:40.559077+0000","last_became_active":"2024-09-16T02:00:32.438238+0000","last_became_peered":"2024-09-16T02:00:32.438238+0000","last_unstale":"2024-09-16T02:01:40.559077+0000","last_undegraded":"2024-09-16T02:01:40.559077+0000","last_fullsized":"2024-09-16T02:01:40.559077+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:52:32.799701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.561050+0000","last_change":"2024-09-16T02:00:34.456299+0000","last_active":"2024-09-16T02:01:40.561050+0000","last_peered":"2024-09-16T02:01:40.561050+0000","last_clean":"2024-09-16T02:01:40.561050+0000","last_became_active":"2024-09-16T02:00:34.455925+0000","last_became_peered":"2024-09-16T02:00:34.455925+0000","last_unstale":"2024-09-16T02:01:40.561050+0000","last_undegraded":"2024-09-16T02:01:40.561050+0000","last_fullsized":"2024-09-16T02:01:40.561050+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:33:27.162664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.565294+0000","last_change":"2024-09-16T02:00:32.442029+0000","last_active":"2024-09-16T02:01:40.565294+0000","last_peered":"2024-09-16T02:01:40.565294+0000","last_clean":"2024-09-16T02:01:40.565294+0000","last_became_active":"2024-09-16T02:00:32.441705+0000","last_became_peered":"2024-09-16T02:00:32.441705+0000","last_unstale":"2024-09-16T02:01:40.565294+0000","last_undegraded":"2024-09-16T02:01:40.565294+0000","last_fullsized":"2024-09-16T02:01:40.565294+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:50:54.725768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.607165+0000","last_change":"2024-09-16T02:00:34.457833+0000","last_active":"2024-09-16T02:01:40.607165+0000","last_peered":"2024-09-16T02:01:40.607165+0000","last_clean":"2024-09-16T02:01:40.607165+0000","last_became_active":"2024-09-16T02:00:34.457453+0000","last_became_peered":"2024-09-16T02:00:34.457453+0000","last_unstale":"2024-09-16T02:01:40.607165+0000","last_undegraded":"2024-09-16T02:01:40.607165+0000","last_fullsized":"2024-09-16T02:01:40.607165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:21.777574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":136,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:41.607968+0000","last_change":"2024-09-16T02:00:32.441341+0000","last_active":"2024-09-16T02:01:41.607968+0000","last_peered":"2024-09-16T02:01:41.607968+0000","last_clean":"2024-09-16T02:01:41.607968+0000","last_became_active":"2024-09-16T02:00:32.441019+0000","last_became_peered":"2024-09-16T02:00:32.441019+0000","last_unstale":"2024-09-16T02:01:41.607968+0000","last_undegraded":"2024-09-16T02:01:41.607968+0000","last_fullsized":"2024-09-16T02:01:41.607968+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:10:10.171714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":110,"num_read_kb":110,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.563417+0000","last_change":"2024-09-16T02:00:34.461746+0000","last_active":"2024-09-16T02:01:40.563417+0000","last_peered":"2024-09-16T02:01:40.563417+0000","last_clean":"2024-09-16T02:01:40.563417+0000","last_became_active":"2024-09-16T02:00:34.461441+0000","last_became_peered":"2024-09-16T02:00:34.461441+0000","last_unstale":"2024-09-16T02:01:40.563417+0000","last_undegraded":"2024-09-16T02:01:40.563417+0000","last_fullsized":"2024-09-16T02:01:40.563417+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:28:14.519159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564386+0000","last_change":"2024-09-16T02:00:32.443129+0000","last_active":"2024-09-16T02:01:40.564386+0000","last_peered":"2024-09-16T02:01:40.564386+0000","last_clean":"2024-09-16T02:01:40.564386+0000","last_became_active":"2024-09-16T02:00:32.442772+0000","last_became_peered":"2024-09-16T02:00:32.442772+0000","last_unstale":"2024-09-16T02:01:40.564386+0000","last_undegraded":"2024-09-16T02:01:40.564386+0000","last_fullsized":"2024-09-16T02:01:40.564386+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:10:03.964583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.562614+0000","last_change":"2024-09-16T02:00:34.457449+0000","last_active":"2024-09-16T02:01:40.562614+0000","last_peered":"2024-09-16T02:01:40.562614+0000","last_clean":"2024-09-16T02:01:40.562614+0000","last_became_active":"2024-09-16T02:00:34.457110+0000","last_became_peered":"2024-09-16T02:00:34.457110+0000","last_unstale":"2024-09-16T02:01:40.562614+0000","last_undegraded":"2024-09-16T02:01:40.562614+0000","last_fullsized":"2024-09-16T02:01:40.562614+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:35.662503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.565563+0000","last_change":"2024-09-16T02:00:32.587419+0000","last_active":"2024-09-16T02:01:40.565563+0000","last_peered":"2024-09-16T02:01:40.565563+0000","last_clean":"2024-09-16T02:01:40.565563+0000","last_became_active":"2024-09-16T02:00:32.587037+0000","last_became_peered":"2024-09-16T02:00:32.587037+0000","last_unstale":"2024-09-16T02:01:40.565563+0000","last_undegraded":"2024-09-16T02:01:40.565563+0000","last_fullsized":"2024-09-16T02:01:40.565563+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:11.835505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.563152+0000","last_change":"2024-09-16T02:00:34.456371+0000","last_active":"2024-09-16T02:01:40.563152+0000","last_peered":"2024-09-16T02:01:40.563152+0000","last_clean":"2024-09-16T02:01:40.563152+0000","last_became_active":"2024-09-16T02:00:34.456013+0000","last_became_peered":"2024-09-16T02:00:34.456013+0000","last_unstale":"2024-09-16T02:01:40.563152+0000","last_undegraded":"2024-09-16T02:01:40.563152+0000","last_fullsized":"2024-09-16T02:01:40.563152+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:30:15.847189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'28","reported_seq":168,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:41.607971+0000","last_change":"2024-09-16T02:00:32.444922+0000","last_active":"2024-09-16T02:01:41.607971+0000","last_peered":"2024-09-16T02:01:41.607971+0000","last_clean":"2024-09-16T02:01:41.607971+0000","last_became_active":"2024-09-16T02:00:32.444540+0000","last_became_peered":"2024-09-16T02:00:32.444540+0000","last_unstale":"2024-09-16T02:01:41.607971+0000","last_undegraded":"2024-09-16T02:01:41.607971+0000","last_fullsized":"2024-09-16T02:01:41.607971+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:59.183952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59211,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":118,"num_read_kb":116,"num_write":30,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564383+0000","last_change":"2024-09-16T02:00:34.453366+0000","last_active":"2024-09-16T02:01:40.564383+0000","last_peered":"2024-09-16T02:01:40.564383+0000","last_clean":"2024-09-16T02:01:40.564383+0000","last_became_active":"2024-09-16T02:00:34.453048+0000","last_became_peered":"2024-09-16T02:00:34.453048+0000","last_unstale":"2024-09-16T02:01:40.564383+0000","last_undegraded":"2024-09-16T02:01:40.564383+0000","last_fullsized":"2024-09-16T02:01:40.564383+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:15:52.721893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.562558+0000","last_change":"2024-09-16T02:00:32.445516+0000","last_active":"2024-09-16T02:01:40.562558+0000","last_peered":"2024-09-16T02:01:40.562558+0000","last_clean":"2024-09-16T02:01:40.562558+0000","last_became_active":"2024-09-16T02:00:32.445092+0000","last_became_peered":"2024-09-16T02:00:32.445092+0000","last_unstale":"2024-09-16T02:01:40.562558+0000","last_undegraded":"2024-09-16T02:01:40.562558+0000","last_fullsized":"2024-09-16T02:01:40.562558+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:54.274692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.559134+0000","last_change":"2024-09-16T02:00:34.448450+0000","last_active":"2024-09-16T02:01:40.559134+0000","last_peered":"2024-09-16T02:01:40.559134+0000","last_clean":"2024-09-16T02:01:40.559134+0000","last_became_active":"2024-09-16T02:00:34.448116+0000","last_became_peered":"2024-09-16T02:00:34.448116+0000","last_unstale":"2024-09-16T02:01:40.559134+0000","last_undegraded":"2024-09-16T02:01:40.559134+0000","last_fullsized":"2024-09-16T02:01:40.559134+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:12:34.834798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.565309+0000","last_change":"2024-09-16T02:00:32.445427+0000","last_active":"2024-09-16T02:01:40.565309+0000","last_peered":"2024-09-16T02:01:40.565309+0000","last_clean":"2024-09-16T02:01:40.565309+0000","last_became_active":"2024-09-16T02:00:32.445058+0000","last_became_peered":"2024-09-16T02:00:32.445058+0000","last_unstale":"2024-09-16T02:01:40.565309+0000","last_undegraded":"2024-09-16T02:01:40.565309+0000","last_fullsized":"2024-09-16T02:01:40.565309+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:35:34.701331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564239+0000","last_change":"2024-09-16T02:00:34.458479+0000","last_active":"2024-09-16T02:01:40.564239+0000","last_peered":"2024-09-16T02:01:40.564239+0000","last_clean":"2024-09-16T02:01:40.564239+0000","last_became_active":"2024-09-16T02:00:34.458135+0000","last_became_peered":"2024-09-16T02:00:34.458135+0000","last_unstale":"2024-09-16T02:01:40.564239+0000","last_undegraded":"2024-09-16T02:01:40.564239+0000","last_fullsized":"2024-09-16T02:01:40.564239+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:37:40.188861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'6","reported_seq":88,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:42.613273+0000","last_change":"2024-09-16T02:00:32.440319+0000","last_active":"2024-09-16T02:01:42.613273+0000","last_peered":"2024-09-16T02:01:42.613273+0000","last_clean":"2024-09-16T02:01:42.613273+0000","last_became_active":"2024-09-16T02:00:32.440005+0000","last_became_peered":"2024-09-16T02:00:32.440005+0000","last_unstale":"2024-09-16T02:01:42.613273+0000","last_undegraded":"2024-09-16T02:01:42.613273+0000","last_fullsized":"2024-09-16T02:01:42.613273+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:07:33.876577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":61,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.563949+0000","last_change":"2024-09-16T02:00:34.460468+0000","last_active":"2024-09-16T02:01:40.563949+0000","last_peered":"2024-09-16T02:01:40.563949+0000","last_clean":"2024-09-16T02:01:40.563949+0000","last_became_active":"2024-09-16T02:00:34.459190+0000","last_became_peered":"2024-09-16T02:00:34.459190+0000","last_unstale":"2024-09-16T02:01:40.563949+0000","last_undegraded":"2024-09-16T02:01:40.563949+0000","last_fullsized":"2024-09-16T02:01:40.563949+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:46:43.654991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.562649+0000","last_change":"2024-09-16T02:00:32.587700+0000","last_active":"2024-09-16T02:01:40.562649+0000","last_peered":"2024-09-16T02:01:40.562649+0000","last_clean":"2024-09-16T02:01:40.562649+0000","last_became_active":"2024-09-16T02:00:32.586964+0000","last_became_peered":"2024-09-16T02:00:32.586964+0000","last_unstale":"2024-09-16T02:01:40.562649+0000","last_undegraded":"2024-09-16T02:01:40.562649+0000","last_fullsized":"2024-09-16T02:01:40.562649+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:41.309918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.565038+0000","last_change":"2024-09-16T02:00:34.455652+0000","last_active":"2024-09-16T02:01:40.565038+0000","last_peered":"2024-09-16T02:01:40.565038+0000","last_clean":"2024-09-16T02:01:40.565038+0000","last_became_active":"2024-09-16T02:00:34.455306+0000","last_became_peered":"2024-09-16T02:00:34.455306+0000","last_unstale":"2024-09-16T02:01:40.565038+0000","last_undegraded":"2024-09-16T02:01:40.565038+0000","last_fullsized":"2024-09-16T02:01:40.565038+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:44:09.505096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564111+0000","last_change":"2024-09-16T02:00:32.441443+0000","last_active":"2024-09-16T02:01:40.564111+0000","last_peered":"2024-09-16T02:01:40.564111+0000","last_clean":"2024-09-16T02:01:40.564111+0000","last_became_active":"2024-09-16T02:00:32.441073+0000","last_became_peered":"2024-09-16T02:00:32.441073+0000","last_unstale":"2024-09-16T02:01:40.564111+0000","last_undegraded":"2024-09-16T02:01:40.564111+0000","last_fullsized":"2024-09-16T02:01:40.564111+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:16.865970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.611317+0000","last_change":"2024-09-16T02:01:40.611317+0000","last_active":"2024-09-16T02:01:40.611317+0000","last_peered":"2024-09-16T02:01:40.611317+0000","last_clean":"2024-09-16T02:01:40.611317+0000","last_became_active":"2024-09-16T02:01:40.610837+0000","last_became_peered":"2024-09-16T02:01:40.610837+0000","last_unstale":"2024-09-16T02:01:40.611317+0000","last_undegraded":"2024-09-16T02:01:40.611317+0000","last_fullsized":"2024-09-16T02:01:40.611317+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:19:55.376738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.617877+0000","last_change":"2024-09-16T02:01:40.617877+0000","last_active":"2024-09-16T02:01:40.617877+0000","last_peered":"2024-09-16T02:01:40.617877+0000","last_clean":"2024-09-16T02:01:40.617877+0000","last_became_active":"2024-09-16T02:01:40.611597+0000","last_became_peered":"2024-09-16T02:01:40.611597+0000","last_unstale":"2024-09-16T02:01:40.617877+0000","last_undegraded":"2024-09-16T02:01:40.617877+0000","last_fullsized":"2024-09-16T02:01:40.617877+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:53:35.796243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564383+0000","last_change":"2024-09-16T02:00:34.461889+0000","last_active":"2024-09-16T02:01:40.564383+0000","last_peered":"2024-09-16T02:01:40.564383+0000","last_clean":"2024-09-16T02:01:40.564383+0000","last_became_active":"2024-09-16T02:00:34.461583+0000","last_became_peered":"2024-09-16T02:00:34.461583+0000","last_unstale":"2024-09-16T02:01:40.564383+0000","last_undegraded":"2024-09-16T02:01:40.564383+0000","last_fullsized":"2024-09-16T02:01:40.564383+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:32.598422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.565617+0000","last_change":"2024-09-16T02:00:32.445201+0000","last_active":"2024-09-16T02:01:40.565617+0000","last_peered":"2024-09-16T02:01:40.565617+0000","last_clean":"2024-09-16T02:01:40.565617+0000","last_became_active":"2024-09-16T02:00:32.444722+0000","last_became_peered":"2024-09-16T02:00:32.444722+0000","last_unstale":"2024-09-16T02:01:40.565617+0000","last_undegraded":"2024-09-16T02:01:40.565617+0000","last_fullsized":"2024-09-16T02:01:40.565617+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:41:11.657028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'1","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.613676+0000","last_change":"2024-09-16T02:01:40.613676+0000","last_active":"2024-09-16T02:01:40.613676+0000","last_peered":"2024-09-16T02:01:40.613676+0000","last_clean":"2024-09-16T02:01:40.613676+0000","last_became_active":"2024-09-16T02:01:40.608260+0000","last_became_peered":"2024-09-16T02:01:40.608260+0000","last_unstale":"2024-09-16T02:01:40.613676+0000","last_undegraded":"2024-09-16T02:01:40.613676+0000","last_fullsized":"2024-09-16T02:01:40.613676+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:30:59.004103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'5","reported_seq":86,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:42.610441+0000","last_change":"2024-09-16T02:00:32.860059+0000","last_active":"2024-09-16T02:01:42.610441+0000","last_peered":"2024-09-16T02:01:42.610441+0000","last_clean":"2024-09-16T02:01:42.610441+0000","last_became_active":"2024-09-16T02:00:32.859617+0000","last_became_peered":"2024-09-16T02:00:32.859617+0000","last_unstale":"2024-09-16T02:01:42.610441+0000","last_undegraded":"2024-09-16T02:01:42.610441+0000","last_fullsized":"2024-09-16T02:01:42.610441+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:41:50.248194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":59,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564754+0000","last_change":"2024-09-16T02:00:34.454344+0000","last_active":"2024-09-16T02:01:40.564754+0000","last_peered":"2024-09-16T02:01:40.564754+0000","last_clean":"2024-09-16T02:01:40.564754+0000","last_became_active":"2024-09-16T02:00:34.454027+0000","last_became_peered":"2024-09-16T02:00:34.454027+0000","last_unstale":"2024-09-16T02:01:40.564754+0000","last_undegraded":"2024-09-16T02:01:40.564754+0000","last_fullsized":"2024-09-16T02:01:40.564754+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:33.020365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":334,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564354+0000","last_change":"2024-09-16T01:59:19.798159+0000","last_active":"2024-09-16T02:01:40.564354+0000","last_peered":"2024-09-16T02:01:40.564354+0000","last_clean":"2024-09-16T02:01:40.564354+0000","last_became_active":"2024-09-16T01:59:19.797688+0000","last_became_peered":"2024-09-16T01:59:19.797688+0000","last_unstale":"2024-09-16T02:01:40.564354+0000","last_undegraded":"2024-09-16T02:01:40.564354+0000","last_fullsized":"2024-09-16T02:01:40.564354+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:55:54.312750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:55:54.312750+0000","last_clean_scrub_stamp":"2024-09-16T01:55:54.312750+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:00:36.860158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.610108+0000","last_change":"2024-09-16T02:00:32.441925+0000","last_active":"2024-09-16T02:01:40.610108+0000","last_peered":"2024-09-16T02:01:40.610108+0000","last_clean":"2024-09-16T02:01:40.610108+0000","last_became_active":"2024-09-16T02:00:32.441565+0000","last_became_peered":"2024-09-16T02:00:32.441565+0000","last_unstale":"2024-09-16T02:01:40.610108+0000","last_undegraded":"2024-09-16T02:01:40.610108+0000","last_fullsized":"2024-09-16T02:01:40.610108+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:23:09.755034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.561404+0000","last_change":"2024-09-16T02:00:34.452986+0000","last_active":"2024-09-16T02:01:40.561404+0000","last_peered":"2024-09-16T02:01:40.561404+0000","last_clean":"2024-09-16T02:01:40.561404+0000","last_became_active":"2024-09-16T02:00:34.452688+0000","last_became_peered":"2024-09-16T02:00:34.452688+0000","last_unstale":"2024-09-16T02:01:40.561404+0000","last_undegraded":"2024-09-16T02:01:40.561404+0000","last_fullsized":"2024-09-16T02:01:40.561404+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:12.958209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564205+0000","last_change":"2024-09-16T02:00:32.443964+0000","last_active":"2024-09-16T02:01:40.564205+0000","last_peered":"2024-09-16T02:01:40.564205+0000","last_clean":"2024-09-16T02:01:40.564205+0000","last_became_active":"2024-09-16T02:00:32.443623+0000","last_became_peered":"2024-09-16T02:00:32.443623+0000","last_unstale":"2024-09-16T02:01:40.564205+0000","last_undegraded":"2024-09-16T02:01:40.564205+0000","last_fullsized":"2024-09-16T02:01:40.564205+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:30:34.406024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.559065+0000","last_change":"2024-09-16T02:00:34.449615+0000","last_active":"2024-09-16T02:01:40.559065+0000","last_peered":"2024-09-16T02:01:40.559065+0000","last_clean":"2024-09-16T02:01:40.559065+0000","last_became_active":"2024-09-16T02:00:34.449109+0000","last_became_peered":"2024-09-16T02:00:34.449109+0000","last_unstale":"2024-09-16T02:01:40.559065+0000","last_undegraded":"2024-09-16T02:01:40.559065+0000","last_fullsized":"2024-09-16T02:01:40.559065+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:45.036177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":85,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:42.612676+0000","last_change":"2024-09-16T02:00:32.587107+0000","last_active":"2024-09-16T02:01:42.612676+0000","last_peered":"2024-09-16T02:01:42.612676+0000","last_clean":"2024-09-16T02:01:42.612676+0000","last_became_active":"2024-09-16T02:00:32.586777+0000","last_became_peered":"2024-09-16T02:00:32.586777+0000","last_unstale":"2024-09-16T02:01:42.612676+0000","last_undegraded":"2024-09-16T02:01:42.612676+0000","last_fullsized":"2024-09-16T02:01:42.612676+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:03.968987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":58,"num_read_kb":58,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.570572+0000","last_change":"2024-09-16T02:01:40.570572+0000","last_active":"2024-09-16T02:01:40.570572+0000","last_peered":"2024-09-16T02:01:40.570572+0000","last_clean":"2024-09-16T02:01:40.570572+0000","last_became_active":"2024-09-16T02:01:40.570176+0000","last_became_peered":"2024-09-16T02:01:40.570176+0000","last_unstale":"2024-09-16T02:01:40.570572+0000","last_undegraded":"2024-09-16T02:01:40.570572+0000","last_fullsized":"2024-09-16T02:01:40.570572+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:24:13.138925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'5","reported_seq":86,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:43.611810+0000","last_change":"2024-09-16T02:00:32.443201+0000","last_active":"2024-09-16T02:01:43.611810+0000","last_peered":"2024-09-16T02:01:43.611810+0000","last_clean":"2024-09-16T02:01:43.611810+0000","last_became_active":"2024-09-16T02:00:32.442778+0000","last_became_peered":"2024-09-16T02:00:32.442778+0000","last_unstale":"2024-09-16T02:01:43.611810+0000","last_undegraded":"2024-09-16T02:01:43.611810+0000","last_fullsized":"2024-09-16T02:01:43.611810+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:43.933817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":59,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560410+0000","last_change":"2024-09-16T02:00:34.463000+0000","last_active":"2024-09-16T02:01:40.560410+0000","last_peered":"2024-09-16T02:01:40.560410+0000","last_clean":"2024-09-16T02:01:40.560410+0000","last_became_active":"2024-09-16T02:00:34.461890+0000","last_became_peered":"2024-09-16T02:00:34.461890+0000","last_unstale":"2024-09-16T02:01:40.560410+0000","last_undegraded":"2024-09-16T02:01:40.560410+0000","last_fullsized":"2024-09-16T02:01:40.560410+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:21:15.161295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.607447+0000","last_change":"2024-09-16T02:00:32.586933+0000","last_active":"2024-09-16T02:01:40.607447+0000","last_peered":"2024-09-16T02:01:40.607447+0000","last_clean":"2024-09-16T02:01:40.607447+0000","last_became_active":"2024-09-16T02:00:32.586565+0000","last_became_peered":"2024-09-16T02:00:32.586565+0000","last_unstale":"2024-09-16T02:01:40.607447+0000","last_undegraded":"2024-09-16T02:01:40.607447+0000","last_fullsized":"2024-09-16T02:01:40.607447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:45:07.130649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560456+0000","last_change":"2024-09-16T02:00:34.460805+0000","last_active":"2024-09-16T02:01:40.560456+0000","last_peered":"2024-09-16T02:01:40.560456+0000","last_clean":"2024-09-16T02:01:40.560456+0000","last_became_active":"2024-09-16T02:00:34.460482+0000","last_became_peered":"2024-09-16T02:00:34.460482+0000","last_unstale":"2024-09-16T02:01:40.560456+0000","last_undegraded":"2024-09-16T02:01:40.560456+0000","last_fullsized":"2024-09-16T02:01:40.560456+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:07:51.565053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564575+0000","last_change":"2024-09-16T02:00:32.587516+0000","last_active":"2024-09-16T02:01:40.564575+0000","last_peered":"2024-09-16T02:01:40.564575+0000","last_clean":"2024-09-16T02:01:40.564575+0000","last_became_active":"2024-09-16T02:00:32.587172+0000","last_became_peered":"2024-09-16T02:00:32.587172+0000","last_unstale":"2024-09-16T02:01:40.564575+0000","last_undegraded":"2024-09-16T02:01:40.564575+0000","last_fullsized":"2024-09-16T02:01:40.564575+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:52:29.018290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.559443+0000","last_change":"2024-09-16T02:00:34.446021+0000","last_active":"2024-09-16T02:01:40.559443+0000","last_peered":"2024-09-16T02:01:40.559443+0000","last_clean":"2024-09-16T02:01:40.559443+0000","last_became_active":"2024-09-16T02:00:34.445732+0000","last_became_peered":"2024-09-16T02:00:34.445732+0000","last_unstale":"2024-09-16T02:01:40.559443+0000","last_undegraded":"2024-09-16T02:01:40.559443+0000","last_fullsized":"2024-09-16T02:01:40.559443+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:51:17.960792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'14","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.607435+0000","last_change":"2024-09-16T02:00:32.439750+0000","last_active":"2024-09-16T02:01:40.607435+0000","last_peered":"2024-09-16T02:01:40.607435+0000","last_clean":"2024-09-16T02:01:40.607435+0000","last_became_active":"2024-09-16T02:00:32.439438+0000","last_became_peered":"2024-09-16T02:00:32.439438+0000","last_unstale":"2024-09-16T02:01:40.607435+0000","last_undegraded":"2024-09-16T02:01:40.607435+0000","last_fullsized":"2024-09-16T02:01:40.607435+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:50.764733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.561712+0000","last_change":"2024-09-16T02:00:34.453810+0000","last_active":"2024-09-16T02:01:40.561712+0000","last_peered":"2024-09-16T02:01:40.561712+0000","last_clean":"2024-09-16T02:01:40.561712+0000","last_became_active":"2024-09-16T02:00:34.453421+0000","last_became_peered":"2024-09-16T02:00:34.453421+0000","last_unstale":"2024-09-16T02:01:40.561712+0000","last_undegraded":"2024-09-16T02:01:40.561712+0000","last_fullsized":"2024-09-16T02:01:40.561712+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:23:50.405749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.610378+0000","last_change":"2024-09-16T02:00:32.860565+0000","last_active":"2024-09-16T02:01:40.610378+0000","last_peered":"2024-09-16T02:01:40.610378+0000","last_clean":"2024-09-16T02:01:40.610378+0000","last_became_active":"2024-09-16T02:00:32.860130+0000","last_became_peered":"2024-09-16T02:00:32.860130+0000","last_unstale":"2024-09-16T02:01:40.610378+0000","last_undegraded":"2024-09-16T02:01:40.610378+0000","last_fullsized":"2024-09-16T02:01:40.610378+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:09:55.759883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564454+0000","last_change":"2024-09-16T02:01:40.564454+0000","last_active":"2024-09-16T02:01:40.564454+0000","last_peered":"2024-09-16T02:01:40.564454+0000","last_clean":"2024-09-16T02:01:40.564454+0000","last_became_active":"2024-09-16T02:01:40.563854+0000","last_became_peered":"2024-09-16T02:01:40.563854+0000","last_unstale":"2024-09-16T02:01:40.564454+0000","last_undegraded":"2024-09-16T02:01:40.564454+0000","last_fullsized":"2024-09-16T02:01:40.564454+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:56:30.610947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.563115+0000","last_change":"2024-09-16T02:00:32.587115+0000","last_active":"2024-09-16T02:01:40.563115+0000","last_peered":"2024-09-16T02:01:40.563115+0000","last_clean":"2024-09-16T02:01:40.563115+0000","last_became_active":"2024-09-16T02:00:32.586835+0000","last_became_peered":"2024-09-16T02:00:32.586835+0000","last_unstale":"2024-09-16T02:01:40.563115+0000","last_undegraded":"2024-09-16T02:01:40.563115+0000","last_fullsized":"2024-09-16T02:01:40.563115+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:12:58.820005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.562587+0000","last_change":"2024-09-16T02:00:34.457673+0000","last_active":"2024-09-16T02:01:40.562587+0000","last_peered":"2024-09-16T02:01:40.562587+0000","last_clean":"2024-09-16T02:01:40.562587+0000","last_became_active":"2024-09-16T02:00:34.457331+0000","last_became_peered":"2024-09-16T02:00:34.457331+0000","last_unstale":"2024-09-16T02:01:40.562587+0000","last_undegraded":"2024-09-16T02:01:40.562587+0000","last_fullsized":"2024-09-16T02:01:40.562587+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:12:16.100036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.561350+0000","last_change":"2024-09-16T02:00:32.437528+0000","last_active":"2024-09-16T02:01:40.561350+0000","last_peered":"2024-09-16T02:01:40.561350+0000","last_clean":"2024-09-16T02:01:40.561350+0000","last_became_active":"2024-09-16T02:00:32.437080+0000","last_became_peered":"2024-09-16T02:00:32.437080+0000","last_unstale":"2024-09-16T02:01:40.561350+0000","last_undegraded":"2024-09-16T02:01:40.561350+0000","last_fullsized":"2024-09-16T02:01:40.561350+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:59:33.908026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560289+0000","last_change":"2024-09-16T02:00:34.453504+0000","last_active":"2024-09-16T02:01:40.560289+0000","last_peered":"2024-09-16T02:01:40.560289+0000","last_clean":"2024-09-16T02:01:40.560289+0000","last_became_active":"2024-09-16T02:00:34.453212+0000","last_became_peered":"2024-09-16T02:00:34.453212+0000","last_unstale":"2024-09-16T02:01:40.560289+0000","last_undegraded":"2024-09-16T02:01:40.560289+0000","last_fullsized":"2024-09-16T02:01:40.560289+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:00:56.414548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.562417+0000","last_change":"2024-09-16T02:00:32.442282+0000","last_active":"2024-09-16T02:01:40.562417+0000","last_peered":"2024-09-16T02:01:40.562417+0000","last_clean":"2024-09-16T02:01:40.562417+0000","last_became_active":"2024-09-16T02:00:32.441910+0000","last_became_peered":"2024-09-16T02:00:32.441910+0000","last_unstale":"2024-09-16T02:01:40.562417+0000","last_undegraded":"2024-09-16T02:01:40.562417+0000","last_fullsized":"2024-09-16T02:01:40.562417+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:01:46.045021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.561467+0000","last_change":"2024-09-16T02:00:34.460177+0000","last_active":"2024-09-16T02:01:40.561467+0000","last_peered":"2024-09-16T02:01:40.561467+0000","last_clean":"2024-09-16T02:01:40.561467+0000","last_became_active":"2024-09-16T02:00:34.459874+0000","last_became_peered":"2024-09-16T02:00:34.459874+0000","last_unstale":"2024-09-16T02:01:40.561467+0000","last_undegraded":"2024-09-16T02:01:40.561467+0000","last_fullsized":"2024-09-16T02:01:40.561467+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:35.855591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.614997+0000","last_change":"2024-09-16T02:01:40.614997+0000","last_active":"2024-09-16T02:01:40.614997+0000","last_peered":"2024-09-16T02:01:40.614997+0000","last_clean":"2024-09-16T02:01:40.614997+0000","last_became_active":"2024-09-16T02:01:40.610889+0000","last_became_peered":"2024-09-16T02:01:40.610889+0000","last_unstale":"2024-09-16T02:01:40.614997+0000","last_undegraded":"2024-09-16T02:01:40.614997+0000","last_fullsized":"2024-09-16T02:01:40.614997+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:39:00.989594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564262+0000","last_change":"2024-09-16T02:00:34.452964+0000","last_active":"2024-09-16T02:01:40.564262+0000","last_peered":"2024-09-16T02:01:40.564262+0000","last_clean":"2024-09-16T02:01:40.564262+0000","last_became_active":"2024-09-16T02:00:34.452617+0000","last_became_peered":"2024-09-16T02:00:34.452617+0000","last_unstale":"2024-09-16T02:01:40.564262+0000","last_undegraded":"2024-09-16T02:01:40.564262+0000","last_fullsized":"2024-09-16T02:01:40.564262+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:04.617970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.565506+0000","last_change":"2024-09-16T02:00:34.456585+0000","last_active":"2024-09-16T02:01:40.565506+0000","last_peered":"2024-09-16T02:01:40.565506+0000","last_clean":"2024-09-16T02:01:40.565506+0000","last_became_active":"2024-09-16T02:00:34.456199+0000","last_became_peered":"2024-09-16T02:00:34.456199+0000","last_unstale":"2024-09-16T02:01:40.565506+0000","last_undegraded":"2024-09-16T02:01:40.565506+0000","last_fullsized":"2024-09-16T02:01:40.565506+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:37:54.376785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.579320+0000","last_change":"2024-09-16T02:01:40.578904+0000","last_active":"2024-09-16T02:01:40.579320+0000","last_peered":"2024-09-16T02:01:40.579320+0000","last_clean":"2024-09-16T02:01:40.579320+0000","last_became_active":"2024-09-16T02:01:40.567221+0000","last_became_peered":"2024-09-16T02:01:40.567221+0000","last_unstale":"2024-09-16T02:01:40.579320+0000","last_undegraded":"2024-09-16T02:01:40.579320+0000","last_fullsized":"2024-09-16T02:01:40.579320+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:01.407145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.563083+0000","last_change":"2024-09-16T02:00:34.460925+0000","last_active":"2024-09-16T02:01:40.563083+0000","last_peered":"2024-09-16T02:01:40.563083+0000","last_clean":"2024-09-16T02:01:40.563083+0000","last_became_active":"2024-09-16T02:00:34.460581+0000","last_became_peered":"2024-09-16T02:00:34.460581+0000","last_unstale":"2024-09-16T02:01:40.563083+0000","last_undegraded":"2024-09-16T02:01:40.563083+0000","last_fullsized":"2024-09-16T02:01:40.563083+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:19:44.460695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564476+0000","last_change":"2024-09-16T02:00:32.443877+0000","last_active":"2024-09-16T02:01:40.564476+0000","last_peered":"2024-09-16T02:01:40.564476+0000","last_clean":"2024-09-16T02:01:40.564476+0000","last_became_active":"2024-09-16T02:00:32.443416+0000","last_became_peered":"2024-09-16T02:00:32.443416+0000","last_unstale":"2024-09-16T02:01:40.564476+0000","last_undegraded":"2024-09-16T02:01:40.564476+0000","last_fullsized":"2024-09-16T02:01:40.564476+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:36:57.367232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.562609+0000","last_change":"2024-09-16T02:00:34.453048+0000","last_active":"2024-09-16T02:01:40.562609+0000","last_peered":"2024-09-16T02:01:40.562609+0000","last_clean":"2024-09-16T02:01:40.562609+0000","last_became_active":"2024-09-16T02:00:34.452752+0000","last_became_peered":"2024-09-16T02:00:34.452752+0000","last_unstale":"2024-09-16T02:01:40.562609+0000","last_undegraded":"2024-09-16T02:01:40.562609+0000","last_fullsized":"2024-09-16T02:01:40.562609+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:55.970195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.561951+0000","last_change":"2024-09-16T02:00:32.586974+0000","last_active":"2024-09-16T02:01:40.561951+0000","last_peered":"2024-09-16T02:01:40.561951+0000","last_clean":"2024-09-16T02:01:40.561951+0000","last_became_active":"2024-09-16T02:00:32.586644+0000","last_became_peered":"2024-09-16T02:00:32.586644+0000","last_unstale":"2024-09-16T02:01:40.561951+0000","last_undegraded":"2024-09-16T02:01:40.561951+0000","last_fullsized":"2024-09-16T02:01:40.561951+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:52.913287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.561774+0000","last_change":"2024-09-16T02:00:34.447548+0000","last_active":"2024-09-16T02:01:40.561774+0000","last_peered":"2024-09-16T02:01:40.561774+0000","last_clean":"2024-09-16T02:01:40.561774+0000","last_became_active":"2024-09-16T02:00:34.447207+0000","last_became_peered":"2024-09-16T02:00:34.447207+0000","last_unstale":"2024-09-16T02:01:40.561774+0000","last_undegraded":"2024-09-16T02:01:40.561774+0000","last_fullsized":"2024-09-16T02:01:40.561774+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:22:02.494343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":95,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:44.620170+0000","last_change":"2024-09-16T02:00:32.441129+0000","last_active":"2024-09-16T02:01:44.620170+0000","last_peered":"2024-09-16T02:01:44.620170+0000","last_clean":"2024-09-16T02:01:44.620170+0000","last_became_active":"2024-09-16T02:00:32.440843+0000","last_became_peered":"2024-09-16T02:00:32.440843+0000","last_unstale":"2024-09-16T02:01:44.620170+0000","last_undegraded":"2024-09-16T02:01:44.620170+0000","last_fullsized":"2024-09-16T02:01:44.620170+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:17.721740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":60,"num_read_kb":60,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66026,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":603,"num_read_kb":372,"num_write":238,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":198078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281576,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612636,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":892928,"data_stored":432750,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841569,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1044480,"data_stored":1082801,"data_compressed":41575,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005094,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":917504,"data_stored":432215,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234027,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":431894,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462960,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":962560,"data_stored":1024996,"data_compressed":41575,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626486,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":897024,"data_stored":431815,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855418,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":974848,"data_stored":1024281,"data_compressed":41575,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018943,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":897024,"data_stored":432069,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247877,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":909312,"data_stored":431523,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444106,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":978944,"data_stored":492159,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":958464,"data_stored":491939,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":917504,"data_stored":433154,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T02:01:49.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:49 smithi137 ceph-mon[34099]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:01:49.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:49 smithi137 ceph-mon[34099]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s; 70 B/s, 1 objects/s recovering 2024-09-16T02:01:49.980 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:49 smithi190 ceph-mon[34762]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:01:49.980 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:49 smithi190 ceph-mon[34762]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s; 70 B/s, 1 objects/s recovering 2024-09-16T02:01:50.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:49 smithi105 ceph-mon[31197]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:01:50.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:49 smithi105 ceph-mon[31197]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s; 70 B/s, 1 objects/s recovering 2024-09-16T02:01:52.707 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T02:01:52.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:52 smithi105 ceph-mon[31197]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 64 B/s, 1 objects/s recovering 2024-09-16T02:01:52.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:52 smithi190 ceph-mon[34762]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 64 B/s, 1 objects/s recovering 2024-09-16T02:01:52.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:52 smithi137 ceph-mon[34099]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 64 B/s, 1 objects/s recovering 2024-09-16T02:01:52.969 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:53.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.481+0000 7effd3db1640 1 -- 172.21.15.105:0/2844402413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effcc102ef0 msgr2=0x7effcc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:53.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.481+0000 7effd3db1640 1 --2- 172.21.15.105:0/2844402413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effcc102ef0 0x7effcc10da40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7effc4009900 tx=0x7effc402a480 comp rx=0 tx=0).stop 2024-09-16T02:01:53.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.481+0000 7effd3db1640 1 -- 172.21.15.105:0/2844402413 shutdown_connections 2024-09-16T02:01:53.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.481+0000 7effd3db1640 1 --2- 172.21.15.105:0/2844402413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effcc102570 0x7effcc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:53.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.481+0000 7effd3db1640 1 --2- 172.21.15.105:0/2844402413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effcc102ef0 0x7effcc10da40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:53.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.481+0000 7effd3db1640 1 --2- 172.21.15.105:0/2844402413 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7effcc100e20 0x7effcc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:53.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.481+0000 7effd3db1640 1 -- 172.21.15.105:0/2844402413 >> 172.21.15.105:0/2844402413 conn(0x7effcc0fc7d0 msgr2=0x7effcc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:53.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.482+0000 7effd3db1640 1 -- 172.21.15.105:0/2844402413 shutdown_connections 2024-09-16T02:01:53.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.482+0000 7effd3db1640 1 -- 172.21.15.105:0/2844402413 wait complete. 2024-09-16T02:01:53.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.482+0000 7effd3db1640 1 Processor -- start 2024-09-16T02:01:53.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.482+0000 7effd3db1640 1 -- start start 2024-09-16T02:01:53.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.482+0000 7effd3db1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7effcc100e20 0x7effcc19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:53.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.482+0000 7effd3db1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effcc102570 0x7effcc19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:53.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.482+0000 7effd3db1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effcc102ef0 0x7effcc19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:53.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.482+0000 7effd3db1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7effcc110e70 con 0x7effcc102570 2024-09-16T02:01:53.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.482+0000 7effd3db1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7effcc110cf0 con 0x7effcc100e20 2024-09-16T02:01:53.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.482+0000 7effd3db1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7effcc110ff0 con 0x7effcc102ef0 2024-09-16T02:01:53.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.483+0000 7effd134c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effcc102570 0x7effcc19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:53.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.483+0000 7effd134c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effcc102570 0x7effcc19ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54806/0 (socket says 172.21.15.105:54806) 2024-09-16T02:01:53.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.483+0000 7effd134c640 1 -- 172.21.15.105:0/3516525552 learned_addr learned my addr 172.21.15.105:0/3516525552 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:01:53.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.483+0000 7effd134c640 1 -- 172.21.15.105:0/3516525552 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effcc102ef0 msgr2=0x7effcc19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:53.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.483+0000 7effd134c640 1 --2- 172.21.15.105:0/3516525552 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effcc102ef0 0x7effcc19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:53.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.483+0000 7effd134c640 1 -- 172.21.15.105:0/3516525552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7effcc100e20 msgr2=0x7effcc19a640 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T02:01:53.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.483+0000 7effd134c640 1 --2- 172.21.15.105:0/3516525552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7effcc100e20 0x7effcc19a640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:53.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.483+0000 7effd134c640 1 -- 172.21.15.105:0/3516525552 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effcc19f5f0 con 0x7effcc102570 2024-09-16T02:01:53.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.483+0000 7effd134c640 1 --2- 172.21.15.105:0/3516525552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effcc102570 0x7effcc19ab80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7effbc00c970 tx=0x7effbc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:53.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.483+0000 7effc2ffd640 1 -- 172.21.15.105:0/3516525552 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc007bb0 con 0x7effcc102570 2024-09-16T02:01:53.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.483+0000 7effc2ffd640 1 -- 172.21.15.105:0/3516525552 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7effbc007d50 con 0x7effcc102570 2024-09-16T02:01:53.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.483+0000 7effd3db1640 1 -- 172.21.15.105:0/3516525552 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effcc19f8e0 con 0x7effcc102570 2024-09-16T02:01:53.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.483+0000 7effc2ffd640 1 -- 172.21.15.105:0/3516525552 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc00f450 con 0x7effcc102570 2024-09-16T02:01:53.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.483+0000 7effd3db1640 1 -- 172.21.15.105:0/3516525552 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effcc1a71c0 con 0x7effcc102570 2024-09-16T02:01:53.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.484+0000 7effd3db1640 1 -- 172.21.15.105:0/3516525552 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effcc101200 con 0x7effcc102570 2024-09-16T02:01:53.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.485+0000 7effc2ffd640 1 -- 172.21.15.105:0/3516525552 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7effbc004f00 con 0x7effcc102570 2024-09-16T02:01:53.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.486+0000 7effc2ffd640 1 --2- 172.21.15.105:0/3516525552 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7effb80a32d0 0x7effb80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:53.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.486+0000 7effd1b4d640 1 --2- 172.21.15.105:0/3516525552 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7effb80a32d0 0x7effb80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:53.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.486+0000 7effc2ffd640 1 -- 172.21.15.105:0/3516525552 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7effbc0cfcc0 con 0x7effcc102570 2024-09-16T02:01:53.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.486+0000 7effd1b4d640 1 --2- 172.21.15.105:0/3516525552 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7effb80a32d0 0x7effb80a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7effc800ab90 tx=0x7effc8009250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:53.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.489+0000 7effc2ffd640 1 -- 172.21.15.105:0/3516525552 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7effbc014030 con 0x7effcc102570 2024-09-16T02:01:53.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.656+0000 7effd3db1640 1 -- 172.21.15.105:0/3516525552 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7effcc10cda0 con 0x7effb80a32d0 2024-09-16T02:01:53.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.663+0000 7effc2ffd640 1 -- 172.21.15.105:0/3516525552 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259021 (secure 0 0 0) 0x7effcc10cda0 con 0x7effb80a32d0 2024-09-16T02:01:53.666 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:01:53.667 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-09-16T02:01:53.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.666+0000 7effd3db1640 1 -- 172.21.15.105:0/3516525552 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7effb80a32d0 msgr2=0x7effb80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:53.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.666+0000 7effd3db1640 1 --2- 172.21.15.105:0/3516525552 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7effb80a32d0 0x7effb80a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7effc800ab90 tx=0x7effc8009250 comp rx=0 tx=0).stop 2024-09-16T02:01:53.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.666+0000 7effd3db1640 1 -- 172.21.15.105:0/3516525552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effcc102570 msgr2=0x7effcc19ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:53.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.666+0000 7effd3db1640 1 --2- 172.21.15.105:0/3516525552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effcc102570 0x7effcc19ab80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7effbc00c970 tx=0x7effbc00ce30 comp rx=0 tx=0).stop 2024-09-16T02:01:53.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.666+0000 7effd3db1640 1 -- 172.21.15.105:0/3516525552 shutdown_connections 2024-09-16T02:01:53.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.666+0000 7effd3db1640 1 --2- 172.21.15.105:0/3516525552 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7effb80a32d0 0x7effb80a5790 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:53.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.666+0000 7effd3db1640 1 --2- 172.21.15.105:0/3516525552 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effcc102ef0 0x7effcc19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:53.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.666+0000 7effd3db1640 1 --2- 172.21.15.105:0/3516525552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effcc102570 0x7effcc19ab80 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:53.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.666+0000 7effd3db1640 1 --2- 172.21.15.105:0/3516525552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7effcc100e20 0x7effcc19a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:53.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.666+0000 7effd3db1640 1 -- 172.21.15.105:0/3516525552 >> 172.21.15.105:0/3516525552 conn(0x7effcc0fc7d0 msgr2=0x7effcc06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:53.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.666+0000 7effd3db1640 1 -- 172.21.15.105:0/3516525552 shutdown_connections 2024-09-16T02:01:53.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:53.666+0000 7effd3db1640 1 -- 172.21.15.105:0/3516525552 wait complete. 2024-09-16T02:01:54.412 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-09-16T02:01:53.463020+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656628,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":799,"num_read_kb":521,"num_write":421,"num_write_kb":3070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1181435,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368624,"kb_used_data":10992,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124573200,"statfs":{"total":1151940427776,"available":1151562956800,"internally_reserved":0,"allocated":11255808,"data_stored":7141596,"data_compressed":176592,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":119,"num_read_kb":64,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003055"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560827+0000","last_change":"2024-09-16T02:00:34.454121+0000","last_active":"2024-09-16T02:01:40.560827+0000","last_peered":"2024-09-16T02:01:40.560827+0000","last_clean":"2024-09-16T02:01:40.560827+0000","last_became_active":"2024-09-16T02:00:34.453810+0000","last_became_peered":"2024-09-16T02:00:34.453810+0000","last_unstale":"2024-09-16T02:01:40.560827+0000","last_undegraded":"2024-09-16T02:01:40.560827+0000","last_fullsized":"2024-09-16T02:01:40.560827+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:42:38.781928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.562651+0000","last_change":"2024-09-16T02:00:34.462020+0000","last_active":"2024-09-16T02:01:40.562651+0000","last_peered":"2024-09-16T02:01:40.562651+0000","last_clean":"2024-09-16T02:01:40.562651+0000","last_became_active":"2024-09-16T02:00:34.461680+0000","last_became_peered":"2024-09-16T02:00:34.461680+0000","last_unstale":"2024-09-16T02:01:40.562651+0000","last_undegraded":"2024-09-16T02:01:40.562651+0000","last_fullsized":"2024-09-16T02:01:40.562651+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:00:14.493643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.562347+0000","last_change":"2024-09-16T02:00:34.453034+0000","last_active":"2024-09-16T02:01:40.562347+0000","last_peered":"2024-09-16T02:01:40.562347+0000","last_clean":"2024-09-16T02:01:40.562347+0000","last_became_active":"2024-09-16T02:00:34.452719+0000","last_became_peered":"2024-09-16T02:00:34.452719+0000","last_unstale":"2024-09-16T02:01:40.562347+0000","last_undegraded":"2024-09-16T02:01:40.562347+0000","last_fullsized":"2024-09-16T02:01:40.562347+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:48:52.478501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.610231+0000","last_change":"2024-09-16T02:00:34.458156+0000","last_active":"2024-09-16T02:01:40.610231+0000","last_peered":"2024-09-16T02:01:40.610231+0000","last_clean":"2024-09-16T02:01:40.610231+0000","last_became_active":"2024-09-16T02:00:34.457831+0000","last_became_peered":"2024-09-16T02:00:34.457831+0000","last_unstale":"2024-09-16T02:01:40.610231+0000","last_undegraded":"2024-09-16T02:01:40.610231+0000","last_fullsized":"2024-09-16T02:01:40.610231+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:35:36.609046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.562621+0000","last_change":"2024-09-16T02:00:34.448638+0000","last_active":"2024-09-16T02:01:40.562621+0000","last_peered":"2024-09-16T02:01:40.562621+0000","last_clean":"2024-09-16T02:01:40.562621+0000","last_became_active":"2024-09-16T02:00:34.447993+0000","last_became_peered":"2024-09-16T02:00:34.447993+0000","last_unstale":"2024-09-16T02:01:40.562621+0000","last_undegraded":"2024-09-16T02:01:40.562621+0000","last_fullsized":"2024-09-16T02:01:40.562621+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:33:04.832053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560794+0000","last_change":"2024-09-16T02:00:34.462651+0000","last_active":"2024-09-16T02:01:40.560794+0000","last_peered":"2024-09-16T02:01:40.560794+0000","last_clean":"2024-09-16T02:01:40.560794+0000","last_became_active":"2024-09-16T02:00:34.461837+0000","last_became_peered":"2024-09-16T02:00:34.461837+0000","last_unstale":"2024-09-16T02:01:40.560794+0000","last_undegraded":"2024-09-16T02:01:40.560794+0000","last_fullsized":"2024-09-16T02:01:40.560794+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:52:13.733591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564160+0000","last_change":"2024-09-16T02:00:34.455362+0000","last_active":"2024-09-16T02:01:40.564160+0000","last_peered":"2024-09-16T02:01:40.564160+0000","last_clean":"2024-09-16T02:01:40.564160+0000","last_became_active":"2024-09-16T02:00:34.455029+0000","last_became_peered":"2024-09-16T02:00:34.455029+0000","last_unstale":"2024-09-16T02:01:40.564160+0000","last_undegraded":"2024-09-16T02:01:40.564160+0000","last_fullsized":"2024-09-16T02:01:40.564160+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:14:08.316845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560955+0000","last_change":"2024-09-16T02:00:34.459578+0000","last_active":"2024-09-16T02:01:40.560955+0000","last_peered":"2024-09-16T02:01:40.560955+0000","last_clean":"2024-09-16T02:01:40.560955+0000","last_became_active":"2024-09-16T02:00:34.459288+0000","last_became_peered":"2024-09-16T02:00:34.459288+0000","last_unstale":"2024-09-16T02:01:40.560955+0000","last_undegraded":"2024-09-16T02:01:40.560955+0000","last_fullsized":"2024-09-16T02:01:40.560955+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:52:28.974276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.563476+0000","last_change":"2024-09-16T02:00:34.463011+0000","last_active":"2024-09-16T02:01:40.563476+0000","last_peered":"2024-09-16T02:01:40.563476+0000","last_clean":"2024-09-16T02:01:40.563476+0000","last_became_active":"2024-09-16T02:00:34.462668+0000","last_became_peered":"2024-09-16T02:00:34.462668+0000","last_unstale":"2024-09-16T02:01:40.563476+0000","last_undegraded":"2024-09-16T02:01:40.563476+0000","last_fullsized":"2024-09-16T02:01:40.563476+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:06:31.655869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560622+0000","last_change":"2024-09-16T02:00:34.462241+0000","last_active":"2024-09-16T02:01:40.560622+0000","last_peered":"2024-09-16T02:01:40.560622+0000","last_clean":"2024-09-16T02:01:40.560622+0000","last_became_active":"2024-09-16T02:00:34.461925+0000","last_became_peered":"2024-09-16T02:00:34.461925+0000","last_unstale":"2024-09-16T02:01:40.560622+0000","last_undegraded":"2024-09-16T02:01:40.560622+0000","last_fullsized":"2024-09-16T02:01:40.560622+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:48:02.414471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564743+0000","last_change":"2024-09-16T02:00:34.466132+0000","last_active":"2024-09-16T02:01:40.564743+0000","last_peered":"2024-09-16T02:01:40.564743+0000","last_clean":"2024-09-16T02:01:40.564743+0000","last_became_active":"2024-09-16T02:00:34.465750+0000","last_became_peered":"2024-09-16T02:00:34.465750+0000","last_unstale":"2024-09-16T02:01:40.564743+0000","last_undegraded":"2024-09-16T02:01:40.564743+0000","last_fullsized":"2024-09-16T02:01:40.564743+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:07:33.072432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.610362+0000","last_change":"2024-09-16T02:00:34.461759+0000","last_active":"2024-09-16T02:01:40.610362+0000","last_peered":"2024-09-16T02:01:40.610362+0000","last_clean":"2024-09-16T02:01:40.610362+0000","last_became_active":"2024-09-16T02:00:34.461385+0000","last_became_peered":"2024-09-16T02:00:34.461385+0000","last_unstale":"2024-09-16T02:01:40.610362+0000","last_undegraded":"2024-09-16T02:01:40.610362+0000","last_fullsized":"2024-09-16T02:01:40.610362+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:00:24.179172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.608361+0000","last_change":"2024-09-16T02:01:40.608361+0000","last_active":"2024-09-16T02:01:40.608361+0000","last_peered":"2024-09-16T02:01:40.608361+0000","last_clean":"2024-09-16T02:01:40.608361+0000","last_became_active":"2024-09-16T02:01:40.607954+0000","last_became_peered":"2024-09-16T02:01:40.607954+0000","last_unstale":"2024-09-16T02:01:40.608361+0000","last_undegraded":"2024-09-16T02:01:40.608361+0000","last_fullsized":"2024-09-16T02:01:40.608361+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:18:31.244969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.562228+0000","last_change":"2024-09-16T02:00:34.460680+0000","last_active":"2024-09-16T02:01:40.562228+0000","last_peered":"2024-09-16T02:01:40.562228+0000","last_clean":"2024-09-16T02:01:40.562228+0000","last_became_active":"2024-09-16T02:00:34.460161+0000","last_became_peered":"2024-09-16T02:00:34.460161+0000","last_unstale":"2024-09-16T02:01:40.562228+0000","last_undegraded":"2024-09-16T02:01:40.562228+0000","last_fullsized":"2024-09-16T02:01:40.562228+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:38:19.848420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.562137+0000","last_change":"2024-09-16T02:00:34.454606+0000","last_active":"2024-09-16T02:01:40.562137+0000","last_peered":"2024-09-16T02:01:40.562137+0000","last_clean":"2024-09-16T02:01:40.562137+0000","last_became_active":"2024-09-16T02:00:34.454293+0000","last_became_peered":"2024-09-16T02:00:34.454293+0000","last_unstale":"2024-09-16T02:01:40.562137+0000","last_undegraded":"2024-09-16T02:01:40.562137+0000","last_fullsized":"2024-09-16T02:01:40.562137+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:55:54.466176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.562971+0000","last_change":"2024-09-16T02:00:34.452940+0000","last_active":"2024-09-16T02:01:40.562971+0000","last_peered":"2024-09-16T02:01:40.562971+0000","last_clean":"2024-09-16T02:01:40.562971+0000","last_became_active":"2024-09-16T02:00:34.452560+0000","last_became_peered":"2024-09-16T02:00:34.452560+0000","last_unstale":"2024-09-16T02:01:40.562971+0000","last_undegraded":"2024-09-16T02:01:40.562971+0000","last_fullsized":"2024-09-16T02:01:40.562971+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:42:11.564264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560911+0000","last_change":"2024-09-16T02:00:34.453457+0000","last_active":"2024-09-16T02:01:40.560911+0000","last_peered":"2024-09-16T02:01:40.560911+0000","last_clean":"2024-09-16T02:01:40.560911+0000","last_became_active":"2024-09-16T02:00:34.453148+0000","last_became_peered":"2024-09-16T02:00:34.453148+0000","last_unstale":"2024-09-16T02:01:40.560911+0000","last_undegraded":"2024-09-16T02:01:40.560911+0000","last_fullsized":"2024-09-16T02:01:40.560911+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:28.456892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.607401+0000","last_change":"2024-09-16T02:00:34.457057+0000","last_active":"2024-09-16T02:01:40.607401+0000","last_peered":"2024-09-16T02:01:40.607401+0000","last_clean":"2024-09-16T02:01:40.607401+0000","last_became_active":"2024-09-16T02:00:34.456751+0000","last_became_peered":"2024-09-16T02:00:34.456751+0000","last_unstale":"2024-09-16T02:01:40.607401+0000","last_undegraded":"2024-09-16T02:01:40.607401+0000","last_fullsized":"2024-09-16T02:01:40.607401+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:32:32.207510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.561837+0000","last_change":"2024-09-16T02:00:34.448434+0000","last_active":"2024-09-16T02:01:40.561837+0000","last_peered":"2024-09-16T02:01:40.561837+0000","last_clean":"2024-09-16T02:01:40.561837+0000","last_became_active":"2024-09-16T02:00:34.448056+0000","last_became_peered":"2024-09-16T02:00:34.448056+0000","last_unstale":"2024-09-16T02:01:40.561837+0000","last_undegraded":"2024-09-16T02:01:40.561837+0000","last_fullsized":"2024-09-16T02:01:40.561837+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:15:28.162159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560368+0000","last_change":"2024-09-16T02:00:34.459499+0000","last_active":"2024-09-16T02:01:40.560368+0000","last_peered":"2024-09-16T02:01:40.560368+0000","last_clean":"2024-09-16T02:01:40.560368+0000","last_became_active":"2024-09-16T02:00:34.459017+0000","last_became_peered":"2024-09-16T02:00:34.459017+0000","last_unstale":"2024-09-16T02:01:40.560368+0000","last_undegraded":"2024-09-16T02:01:40.560368+0000","last_fullsized":"2024-09-16T02:01:40.560368+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:41:36.866100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.559296+0000","last_change":"2024-09-16T02:00:34.448640+0000","last_active":"2024-09-16T02:01:40.559296+0000","last_peered":"2024-09-16T02:01:40.559296+0000","last_clean":"2024-09-16T02:01:40.559296+0000","last_became_active":"2024-09-16T02:00:34.447737+0000","last_became_peered":"2024-09-16T02:00:34.447737+0000","last_unstale":"2024-09-16T02:01:40.559296+0000","last_undegraded":"2024-09-16T02:01:40.559296+0000","last_fullsized":"2024-09-16T02:01:40.559296+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:20.192650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.563552+0000","last_change":"2024-09-16T02:00:34.451718+0000","last_active":"2024-09-16T02:01:40.563552+0000","last_peered":"2024-09-16T02:01:40.563552+0000","last_clean":"2024-09-16T02:01:40.563552+0000","last_became_active":"2024-09-16T02:00:34.451389+0000","last_became_peered":"2024-09-16T02:00:34.451389+0000","last_unstale":"2024-09-16T02:01:40.563552+0000","last_undegraded":"2024-09-16T02:01:40.563552+0000","last_fullsized":"2024-09-16T02:01:40.563552+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:13:19.296178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.561207+0000","last_change":"2024-09-16T02:00:34.458379+0000","last_active":"2024-09-16T02:01:40.561207+0000","last_peered":"2024-09-16T02:01:40.561207+0000","last_clean":"2024-09-16T02:01:40.561207+0000","last_became_active":"2024-09-16T02:00:34.458038+0000","last_became_peered":"2024-09-16T02:00:34.458038+0000","last_unstale":"2024-09-16T02:01:40.561207+0000","last_undegraded":"2024-09-16T02:01:40.561207+0000","last_fullsized":"2024-09-16T02:01:40.561207+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:34.669470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.565143+0000","last_change":"2024-09-16T02:00:34.450927+0000","last_active":"2024-09-16T02:01:40.565143+0000","last_peered":"2024-09-16T02:01:40.565143+0000","last_clean":"2024-09-16T02:01:40.565143+0000","last_became_active":"2024-09-16T02:00:34.447669+0000","last_became_peered":"2024-09-16T02:00:34.447669+0000","last_unstale":"2024-09-16T02:01:40.565143+0000","last_undegraded":"2024-09-16T02:01:40.565143+0000","last_fullsized":"2024-09-16T02:01:40.565143+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:57.861633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.561666+0000","last_change":"2024-09-16T02:00:34.454827+0000","last_active":"2024-09-16T02:01:40.561666+0000","last_peered":"2024-09-16T02:01:40.561666+0000","last_clean":"2024-09-16T02:01:40.561666+0000","last_became_active":"2024-09-16T02:00:34.454549+0000","last_became_peered":"2024-09-16T02:00:34.454549+0000","last_unstale":"2024-09-16T02:01:40.561666+0000","last_undegraded":"2024-09-16T02:01:40.561666+0000","last_fullsized":"2024-09-16T02:01:40.561666+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:53:45.440860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.563093+0000","last_change":"2024-09-16T02:00:34.455121+0000","last_active":"2024-09-16T02:01:40.563093+0000","last_peered":"2024-09-16T02:01:40.563093+0000","last_clean":"2024-09-16T02:01:40.563093+0000","last_became_active":"2024-09-16T02:00:34.454665+0000","last_became_peered":"2024-09-16T02:00:34.454665+0000","last_unstale":"2024-09-16T02:01:40.563093+0000","last_undegraded":"2024-09-16T02:01:40.563093+0000","last_fullsized":"2024-09-16T02:01:40.563093+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:31:42.771230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560656+0000","last_change":"2024-09-16T02:00:34.458777+0000","last_active":"2024-09-16T02:01:40.560656+0000","last_peered":"2024-09-16T02:01:40.560656+0000","last_clean":"2024-09-16T02:01:40.560656+0000","last_became_active":"2024-09-16T02:00:34.457954+0000","last_became_peered":"2024-09-16T02:00:34.457954+0000","last_unstale":"2024-09-16T02:01:40.560656+0000","last_undegraded":"2024-09-16T02:01:40.560656+0000","last_fullsized":"2024-09-16T02:01:40.560656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:58:28.032293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.559354+0000","last_change":"2024-09-16T02:00:34.444712+0000","last_active":"2024-09-16T02:01:40.559354+0000","last_peered":"2024-09-16T02:01:40.559354+0000","last_clean":"2024-09-16T02:01:40.559354+0000","last_became_active":"2024-09-16T02:00:34.444421+0000","last_became_peered":"2024-09-16T02:00:34.444421+0000","last_unstale":"2024-09-16T02:01:40.559354+0000","last_undegraded":"2024-09-16T02:01:40.559354+0000","last_fullsized":"2024-09-16T02:01:40.559354+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:36.557558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.562379+0000","last_change":"2024-09-16T02:00:34.454570+0000","last_active":"2024-09-16T02:01:40.562379+0000","last_peered":"2024-09-16T02:01:40.562379+0000","last_clean":"2024-09-16T02:01:40.562379+0000","last_became_active":"2024-09-16T02:00:34.454246+0000","last_became_peered":"2024-09-16T02:00:34.454246+0000","last_unstale":"2024-09-16T02:01:40.562379+0000","last_undegraded":"2024-09-16T02:01:40.562379+0000","last_fullsized":"2024-09-16T02:01:40.562379+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:36:18.634907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.561677+0000","last_change":"2024-09-16T02:00:34.449680+0000","last_active":"2024-09-16T02:01:40.561677+0000","last_peered":"2024-09-16T02:01:40.561677+0000","last_clean":"2024-09-16T02:01:40.561677+0000","last_became_active":"2024-09-16T02:00:34.449184+0000","last_became_peered":"2024-09-16T02:00:34.449184+0000","last_unstale":"2024-09-16T02:01:40.561677+0000","last_undegraded":"2024-09-16T02:01:40.561677+0000","last_fullsized":"2024-09-16T02:01:40.561677+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:28:34.867820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.562212+0000","last_change":"2024-09-16T02:00:34.455362+0000","last_active":"2024-09-16T02:01:40.562212+0000","last_peered":"2024-09-16T02:01:40.562212+0000","last_clean":"2024-09-16T02:01:40.562212+0000","last_became_active":"2024-09-16T02:00:34.455018+0000","last_became_peered":"2024-09-16T02:00:34.455018+0000","last_unstale":"2024-09-16T02:01:40.562212+0000","last_undegraded":"2024-09-16T02:01:40.562212+0000","last_fullsized":"2024-09-16T02:01:40.562212+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:38.278418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560207+0000","last_change":"2024-09-16T02:00:34.459426+0000","last_active":"2024-09-16T02:01:40.560207+0000","last_peered":"2024-09-16T02:01:40.560207+0000","last_clean":"2024-09-16T02:01:40.560207+0000","last_became_active":"2024-09-16T02:00:34.459082+0000","last_became_peered":"2024-09-16T02:00:34.459082+0000","last_unstale":"2024-09-16T02:01:40.560207+0000","last_undegraded":"2024-09-16T02:01:40.560207+0000","last_fullsized":"2024-09-16T02:01:40.560207+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:59:42.226065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.563365+0000","last_change":"2024-09-16T02:00:34.450936+0000","last_active":"2024-09-16T02:01:40.563365+0000","last_peered":"2024-09-16T02:01:40.563365+0000","last_clean":"2024-09-16T02:01:40.563365+0000","last_became_active":"2024-09-16T02:00:34.450503+0000","last_became_peered":"2024-09-16T02:00:34.450503+0000","last_unstale":"2024-09-16T02:01:40.563365+0000","last_undegraded":"2024-09-16T02:01:40.563365+0000","last_fullsized":"2024-09-16T02:01:40.563365+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:35:55.806193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'5","reported_seq":91,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:47.617495+0000","last_change":"2024-09-16T02:00:32.443193+0000","last_active":"2024-09-16T02:01:47.617495+0000","last_peered":"2024-09-16T02:01:47.617495+0000","last_clean":"2024-09-16T02:01:47.617495+0000","last_became_active":"2024-09-16T02:00:32.442863+0000","last_became_peered":"2024-09-16T02:00:32.442863+0000","last_unstale":"2024-09-16T02:01:47.617495+0000","last_undegraded":"2024-09-16T02:01:47.617495+0000","last_fullsized":"2024-09-16T02:01:47.617495+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:19.734946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.570518+0000","last_change":"2024-09-16T02:01:40.570518+0000","last_active":"2024-09-16T02:01:40.570518+0000","last_peered":"2024-09-16T02:01:40.570518+0000","last_clean":"2024-09-16T02:01:40.570518+0000","last_became_active":"2024-09-16T02:01:40.570124+0000","last_became_peered":"2024-09-16T02:01:40.570124+0000","last_unstale":"2024-09-16T02:01:40.570518+0000","last_undegraded":"2024-09-16T02:01:40.570518+0000","last_fullsized":"2024-09-16T02:01:40.570518+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:44:21.062742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.572282+0000","last_change":"2024-09-16T02:01:40.572282+0000","last_active":"2024-09-16T02:01:40.572282+0000","last_peered":"2024-09-16T02:01:40.572282+0000","last_clean":"2024-09-16T02:01:40.572282+0000","last_became_active":"2024-09-16T02:01:40.564577+0000","last_became_peered":"2024-09-16T02:01:40.564577+0000","last_unstale":"2024-09-16T02:01:40.572282+0000","last_undegraded":"2024-09-16T02:01:40.572282+0000","last_fullsized":"2024-09-16T02:01:40.572282+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:19:40.834173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.559077+0000","last_change":"2024-09-16T02:00:32.438530+0000","last_active":"2024-09-16T02:01:40.559077+0000","last_peered":"2024-09-16T02:01:40.559077+0000","last_clean":"2024-09-16T02:01:40.559077+0000","last_became_active":"2024-09-16T02:00:32.438238+0000","last_became_peered":"2024-09-16T02:00:32.438238+0000","last_unstale":"2024-09-16T02:01:40.559077+0000","last_undegraded":"2024-09-16T02:01:40.559077+0000","last_fullsized":"2024-09-16T02:01:40.559077+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:52:32.799701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.561050+0000","last_change":"2024-09-16T02:00:34.456299+0000","last_active":"2024-09-16T02:01:40.561050+0000","last_peered":"2024-09-16T02:01:40.561050+0000","last_clean":"2024-09-16T02:01:40.561050+0000","last_became_active":"2024-09-16T02:00:34.455925+0000","last_became_peered":"2024-09-16T02:00:34.455925+0000","last_unstale":"2024-09-16T02:01:40.561050+0000","last_undegraded":"2024-09-16T02:01:40.561050+0000","last_fullsized":"2024-09-16T02:01:40.561050+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:33:27.162664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.565294+0000","last_change":"2024-09-16T02:00:32.442029+0000","last_active":"2024-09-16T02:01:40.565294+0000","last_peered":"2024-09-16T02:01:40.565294+0000","last_clean":"2024-09-16T02:01:40.565294+0000","last_became_active":"2024-09-16T02:00:32.441705+0000","last_became_peered":"2024-09-16T02:00:32.441705+0000","last_unstale":"2024-09-16T02:01:40.565294+0000","last_undegraded":"2024-09-16T02:01:40.565294+0000","last_fullsized":"2024-09-16T02:01:40.565294+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:50:54.725768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.607165+0000","last_change":"2024-09-16T02:00:34.457833+0000","last_active":"2024-09-16T02:01:40.607165+0000","last_peered":"2024-09-16T02:01:40.607165+0000","last_clean":"2024-09-16T02:01:40.607165+0000","last_became_active":"2024-09-16T02:00:34.457453+0000","last_became_peered":"2024-09-16T02:00:34.457453+0000","last_unstale":"2024-09-16T02:01:40.607165+0000","last_undegraded":"2024-09-16T02:01:40.607165+0000","last_fullsized":"2024-09-16T02:01:40.607165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:21.777574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":156,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:51.621908+0000","last_change":"2024-09-16T02:00:32.441341+0000","last_active":"2024-09-16T02:01:51.621908+0000","last_peered":"2024-09-16T02:01:51.621908+0000","last_clean":"2024-09-16T02:01:51.621908+0000","last_became_active":"2024-09-16T02:00:32.441019+0000","last_became_peered":"2024-09-16T02:00:32.441019+0000","last_unstale":"2024-09-16T02:01:51.621908+0000","last_undegraded":"2024-09-16T02:01:51.621908+0000","last_fullsized":"2024-09-16T02:01:51.621908+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:10:10.171714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":130,"num_read_kb":130,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.563417+0000","last_change":"2024-09-16T02:00:34.461746+0000","last_active":"2024-09-16T02:01:40.563417+0000","last_peered":"2024-09-16T02:01:40.563417+0000","last_clean":"2024-09-16T02:01:40.563417+0000","last_became_active":"2024-09-16T02:00:34.461441+0000","last_became_peered":"2024-09-16T02:00:34.461441+0000","last_unstale":"2024-09-16T02:01:40.563417+0000","last_undegraded":"2024-09-16T02:01:40.563417+0000","last_fullsized":"2024-09-16T02:01:40.563417+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:28:14.519159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564386+0000","last_change":"2024-09-16T02:00:32.443129+0000","last_active":"2024-09-16T02:01:40.564386+0000","last_peered":"2024-09-16T02:01:40.564386+0000","last_clean":"2024-09-16T02:01:40.564386+0000","last_became_active":"2024-09-16T02:00:32.442772+0000","last_became_peered":"2024-09-16T02:00:32.442772+0000","last_unstale":"2024-09-16T02:01:40.564386+0000","last_undegraded":"2024-09-16T02:01:40.564386+0000","last_fullsized":"2024-09-16T02:01:40.564386+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:10:03.964583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.562614+0000","last_change":"2024-09-16T02:00:34.457449+0000","last_active":"2024-09-16T02:01:40.562614+0000","last_peered":"2024-09-16T02:01:40.562614+0000","last_clean":"2024-09-16T02:01:40.562614+0000","last_became_active":"2024-09-16T02:00:34.457110+0000","last_became_peered":"2024-09-16T02:00:34.457110+0000","last_unstale":"2024-09-16T02:01:40.562614+0000","last_undegraded":"2024-09-16T02:01:40.562614+0000","last_fullsized":"2024-09-16T02:01:40.562614+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:35.662503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.565563+0000","last_change":"2024-09-16T02:00:32.587419+0000","last_active":"2024-09-16T02:01:40.565563+0000","last_peered":"2024-09-16T02:01:40.565563+0000","last_clean":"2024-09-16T02:01:40.565563+0000","last_became_active":"2024-09-16T02:00:32.587037+0000","last_became_peered":"2024-09-16T02:00:32.587037+0000","last_unstale":"2024-09-16T02:01:40.565563+0000","last_undegraded":"2024-09-16T02:01:40.565563+0000","last_fullsized":"2024-09-16T02:01:40.565563+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:11.835505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.563152+0000","last_change":"2024-09-16T02:00:34.456371+0000","last_active":"2024-09-16T02:01:40.563152+0000","last_peered":"2024-09-16T02:01:40.563152+0000","last_clean":"2024-09-16T02:01:40.563152+0000","last_became_active":"2024-09-16T02:00:34.456013+0000","last_became_peered":"2024-09-16T02:00:34.456013+0000","last_unstale":"2024-09-16T02:01:40.563152+0000","last_undegraded":"2024-09-16T02:01:40.563152+0000","last_fullsized":"2024-09-16T02:01:40.563152+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:30:15.847189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'28","reported_seq":188,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:51.621773+0000","last_change":"2024-09-16T02:00:32.444922+0000","last_active":"2024-09-16T02:01:51.621773+0000","last_peered":"2024-09-16T02:01:51.621773+0000","last_clean":"2024-09-16T02:01:51.621773+0000","last_became_active":"2024-09-16T02:00:32.444540+0000","last_became_peered":"2024-09-16T02:00:32.444540+0000","last_unstale":"2024-09-16T02:01:51.621773+0000","last_undegraded":"2024-09-16T02:01:51.621773+0000","last_fullsized":"2024-09-16T02:01:51.621773+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:59.183952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59211,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":138,"num_read_kb":126,"num_write":30,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564383+0000","last_change":"2024-09-16T02:00:34.453366+0000","last_active":"2024-09-16T02:01:40.564383+0000","last_peered":"2024-09-16T02:01:40.564383+0000","last_clean":"2024-09-16T02:01:40.564383+0000","last_became_active":"2024-09-16T02:00:34.453048+0000","last_became_peered":"2024-09-16T02:00:34.453048+0000","last_unstale":"2024-09-16T02:01:40.564383+0000","last_undegraded":"2024-09-16T02:01:40.564383+0000","last_fullsized":"2024-09-16T02:01:40.564383+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:15:52.721893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.562558+0000","last_change":"2024-09-16T02:00:32.445516+0000","last_active":"2024-09-16T02:01:40.562558+0000","last_peered":"2024-09-16T02:01:40.562558+0000","last_clean":"2024-09-16T02:01:40.562558+0000","last_became_active":"2024-09-16T02:00:32.445092+0000","last_became_peered":"2024-09-16T02:00:32.445092+0000","last_unstale":"2024-09-16T02:01:40.562558+0000","last_undegraded":"2024-09-16T02:01:40.562558+0000","last_fullsized":"2024-09-16T02:01:40.562558+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:54.274692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.559134+0000","last_change":"2024-09-16T02:00:34.448450+0000","last_active":"2024-09-16T02:01:40.559134+0000","last_peered":"2024-09-16T02:01:40.559134+0000","last_clean":"2024-09-16T02:01:40.559134+0000","last_became_active":"2024-09-16T02:00:34.448116+0000","last_became_peered":"2024-09-16T02:00:34.448116+0000","last_unstale":"2024-09-16T02:01:40.559134+0000","last_undegraded":"2024-09-16T02:01:40.559134+0000","last_fullsized":"2024-09-16T02:01:40.559134+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:12:34.834798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.565309+0000","last_change":"2024-09-16T02:00:32.445427+0000","last_active":"2024-09-16T02:01:40.565309+0000","last_peered":"2024-09-16T02:01:40.565309+0000","last_clean":"2024-09-16T02:01:40.565309+0000","last_became_active":"2024-09-16T02:00:32.445058+0000","last_became_peered":"2024-09-16T02:00:32.445058+0000","last_unstale":"2024-09-16T02:01:40.565309+0000","last_undegraded":"2024-09-16T02:01:40.565309+0000","last_fullsized":"2024-09-16T02:01:40.565309+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:35:34.701331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564239+0000","last_change":"2024-09-16T02:00:34.458479+0000","last_active":"2024-09-16T02:01:40.564239+0000","last_peered":"2024-09-16T02:01:40.564239+0000","last_clean":"2024-09-16T02:01:40.564239+0000","last_became_active":"2024-09-16T02:00:34.458135+0000","last_became_peered":"2024-09-16T02:00:34.458135+0000","last_unstale":"2024-09-16T02:01:40.564239+0000","last_undegraded":"2024-09-16T02:01:40.564239+0000","last_fullsized":"2024-09-16T02:01:40.564239+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:37:40.188861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'6","reported_seq":93,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:47.621154+0000","last_change":"2024-09-16T02:00:32.440319+0000","last_active":"2024-09-16T02:01:47.621154+0000","last_peered":"2024-09-16T02:01:47.621154+0000","last_clean":"2024-09-16T02:01:47.621154+0000","last_became_active":"2024-09-16T02:00:32.440005+0000","last_became_peered":"2024-09-16T02:00:32.440005+0000","last_unstale":"2024-09-16T02:01:47.621154+0000","last_undegraded":"2024-09-16T02:01:47.621154+0000","last_fullsized":"2024-09-16T02:01:47.621154+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:07:33.876577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.563949+0000","last_change":"2024-09-16T02:00:34.460468+0000","last_active":"2024-09-16T02:01:40.563949+0000","last_peered":"2024-09-16T02:01:40.563949+0000","last_clean":"2024-09-16T02:01:40.563949+0000","last_became_active":"2024-09-16T02:00:34.459190+0000","last_became_peered":"2024-09-16T02:00:34.459190+0000","last_unstale":"2024-09-16T02:01:40.563949+0000","last_undegraded":"2024-09-16T02:01:40.563949+0000","last_fullsized":"2024-09-16T02:01:40.563949+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:46:43.654991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.562649+0000","last_change":"2024-09-16T02:00:32.587700+0000","last_active":"2024-09-16T02:01:40.562649+0000","last_peered":"2024-09-16T02:01:40.562649+0000","last_clean":"2024-09-16T02:01:40.562649+0000","last_became_active":"2024-09-16T02:00:32.586964+0000","last_became_peered":"2024-09-16T02:00:32.586964+0000","last_unstale":"2024-09-16T02:01:40.562649+0000","last_undegraded":"2024-09-16T02:01:40.562649+0000","last_fullsized":"2024-09-16T02:01:40.562649+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:41.309918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.565038+0000","last_change":"2024-09-16T02:00:34.455652+0000","last_active":"2024-09-16T02:01:40.565038+0000","last_peered":"2024-09-16T02:01:40.565038+0000","last_clean":"2024-09-16T02:01:40.565038+0000","last_became_active":"2024-09-16T02:00:34.455306+0000","last_became_peered":"2024-09-16T02:00:34.455306+0000","last_unstale":"2024-09-16T02:01:40.565038+0000","last_undegraded":"2024-09-16T02:01:40.565038+0000","last_fullsized":"2024-09-16T02:01:40.565038+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:44:09.505096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564111+0000","last_change":"2024-09-16T02:00:32.441443+0000","last_active":"2024-09-16T02:01:40.564111+0000","last_peered":"2024-09-16T02:01:40.564111+0000","last_clean":"2024-09-16T02:01:40.564111+0000","last_became_active":"2024-09-16T02:00:32.441073+0000","last_became_peered":"2024-09-16T02:00:32.441073+0000","last_unstale":"2024-09-16T02:01:40.564111+0000","last_undegraded":"2024-09-16T02:01:40.564111+0000","last_fullsized":"2024-09-16T02:01:40.564111+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:16.865970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.611317+0000","last_change":"2024-09-16T02:01:40.611317+0000","last_active":"2024-09-16T02:01:40.611317+0000","last_peered":"2024-09-16T02:01:40.611317+0000","last_clean":"2024-09-16T02:01:40.611317+0000","last_became_active":"2024-09-16T02:01:40.610837+0000","last_became_peered":"2024-09-16T02:01:40.610837+0000","last_unstale":"2024-09-16T02:01:40.611317+0000","last_undegraded":"2024-09-16T02:01:40.611317+0000","last_fullsized":"2024-09-16T02:01:40.611317+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:19:55.376738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.617877+0000","last_change":"2024-09-16T02:01:40.617877+0000","last_active":"2024-09-16T02:01:40.617877+0000","last_peered":"2024-09-16T02:01:40.617877+0000","last_clean":"2024-09-16T02:01:40.617877+0000","last_became_active":"2024-09-16T02:01:40.611597+0000","last_became_peered":"2024-09-16T02:01:40.611597+0000","last_unstale":"2024-09-16T02:01:40.617877+0000","last_undegraded":"2024-09-16T02:01:40.617877+0000","last_fullsized":"2024-09-16T02:01:40.617877+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:53:35.796243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564383+0000","last_change":"2024-09-16T02:00:34.461889+0000","last_active":"2024-09-16T02:01:40.564383+0000","last_peered":"2024-09-16T02:01:40.564383+0000","last_clean":"2024-09-16T02:01:40.564383+0000","last_became_active":"2024-09-16T02:00:34.461583+0000","last_became_peered":"2024-09-16T02:00:34.461583+0000","last_unstale":"2024-09-16T02:01:40.564383+0000","last_undegraded":"2024-09-16T02:01:40.564383+0000","last_fullsized":"2024-09-16T02:01:40.564383+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:32.598422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.565617+0000","last_change":"2024-09-16T02:00:32.445201+0000","last_active":"2024-09-16T02:01:40.565617+0000","last_peered":"2024-09-16T02:01:40.565617+0000","last_clean":"2024-09-16T02:01:40.565617+0000","last_became_active":"2024-09-16T02:00:32.444722+0000","last_became_peered":"2024-09-16T02:00:32.444722+0000","last_unstale":"2024-09-16T02:01:40.565617+0000","last_undegraded":"2024-09-16T02:01:40.565617+0000","last_fullsized":"2024-09-16T02:01:40.565617+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:41:11.657028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'1","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.613676+0000","last_change":"2024-09-16T02:01:40.613676+0000","last_active":"2024-09-16T02:01:40.613676+0000","last_peered":"2024-09-16T02:01:40.613676+0000","last_clean":"2024-09-16T02:01:40.613676+0000","last_became_active":"2024-09-16T02:01:40.608260+0000","last_became_peered":"2024-09-16T02:01:40.608260+0000","last_unstale":"2024-09-16T02:01:40.613676+0000","last_undegraded":"2024-09-16T02:01:40.613676+0000","last_fullsized":"2024-09-16T02:01:40.613676+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:30:59.004103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'5","reported_seq":91,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:47.626057+0000","last_change":"2024-09-16T02:00:32.860059+0000","last_active":"2024-09-16T02:01:47.626057+0000","last_peered":"2024-09-16T02:01:47.626057+0000","last_clean":"2024-09-16T02:01:47.626057+0000","last_became_active":"2024-09-16T02:00:32.859617+0000","last_became_peered":"2024-09-16T02:00:32.859617+0000","last_unstale":"2024-09-16T02:01:47.626057+0000","last_undegraded":"2024-09-16T02:01:47.626057+0000","last_fullsized":"2024-09-16T02:01:47.626057+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:41:50.248194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":64,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564754+0000","last_change":"2024-09-16T02:00:34.454344+0000","last_active":"2024-09-16T02:01:40.564754+0000","last_peered":"2024-09-16T02:01:40.564754+0000","last_clean":"2024-09-16T02:01:40.564754+0000","last_became_active":"2024-09-16T02:00:34.454027+0000","last_became_peered":"2024-09-16T02:00:34.454027+0000","last_unstale":"2024-09-16T02:01:40.564754+0000","last_undegraded":"2024-09-16T02:01:40.564754+0000","last_fullsized":"2024-09-16T02:01:40.564754+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:33.020365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":334,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564354+0000","last_change":"2024-09-16T01:59:19.798159+0000","last_active":"2024-09-16T02:01:40.564354+0000","last_peered":"2024-09-16T02:01:40.564354+0000","last_clean":"2024-09-16T02:01:40.564354+0000","last_became_active":"2024-09-16T01:59:19.797688+0000","last_became_peered":"2024-09-16T01:59:19.797688+0000","last_unstale":"2024-09-16T02:01:40.564354+0000","last_undegraded":"2024-09-16T02:01:40.564354+0000","last_fullsized":"2024-09-16T02:01:40.564354+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:55:54.312750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:55:54.312750+0000","last_clean_scrub_stamp":"2024-09-16T01:55:54.312750+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:00:36.860158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.610108+0000","last_change":"2024-09-16T02:00:32.441925+0000","last_active":"2024-09-16T02:01:40.610108+0000","last_peered":"2024-09-16T02:01:40.610108+0000","last_clean":"2024-09-16T02:01:40.610108+0000","last_became_active":"2024-09-16T02:00:32.441565+0000","last_became_peered":"2024-09-16T02:00:32.441565+0000","last_unstale":"2024-09-16T02:01:40.610108+0000","last_undegraded":"2024-09-16T02:01:40.610108+0000","last_fullsized":"2024-09-16T02:01:40.610108+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:23:09.755034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.561404+0000","last_change":"2024-09-16T02:00:34.452986+0000","last_active":"2024-09-16T02:01:40.561404+0000","last_peered":"2024-09-16T02:01:40.561404+0000","last_clean":"2024-09-16T02:01:40.561404+0000","last_became_active":"2024-09-16T02:00:34.452688+0000","last_became_peered":"2024-09-16T02:00:34.452688+0000","last_unstale":"2024-09-16T02:01:40.561404+0000","last_undegraded":"2024-09-16T02:01:40.561404+0000","last_fullsized":"2024-09-16T02:01:40.561404+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:12.958209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564205+0000","last_change":"2024-09-16T02:00:32.443964+0000","last_active":"2024-09-16T02:01:40.564205+0000","last_peered":"2024-09-16T02:01:40.564205+0000","last_clean":"2024-09-16T02:01:40.564205+0000","last_became_active":"2024-09-16T02:00:32.443623+0000","last_became_peered":"2024-09-16T02:00:32.443623+0000","last_unstale":"2024-09-16T02:01:40.564205+0000","last_undegraded":"2024-09-16T02:01:40.564205+0000","last_fullsized":"2024-09-16T02:01:40.564205+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:30:34.406024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.559065+0000","last_change":"2024-09-16T02:00:34.449615+0000","last_active":"2024-09-16T02:01:40.559065+0000","last_peered":"2024-09-16T02:01:40.559065+0000","last_clean":"2024-09-16T02:01:40.559065+0000","last_became_active":"2024-09-16T02:00:34.449109+0000","last_became_peered":"2024-09-16T02:00:34.449109+0000","last_unstale":"2024-09-16T02:01:40.559065+0000","last_undegraded":"2024-09-16T02:01:40.559065+0000","last_fullsized":"2024-09-16T02:01:40.559065+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:45.036177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":90,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:47.620527+0000","last_change":"2024-09-16T02:00:32.587107+0000","last_active":"2024-09-16T02:01:47.620527+0000","last_peered":"2024-09-16T02:01:47.620527+0000","last_clean":"2024-09-16T02:01:47.620527+0000","last_became_active":"2024-09-16T02:00:32.586777+0000","last_became_peered":"2024-09-16T02:00:32.586777+0000","last_unstale":"2024-09-16T02:01:47.620527+0000","last_undegraded":"2024-09-16T02:01:47.620527+0000","last_fullsized":"2024-09-16T02:01:47.620527+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:03.968987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":63,"num_read_kb":63,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.570572+0000","last_change":"2024-09-16T02:01:40.570572+0000","last_active":"2024-09-16T02:01:40.570572+0000","last_peered":"2024-09-16T02:01:40.570572+0000","last_clean":"2024-09-16T02:01:40.570572+0000","last_became_active":"2024-09-16T02:01:40.570176+0000","last_became_peered":"2024-09-16T02:01:40.570176+0000","last_unstale":"2024-09-16T02:01:40.570572+0000","last_undegraded":"2024-09-16T02:01:40.570572+0000","last_fullsized":"2024-09-16T02:01:40.570572+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:24:13.138925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'5","reported_seq":91,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:48.625860+0000","last_change":"2024-09-16T02:00:32.443201+0000","last_active":"2024-09-16T02:01:48.625860+0000","last_peered":"2024-09-16T02:01:48.625860+0000","last_clean":"2024-09-16T02:01:48.625860+0000","last_became_active":"2024-09-16T02:00:32.442778+0000","last_became_peered":"2024-09-16T02:00:32.442778+0000","last_unstale":"2024-09-16T02:01:48.625860+0000","last_undegraded":"2024-09-16T02:01:48.625860+0000","last_fullsized":"2024-09-16T02:01:48.625860+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:43.933817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560410+0000","last_change":"2024-09-16T02:00:34.463000+0000","last_active":"2024-09-16T02:01:40.560410+0000","last_peered":"2024-09-16T02:01:40.560410+0000","last_clean":"2024-09-16T02:01:40.560410+0000","last_became_active":"2024-09-16T02:00:34.461890+0000","last_became_peered":"2024-09-16T02:00:34.461890+0000","last_unstale":"2024-09-16T02:01:40.560410+0000","last_undegraded":"2024-09-16T02:01:40.560410+0000","last_fullsized":"2024-09-16T02:01:40.560410+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:21:15.161295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.607447+0000","last_change":"2024-09-16T02:00:32.586933+0000","last_active":"2024-09-16T02:01:40.607447+0000","last_peered":"2024-09-16T02:01:40.607447+0000","last_clean":"2024-09-16T02:01:40.607447+0000","last_became_active":"2024-09-16T02:00:32.586565+0000","last_became_peered":"2024-09-16T02:00:32.586565+0000","last_unstale":"2024-09-16T02:01:40.607447+0000","last_undegraded":"2024-09-16T02:01:40.607447+0000","last_fullsized":"2024-09-16T02:01:40.607447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:45:07.130649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560456+0000","last_change":"2024-09-16T02:00:34.460805+0000","last_active":"2024-09-16T02:01:40.560456+0000","last_peered":"2024-09-16T02:01:40.560456+0000","last_clean":"2024-09-16T02:01:40.560456+0000","last_became_active":"2024-09-16T02:00:34.460482+0000","last_became_peered":"2024-09-16T02:00:34.460482+0000","last_unstale":"2024-09-16T02:01:40.560456+0000","last_undegraded":"2024-09-16T02:01:40.560456+0000","last_fullsized":"2024-09-16T02:01:40.560456+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:07:51.565053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564575+0000","last_change":"2024-09-16T02:00:32.587516+0000","last_active":"2024-09-16T02:01:40.564575+0000","last_peered":"2024-09-16T02:01:40.564575+0000","last_clean":"2024-09-16T02:01:40.564575+0000","last_became_active":"2024-09-16T02:00:32.587172+0000","last_became_peered":"2024-09-16T02:00:32.587172+0000","last_unstale":"2024-09-16T02:01:40.564575+0000","last_undegraded":"2024-09-16T02:01:40.564575+0000","last_fullsized":"2024-09-16T02:01:40.564575+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:52:29.018290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.559443+0000","last_change":"2024-09-16T02:00:34.446021+0000","last_active":"2024-09-16T02:01:40.559443+0000","last_peered":"2024-09-16T02:01:40.559443+0000","last_clean":"2024-09-16T02:01:40.559443+0000","last_became_active":"2024-09-16T02:00:34.445732+0000","last_became_peered":"2024-09-16T02:00:34.445732+0000","last_unstale":"2024-09-16T02:01:40.559443+0000","last_undegraded":"2024-09-16T02:01:40.559443+0000","last_fullsized":"2024-09-16T02:01:40.559443+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:51:17.960792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'14","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.607435+0000","last_change":"2024-09-16T02:00:32.439750+0000","last_active":"2024-09-16T02:01:40.607435+0000","last_peered":"2024-09-16T02:01:40.607435+0000","last_clean":"2024-09-16T02:01:40.607435+0000","last_became_active":"2024-09-16T02:00:32.439438+0000","last_became_peered":"2024-09-16T02:00:32.439438+0000","last_unstale":"2024-09-16T02:01:40.607435+0000","last_undegraded":"2024-09-16T02:01:40.607435+0000","last_fullsized":"2024-09-16T02:01:40.607435+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:50.764733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.561712+0000","last_change":"2024-09-16T02:00:34.453810+0000","last_active":"2024-09-16T02:01:40.561712+0000","last_peered":"2024-09-16T02:01:40.561712+0000","last_clean":"2024-09-16T02:01:40.561712+0000","last_became_active":"2024-09-16T02:00:34.453421+0000","last_became_peered":"2024-09-16T02:00:34.453421+0000","last_unstale":"2024-09-16T02:01:40.561712+0000","last_undegraded":"2024-09-16T02:01:40.561712+0000","last_fullsized":"2024-09-16T02:01:40.561712+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:23:50.405749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.610378+0000","last_change":"2024-09-16T02:00:32.860565+0000","last_active":"2024-09-16T02:01:40.610378+0000","last_peered":"2024-09-16T02:01:40.610378+0000","last_clean":"2024-09-16T02:01:40.610378+0000","last_became_active":"2024-09-16T02:00:32.860130+0000","last_became_peered":"2024-09-16T02:00:32.860130+0000","last_unstale":"2024-09-16T02:01:40.610378+0000","last_undegraded":"2024-09-16T02:01:40.610378+0000","last_fullsized":"2024-09-16T02:01:40.610378+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:09:55.759883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564454+0000","last_change":"2024-09-16T02:01:40.564454+0000","last_active":"2024-09-16T02:01:40.564454+0000","last_peered":"2024-09-16T02:01:40.564454+0000","last_clean":"2024-09-16T02:01:40.564454+0000","last_became_active":"2024-09-16T02:01:40.563854+0000","last_became_peered":"2024-09-16T02:01:40.563854+0000","last_unstale":"2024-09-16T02:01:40.564454+0000","last_undegraded":"2024-09-16T02:01:40.564454+0000","last_fullsized":"2024-09-16T02:01:40.564454+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:56:30.610947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.563115+0000","last_change":"2024-09-16T02:00:32.587115+0000","last_active":"2024-09-16T02:01:40.563115+0000","last_peered":"2024-09-16T02:01:40.563115+0000","last_clean":"2024-09-16T02:01:40.563115+0000","last_became_active":"2024-09-16T02:00:32.586835+0000","last_became_peered":"2024-09-16T02:00:32.586835+0000","last_unstale":"2024-09-16T02:01:40.563115+0000","last_undegraded":"2024-09-16T02:01:40.563115+0000","last_fullsized":"2024-09-16T02:01:40.563115+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:12:58.820005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.562587+0000","last_change":"2024-09-16T02:00:34.457673+0000","last_active":"2024-09-16T02:01:40.562587+0000","last_peered":"2024-09-16T02:01:40.562587+0000","last_clean":"2024-09-16T02:01:40.562587+0000","last_became_active":"2024-09-16T02:00:34.457331+0000","last_became_peered":"2024-09-16T02:00:34.457331+0000","last_unstale":"2024-09-16T02:01:40.562587+0000","last_undegraded":"2024-09-16T02:01:40.562587+0000","last_fullsized":"2024-09-16T02:01:40.562587+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:12:16.100036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.561350+0000","last_change":"2024-09-16T02:00:32.437528+0000","last_active":"2024-09-16T02:01:40.561350+0000","last_peered":"2024-09-16T02:01:40.561350+0000","last_clean":"2024-09-16T02:01:40.561350+0000","last_became_active":"2024-09-16T02:00:32.437080+0000","last_became_peered":"2024-09-16T02:00:32.437080+0000","last_unstale":"2024-09-16T02:01:40.561350+0000","last_undegraded":"2024-09-16T02:01:40.561350+0000","last_fullsized":"2024-09-16T02:01:40.561350+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:59:33.908026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.560289+0000","last_change":"2024-09-16T02:00:34.453504+0000","last_active":"2024-09-16T02:01:40.560289+0000","last_peered":"2024-09-16T02:01:40.560289+0000","last_clean":"2024-09-16T02:01:40.560289+0000","last_became_active":"2024-09-16T02:00:34.453212+0000","last_became_peered":"2024-09-16T02:00:34.453212+0000","last_unstale":"2024-09-16T02:01:40.560289+0000","last_undegraded":"2024-09-16T02:01:40.560289+0000","last_fullsized":"2024-09-16T02:01:40.560289+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:00:56.414548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.562417+0000","last_change":"2024-09-16T02:00:32.442282+0000","last_active":"2024-09-16T02:01:40.562417+0000","last_peered":"2024-09-16T02:01:40.562417+0000","last_clean":"2024-09-16T02:01:40.562417+0000","last_became_active":"2024-09-16T02:00:32.441910+0000","last_became_peered":"2024-09-16T02:00:32.441910+0000","last_unstale":"2024-09-16T02:01:40.562417+0000","last_undegraded":"2024-09-16T02:01:40.562417+0000","last_fullsized":"2024-09-16T02:01:40.562417+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:01:46.045021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.561467+0000","last_change":"2024-09-16T02:00:34.460177+0000","last_active":"2024-09-16T02:01:40.561467+0000","last_peered":"2024-09-16T02:01:40.561467+0000","last_clean":"2024-09-16T02:01:40.561467+0000","last_became_active":"2024-09-16T02:00:34.459874+0000","last_became_peered":"2024-09-16T02:00:34.459874+0000","last_unstale":"2024-09-16T02:01:40.561467+0000","last_undegraded":"2024-09-16T02:01:40.561467+0000","last_fullsized":"2024-09-16T02:01:40.561467+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:35.855591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.614997+0000","last_change":"2024-09-16T02:01:40.614997+0000","last_active":"2024-09-16T02:01:40.614997+0000","last_peered":"2024-09-16T02:01:40.614997+0000","last_clean":"2024-09-16T02:01:40.614997+0000","last_became_active":"2024-09-16T02:01:40.610889+0000","last_became_peered":"2024-09-16T02:01:40.610889+0000","last_unstale":"2024-09-16T02:01:40.614997+0000","last_undegraded":"2024-09-16T02:01:40.614997+0000","last_fullsized":"2024-09-16T02:01:40.614997+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:39:00.989594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564262+0000","last_change":"2024-09-16T02:00:34.452964+0000","last_active":"2024-09-16T02:01:40.564262+0000","last_peered":"2024-09-16T02:01:40.564262+0000","last_clean":"2024-09-16T02:01:40.564262+0000","last_became_active":"2024-09-16T02:00:34.452617+0000","last_became_peered":"2024-09-16T02:00:34.452617+0000","last_unstale":"2024-09-16T02:01:40.564262+0000","last_undegraded":"2024-09-16T02:01:40.564262+0000","last_fullsized":"2024-09-16T02:01:40.564262+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:04.617970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.565506+0000","last_change":"2024-09-16T02:00:34.456585+0000","last_active":"2024-09-16T02:01:40.565506+0000","last_peered":"2024-09-16T02:01:40.565506+0000","last_clean":"2024-09-16T02:01:40.565506+0000","last_became_active":"2024-09-16T02:00:34.456199+0000","last_became_peered":"2024-09-16T02:00:34.456199+0000","last_unstale":"2024-09-16T02:01:40.565506+0000","last_undegraded":"2024-09-16T02:01:40.565506+0000","last_fullsized":"2024-09-16T02:01:40.565506+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:37:54.376785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.579320+0000","last_change":"2024-09-16T02:01:40.578904+0000","last_active":"2024-09-16T02:01:40.579320+0000","last_peered":"2024-09-16T02:01:40.579320+0000","last_clean":"2024-09-16T02:01:40.579320+0000","last_became_active":"2024-09-16T02:01:40.567221+0000","last_became_peered":"2024-09-16T02:01:40.567221+0000","last_unstale":"2024-09-16T02:01:40.579320+0000","last_undegraded":"2024-09-16T02:01:40.579320+0000","last_fullsized":"2024-09-16T02:01:40.579320+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:01.407145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.563083+0000","last_change":"2024-09-16T02:00:34.460925+0000","last_active":"2024-09-16T02:01:40.563083+0000","last_peered":"2024-09-16T02:01:40.563083+0000","last_clean":"2024-09-16T02:01:40.563083+0000","last_became_active":"2024-09-16T02:00:34.460581+0000","last_became_peered":"2024-09-16T02:00:34.460581+0000","last_unstale":"2024-09-16T02:01:40.563083+0000","last_undegraded":"2024-09-16T02:01:40.563083+0000","last_fullsized":"2024-09-16T02:01:40.563083+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:19:44.460695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564476+0000","last_change":"2024-09-16T02:00:32.443877+0000","last_active":"2024-09-16T02:01:40.564476+0000","last_peered":"2024-09-16T02:01:40.564476+0000","last_clean":"2024-09-16T02:01:40.564476+0000","last_became_active":"2024-09-16T02:00:32.443416+0000","last_became_peered":"2024-09-16T02:00:32.443416+0000","last_unstale":"2024-09-16T02:01:40.564476+0000","last_undegraded":"2024-09-16T02:01:40.564476+0000","last_fullsized":"2024-09-16T02:01:40.564476+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:36:57.367232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.562609+0000","last_change":"2024-09-16T02:00:34.453048+0000","last_active":"2024-09-16T02:01:40.562609+0000","last_peered":"2024-09-16T02:01:40.562609+0000","last_clean":"2024-09-16T02:01:40.562609+0000","last_became_active":"2024-09-16T02:00:34.452752+0000","last_became_peered":"2024-09-16T02:00:34.452752+0000","last_unstale":"2024-09-16T02:01:40.562609+0000","last_undegraded":"2024-09-16T02:01:40.562609+0000","last_fullsized":"2024-09-16T02:01:40.562609+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:55.970195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.561951+0000","last_change":"2024-09-16T02:00:32.586974+0000","last_active":"2024-09-16T02:01:40.561951+0000","last_peered":"2024-09-16T02:01:40.561951+0000","last_clean":"2024-09-16T02:01:40.561951+0000","last_became_active":"2024-09-16T02:00:32.586644+0000","last_became_peered":"2024-09-16T02:00:32.586644+0000","last_unstale":"2024-09-16T02:01:40.561951+0000","last_undegraded":"2024-09-16T02:01:40.561951+0000","last_fullsized":"2024-09-16T02:01:40.561951+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:52.913287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.561774+0000","last_change":"2024-09-16T02:00:34.447548+0000","last_active":"2024-09-16T02:01:40.561774+0000","last_peered":"2024-09-16T02:01:40.561774+0000","last_clean":"2024-09-16T02:01:40.561774+0000","last_became_active":"2024-09-16T02:00:34.447207+0000","last_became_peered":"2024-09-16T02:00:34.447207+0000","last_unstale":"2024-09-16T02:01:40.561774+0000","last_undegraded":"2024-09-16T02:01:40.561774+0000","last_fullsized":"2024-09-16T02:01:40.561774+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:22:02.494343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":100,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:49.626501+0000","last_change":"2024-09-16T02:00:32.441129+0000","last_active":"2024-09-16T02:01:49.626501+0000","last_peered":"2024-09-16T02:01:49.626501+0000","last_clean":"2024-09-16T02:01:49.626501+0000","last_became_active":"2024-09-16T02:00:32.440843+0000","last_became_peered":"2024-09-16T02:00:32.440843+0000","last_unstale":"2024-09-16T02:01:49.626501+0000","last_undegraded":"2024-09-16T02:01:49.626501+0000","last_fullsized":"2024-09-16T02:01:49.626501+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:17.721740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":65,"num_read_kb":65,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66026,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":673,"num_read_kb":412,"num_write":238,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":198078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281576,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612638,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":892928,"data_stored":432750,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841571,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1044480,"data_stored":1082801,"data_compressed":41575,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005095,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":917504,"data_stored":432215,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234028,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":431894,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462961,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":962560,"data_stored":1024996,"data_compressed":41575,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626487,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":897024,"data_stored":431815,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855419,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":974848,"data_stored":1024281,"data_compressed":41575,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018944,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":897024,"data_stored":432069,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247878,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":909312,"data_stored":431523,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444107,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":978944,"data_stored":492159,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":958464,"data_stored":491939,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":917504,"data_stored":433154,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T02:01:54.417 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-16T02:01:54.417 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-16T02:01:54.417 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-16T02:01:54.419 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-16T02:01:54.669 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:54.723 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:54 smithi105 ceph-mon[31197]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s; 58 B/s, 0 objects/s recovering 2024-09-16T02:01:54.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:54 smithi190 ceph-mon[34762]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s; 58 B/s, 0 objects/s recovering 2024-09-16T02:01:54.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:54 smithi137 ceph-mon[34099]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s; 58 B/s, 0 objects/s recovering 2024-09-16T02:01:55.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.181+0000 7f9a10861640 1 -- 172.21.15.105:0/1259897440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a0c102ef0 msgr2=0x7f9a0c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:55.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.181+0000 7f9a10861640 1 --2- 172.21.15.105:0/1259897440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a0c102ef0 0x7f9a0c10da40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9a04009ca0 tx=0x7f9a0402a950 comp rx=0 tx=0).stop 2024-09-16T02:01:55.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.181+0000 7f9a10861640 1 -- 172.21.15.105:0/1259897440 shutdown_connections 2024-09-16T02:01:55.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.181+0000 7f9a10861640 1 --2- 172.21.15.105:0/1259897440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a0c102570 0x7f9a0c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:55.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.181+0000 7f9a10861640 1 --2- 172.21.15.105:0/1259897440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a0c102ef0 0x7f9a0c10da40 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:55.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.181+0000 7f9a10861640 1 --2- 172.21.15.105:0/1259897440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a0c100e20 0x7f9a0c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:55.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.181+0000 7f9a10861640 1 -- 172.21.15.105:0/1259897440 >> 172.21.15.105:0/1259897440 conn(0x7f9a0c0fc7d0 msgr2=0x7f9a0c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:55.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.181+0000 7f9a10861640 1 -- 172.21.15.105:0/1259897440 shutdown_connections 2024-09-16T02:01:55.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.181+0000 7f9a10861640 1 -- 172.21.15.105:0/1259897440 wait complete. 2024-09-16T02:01:55.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.182+0000 7f9a10861640 1 Processor -- start 2024-09-16T02:01:55.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.182+0000 7f9a10861640 1 -- start start 2024-09-16T02:01:55.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.182+0000 7f9a10861640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a0c100e20 0x7f9a0c199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:55.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.182+0000 7f9a10861640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a0c102570 0x7f9a0c1996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:55.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.182+0000 7f9a10861640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a0c102ef0 0x7f9a0c1a4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:55.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.182+0000 7f9a10861640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0c110e30 con 0x7f9a0c102570 2024-09-16T02:01:55.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.182+0000 7f9a10861640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0c110cb0 con 0x7f9a0c100e20 2024-09-16T02:01:55.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.182+0000 7f9a10861640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0c110fb0 con 0x7f9a0c102ef0 2024-09-16T02:01:55.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.183+0000 7f9a01d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a0c102570 0x7f9a0c1996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:55.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.183+0000 7f9a01d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a0c102570 0x7f9a0c1996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54820/0 (socket says 172.21.15.105:54820) 2024-09-16T02:01:55.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.183+0000 7f9a01d9b640 1 -- 172.21.15.105:0/1658370625 learned_addr learned my addr 172.21.15.105:0/1658370625 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:01:55.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.183+0000 7f9a0a59c640 1 --2- 172.21.15.105:0/1658370625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a0c100e20 0x7f9a0c199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:55.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.183+0000 7f9a01d9b640 1 -- 172.21.15.105:0/1658370625 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a0c102ef0 msgr2=0x7f9a0c1a4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:55.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.183+0000 7f9a0ad9d640 1 --2- 172.21.15.105:0/1658370625 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a0c102ef0 0x7f9a0c1a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:55.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.183+0000 7f9a01d9b640 1 --2- 172.21.15.105:0/1658370625 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a0c102ef0 0x7f9a0c1a4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:55.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.183+0000 7f9a01d9b640 1 -- 172.21.15.105:0/1658370625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a0c100e20 msgr2=0x7f9a0c199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:55.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.183+0000 7f9a01d9b640 1 --2- 172.21.15.105:0/1658370625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a0c100e20 0x7f9a0c199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:55.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.183+0000 7f9a01d9b640 1 -- 172.21.15.105:0/1658370625 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a0c1a4740 con 0x7f9a0c102570 2024-09-16T02:01:55.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.183+0000 7f9a0a59c640 1 --2- 172.21.15.105:0/1658370625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a0c100e20 0x7f9a0c199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:01:55.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.183+0000 7f9a01d9b640 1 --2- 172.21.15.105:0/1658370625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a0c102570 0x7f9a0c1996c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f99f400c920 tx=0x7f99f400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:55.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.183+0000 7f9a03fff640 1 -- 172.21.15.105:0/1658370625 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f4007c70 con 0x7f9a0c102570 2024-09-16T02:01:55.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.183+0000 7f9a10861640 1 -- 172.21.15.105:0/1658370625 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a0c1a4910 con 0x7f9a0c102570 2024-09-16T02:01:55.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.183+0000 7f9a03fff640 1 -- 172.21.15.105:0/1658370625 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99f4007e10 con 0x7f9a0c102570 2024-09-16T02:01:55.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.183+0000 7f9a03fff640 1 -- 172.21.15.105:0/1658370625 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f4018820 con 0x7f9a0c102570 2024-09-16T02:01:55.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.183+0000 7f9a10861640 1 -- 172.21.15.105:0/1658370625 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a0c1a4e50 con 0x7f9a0c102570 2024-09-16T02:01:55.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.184+0000 7f9a10861640 1 -- 172.21.15.105:0/1658370625 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a0c101200 con 0x7f9a0c102570 2024-09-16T02:01:55.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.185+0000 7f9a03fff640 1 -- 172.21.15.105:0/1658370625 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f99f40189c0 con 0x7f9a0c102570 2024-09-16T02:01:55.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.186+0000 7f9a03fff640 1 --2- 172.21.15.105:0/1658370625 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f99f80a32d0 0x7f99f80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:55.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.186+0000 7f9a0a59c640 1 --2- 172.21.15.105:0/1658370625 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f99f80a32d0 0x7f99f80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:55.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.186+0000 7f9a03fff640 1 -- 172.21.15.105:0/1658370625 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f99f40cfc80 con 0x7f9a0c102570 2024-09-16T02:01:55.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.186+0000 7f9a0a59c640 1 --2- 172.21.15.105:0/1658370625 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f99f80a32d0 0x7f99f80a5790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f99fc009b40 tx=0x7f99fc005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:55.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.190+0000 7f9a03fff640 1 -- 172.21.15.105:0/1658370625 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f99f4014030 con 0x7f9a0c102570 2024-09-16T02:01:55.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.392+0000 7f9a10861640 1 -- 172.21.15.105:0/1658370625 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f9a0c1a50e0 con 0x7f9a0c102570 2024-09-16T02:01:55.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.392+0000 7f9a03fff640 1 -- 172.21.15.105:0/1658370625 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f99f409c820 con 0x7f9a0c102570 2024-09-16T02:01:55.394 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:01:55.394 INFO:teuthology.orchestra.run.smithi105.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-16T02:01:55.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.395+0000 7f9a10861640 1 -- 172.21.15.105:0/1658370625 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f99f80a32d0 msgr2=0x7f99f80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:55.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.395+0000 7f9a10861640 1 --2- 172.21.15.105:0/1658370625 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f99f80a32d0 0x7f99f80a5790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f99fc009b40 tx=0x7f99fc005e50 comp rx=0 tx=0).stop 2024-09-16T02:01:55.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.395+0000 7f9a10861640 1 -- 172.21.15.105:0/1658370625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a0c102570 msgr2=0x7f9a0c1996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:55.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.395+0000 7f9a10861640 1 --2- 172.21.15.105:0/1658370625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a0c102570 0x7f9a0c1996c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f99f400c920 tx=0x7f99f400cde0 comp rx=0 tx=0).stop 2024-09-16T02:01:55.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.395+0000 7f9a10861640 1 -- 172.21.15.105:0/1658370625 shutdown_connections 2024-09-16T02:01:55.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.395+0000 7f9a10861640 1 --2- 172.21.15.105:0/1658370625 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f99f80a32d0 0x7f99f80a5790 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:55.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.395+0000 7f9a10861640 1 --2- 172.21.15.105:0/1658370625 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a0c102ef0 0x7f9a0c1a4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:55.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.395+0000 7f9a10861640 1 --2- 172.21.15.105:0/1658370625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a0c102570 0x7f9a0c1996c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:55.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.395+0000 7f9a10861640 1 --2- 172.21.15.105:0/1658370625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a0c100e20 0x7f9a0c199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:55.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.395+0000 7f9a10861640 1 -- 172.21.15.105:0/1658370625 >> 172.21.15.105:0/1658370625 conn(0x7f9a0c0fc7d0 msgr2=0x7f9a0c106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:55.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.396+0000 7f9a10861640 1 -- 172.21.15.105:0/1658370625 shutdown_connections 2024-09-16T02:01:55.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:55.396+0000 7f9a10861640 1 -- 172.21.15.105:0/1658370625 wait complete. 2024-09-16T02:01:55.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:55 smithi105 ceph-mon[31197]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:01:55.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:55 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/1658370625' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-16T02:01:55.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:55 smithi190 ceph-mon[34762]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:01:55.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:55 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/1658370625' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-16T02:01:55.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:55 smithi137 ceph-mon[34099]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:01:55.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:55 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/1658370625' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-16T02:01:56.104 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-16T02:01:56.105 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:01:56.350 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:56.780 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:56 smithi105 ceph-mon[31197]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-09-16T02:01:56.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:56 smithi190 ceph-mon[34762]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-09-16T02:01:56.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:56.857+0000 7f31b8ce1640 1 -- 172.21.15.105:0/2897075825 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31b4106f10 msgr2=0x7f31b410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:56.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:56.857+0000 7f31b8ce1640 1 --2- 172.21.15.105:0/2897075825 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31b4106f10 0x7f31b410ea90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f31ac009ca0 tx=0x7f31ac02a8c0 comp rx=0 tx=0).stop 2024-09-16T02:01:56.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:56.857+0000 7f31b8ce1640 1 -- 172.21.15.105:0/2897075825 shutdown_connections 2024-09-16T02:01:56.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:56.857+0000 7f31b8ce1640 1 --2- 172.21.15.105:0/2897075825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31b4102790 0x7f31b41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:56.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:56.857+0000 7f31b8ce1640 1 --2- 172.21.15.105:0/2897075825 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31b4106f10 0x7f31b410ea90 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:56.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:56.857+0000 7f31b8ce1640 1 --2- 172.21.15.105:0/2897075825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f31b4101de0 0x7f31b41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:56.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:56.857+0000 7f31b8ce1640 1 -- 172.21.15.105:0/2897075825 >> 172.21.15.105:0/2897075825 conn(0x7f31b40fd8c0 msgr2=0x7f31b40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:56.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:56.857+0000 7f31b8ce1640 1 -- 172.21.15.105:0/2897075825 shutdown_connections 2024-09-16T02:01:56.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:56.857+0000 7f31b8ce1640 1 -- 172.21.15.105:0/2897075825 wait complete. 2024-09-16T02:01:56.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:56.858+0000 7f31b8ce1640 1 Processor -- start 2024-09-16T02:01:56.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:56.858+0000 7f31b8ce1640 1 -- start start 2024-09-16T02:01:56.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:56.858+0000 7f31b8ce1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f31b4101de0 0x7f31b40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:56.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:56.858+0000 7f31b8ce1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31b4102790 0x7f31b4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:56.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:56.858+0000 7f31b8ce1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31b4106f10 0x7f31b4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:56.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:56.858+0000 7f31b8ce1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f31b4111ec0 con 0x7f31b4102790 2024-09-16T02:01:56.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:56.858+0000 7f31b8ce1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f31b4111d40 con 0x7f31b4101de0 2024-09-16T02:01:56.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:56.858+0000 7f31b8ce1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f31b4112040 con 0x7f31b4106f10 2024-09-16T02:01:56.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:56.859+0000 7f31b1d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31b4102790 0x7f31b4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:56.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:56.859+0000 7f31b1d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31b4102790 0x7f31b4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54848/0 (socket says 172.21.15.105:54848) 2024-09-16T02:01:56.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:56.859+0000 7f31b1d9b640 1 -- 172.21.15.105:0/1372101967 learned_addr learned my addr 172.21.15.105:0/1372101967 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:01:56.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:56.859+0000 7f31b259c640 1 --2- 172.21.15.105:0/1372101967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f31b4101de0 0x7f31b40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:56.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:56.859+0000 7f31b1d9b640 1 -- 172.21.15.105:0/1372101967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31b4106f10 msgr2=0x7f31b4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:56.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:56.859+0000 7f31b1d9b640 1 --2- 172.21.15.105:0/1372101967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31b4106f10 0x7f31b4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:56.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:56.859+0000 7f31b1d9b640 1 -- 172.21.15.105:0/1372101967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f31b4101de0 msgr2=0x7f31b40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:56.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:56.859+0000 7f31b1d9b640 1 --2- 172.21.15.105:0/1372101967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f31b4101de0 0x7f31b40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:56.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:56.859+0000 7f31b1d9b640 1 -- 172.21.15.105:0/1372101967 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31b41a6940 con 0x7f31b4102790 2024-09-16T02:01:56.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:56.859+0000 7f31b1d9b640 1 --2- 172.21.15.105:0/1372101967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31b4102790 0x7f31b4074a30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f31a800b4a0 tx=0x7f31a800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:56.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:56.859+0000 7f31a37fe640 1 -- 172.21.15.105:0/1372101967 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a8015020 con 0x7f31b4102790 2024-09-16T02:01:56.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:56.859+0000 7f31a37fe640 1 -- 172.21.15.105:0/1372101967 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f31a8011ce0 con 0x7f31b4102790 2024-09-16T02:01:56.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:56.859+0000 7f31b8ce1640 1 -- 172.21.15.105:0/1372101967 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31b41a6c30 con 0x7f31b4102790 2024-09-16T02:01:56.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:56.859+0000 7f31a37fe640 1 -- 172.21.15.105:0/1372101967 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a8010dc0 con 0x7f31b4102790 2024-09-16T02:01:56.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:56.859+0000 7f31b8ce1640 1 -- 172.21.15.105:0/1372101967 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31b41a7120 con 0x7f31b4102790 2024-09-16T02:01:56.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:56.860+0000 7f31b8ce1640 1 -- 172.21.15.105:0/1372101967 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31b41021c0 con 0x7f31b4102790 2024-09-16T02:01:56.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:56.861+0000 7f31a37fe640 1 -- 172.21.15.105:0/1372101967 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f31a8018020 con 0x7f31b4102790 2024-09-16T02:01:56.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:56.862+0000 7f31a37fe640 1 --2- 172.21.15.105:0/1372101967 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f319c0a32d0 0x7f319c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:56.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:56.862+0000 7f31b259c640 1 --2- 172.21.15.105:0/1372101967 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f319c0a32d0 0x7f319c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:56.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:56.862+0000 7f31a37fe640 1 -- 172.21.15.105:0/1372101967 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f31a80cf870 con 0x7f31b4102790 2024-09-16T02:01:56.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:56.863+0000 7f31b259c640 1 --2- 172.21.15.105:0/1372101967 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f319c0a32d0 0x7f319c0a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f31a4009990 tx=0x7f31a4009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:56.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:56.865+0000 7f31a37fe640 1 -- 172.21.15.105:0/1372101967 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f31a801f020 con 0x7f31b4102790 2024-09-16T02:01:56.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:56 smithi137 ceph-mon[34099]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-09-16T02:01:57.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:57.067+0000 7f31b8ce1640 1 -- 172.21.15.105:0/1372101967 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31b4075a20 con 0x7f31b4102790 2024-09-16T02:01:57.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:57.067+0000 7f31a37fe640 1 -- 172.21.15.105:0/1372101967 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11991 (secure 0 0 0) 0x7f31b4075a20 con 0x7f31b4102790 2024-09-16T02:01:57.069 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:01:57.070 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":27,"btime":"2024-09-16T02:00:47:526725+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2057699920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2057699920},{"type":"v1","addr":"172.21.15.105:6835","nonce":2057699920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1764886948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1764886948},{"type":"v1","addr":"172.21.15.105:6837","nonce":1764886948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T02:00:35.112266+0000","modified":"2024-09-16T02:00:47.526704+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24469,"mds_2":24461,"mds_3":24457,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/4112062361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":4112062361},{"type":"v1","addr":"172.21.15.105:6839","nonce":4112062361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6841/163156514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":163156514},{"type":"v1","addr":"172.21.15.105:6841","nonce":163156514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/683519097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":683519097},{"type":"v1","addr":"172.21.15.190:6833","nonce":683519097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3025442862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3025442862},{"type":"v1","addr":"172.21.15.137:6833","nonce":3025442862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/763783602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":763783602},{"type":"v1","addr":"172.21.15.137:6835","nonce":763783602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.190:6835/1851042281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1851042281},{"type":"v1","addr":"172.21.15.190:6835","nonce":1851042281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/3181257549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3181257549},{"type":"v1","addr":"172.21.15.190:6837","nonce":3181257549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6837/2788541030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2788541030},{"type":"v1","addr":"172.21.15.137:6837","nonce":2788541030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.190:6839/1222573537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":1222573537},{"type":"v1","addr":"172.21.15.190:6839","nonce":1222573537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/3454924009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3454924009},{"type":"v1","addr":"172.21.15.137:6839","nonce":3454924009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24469,24461,24457,24445]},"id":1}]} 2024-09-16T02:01:57.071 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 27 2024-09-16T02:01:57.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:57.069+0000 7f31b8ce1640 1 -- 172.21.15.105:0/1372101967 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f319c0a32d0 msgr2=0x7f319c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:57.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:57.069+0000 7f31b8ce1640 1 --2- 172.21.15.105:0/1372101967 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f319c0a32d0 0x7f319c0a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f31a4009990 tx=0x7f31a4009110 comp rx=0 tx=0).stop 2024-09-16T02:01:57.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:57.069+0000 7f31b8ce1640 1 -- 172.21.15.105:0/1372101967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31b4102790 msgr2=0x7f31b4074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:57.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:57.069+0000 7f31b8ce1640 1 --2- 172.21.15.105:0/1372101967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31b4102790 0x7f31b4074a30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f31a800b4a0 tx=0x7f31a800b8a0 comp rx=0 tx=0).stop 2024-09-16T02:01:57.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:57.069+0000 7f31b8ce1640 1 -- 172.21.15.105:0/1372101967 shutdown_connections 2024-09-16T02:01:57.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:57.069+0000 7f31b8ce1640 1 --2- 172.21.15.105:0/1372101967 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f319c0a32d0 0x7f319c0a5790 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:57.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:57.069+0000 7f31b8ce1640 1 --2- 172.21.15.105:0/1372101967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31b4106f10 0x7f31b4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:57.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:57.069+0000 7f31b8ce1640 1 --2- 172.21.15.105:0/1372101967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31b4102790 0x7f31b4074a30 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:57.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:57.069+0000 7f31b8ce1640 1 --2- 172.21.15.105:0/1372101967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f31b4101de0 0x7f31b40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:57.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:57.069+0000 7f31b8ce1640 1 -- 172.21.15.105:0/1372101967 >> 172.21.15.105:0/1372101967 conn(0x7f31b40fd8c0 msgr2=0x7f31b40fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:57.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:57.070+0000 7f31b8ce1640 1 -- 172.21.15.105:0/1372101967 shutdown_connections 2024-09-16T02:01:57.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:57.070+0000 7f31b8ce1640 1 -- 172.21.15.105:0/1372101967 wait complete. 2024-09-16T02:01:57.761 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:57 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/1372101967' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:01:57.793 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:01:57.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:57 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/1372101967' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:01:57.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:57 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/1372101967' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:01:58.041 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:58.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.500+0000 7fa3b8899640 1 -- 172.21.15.105:0/1470167621 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3b4102570 msgr2=0x7fa3b41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:58.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.500+0000 7fa3b8899640 1 --2- 172.21.15.105:0/1470167621 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3b4102570 0x7fa3b41029b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa39c009900 tx=0x7fa39c02a480 comp rx=0 tx=0).stop 2024-09-16T02:01:58.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.500+0000 7fa3b8899640 1 -- 172.21.15.105:0/1470167621 shutdown_connections 2024-09-16T02:01:58.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.500+0000 7fa3b8899640 1 --2- 172.21.15.105:0/1470167621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3b4102ef0 0x7fa3b410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:58.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.500+0000 7fa3b8899640 1 --2- 172.21.15.105:0/1470167621 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3b4102570 0x7fa3b41029b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:58.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.500+0000 7fa3b8899640 1 --2- 172.21.15.105:0/1470167621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3b4100e20 0x7fa3b4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:58.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.500+0000 7fa3b8899640 1 -- 172.21.15.105:0/1470167621 >> 172.21.15.105:0/1470167621 conn(0x7fa3b40fc7d0 msgr2=0x7fa3b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:58.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.500+0000 7fa3b8899640 1 -- 172.21.15.105:0/1470167621 shutdown_connections 2024-09-16T02:01:58.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.500+0000 7fa3b8899640 1 -- 172.21.15.105:0/1470167621 wait complete. 2024-09-16T02:01:58.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.501+0000 7fa3b8899640 1 Processor -- start 2024-09-16T02:01:58.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.501+0000 7fa3b8899640 1 -- start start 2024-09-16T02:01:58.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.501+0000 7fa3b8899640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3b4100e20 0x7fa3b4197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:58.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.501+0000 7fa3b8899640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3b4102570 0x7fa3b4197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:58.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.501+0000 7fa3b8899640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3b4102ef0 0x7fa3b419bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:58.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.501+0000 7fa3b8899640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b4110e30 con 0x7fa3b4102ef0 2024-09-16T02:01:58.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.501+0000 7fa3b8899640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b4110cb0 con 0x7fa3b4100e20 2024-09-16T02:01:58.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.501+0000 7fa3b8899640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b4110fb0 con 0x7fa3b4102570 2024-09-16T02:01:58.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.501+0000 7fa3b2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3b4102ef0 0x7fa3b419bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:58.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.501+0000 7fa3b2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3b4102ef0 0x7fa3b419bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57650/0 (socket says 172.21.15.105:57650) 2024-09-16T02:01:58.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.501+0000 7fa3b2d9d640 1 -- 172.21.15.105:0/3606901793 learned_addr learned my addr 172.21.15.105:0/3606901793 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:01:58.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.501+0000 7fa3a9d9b640 1 --2- 172.21.15.105:0/3606901793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3b4102570 0x7fa3b4197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:58.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.501+0000 7fa3b259c640 1 --2- 172.21.15.105:0/3606901793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3b4100e20 0x7fa3b4197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:58.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.502+0000 7fa3b2d9d640 1 -- 172.21.15.105:0/3606901793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3b4102570 msgr2=0x7fa3b4197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:58.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.502+0000 7fa3b2d9d640 1 --2- 172.21.15.105:0/3606901793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3b4102570 0x7fa3b4197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:58.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.502+0000 7fa3b2d9d640 1 -- 172.21.15.105:0/3606901793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3b4100e20 msgr2=0x7fa3b4197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:58.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.502+0000 7fa3b2d9d640 1 --2- 172.21.15.105:0/3606901793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3b4100e20 0x7fa3b4197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:58.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.502+0000 7fa3b2d9d640 1 -- 172.21.15.105:0/3606901793 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3b419c2b0 con 0x7fa3b4102ef0 2024-09-16T02:01:58.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.502+0000 7fa3b2d9d640 1 --2- 172.21.15.105:0/3606901793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3b4102ef0 0x7fa3b419bbd0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00eaa0 tx=0x7fa3ac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:58.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.502+0000 7fa3abfff640 1 -- 172.21.15.105:0/3606901793 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac00cdb0 con 0x7fa3b4102ef0 2024-09-16T02:01:58.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.502+0000 7fa3b8899640 1 -- 172.21.15.105:0/3606901793 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3b419c5a0 con 0x7fa3b4102ef0 2024-09-16T02:01:58.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.502+0000 7fa3abfff640 1 -- 172.21.15.105:0/3606901793 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3ac005020 con 0x7fa3b4102ef0 2024-09-16T02:01:58.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.502+0000 7fa3abfff640 1 -- 172.21.15.105:0/3606901793 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac0116d0 con 0x7fa3b4102ef0 2024-09-16T02:01:58.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.502+0000 7fa3b8899640 1 -- 172.21.15.105:0/3606901793 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3b4069f70 con 0x7fa3b4102ef0 2024-09-16T02:01:58.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.503+0000 7fa3b8899640 1 -- 172.21.15.105:0/3606901793 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3b41021b0 con 0x7fa3b4102ef0 2024-09-16T02:01:58.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.504+0000 7fa3abfff640 1 -- 172.21.15.105:0/3606901793 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fa3ac0051d0 con 0x7fa3b4102ef0 2024-09-16T02:01:58.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.505+0000 7fa3abfff640 1 --2- 172.21.15.105:0/3606901793 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa3a00a32d0 0x7fa3a00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:58.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.505+0000 7fa3b259c640 1 --2- 172.21.15.105:0/3606901793 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa3a00a32d0 0x7fa3a00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:58.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.505+0000 7fa3abfff640 1 -- 172.21.15.105:0/3606901793 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fa3ac0cfbc0 con 0x7fa3b4102ef0 2024-09-16T02:01:58.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.505+0000 7fa3b259c640 1 --2- 172.21.15.105:0/3606901793 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa3a00a32d0 0x7fa3a00a5790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa3a400bbe0 tx=0x7fa3a400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:58.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.509+0000 7fa3abfff640 1 -- 172.21.15.105:0/3606901793 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa3ac016030 con 0x7fa3b4102ef0 2024-09-16T02:01:58.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.713+0000 7fa3b8899640 1 -- 172.21.15.105:0/3606901793 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3b406a9f0 con 0x7fa3b4102ef0 2024-09-16T02:01:58.715 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:58 smithi105 ceph-mon[31197]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-09-16T02:01:58.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.714+0000 7fa3abfff640 1 -- 172.21.15.105:0/3606901793 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11991 (secure 0 0 0) 0x7fa3ac09c110 con 0x7fa3b4102ef0 2024-09-16T02:01:58.715 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:01:58.715 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":27,"btime":"2024-09-16T02:00:47:526725+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2057699920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2057699920},{"type":"v1","addr":"172.21.15.105:6835","nonce":2057699920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1764886948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1764886948},{"type":"v1","addr":"172.21.15.105:6837","nonce":1764886948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T02:00:35.112266+0000","modified":"2024-09-16T02:00:47.526704+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24469,"mds_2":24461,"mds_3":24457,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/4112062361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":4112062361},{"type":"v1","addr":"172.21.15.105:6839","nonce":4112062361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6841/163156514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":163156514},{"type":"v1","addr":"172.21.15.105:6841","nonce":163156514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/683519097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":683519097},{"type":"v1","addr":"172.21.15.190:6833","nonce":683519097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3025442862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3025442862},{"type":"v1","addr":"172.21.15.137:6833","nonce":3025442862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/763783602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":763783602},{"type":"v1","addr":"172.21.15.137:6835","nonce":763783602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.190:6835/1851042281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1851042281},{"type":"v1","addr":"172.21.15.190:6835","nonce":1851042281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/3181257549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3181257549},{"type":"v1","addr":"172.21.15.190:6837","nonce":3181257549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6837/2788541030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2788541030},{"type":"v1","addr":"172.21.15.137:6837","nonce":2788541030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.190:6839/1222573537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":1222573537},{"type":"v1","addr":"172.21.15.190:6839","nonce":1222573537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/3454924009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3454924009},{"type":"v1","addr":"172.21.15.137:6839","nonce":3454924009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24469,24461,24457,24445]},"id":1}]} 2024-09-16T02:01:58.716 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 27 2024-09-16T02:01:58.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.716+0000 7fa3b8899640 1 -- 172.21.15.105:0/3606901793 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa3a00a32d0 msgr2=0x7fa3a00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:58.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.716+0000 7fa3b8899640 1 --2- 172.21.15.105:0/3606901793 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa3a00a32d0 0x7fa3a00a5790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa3a400bbe0 tx=0x7fa3a400b390 comp rx=0 tx=0).stop 2024-09-16T02:01:58.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.716+0000 7fa3b8899640 1 -- 172.21.15.105:0/3606901793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3b4102ef0 msgr2=0x7fa3b419bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:58.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.716+0000 7fa3b8899640 1 --2- 172.21.15.105:0/3606901793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3b4102ef0 0x7fa3b419bbd0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00eaa0 tx=0x7fa3ac00ef60 comp rx=0 tx=0).stop 2024-09-16T02:01:58.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.716+0000 7fa3b8899640 1 -- 172.21.15.105:0/3606901793 shutdown_connections 2024-09-16T02:01:58.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.716+0000 7fa3b8899640 1 --2- 172.21.15.105:0/3606901793 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa3a00a32d0 0x7fa3a00a5790 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:58.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.716+0000 7fa3b8899640 1 --2- 172.21.15.105:0/3606901793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3b4102570 0x7fa3b4197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:58.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.716+0000 7fa3b8899640 1 --2- 172.21.15.105:0/3606901793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3b4102ef0 0x7fa3b419bbd0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:58.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.716+0000 7fa3b8899640 1 --2- 172.21.15.105:0/3606901793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3b4100e20 0x7fa3b4197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:58.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.716+0000 7fa3b8899640 1 -- 172.21.15.105:0/3606901793 >> 172.21.15.105:0/3606901793 conn(0x7fa3b40fc7d0 msgr2=0x7fa3b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:58.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.716+0000 7fa3b8899640 1 -- 172.21.15.105:0/3606901793 shutdown_connections 2024-09-16T02:01:58.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:01:58.716+0000 7fa3b8899640 1 -- 172.21.15.105:0/3606901793 wait complete. 2024-09-16T02:01:58.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:58 smithi190 ceph-mon[34762]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-09-16T02:01:58.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:58 smithi137 ceph-mon[34099]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-09-16T02:01:59.385 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T02:01:59.634 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:02:00.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:01:59 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/3606901793' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:02:00.646 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:01:59 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/3606901793' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:02:00.646 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:01:59 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/3606901793' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:02:00.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.117+0000 7f184472f640 1 -- 172.21.15.105:0/3590910222 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f183c1027c0 msgr2=0x7f183c106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:00.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.117+0000 7f184472f640 1 --2- 172.21.15.105:0/3590910222 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f183c1027c0 0x7f183c106810 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1830009900 tx=0x7f183002a410 comp rx=0 tx=0).stop 2024-09-16T02:02:00.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.117+0000 7f184472f640 1 -- 172.21.15.105:0/3590910222 shutdown_connections 2024-09-16T02:02:00.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.117+0000 7f184472f640 1 --2- 172.21.15.105:0/3590910222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f183c106f40 0x7f183c10eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:00.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.117+0000 7f184472f640 1 --2- 172.21.15.105:0/3590910222 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f183c1027c0 0x7f183c106810 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:00.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.117+0000 7f184472f640 1 --2- 172.21.15.105:0/3590910222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f183c101e10 0x7f183c1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:00.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.118+0000 7f184472f640 1 -- 172.21.15.105:0/3590910222 >> 172.21.15.105:0/3590910222 conn(0x7f183c0fd8d0 msgr2=0x7f183c0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:00.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.118+0000 7f184472f640 1 -- 172.21.15.105:0/3590910222 shutdown_connections 2024-09-16T02:02:00.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.118+0000 7f184472f640 1 -- 172.21.15.105:0/3590910222 wait complete. 2024-09-16T02:02:00.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.118+0000 7f184472f640 1 Processor -- start 2024-09-16T02:02:00.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.118+0000 7f184472f640 1 -- start start 2024-09-16T02:02:00.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.118+0000 7f184472f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f183c101e10 0x7f183c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:00.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.118+0000 7f184472f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f183c1027c0 0x7f183c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:00.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.118+0000 7f184472f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f183c106f40 0x7f183c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:00.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.118+0000 7f184472f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f183c111e70 con 0x7f183c106f40 2024-09-16T02:02:00.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.118+0000 7f184472f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f183c111cf0 con 0x7f183c1027c0 2024-09-16T02:02:00.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.118+0000 7f184472f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f183c111ff0 con 0x7f183c101e10 2024-09-16T02:02:00.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.118+0000 7f1842ccc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f183c106f40 0x7f183c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:00.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.119+0000 7f1842ccc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f183c106f40 0x7f183c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57660/0 (socket says 172.21.15.105:57660) 2024-09-16T02:02:00.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.119+0000 7f1842ccc640 1 -- 172.21.15.105:0/1576502892 learned_addr learned my addr 172.21.15.105:0/1576502892 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:00.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.119+0000 7f1841cca640 1 --2- 172.21.15.105:0/1576502892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f183c1027c0 0x7f183c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:00.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.119+0000 7f18424cb640 1 --2- 172.21.15.105:0/1576502892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f183c101e10 0x7f183c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:00.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.119+0000 7f1842ccc640 1 -- 172.21.15.105:0/1576502892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f183c101e10 msgr2=0x7f183c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:00.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.119+0000 7f1842ccc640 1 --2- 172.21.15.105:0/1576502892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f183c101e10 0x7f183c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:00.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.119+0000 7f1842ccc640 1 -- 172.21.15.105:0/1576502892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f183c1027c0 msgr2=0x7f183c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:00.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.119+0000 7f1842ccc640 1 --2- 172.21.15.105:0/1576502892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f183c1027c0 0x7f183c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:00.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.119+0000 7f1842ccc640 1 -- 172.21.15.105:0/1576502892 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f183c1a6850 con 0x7f183c106f40 2024-09-16T02:02:00.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.119+0000 7f1842ccc640 1 --2- 172.21.15.105:0/1576502892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f183c106f40 0x7f183c074f70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1834009fd0 tx=0x7f183400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:00.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.119+0000 7f183b7fe640 1 -- 172.21.15.105:0/1576502892 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1834011830 con 0x7f183c106f40 2024-09-16T02:02:00.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.119+0000 7f183b7fe640 1 -- 172.21.15.105:0/1576502892 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1834016070 con 0x7f183c106f40 2024-09-16T02:02:00.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.119+0000 7f184472f640 1 -- 172.21.15.105:0/1576502892 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f183c1a6b40 con 0x7f183c106f40 2024-09-16T02:02:00.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.119+0000 7f183b7fe640 1 -- 172.21.15.105:0/1576502892 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1834010de0 con 0x7f183c106f40 2024-09-16T02:02:00.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.119+0000 7f184472f640 1 -- 172.21.15.105:0/1576502892 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f183c1a70a0 con 0x7f183c106f40 2024-09-16T02:02:00.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.120+0000 7f184472f640 1 -- 172.21.15.105:0/1576502892 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f183c1021f0 con 0x7f183c106f40 2024-09-16T02:02:00.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.121+0000 7f183b7fe640 1 -- 172.21.15.105:0/1576502892 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f18340119d0 con 0x7f183c106f40 2024-09-16T02:02:00.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.122+0000 7f183b7fe640 1 --2- 172.21.15.105:0/1576502892 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f18240a3350 0x7f18240a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:00.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.122+0000 7f18424cb640 1 --2- 172.21.15.105:0/1576502892 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f18240a3350 0x7f18240a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:00.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.122+0000 7f183b7fe640 1 -- 172.21.15.105:0/1576502892 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f18340cf6d0 con 0x7f183c106f40 2024-09-16T02:02:00.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.122+0000 7f18424cb640 1 --2- 172.21.15.105:0/1576502892 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f18240a3350 0x7f18240a5810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f182c007350 tx=0x7f182c0072e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:00.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.125+0000 7f183b7fe640 1 -- 172.21.15.105:0/1576502892 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f183409c380 con 0x7f183c106f40 2024-09-16T02:02:00.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.294+0000 7f184472f640 1 -- 172.21.15.105:0/1576502892 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f183c075a20 con 0x7f183c106f40 2024-09-16T02:02:00.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.295+0000 7f183b7fe640 1 -- 172.21.15.105:0/1576502892 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22850 (secure 0 0 0) 0x7f183c075a20 con 0x7f183c106f40 2024-09-16T02:02:00.651 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:02:00.651 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":81,"fsid":"423b7a2a-73ce-11ef-bceb-c7b262605968","created":"2024-09-16T01:52:44.388033+0000","modified":"2024-09-16T02:01:40.556339+0000","last_up_change":"2024-09-16T01:59:42.218034+0000","last_in_change":"2024-09-16T01:59:18.785252+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T01:55:53.680808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T02:00:31.286117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T02:00:33.310045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a60d6c72-da8b-4b02-bb33-672be795e7f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6803","nonce":1226660290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6805","nonce":1226660290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6809","nonce":1226660290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6807","nonce":1226660290}]},"public_addr":"172.21.15.105:6803/1226660290","cluster_addr":"172.21.15.105:6805/1226660290","heartbeat_back_addr":"172.21.15.105:6809/1226660290","heartbeat_front_addr":"172.21.15.105:6807/1226660290","state":["exists","up"]},{"osd":1,"uuid":"b1d9ae84-b9a0-4a7e-b0fe-c402d46809fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6801","nonce":34753956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6803","nonce":34753956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6807","nonce":34753956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6805","nonce":34753956}]},"public_addr":"172.21.15.137:6801/34753956","cluster_addr":"172.21.15.137:6803/34753956","heartbeat_back_addr":"172.21.15.137:6807/34753956","heartbeat_front_addr":"172.21.15.137:6805/34753956","state":["exists","up"]},{"osd":2,"uuid":"137faae6-2e8b-44b5-8976-3d95a6eef1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6801","nonce":1159531918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6803","nonce":1159531918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6807","nonce":1159531918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6805","nonce":1159531918}]},"public_addr":"172.21.15.190:6801/1159531918","cluster_addr":"172.21.15.190:6803/1159531918","heartbeat_back_addr":"172.21.15.190:6807/1159531918","heartbeat_front_addr":"172.21.15.190:6805/1159531918","state":["exists","up"]},{"osd":3,"uuid":"5743c37b-5cd8-42b6-81c8-5cf5afc71e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6811","nonce":604905679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6813","nonce":604905679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6817","nonce":604905679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6815","nonce":604905679}]},"public_addr":"172.21.15.105:6811/604905679","cluster_addr":"172.21.15.105:6813/604905679","heartbeat_back_addr":"172.21.15.105:6817/604905679","heartbeat_front_addr":"172.21.15.105:6815/604905679","state":["exists","up"]},{"osd":4,"uuid":"e439d4a2-b54c-4de9-97f6-3510afe57f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6809","nonce":3758109169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6811","nonce":3758109169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6815","nonce":3758109169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6813","nonce":3758109169}]},"public_addr":"172.21.15.137:6809/3758109169","cluster_addr":"172.21.15.137:6811/3758109169","heartbeat_back_addr":"172.21.15.137:6815/3758109169","heartbeat_front_addr":"172.21.15.137:6813/3758109169","state":["exists","up"]},{"osd":5,"uuid":"d1e2a7d5-99f8-40fa-af6e-0305cc0af677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6809","nonce":2086773754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6811","nonce":2086773754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6815","nonce":2086773754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6813","nonce":2086773754}]},"public_addr":"172.21.15.190:6809/2086773754","cluster_addr":"172.21.15.190:6811/2086773754","heartbeat_back_addr":"172.21.15.190:6815/2086773754","heartbeat_front_addr":"172.21.15.190:6813/2086773754","state":["exists","up"]},{"osd":6,"uuid":"cd418159-a0f6-44c7-9213-c010b132fb56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6819","nonce":4012018725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6821","nonce":4012018725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6825","nonce":4012018725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6823","nonce":4012018725}]},"public_addr":"172.21.15.105:6819/4012018725","cluster_addr":"172.21.15.105:6821/4012018725","heartbeat_back_addr":"172.21.15.105:6825/4012018725","heartbeat_front_addr":"172.21.15.105:6823/4012018725","state":["exists","up"]},{"osd":7,"uuid":"76c15863-801e-4a72-be73-ba9df46c15b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6817","nonce":888678288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6819","nonce":888678288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6823","nonce":888678288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6821","nonce":888678288}]},"public_addr":"172.21.15.137:6817/888678288","cluster_addr":"172.21.15.137:6819/888678288","heartbeat_back_addr":"172.21.15.137:6823/888678288","heartbeat_front_addr":"172.21.15.137:6821/888678288","state":["exists","up"]},{"osd":8,"uuid":"6718597c-6f42-48ee-9493-edb2396fe707","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6817","nonce":2132638341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6819","nonce":2132638341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6823","nonce":2132638341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6821","nonce":2132638341}]},"public_addr":"172.21.15.190:6817/2132638341","cluster_addr":"172.21.15.190:6819/2132638341","heartbeat_back_addr":"172.21.15.190:6823/2132638341","heartbeat_front_addr":"172.21.15.190:6821/2132638341","state":["exists","up"]},{"osd":9,"uuid":"5178edc0-a2e8-491c-821c-6f41ec5aeb2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6827","nonce":3375034979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6829","nonce":3375034979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6833","nonce":3375034979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6831","nonce":3375034979}]},"public_addr":"172.21.15.105:6827/3375034979","cluster_addr":"172.21.15.105:6829/3375034979","heartbeat_back_addr":"172.21.15.105:6833/3375034979","heartbeat_front_addr":"172.21.15.105:6831/3375034979","state":["exists","up"]},{"osd":10,"uuid":"ad145d3e-a6e2-4dd7-bea3-bcd01eb3413e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6825","nonce":318562234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6827","nonce":318562234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6831","nonce":318562234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6829","nonce":318562234}]},"public_addr":"172.21.15.137:6825/318562234","cluster_addr":"172.21.15.137:6827/318562234","heartbeat_back_addr":"172.21.15.137:6831/318562234","heartbeat_front_addr":"172.21.15.137:6829/318562234","state":["exists","up"]},{"osd":11,"uuid":"6d44bced-3a8b-4763-a93e-0ea34acb4cb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6825","nonce":3820460689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6827","nonce":3820460689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6831","nonce":3820460689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6829","nonce":3820460689}]},"public_addr":"172.21.15.190:6825/3820460689","cluster_addr":"172.21.15.190:6827/3820460689","heartbeat_back_addr":"172.21.15.190:6831/3820460689","heartbeat_front_addr":"172.21.15.190:6829/3820460689","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:55:02.759033+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:55:26.829142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:55:51.120200+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:56:17.277870+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:56:42.313701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:07.013416+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:32.477916+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:58.643764+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:23.454908+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:48.869082+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:14.921573+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:40.908993+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":7}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:0/2453739657":"2024-09-17T01:53:39.351314+0000","172.21.15.105:6801/1469189484":"2024-09-17T01:53:39.351314+0000","172.21.15.105:0/3466166879":"2024-09-17T01:53:39.351314+0000","172.21.15.105:6801/1268348742":"2024-09-17T01:53:15.522227+0000","172.21.15.105:0/2897248478":"2024-09-17T01:53:39.351314+0000","172.21.15.105:0/3709363371":"2024-09-17T01:53:15.522227+0000","172.21.15.105:6800/1268348742":"2024-09-17T01:53:15.522227+0000","172.21.15.105:6800/1469189484":"2024-09-17T01:53:39.351314+0000","172.21.15.105:0/1198176299":"2024-09-17T01:53:15.522227+0000","172.21.15.105:0/3031917848":"2024-09-17T01:53:15.522227+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T02:02:00.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.297+0000 7f184472f640 1 -- 172.21.15.105:0/1576502892 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f18240a3350 msgr2=0x7f18240a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:00.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.297+0000 7f184472f640 1 --2- 172.21.15.105:0/1576502892 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f18240a3350 0x7f18240a5810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f182c007350 tx=0x7f182c0072e0 comp rx=0 tx=0).stop 2024-09-16T02:02:00.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.297+0000 7f184472f640 1 -- 172.21.15.105:0/1576502892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f183c106f40 msgr2=0x7f183c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:00.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.297+0000 7f184472f640 1 --2- 172.21.15.105:0/1576502892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f183c106f40 0x7f183c074f70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1834009fd0 tx=0x7f183400b8e0 comp rx=0 tx=0).stop 2024-09-16T02:02:00.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.297+0000 7f184472f640 1 -- 172.21.15.105:0/1576502892 shutdown_connections 2024-09-16T02:02:00.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.297+0000 7f184472f640 1 --2- 172.21.15.105:0/1576502892 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f18240a3350 0x7f18240a5810 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:00.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.297+0000 7f184472f640 1 --2- 172.21.15.105:0/1576502892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f183c106f40 0x7f183c074f70 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:00.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.297+0000 7f184472f640 1 --2- 172.21.15.105:0/1576502892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f183c1027c0 0x7f183c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:00.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.297+0000 7f184472f640 1 --2- 172.21.15.105:0/1576502892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f183c101e10 0x7f183c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:00.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.297+0000 7f184472f640 1 -- 172.21.15.105:0/1576502892 >> 172.21.15.105:0/1576502892 conn(0x7f183c0fd8d0 msgr2=0x7f183c0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:00.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.297+0000 7f184472f640 1 -- 172.21.15.105:0/1576502892 shutdown_connections 2024-09-16T02:02:00.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:00.297+0000 7f184472f640 1 -- 172.21.15.105:0/1576502892 wait complete. 2024-09-16T02:02:00.781 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:00 smithi105 ceph-mon[31197]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-16T02:02:00.782 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:00 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/1576502892' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:02:00.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:00 smithi190 ceph-mon[34762]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-16T02:02:00.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:00 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/1576502892' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:02:00.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:00 smithi137 ceph-mon[34099]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-16T02:02:00.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:00 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/1576502892' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:02:00.999 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:02:01.244 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:02:01.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.735+0000 7f079e2d1640 1 -- 172.21.15.105:0/3334629438 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0798102790 msgr2=0x7f07981067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:01.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.735+0000 7f079e2d1640 1 --2- 172.21.15.105:0/3334629438 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0798102790 0x7f07981067e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f078c009900 tx=0x7f078c02a440 comp rx=0 tx=0).stop 2024-09-16T02:02:01.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.735+0000 7f079e2d1640 1 -- 172.21.15.105:0/3334629438 shutdown_connections 2024-09-16T02:02:01.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.735+0000 7f079e2d1640 1 --2- 172.21.15.105:0/3334629438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0798106f10 0x7f079810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:01.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.735+0000 7f079e2d1640 1 --2- 172.21.15.105:0/3334629438 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0798102790 0x7f07981067e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:01.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.735+0000 7f079e2d1640 1 --2- 172.21.15.105:0/3334629438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0798101de0 0x7f07981021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:01.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.735+0000 7f079e2d1640 1 -- 172.21.15.105:0/3334629438 >> 172.21.15.105:0/3334629438 conn(0x7f07980fd8c0 msgr2=0x7f07980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:01.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.735+0000 7f079e2d1640 1 -- 172.21.15.105:0/3334629438 shutdown_connections 2024-09-16T02:02:01.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.735+0000 7f079e2d1640 1 -- 172.21.15.105:0/3334629438 wait complete. 2024-09-16T02:02:01.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.736+0000 7f079e2d1640 1 Processor -- start 2024-09-16T02:02:01.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.736+0000 7f079e2d1640 1 -- start start 2024-09-16T02:02:01.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.736+0000 7f079e2d1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0798101de0 0x7f07980744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:01.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.736+0000 7f079e2d1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0798102790 0x7f0798074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:01.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.736+0000 7f079e2d1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0798106f10 0x7f0798074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:01.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.736+0000 7f079e2d1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0798111f00 con 0x7f0798101de0 2024-09-16T02:02:01.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.736+0000 7f079e2d1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0798111d80 con 0x7f0798102790 2024-09-16T02:02:01.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.736+0000 7f079e2d1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0798112080 con 0x7f0798106f10 2024-09-16T02:02:01.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.736+0000 7f0797fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0798101de0 0x7f07980744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:01.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.736+0000 7f0797fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0798101de0 0x7f07980744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57678/0 (socket says 172.21.15.105:57678) 2024-09-16T02:02:01.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.736+0000 7f0797fff640 1 -- 172.21.15.105:0/1041628056 learned_addr learned my addr 172.21.15.105:0/1041628056 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:01.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.736+0000 7f07977fe640 1 --2- 172.21.15.105:0/1041628056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0798102790 0x7f0798074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:01.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.736+0000 7f079c86e640 1 --2- 172.21.15.105:0/1041628056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0798106f10 0x7f0798074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:01.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.737+0000 7f0797fff640 1 -- 172.21.15.105:0/1041628056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0798106f10 msgr2=0x7f0798074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:01.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.737+0000 7f0797fff640 1 --2- 172.21.15.105:0/1041628056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0798106f10 0x7f0798074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:01.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.737+0000 7f0797fff640 1 -- 172.21.15.105:0/1041628056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0798102790 msgr2=0x7f0798074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:01.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.737+0000 7f0797fff640 1 --2- 172.21.15.105:0/1041628056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0798102790 0x7f0798074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:01.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.737+0000 7f0797fff640 1 -- 172.21.15.105:0/1041628056 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07981a6a20 con 0x7f0798101de0 2024-09-16T02:02:01.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.737+0000 7f0797fff640 1 --2- 172.21.15.105:0/1041628056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0798101de0 0x7f07980744f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f078800b450 tx=0x7f078800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:01.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.737+0000 7f07957fa640 1 -- 172.21.15.105:0/1041628056 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0788015020 con 0x7f0798101de0 2024-09-16T02:02:01.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.737+0000 7f07957fa640 1 -- 172.21.15.105:0/1041628056 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0788011d40 con 0x7f0798101de0 2024-09-16T02:02:01.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.737+0000 7f07957fa640 1 -- 172.21.15.105:0/1041628056 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078800f340 con 0x7f0798101de0 2024-09-16T02:02:01.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.737+0000 7f079e2d1640 1 -- 172.21.15.105:0/1041628056 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07981a6d10 con 0x7f0798101de0 2024-09-16T02:02:01.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.737+0000 7f079e2d1640 1 -- 172.21.15.105:0/1041628056 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07981a7210 con 0x7f0798101de0 2024-09-16T02:02:01.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.738+0000 7f079e2d1640 1 -- 172.21.15.105:0/1041628056 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0798102240 con 0x7f0798101de0 2024-09-16T02:02:01.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.739+0000 7f07957fa640 1 -- 172.21.15.105:0/1041628056 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f0788018020 con 0x7f0798101de0 2024-09-16T02:02:01.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.740+0000 7f07957fa640 1 --2- 172.21.15.105:0/1041628056 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f07800a3350 0x7f07800a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:01.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.740+0000 7f07977fe640 1 --2- 172.21.15.105:0/1041628056 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f07800a3350 0x7f07800a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:01.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.740+0000 7f07957fa640 1 -- 172.21.15.105:0/1041628056 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f07880cf9b0 con 0x7f0798101de0 2024-09-16T02:02:01.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.740+0000 7f07977fe640 1 --2- 172.21.15.105:0/1041628056 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f07800a3350 0x7f07800a5810 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f078c031440 tx=0x7f078c0095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:01.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.743+0000 7f07957fa640 1 -- 172.21.15.105:0/1041628056 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f078801f020 con 0x7f0798101de0 2024-09-16T02:02:01.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.943+0000 7f079e2d1640 1 -- 172.21.15.105:0/1041628056 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0798075a20 con 0x7f0798101de0 2024-09-16T02:02:01.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.944+0000 7f07957fa640 1 -- 172.21.15.105:0/1041628056 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11991 (secure 0 0 0) 0x7f0798075a20 con 0x7f0798101de0 2024-09-16T02:02:01.945 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:02:01.945 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":27,"btime":"2024-09-16T02:00:47:526725+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2057699920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2057699920},{"type":"v1","addr":"172.21.15.105:6835","nonce":2057699920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1764886948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1764886948},{"type":"v1","addr":"172.21.15.105:6837","nonce":1764886948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T02:00:35.112266+0000","modified":"2024-09-16T02:00:47.526704+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24469,"mds_2":24461,"mds_3":24457,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/4112062361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":4112062361},{"type":"v1","addr":"172.21.15.105:6839","nonce":4112062361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6841/163156514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":163156514},{"type":"v1","addr":"172.21.15.105:6841","nonce":163156514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/683519097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":683519097},{"type":"v1","addr":"172.21.15.190:6833","nonce":683519097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3025442862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3025442862},{"type":"v1","addr":"172.21.15.137:6833","nonce":3025442862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/763783602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":763783602},{"type":"v1","addr":"172.21.15.137:6835","nonce":763783602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.190:6835/1851042281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1851042281},{"type":"v1","addr":"172.21.15.190:6835","nonce":1851042281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/3181257549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3181257549},{"type":"v1","addr":"172.21.15.190:6837","nonce":3181257549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6837/2788541030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2788541030},{"type":"v1","addr":"172.21.15.137:6837","nonce":2788541030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.190:6839/1222573537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":1222573537},{"type":"v1","addr":"172.21.15.190:6839","nonce":1222573537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/3454924009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3454924009},{"type":"v1","addr":"172.21.15.137:6839","nonce":3454924009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24469,24461,24457,24445]},"id":1}]} 2024-09-16T02:02:01.945 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 27 2024-09-16T02:02:01.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.946+0000 7f079e2d1640 1 -- 172.21.15.105:0/1041628056 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f07800a3350 msgr2=0x7f07800a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:01.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.946+0000 7f079e2d1640 1 --2- 172.21.15.105:0/1041628056 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f07800a3350 0x7f07800a5810 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f078c031440 tx=0x7f078c0095f0 comp rx=0 tx=0).stop 2024-09-16T02:02:01.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.946+0000 7f079e2d1640 1 -- 172.21.15.105:0/1041628056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0798101de0 msgr2=0x7f07980744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:01.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.946+0000 7f079e2d1640 1 --2- 172.21.15.105:0/1041628056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0798101de0 0x7f07980744f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f078800b450 tx=0x7f078800b910 comp rx=0 tx=0).stop 2024-09-16T02:02:01.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.946+0000 7f079e2d1640 1 -- 172.21.15.105:0/1041628056 shutdown_connections 2024-09-16T02:02:01.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.946+0000 7f079e2d1640 1 --2- 172.21.15.105:0/1041628056 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f07800a3350 0x7f07800a5810 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:01.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.946+0000 7f079e2d1640 1 --2- 172.21.15.105:0/1041628056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0798106f10 0x7f0798074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:01.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.946+0000 7f079e2d1640 1 --2- 172.21.15.105:0/1041628056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0798102790 0x7f0798074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:01.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.946+0000 7f079e2d1640 1 --2- 172.21.15.105:0/1041628056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0798101de0 0x7f07980744f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:01.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.946+0000 7f079e2d1640 1 -- 172.21.15.105:0/1041628056 >> 172.21.15.105:0/1041628056 conn(0x7f07980fd8c0 msgr2=0x7f07980fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:01.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.946+0000 7f079e2d1640 1 -- 172.21.15.105:0/1041628056 shutdown_connections 2024-09-16T02:02:01.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:01.946+0000 7f079e2d1640 1 -- 172.21.15.105:0/1041628056 wait complete. 2024-09-16T02:02:02.571 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:02 smithi105 ceph-mon[31197]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T02:02:02.571 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:02 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/1041628056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:02:02.598 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-16T02:00:35.112266+0000', 'modified': '2024-09-16T02:00:47.526704+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24482, 'mds_1': 24469, 'mds_2': 24461, 'mds_3': 24457, 'mds_4': 24445}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14613': {'gid': 14613, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.105:6839/4112062361', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6838', 'nonce': 4112062361}, {'type': 'v1', 'addr': '172.21.15.105:6839', 'nonce': 4112062361}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14625': {'gid': 14625, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.105:6841/163156514', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6840', 'nonce': 163156514}, {'type': 'v1', 'addr': '172.21.15.105:6841', 'nonce': 163156514}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24421': {'gid': 24421, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.190:6833/683519097', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6832', 'nonce': 683519097}, {'type': 'v1', 'addr': '172.21.15.190:6833', 'nonce': 683519097}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24422': {'gid': 24422, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.137:6833/3025442862', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6832', 'nonce': 3025442862}, {'type': 'v1', 'addr': '172.21.15.137:6833', 'nonce': 3025442862}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.137:6835/763783602', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6834', 'nonce': 763783602}, {'type': 'v1', 'addr': '172.21.15.137:6835', 'nonce': 763783602}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.190:6835/1851042281', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6834', 'nonce': 1851042281}, {'type': 'v1', 'addr': '172.21.15.190:6835', 'nonce': 1851042281}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.190:6837/3181257549', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6836', 'nonce': 3181257549}, {'type': 'v1', 'addr': '172.21.15.190:6837', 'nonce': 3181257549}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24461': {'gid': 24461, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.137:6837/2788541030', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6836', 'nonce': 2788541030}, {'type': 'v1', 'addr': '172.21.15.137:6837', 'nonce': 2788541030}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.190:6839/1222573537', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6838', 'nonce': 1222573537}, {'type': 'v1', 'addr': '172.21.15.190:6839', 'nonce': 1222573537}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.137:6839/3454924009', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6838', 'nonce': 3454924009}, {'type': 'v1', 'addr': '172.21.15.137:6839', 'nonce': 3454924009}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24482, 'qdb_cluster': [24482, 24469, 24461, 24457, 24445]} 2024-09-16T02:02:02.598 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-16T02:02:02.598 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.f status 2024-09-16T02:02:02.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:02 smithi190 ceph-mon[34762]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T02:02:02.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:02 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/1041628056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:02:02.844 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:02:02.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:02 smithi137 ceph-mon[34099]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T02:02:02.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:02 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/1041628056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:02:03.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.361+0000 7f59c1b42640 1 -- 172.21.15.105:0/1067538464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59bc10e500 msgr2=0x7f59bc1108f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:03.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.361+0000 7f59c1b42640 1 --2- 172.21.15.105:0/1067538464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59bc10e500 0x7f59bc1108f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009900 tx=0x7f59b402a410 comp rx=0 tx=0).stop 2024-09-16T02:02:03.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.361+0000 7f59c1b42640 1 -- 172.21.15.105:0/1067538464 shutdown_connections 2024-09-16T02:02:03.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.361+0000 7f59c1b42640 1 --2- 172.21.15.105:0/1067538464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59bc101300 0x7f59bc10dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:03.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.361+0000 7f59c1b42640 1 --2- 172.21.15.105:0/1067538464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59bc10e500 0x7f59bc1108f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:03.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.361+0000 7f59c1b42640 1 --2- 172.21.15.105:0/1067538464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59bc100950 0x7f59bc100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:03.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.361+0000 7f59c1b42640 1 -- 172.21.15.105:0/1067538464 >> 172.21.15.105:0/1067538464 conn(0x7f59bc0fc7a0 msgr2=0x7f59bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:03.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.361+0000 7f59c1b42640 1 -- 172.21.15.105:0/1067538464 shutdown_connections 2024-09-16T02:02:03.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.361+0000 7f59c1b42640 1 -- 172.21.15.105:0/1067538464 wait complete. 2024-09-16T02:02:03.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.362+0000 7f59c1b42640 1 Processor -- start 2024-09-16T02:02:03.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.362+0000 7f59c1b42640 1 -- start start 2024-09-16T02:02:03.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.362+0000 7f59c1b42640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59bc100950 0x7f59bc19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:03.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.362+0000 7f59c1b42640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59bc101300 0x7f59bc19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:03.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.362+0000 7f59c1b42640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59bc10e500 0x7f59bc19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:03.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.362+0000 7f59c1b42640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc1140c0 con 0x7f59bc101300 2024-09-16T02:02:03.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.362+0000 7f59c1b42640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc113f40 con 0x7f59bc10e500 2024-09-16T02:02:03.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.362+0000 7f59c1b42640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc114240 con 0x7f59bc100950 2024-09-16T02:02:03.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.362+0000 7f59baffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59bc101300 0x7f59bc19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:03.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.362+0000 7f59baffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59bc101300 0x7f59bc19ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57700/0 (socket says 172.21.15.105:57700) 2024-09-16T02:02:03.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.362+0000 7f59baffd640 1 -- 172.21.15.105:0/3248453304 learned_addr learned my addr 172.21.15.105:0/3248453304 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:03.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.362+0000 7f59bb7fe640 1 --2- 172.21.15.105:0/3248453304 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59bc100950 0x7f59bc19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:03.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.362+0000 7f59bbfff640 1 --2- 172.21.15.105:0/3248453304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59bc10e500 0x7f59bc19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:03.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.363+0000 7f59baffd640 1 -- 172.21.15.105:0/3248453304 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59bc100950 msgr2=0x7f59bc19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:03.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.363+0000 7f59baffd640 1 --2- 172.21.15.105:0/3248453304 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59bc100950 0x7f59bc19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:03.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.363+0000 7f59baffd640 1 -- 172.21.15.105:0/3248453304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59bc10e500 msgr2=0x7f59bc19eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:03.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.363+0000 7f59baffd640 1 --2- 172.21.15.105:0/3248453304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59bc10e500 0x7f59bc19eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:03.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.363+0000 7f59baffd640 1 -- 172.21.15.105:0/3248453304 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59bc19f5b0 con 0x7f59bc101300 2024-09-16T02:02:03.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.363+0000 7f59baffd640 1 --2- 172.21.15.105:0/3248453304 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59bc101300 0x7f59bc19ab40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f59b000b4a0 tx=0x7f59b000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:03.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.363+0000 7f59b8ff9640 1 -- 172.21.15.105:0/3248453304 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b0014070 con 0x7f59bc101300 2024-09-16T02:02:03.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.363+0000 7f59c1b42640 1 -- 172.21.15.105:0/3248453304 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59bc19f8a0 con 0x7f59bc101300 2024-09-16T02:02:03.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.363+0000 7f59b8ff9640 1 -- 172.21.15.105:0/3248453304 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59b0010ac0 con 0x7f59bc101300 2024-09-16T02:02:03.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.363+0000 7f59b8ff9640 1 -- 172.21.15.105:0/3248453304 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b0017c30 con 0x7f59bc101300 2024-09-16T02:02:03.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.363+0000 7f59c1b42640 1 -- 172.21.15.105:0/3248453304 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59bc1a7180 con 0x7f59bc101300 2024-09-16T02:02:03.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.365+0000 7f59b8ff9640 1 -- 172.21.15.105:0/3248453304 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f59b0017070 con 0x7f59bc101300 2024-09-16T02:02:03.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.365+0000 7f59c3f1b740 1 Processor -- start 2024-09-16T02:02:03.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.365+0000 7f59c3f1b740 1 -- start start 2024-09-16T02:02:03.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.366+0000 7f59c3f1b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8e9d0f9d0 0x55e8e9d0fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:03.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.366+0000 7f59c3f1b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8e9d102f0 0x55e8e9d1a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:03.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.366+0000 7f59c3f1b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e9d1abb0 0x55e8e9d1cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:03.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.366+0000 7f59c3f1b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e8e9c76090 con 0x55e8e9d102f0 2024-09-16T02:02:03.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.366+0000 7f59c3f1b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e8e9c1da70 con 0x55e8e9d1abb0 2024-09-16T02:02:03.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.366+0000 7f59c3f1b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e8e9c72050 con 0x55e8e9d0f9d0 2024-09-16T02:02:03.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.366+0000 7f59bb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8e9d102f0 0x55e8e9d1a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:03.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.366+0000 7f59bb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8e9d102f0 0x55e8e9d1a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57710/0 (socket says 172.21.15.105:57710) 2024-09-16T02:02:03.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.366+0000 7f59bb7fe640 1 -- 172.21.15.105:0/877386483 learned_addr learned my addr 172.21.15.105:0/877386483 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:03.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.366+0000 7f59b8ff9640 1 --2- 172.21.15.105:0/3248453304 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f59a40a32d0 0x7f59a40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:03.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.366+0000 7f59bbfff640 1 --2- 172.21.15.105:0/877386483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8e9d0f9d0 0x55e8e9d0fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:03.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.366+0000 7f59baffd640 1 --2- 172.21.15.105:0/877386483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e9d1abb0 0x55e8e9d1cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:03.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.366+0000 7f59bbfff640 1 --2- 172.21.15.105:0/3248453304 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f59a40a32d0 0x7f59a40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:03.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.366+0000 7f59bb7fe640 1 -- 172.21.15.105:0/877386483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8e9d0f9d0 msgr2=0x55e8e9d0fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:03.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.366+0000 7f59b8ff9640 1 -- 172.21.15.105:0/3248453304 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f59b00cee50 con 0x7f59bc101300 2024-09-16T02:02:03.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.366+0000 7f59bb7fe640 1 --2- 172.21.15.105:0/877386483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8e9d0f9d0 0x55e8e9d0fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:03.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.366+0000 7f59bb7fe640 1 -- 172.21.15.105:0/877386483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e9d1abb0 msgr2=0x55e8e9d1cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:03.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.366+0000 7f59bb7fe640 1 --2- 172.21.15.105:0/877386483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e9d1abb0 0x55e8e9d1cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:03.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.366+0000 7f59bb7fe640 1 -- 172.21.15.105:0/877386483 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8e9c2a130 con 0x55e8e9d102f0 2024-09-16T02:02:03.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.366+0000 7f59baffd640 1 --2- 172.21.15.105:0/877386483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e9d1abb0 0x55e8e9d1cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:02:03.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.366+0000 7f59bb7fe640 1 --2- 172.21.15.105:0/877386483 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8e9d102f0 0x55e8e9d1a670 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f59ac00ae90 tx=0x7f59ac00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62c4962e25aea0da server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:03.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.367+0000 7f59bbfff640 1 --2- 172.21.15.105:0/3248453304 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f59a40a32d0 0x7f59a40a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f59b402a930 tx=0x7f59b402a410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:03.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.367+0000 7f59a1ffb640 1 -- 172.21.15.105:0/877386483 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac011850 con 0x55e8e9d102f0 2024-09-16T02:02:03.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.367+0000 7f59a1ffb640 1 -- 172.21.15.105:0/877386483 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59ac01a480 con 0x55e8e9d102f0 2024-09-16T02:02:03.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.367+0000 7f59a1ffb640 1 -- 172.21.15.105:0/877386483 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac010630 con 0x55e8e9d102f0 2024-09-16T02:02:03.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.367+0000 7f59c3f1b740 1 -- 172.21.15.105:0/877386483 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8e9d102f0 msgr2=0x55e8e9d1a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:03.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.367+0000 7f59c3f1b740 1 --2- 172.21.15.105:0/877386483 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8e9d102f0 0x55e8e9d1a670 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f59ac00ae90 tx=0x7f59ac00f730 comp rx=0 tx=0).stop 2024-09-16T02:02:03.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.367+0000 7f59c3f1b740 1 -- 172.21.15.105:0/877386483 shutdown_connections 2024-09-16T02:02:03.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.367+0000 7f59c3f1b740 1 --2- 172.21.15.105:0/877386483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e9d1abb0 0x55e8e9d1cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:03.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.367+0000 7f59c3f1b740 1 --2- 172.21.15.105:0/877386483 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8e9d102f0 0x55e8e9d1a670 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:03.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.367+0000 7f59c3f1b740 1 --2- 172.21.15.105:0/877386483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8e9d0f9d0 0x55e8e9d0fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:03.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.367+0000 7f59c3f1b740 1 -- 172.21.15.105:0/877386483 >> 172.21.15.105:0/877386483 conn(0x55e8e9d0bf60 msgr2=0x55e8e9d0c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:03.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.367+0000 7f59c3f1b740 1 -- 172.21.15.105:0/877386483 shutdown_connections 2024-09-16T02:02:03.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.367+0000 7f59c3f1b740 1 -- 172.21.15.105:0/877386483 wait complete. 2024-09-16T02:02:03.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.368+0000 7f59c3f1b740 1 -- start start 2024-09-16T02:02:03.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.368+0000 7f59c3f1b740 1 Processor -- start 2024-09-16T02:02:03.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.368+0000 7f59c3f1b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8e9d0f9d0 0x55e8e9d1a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:03.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.368+0000 7f59c3f1b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e9d102f0 0x55e8e9d17790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:03.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.368+0000 7f59c3f1b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8e9d1abb0 0x55e8e9d17cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:03.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.368+0000 7f59c3f1b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e8e9d1d780 con 0x55e8e9d0f9d0 2024-09-16T02:02:03.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.368+0000 7f59c3f1b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e8e9c30f20 con 0x55e8e9d102f0 2024-09-16T02:02:03.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.368+0000 7f59c3f1b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e8e9d1d900 con 0x55e8e9d1abb0 2024-09-16T02:02:03.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.368+0000 7f59c3f1b740 10 client.0 _ll_register_callbacks cb 0x55e8e9bfa030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:02:03.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.368+0000 7f59bb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8e9d0f9d0 0x55e8e9d1a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:03.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.368+0000 7f59bb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8e9d0f9d0 0x55e8e9d1a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57716/0 (socket says 172.21.15.105:57716) 2024-09-16T02:02:03.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.368+0000 7f59bb7fe640 1 -- 172.21.15.105:0/1782947365 learned_addr learned my addr 172.21.15.105:0/1782947365 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:03.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.368+0000 7f59bbfff640 1 --2- 172.21.15.105:0/1782947365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8e9d1abb0 0x55e8e9d17cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:03.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.369+0000 7f59bb7fe640 1 -- 172.21.15.105:0/1782947365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8e9d1abb0 msgr2=0x55e8e9d17cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:03.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.369+0000 7f59bb7fe640 1 --2- 172.21.15.105:0/1782947365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8e9d1abb0 0x55e8e9d17cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:03.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.369+0000 7f59bb7fe640 1 -- 172.21.15.105:0/1782947365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e9d102f0 msgr2=0x55e8e9d17790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:03.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.369+0000 7f59bb7fe640 1 --2- 172.21.15.105:0/1782947365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e9d102f0 0x55e8e9d17790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:03.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.369+0000 7f59bb7fe640 1 -- 172.21.15.105:0/1782947365 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8e99e72b0 con 0x55e8e9d0f9d0 2024-09-16T02:02:03.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.369+0000 7f59bb7fe640 1 --2- 172.21.15.105:0/1782947365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8e9d0f9d0 0x55e8e9d1a440 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f59ac01adf0 tx=0x7f59ac00ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a3dedc586938aa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:03.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.369+0000 7f599affd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:02:03.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.369+0000 7f599affd640 1 -- 172.21.15.105:0/1782947365 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac028060 con 0x55e8e9d0f9d0 2024-09-16T02:02:03.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.369+0000 7f599affd640 1 -- 172.21.15.105:0/1782947365 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f59ac01a8c0 con 0x55e8e9d0f9d0 2024-09-16T02:02:03.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.369+0000 7f59c3f1b740 1 -- 172.21.15.105:0/1782947365 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59ac01a480 con 0x55e8e9d0f9d0 2024-09-16T02:02:03.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.369+0000 7f599affd640 1 -- 172.21.15.105:0/1782947365 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f59ac004a40 con 0x55e8e9d0f9d0 2024-09-16T02:02:03.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.369+0000 7f59c3f1b740 10 client.15006 fetch_fsmap learned FSMap version 27 2024-09-16T02:02:03.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.369+0000 7f59c3f1b740 1 -- 172.21.15.105:0/1782947365 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e8e9d29000 con 0x55e8e9d0f9d0 2024-09-16T02:02:03.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.369+0000 7f599affd640 1 -- 172.21.15.105:0/1782947365 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59ac01a480 con 0x55e8e9d0f9d0 2024-09-16T02:02:03.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.369+0000 7f599affd640 1 -- 172.21.15.105:0/1782947365 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f59ac005040 con 0x55e8e9d0f9d0 2024-09-16T02:02:03.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.369+0000 7f59c3f1b740 10 client.15006 fetch_fsmap finished waiting for FSMap version 27 2024-09-16T02:02:03.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.369+0000 7f59c3f1b740 10 client.15006 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-16T02:02:03.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.369+0000 7f59c3f1b740 20 client.15006 populate_metadata read hostname 'smithi105' 2024-09-16T02:02:03.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.370+0000 7f59c3f1b740 1 --2- 172.21.15.105:0/1782947365 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x55e8e9d2b230 0x55e8e9d2c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:03.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.370+0000 7f59c3f1b740 4 client.15006 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:02:03.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.370+0000 7f59c3f1b740 1 -- 172.21.15.105:0/1782947365 --> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8e9c83600 con 0x55e8e9d2b230 2024-09-16T02:02:03.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.370+0000 7f59baffd640 1 --2- 172.21.15.105:0/1782947365 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x55e8e9d2b230 0x55e8e9d2c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:03.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.370+0000 7f59baffd640 1 --2- 172.21.15.105:0/1782947365 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x55e8e9d2b230 0x55e8e9d2c6a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=50d6bd6f41042a40 server_cookie=c0fe3dd1632327dd in_seq=0 out_seq=0 2024-09-16T02:02:03.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.370+0000 7f599affd640 10 client.15006 ms_handle_connect on v2:172.21.15.190:6834/1851042281 2024-09-16T02:02:03.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.371+0000 7f599affd640 1 -- 172.21.15.105:0/1782947365 <== mds.4 v2:172.21.15.190:6834/1851042281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e8e9c83600 con 0x55e8e9d2b230 2024-09-16T02:02:03.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.371+0000 7f599affd640 10 client.15006 handle_command_reply: tid=0 2024-09-16T02:02:03.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.371+0000 7f59c3f1b740 1 client.15006 shutdown 2024-09-16T02:02:03.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.371+0000 7f59c3f1b740 1 -- 172.21.15.105:0/1782947365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8e9d0f9d0 msgr2=0x55e8e9d1a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:03.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.371+0000 7f59c3f1b740 1 --2- 172.21.15.105:0/1782947365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8e9d0f9d0 0x55e8e9d1a440 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f59ac01adf0 tx=0x7f59ac00ac00 comp rx=0 tx=0).stop 2024-09-16T02:02:03.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.371+0000 7f59c3f1b740 1 -- 172.21.15.105:0/1782947365 shutdown_connections 2024-09-16T02:02:03.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.371+0000 7f59c3f1b740 1 --2- 172.21.15.105:0/1782947365 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x55e8e9d2b230 0x55e8e9d2c6a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:03.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.371+0000 7f59c3f1b740 1 --2- 172.21.15.105:0/1782947365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8e9d1abb0 0x55e8e9d17cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:03.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.371+0000 7f599affd640 0 client.15006 ms_handle_reset on v2:172.21.15.190:6834/1851042281 2024-09-16T02:02:03.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.371+0000 7f59c3f1b740 1 --2- 172.21.15.105:0/1782947365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e9d102f0 0x55e8e9d17790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:03.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.371+0000 7f59c3f1b740 1 --2- 172.21.15.105:0/1782947365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8e9d0f9d0 0x55e8e9d1a440 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:03.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.371+0000 7f59c3f1b740 1 -- 172.21.15.105:0/1782947365 >> 172.21.15.105:0/1782947365 conn(0x55e8e9d0bf60 msgr2=0x55e8e9d11c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:03.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.372+0000 7f59c3f1b740 1 -- 172.21.15.105:0/1782947365 shutdown_connections 2024-09-16T02:02:03.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.372+0000 7f59c3f1b740 1 -- 172.21.15.105:0/1782947365 wait complete. 2024-09-16T02:02:03.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.372+0000 7f59c3f1b740 20 client.15006 trim_cache size 0 max 16384 2024-09-16T02:02:03.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.389+0000 7f59c3f1b740 1 Processor -- start 2024-09-16T02:02:03.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.389+0000 7f59c3f1b740 1 -- start start 2024-09-16T02:02:03.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.389+0000 7f59c3f1b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8e9d0f9d0 0x55e8e9d14a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:03.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.389+0000 7f59c3f1b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8e9d102f0 0x55e8e9d14f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:03.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.389+0000 7f59c3f1b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e9d1abb0 0x55e8e9d25b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:03.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.390+0000 7f59c3f1b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e8e9d2a9e0 con 0x55e8e9d0f9d0 2024-09-16T02:02:03.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.390+0000 7f59c3f1b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e8e9d2b0b0 con 0x55e8e9d1abb0 2024-09-16T02:02:03.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.390+0000 7f59c3f1b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e8e9d2b780 con 0x55e8e9d102f0 2024-09-16T02:02:03.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.390+0000 7f59bb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8e9d0f9d0 0x55e8e9d14a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:03.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.390+0000 7f59bb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8e9d0f9d0 0x55e8e9d14a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57730/0 (socket says 172.21.15.105:57730) 2024-09-16T02:02:03.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.390+0000 7f59bb7fe640 1 -- 172.21.15.105:0/1929241872 learned_addr learned my addr 172.21.15.105:0/1929241872 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:03.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.390+0000 7f59bb7fe640 1 -- 172.21.15.105:0/1929241872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8e9d102f0 msgr2=0x55e8e9d14f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:03.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.390+0000 7f59bb7fe640 1 --2- 172.21.15.105:0/1929241872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8e9d102f0 0x55e8e9d14f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:03.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.390+0000 7f59bb7fe640 1 -- 172.21.15.105:0/1929241872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e9d1abb0 msgr2=0x55e8e9d25b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:03.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.390+0000 7f59bb7fe640 1 --2- 172.21.15.105:0/1929241872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e9d1abb0 0x55e8e9d25b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:03.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.390+0000 7f59bb7fe640 1 -- 172.21.15.105:0/1929241872 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8e9d0f750 con 0x55e8e9d0f9d0 2024-09-16T02:02:03.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.390+0000 7f59bb7fe640 1 --2- 172.21.15.105:0/1929241872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8e9d0f9d0 0x55e8e9d14a10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f59ac005c40 tx=0x7f59ac005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe8388a4e0ffc63 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:03.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.390+0000 7f59a17fa640 1 -- 172.21.15.105:0/1929241872 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac028070 con 0x55e8e9d0f9d0 2024-09-16T02:02:03.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.390+0000 7f59a17fa640 1 -- 172.21.15.105:0/1929241872 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59ac033020 con 0x55e8e9d0f9d0 2024-09-16T02:02:03.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.390+0000 7f59a17fa640 1 -- 172.21.15.105:0/1929241872 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac01a8c0 con 0x55e8e9d0f9d0 2024-09-16T02:02:03.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.391+0000 7f59c3f1b740 1 -- 172.21.15.105:0/1929241872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8e9d0f9d0 msgr2=0x55e8e9d14a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:03.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.391+0000 7f59c3f1b740 1 --2- 172.21.15.105:0/1929241872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8e9d0f9d0 0x55e8e9d14a10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f59ac005c40 tx=0x7f59ac005e10 comp rx=0 tx=0).stop 2024-09-16T02:02:03.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.391+0000 7f59c3f1b740 1 -- 172.21.15.105:0/1929241872 shutdown_connections 2024-09-16T02:02:03.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.391+0000 7f59c3f1b740 1 --2- 172.21.15.105:0/1929241872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8e9d102f0 0x55e8e9d14f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:03.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.391+0000 7f59c3f1b740 1 --2- 172.21.15.105:0/1929241872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e9d1abb0 0x55e8e9d25b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:03.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.391+0000 7f59c3f1b740 1 --2- 172.21.15.105:0/1929241872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8e9d0f9d0 0x55e8e9d14a10 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:03.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.391+0000 7f59c3f1b740 1 -- 172.21.15.105:0/1929241872 >> 172.21.15.105:0/1929241872 conn(0x55e8e9d0bf60 msgr2=0x55e8e9d16060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:03.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.391+0000 7f59c3f1b740 1 -- 172.21.15.105:0/1929241872 shutdown_connections 2024-09-16T02:02:03.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.391+0000 7f59c3f1b740 1 -- 172.21.15.105:0/1929241872 wait complete. 2024-09-16T02:02:03.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.391+0000 7f59c3f1b740 1 -- start start 2024-09-16T02:02:03.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.392+0000 7f59c3f1b740 1 Processor -- start 2024-09-16T02:02:03.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.392+0000 7f59c3f1b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8e9d0f9d0 0x55e8e9d2ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:03.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.392+0000 7f59c3f1b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e9d102f0 0x55e8e9d30500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:03.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.392+0000 7f59c3f1b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8e9d1abb0 0x55e8e9d8d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:03.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.392+0000 7f59c3f1b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e8e9d2c520 con 0x55e8e9d0f9d0 2024-09-16T02:02:03.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.392+0000 7f59c3f1b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e8e9d2be50 con 0x55e8e9d102f0 2024-09-16T02:02:03.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.392+0000 7f59c3f1b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e8e9d2cbf0 con 0x55e8e9d1abb0 2024-09-16T02:02:03.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.392+0000 7f59c3f1b740 10 client.0 _ll_register_callbacks cb 0x7f59ac01af50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:02:03.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.392+0000 7f59bb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8e9d0f9d0 0x55e8e9d2ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:03.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.392+0000 7f59bb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8e9d0f9d0 0x55e8e9d2ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57746/0 (socket says 172.21.15.105:57746) 2024-09-16T02:02:03.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.392+0000 7f59bb7fe640 1 -- 172.21.15.105:0/1661075184 learned_addr learned my addr 172.21.15.105:0/1661075184 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:03.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.392+0000 7f59baffd640 1 --2- 172.21.15.105:0/1661075184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e9d102f0 0x55e8e9d30500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:03.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.392+0000 7f59bbfff640 1 --2- 172.21.15.105:0/1661075184 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8e9d1abb0 0x55e8e9d8d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:03.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.392+0000 7f59bb7fe640 1 -- 172.21.15.105:0/1661075184 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8e9d1abb0 msgr2=0x55e8e9d8d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:03.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.392+0000 7f59bb7fe640 1 --2- 172.21.15.105:0/1661075184 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8e9d1abb0 0x55e8e9d8d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:03.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.392+0000 7f59bb7fe640 1 -- 172.21.15.105:0/1661075184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e9d102f0 msgr2=0x55e8e9d30500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:03.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.392+0000 7f59bb7fe640 1 --2- 172.21.15.105:0/1661075184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e9d102f0 0x55e8e9d30500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:03.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.392+0000 7f59bb7fe640 1 -- 172.21.15.105:0/1661075184 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8e9d8dfb0 con 0x55e8e9d0f9d0 2024-09-16T02:02:03.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.392+0000 7f59baffd640 1 --2- 172.21.15.105:0/1661075184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e9d102f0 0x55e8e9d30500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:02:03.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.392+0000 7f59bb7fe640 1 --2- 172.21.15.105:0/1661075184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8e9d0f9d0 0x55e8e9d2ffc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f59ac01aef0 tx=0x7f59ac00ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af90e253e3b5298f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:03.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.392+0000 7f599affd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:02:03.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.393+0000 7f599affd640 1 -- 172.21.15.105:0/1661075184 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac028040 con 0x55e8e9d0f9d0 2024-09-16T02:02:03.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.393+0000 7f59c3f1b740 1 -- 172.21.15.105:0/1661075184 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59ac033020 con 0x55e8e9d0f9d0 2024-09-16T02:02:03.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.393+0000 7f599affd640 1 -- 172.21.15.105:0/1661075184 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f59ac034070 con 0x55e8e9d0f9d0 2024-09-16T02:02:03.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.393+0000 7f599affd640 1 -- 172.21.15.105:0/1661075184 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f59ac003970 con 0x55e8e9d0f9d0 2024-09-16T02:02:03.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.393+0000 7f59c3f1b740 10 client.15012 fetch_fsmap learned FSMap version 27 2024-09-16T02:02:03.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.393+0000 7f59c3f1b740 1 -- 172.21.15.105:0/1661075184 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e8e9d95360 con 0x55e8e9d0f9d0 2024-09-16T02:02:03.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.393+0000 7f599affd640 1 -- 172.21.15.105:0/1661075184 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59ac033020 con 0x55e8e9d0f9d0 2024-09-16T02:02:03.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.393+0000 7f599affd640 1 -- 172.21.15.105:0/1661075184 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f59ac011ae0 con 0x55e8e9d0f9d0 2024-09-16T02:02:03.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.393+0000 7f59c3f1b740 10 client.15012 fetch_fsmap finished waiting for FSMap version 27 2024-09-16T02:02:03.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.393+0000 7f59c3f1b740 10 client.15012 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-16T02:02:03.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.393+0000 7f59c3f1b740 20 client.15012 populate_metadata read hostname 'smithi105' 2024-09-16T02:02:03.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.393+0000 7f59c3f1b740 1 --2- 172.21.15.105:0/1661075184 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x55e8e9d2b230 0x55e8e9d98870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:03.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.393+0000 7f59c3f1b740 4 client.15012 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T02:02:03.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.393+0000 7f59c3f1b740 1 -- 172.21.15.105:0/1661075184 --> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] -- command(tid 0: {"prefix": "status"}) -- 0x55e8e9c7db10 con 0x55e8e9d2b230 2024-09-16T02:02:03.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.393+0000 7f59baffd640 1 --2- 172.21.15.105:0/1661075184 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x55e8e9d2b230 0x55e8e9d98870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:03.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.394+0000 7f599affd640 10 client.15012 ms_handle_connect on v2:172.21.15.190:6834/1851042281 2024-09-16T02:02:03.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.394+0000 7f59baffd640 1 --2- 172.21.15.105:0/1661075184 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x55e8e9d2b230 0x55e8e9d98870 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=686657728c581be server_cookie=94176784c8bf5ee5 in_seq=0 out_seq=0 2024-09-16T02:02:03.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.394+0000 7f599affd640 1 -- 172.21.15.105:0/1661075184 <== mds.4 v2:172.21.15.190:6834/1851042281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55e8e9c7db10 con 0x55e8e9d2b230 2024-09-16T02:02:03.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.394+0000 7f599affd640 10 client.15012 handle_command_reply: tid=0 2024-09-16T02:02:03.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.394+0000 7f59c3f1b740 1 client.15012 shutdown 2024-09-16T02:02:03.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.394+0000 7f59c3f1b740 1 -- 172.21.15.105:0/1661075184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8e9d0f9d0 msgr2=0x55e8e9d2ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:03.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.394+0000 7f59c3f1b740 1 --2- 172.21.15.105:0/1661075184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8e9d0f9d0 0x55e8e9d2ffc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f59ac01aef0 tx=0x7f59ac00ae50 comp rx=0 tx=0).stop 2024-09-16T02:02:03.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.394+0000 7f59c3f1b740 1 -- 172.21.15.105:0/1661075184 shutdown_connections 2024-09-16T02:02:03.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.394+0000 7f59c3f1b740 1 --2- 172.21.15.105:0/1661075184 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x55e8e9d2b230 0x55e8e9d98870 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:03.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.394+0000 7f59c3f1b740 1 --2- 172.21.15.105:0/1661075184 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8e9d1abb0 0x55e8e9d8d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:03.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.394+0000 7f59c3f1b740 1 --2- 172.21.15.105:0/1661075184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e9d102f0 0x55e8e9d30500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:03.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.394+0000 7f59c3f1b740 1 --2- 172.21.15.105:0/1661075184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8e9d0f9d0 0x55e8e9d2ffc0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:03.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.394+0000 7f59c3f1b740 1 -- 172.21.15.105:0/1661075184 >> 172.21.15.105:0/1661075184 conn(0x55e8e9d0bf60 msgr2=0x55e8e9d25b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:03.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.394+0000 7f599affd640 0 client.15012 ms_handle_reset on v2:172.21.15.190:6834/1851042281 2024-09-16T02:02:03.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.394+0000 7f59c3f1b740 1 -- 172.21.15.105:0/1661075184 shutdown_connections 2024-09-16T02:02:03.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.394+0000 7f59c3f1b740 1 -- 172.21.15.105:0/1661075184 wait complete. 2024-09-16T02:02:03.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.394+0000 7f59c3f1b740 20 client.15012 trim_cache size 0 max 16384 2024-09-16T02:02:03.415 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-16T02:02:03.415 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "423b7a2a-73ce-11ef-bceb-c7b262605968", 2024-09-16T02:02:03.415 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 4, 2024-09-16T02:02:03.415 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 24445, 2024-09-16T02:02:03.415 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2024-09-16T02:02:03.415 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2024-09-16T02:02:03.415 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2024-09-16T02:02:03.415 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 76.876518637000004, 2024-09-16T02:02:03.415 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 27, 2024-09-16T02:02:03.415 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 81, 2024-09-16T02:02:03.416 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 79, 2024-09-16T02:02:03.416 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 116.182641042 2024-09-16T02:02:03.416 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-16T02:02:03.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.395+0000 7f59c1b42640 1 -- 172.21.15.105:0/3248453304 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f59a40a32d0 msgr2=0x7f59a40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:03.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.395+0000 7f59c1b42640 1 --2- 172.21.15.105:0/3248453304 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f59a40a32d0 0x7f59a40a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f59b402a930 tx=0x7f59b402a410 comp rx=0 tx=0).stop 2024-09-16T02:02:03.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.395+0000 7f59c1b42640 1 -- 172.21.15.105:0/3248453304 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59bc101300 msgr2=0x7f59bc19ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:03.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.395+0000 7f59c1b42640 1 --2- 172.21.15.105:0/3248453304 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59bc101300 0x7f59bc19ab40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f59b000b4a0 tx=0x7f59b000b960 comp rx=0 tx=0).stop 2024-09-16T02:02:03.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.395+0000 7f59c1b42640 1 -- 172.21.15.105:0/3248453304 shutdown_connections 2024-09-16T02:02:03.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.395+0000 7f59c1b42640 1 --2- 172.21.15.105:0/3248453304 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f59a40a32d0 0x7f59a40a5790 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:03.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.395+0000 7f59c1b42640 1 --2- 172.21.15.105:0/3248453304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59bc10e500 0x7f59bc19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:03.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.395+0000 7f59c1b42640 1 --2- 172.21.15.105:0/3248453304 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59bc101300 0x7f59bc19ab40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:03.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.395+0000 7f59c1b42640 1 --2- 172.21.15.105:0/3248453304 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59bc100950 0x7f59bc19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:03.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.395+0000 7f59c1b42640 1 -- 172.21.15.105:0/3248453304 >> 172.21.15.105:0/3248453304 conn(0x7f59bc0fc7a0 msgr2=0x7f59bc0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:03.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.395+0000 7f59c1b42640 1 -- 172.21.15.105:0/3248453304 shutdown_connections 2024-09-16T02:02:03.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:03.395+0000 7f59c1b42640 1 -- 172.21.15.105:0/3248453304 wait complete. 2024-09-16T02:02:04.191 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-16T02:02:04.436 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:02:04.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:04 smithi190 ceph-mon[34762]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-16T02:02:04.851 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:04 smithi105 ceph-mon[31197]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-16T02:02:04.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:04 smithi137 ceph-mon[34099]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-16T02:02:04.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.930+0000 7f1ad285a640 1 -- 172.21.15.105:0/153919802 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1acc106f20 msgr2=0x7f1acc10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:04.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.930+0000 7f1ad285a640 1 --2- 172.21.15.105:0/153919802 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1acc106f20 0x7f1acc10eaa0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4009ca0 tx=0x7f1ac402a8c0 comp rx=0 tx=0).stop 2024-09-16T02:02:04.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.930+0000 7f1ad285a640 1 -- 172.21.15.105:0/153919802 shutdown_connections 2024-09-16T02:02:04.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.930+0000 7f1ad285a640 1 --2- 172.21.15.105:0/153919802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1acc1027a0 0x7f1acc1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:04.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.930+0000 7f1ad285a640 1 --2- 172.21.15.105:0/153919802 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1acc106f20 0x7f1acc10eaa0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:04.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.930+0000 7f1ad285a640 1 --2- 172.21.15.105:0/153919802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1acc101df0 0x7f1acc1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:04.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.930+0000 7f1ad285a640 1 -- 172.21.15.105:0/153919802 >> 172.21.15.105:0/153919802 conn(0x7f1acc0fd8b0 msgr2=0x7f1acc0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:04.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.930+0000 7f1ad285a640 1 -- 172.21.15.105:0/153919802 shutdown_connections 2024-09-16T02:02:04.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.930+0000 7f1ad285a640 1 -- 172.21.15.105:0/153919802 wait complete. 2024-09-16T02:02:04.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.931+0000 7f1ad285a640 1 Processor -- start 2024-09-16T02:02:04.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.931+0000 7f1ad285a640 1 -- start start 2024-09-16T02:02:04.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.931+0000 7f1ad285a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1acc101df0 0x7f1acc198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:04.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.931+0000 7f1ad285a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1acc1027a0 0x7f1acc198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:04.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.931+0000 7f1ad285a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1acc106f20 0x7f1acc19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:04.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.931+0000 7f1ad285a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc111ed0 con 0x7f1acc101df0 2024-09-16T02:02:04.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.931+0000 7f1ad285a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc111d50 con 0x7f1acc106f20 2024-09-16T02:02:04.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.931+0000 7f1ad285a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc112050 con 0x7f1acc1027a0 2024-09-16T02:02:04.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.931+0000 7f1acbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1acc101df0 0x7f1acc198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:04.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.931+0000 7f1acbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1acc101df0 0x7f1acc198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57758/0 (socket says 172.21.15.105:57758) 2024-09-16T02:02:04.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.931+0000 7f1acbfff640 1 -- 172.21.15.105:0/3923023983 learned_addr learned my addr 172.21.15.105:0/3923023983 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:04.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.932+0000 7f1acb7fe640 1 --2- 172.21.15.105:0/3923023983 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1acc1027a0 0x7f1acc198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:04.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.932+0000 7f1ad0df7640 1 --2- 172.21.15.105:0/3923023983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1acc106f20 0x7f1acc19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:04.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.932+0000 7f1acbfff640 1 -- 172.21.15.105:0/3923023983 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1acc1027a0 msgr2=0x7f1acc198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:04.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.932+0000 7f1acbfff640 1 --2- 172.21.15.105:0/3923023983 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1acc1027a0 0x7f1acc198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:04.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.932+0000 7f1acbfff640 1 -- 172.21.15.105:0/3923023983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1acc106f20 msgr2=0x7f1acc19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:04.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.932+0000 7f1acbfff640 1 --2- 172.21.15.105:0/3923023983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1acc106f20 0x7f1acc19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:04.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.932+0000 7f1acbfff640 1 -- 172.21.15.105:0/3923023983 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1acc19d3e0 con 0x7f1acc101df0 2024-09-16T02:02:04.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.932+0000 7f1acb7fe640 1 --2- 172.21.15.105:0/3923023983 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1acc1027a0 0x7f1acc198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:02:04.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.932+0000 7f1ad0df7640 1 --2- 172.21.15.105:0/3923023983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1acc106f20 0x7f1acc19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:02:04.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.932+0000 7f1acbfff640 1 --2- 172.21.15.105:0/3923023983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1acc101df0 0x7f1acc198430 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1abc00b4a0 tx=0x7f1abc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:04.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.932+0000 7f1ac97fa640 1 -- 172.21.15.105:0/3923023983 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1abc014070 con 0x7f1acc101df0 2024-09-16T02:02:04.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.932+0000 7f1ac97fa640 1 -- 172.21.15.105:0/3923023983 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1abc010ac0 con 0x7f1acc101df0 2024-09-16T02:02:04.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.932+0000 7f1ad285a640 1 -- 172.21.15.105:0/3923023983 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1acc19d6d0 con 0x7f1acc101df0 2024-09-16T02:02:04.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.932+0000 7f1ac97fa640 1 -- 172.21.15.105:0/3923023983 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1abc017c30 con 0x7f1acc101df0 2024-09-16T02:02:04.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.932+0000 7f1ad285a640 1 -- 172.21.15.105:0/3923023983 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1acc1a4fb0 con 0x7f1acc101df0 2024-09-16T02:02:04.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.934+0000 7f1ac97fa640 1 -- 172.21.15.105:0/3923023983 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f1abc017070 con 0x7f1acc101df0 2024-09-16T02:02:04.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.935+0000 7f1ad4c33740 1 Processor -- start 2024-09-16T02:02:04.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.935+0000 7f1ad4c33740 1 -- start start 2024-09-16T02:02:04.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.935+0000 7f1ad4c33740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2aa2439d0 0x55e2aa243db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:04.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.935+0000 7f1ad4c33740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aa2442f0 0x55e2aa24e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:04.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.935+0000 7f1ad4c33740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2aa24ebb0 0x55e2aa250fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:04.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.935+0000 7f1ad4c33740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e2aa1aa090 con 0x55e2aa2439d0 2024-09-16T02:02:04.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.935+0000 7f1ad4c33740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e2aa151a70 con 0x55e2aa24ebb0 2024-09-16T02:02:04.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.935+0000 7f1ad4c33740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e2aa1a6050 con 0x55e2aa2442f0 2024-09-16T02:02:04.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.935+0000 7f1ac97fa640 1 --2- 172.21.15.105:0/3923023983 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f1ab40a3350 0x7f1ab40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:04.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.935+0000 7f1ad0df7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2aa2439d0 0x55e2aa243db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:04.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.935+0000 7f1ad0df7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2aa2439d0 0x55e2aa243db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57772/0 (socket says 172.21.15.105:57772) 2024-09-16T02:02:04.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.935+0000 7f1ad0df7640 1 -- 172.21.15.105:0/981172606 learned_addr learned my addr 172.21.15.105:0/981172606 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:04.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.935+0000 7f1ac97fa640 1 -- 172.21.15.105:0/3923023983 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f1abc0ceff0 con 0x7f1acc101df0 2024-09-16T02:02:04.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.935+0000 7f1ad0df7640 1 --2- 172.21.15.105:0/3923023983 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f1ab40a3350 0x7f1ab40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:04.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.935+0000 7f1acb7fe640 1 --2- 172.21.15.105:0/981172606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2aa24ebb0 0x55e2aa250fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:04.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.935+0000 7f1acbfff640 1 --2- 172.21.15.105:0/981172606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aa2442f0 0x55e2aa24e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:04.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.935+0000 7f1ad0df7640 1 -- 172.21.15.105:0/981172606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aa2442f0 msgr2=0x55e2aa24e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:04.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.935+0000 7f1ad0df7640 1 --2- 172.21.15.105:0/981172606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aa2442f0 0x55e2aa24e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:04.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.935+0000 7f1ad0df7640 1 -- 172.21.15.105:0/981172606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2aa24ebb0 msgr2=0x55e2aa250fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:04.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.935+0000 7f1ad0df7640 1 --2- 172.21.15.105:0/981172606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2aa24ebb0 0x55e2aa250fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:04.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.935+0000 7f1ad0df7640 1 -- 172.21.15.105:0/981172606 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2aa15e130 con 0x55e2aa2439d0 2024-09-16T02:02:04.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.935+0000 7f1acb7fe640 1 --2- 172.21.15.105:0/981172606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2aa24ebb0 0x55e2aa250fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:02:04.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.935+0000 7f1acbfff640 1 --2- 172.21.15.105:0/981172606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aa2442f0 0x55e2aa24e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:02:04.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.936+0000 7f1ad0df7640 1 --2- 172.21.15.105:0/981172606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2aa2439d0 0x55e2aa243db0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4032d90 tx=0x7f1ac4032dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21cb3a9cc3bdacb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:04.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.936+0000 7f1ad0df7640 1 --2- 172.21.15.105:0/3923023983 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f1ab40a3350 0x7f1ab40a5810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4032e30 tx=0x7f1ac4042000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:04.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.936+0000 7f1ab27fc640 1 -- 172.21.15.105:0/981172606 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac404d070 con 0x55e2aa2439d0 2024-09-16T02:02:04.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.936+0000 7f1ab27fc640 1 -- 172.21.15.105:0/981172606 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ac4036050 con 0x55e2aa2439d0 2024-09-16T02:02:04.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.936+0000 7f1ab27fc640 1 -- 172.21.15.105:0/981172606 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac40045b0 con 0x55e2aa2439d0 2024-09-16T02:02:04.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.937+0000 7f1ad4c33740 1 -- 172.21.15.105:0/981172606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2aa2439d0 msgr2=0x55e2aa243db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:04.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.937+0000 7f1ad4c33740 1 --2- 172.21.15.105:0/981172606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2aa2439d0 0x55e2aa243db0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4032d90 tx=0x7f1ac4032dc0 comp rx=0 tx=0).stop 2024-09-16T02:02:04.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.937+0000 7f1ad4c33740 1 -- 172.21.15.105:0/981172606 shutdown_connections 2024-09-16T02:02:04.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.937+0000 7f1ad4c33740 1 --2- 172.21.15.105:0/981172606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aa2442f0 0x55e2aa24e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:04.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.937+0000 7f1ad4c33740 1 --2- 172.21.15.105:0/981172606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2aa24ebb0 0x55e2aa250fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:04.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.937+0000 7f1ad4c33740 1 --2- 172.21.15.105:0/981172606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2aa2439d0 0x55e2aa243db0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:04.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.937+0000 7f1ad4c33740 1 -- 172.21.15.105:0/981172606 >> 172.21.15.105:0/981172606 conn(0x55e2aa23ff60 msgr2=0x55e2aa240340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:04.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.937+0000 7f1ad4c33740 1 -- 172.21.15.105:0/981172606 shutdown_connections 2024-09-16T02:02:04.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.937+0000 7f1ad4c33740 1 -- 172.21.15.105:0/981172606 wait complete. 2024-09-16T02:02:04.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.937+0000 7f1ad4c33740 1 -- start start 2024-09-16T02:02:04.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.937+0000 7f1ad4c33740 1 Processor -- start 2024-09-16T02:02:04.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.937+0000 7f1ad4c33740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aa2439d0 0x55e2aa24e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:04.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.937+0000 7f1ad4c33740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2aa2442f0 0x55e2aa24b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:04.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.937+0000 7f1ad4c33740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2aa24ebb0 0x55e2aa24bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:04.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.937+0000 7f1ad4c33740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e2aa251780 con 0x55e2aa2442f0 2024-09-16T02:02:04.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.937+0000 7f1ad4c33740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e2aa164f20 con 0x55e2aa24ebb0 2024-09-16T02:02:04.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.937+0000 7f1ad4c33740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e2aa251900 con 0x55e2aa2439d0 2024-09-16T02:02:04.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.938+0000 7f1ad4c33740 10 client.0 _ll_register_callbacks cb 0x55e2aa12e030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:02:04.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.938+0000 7f1acb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2aa2442f0 0x55e2aa24b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:04.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.938+0000 7f1acb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2aa2442f0 0x55e2aa24b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57786/0 (socket says 172.21.15.105:57786) 2024-09-16T02:02:04.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.938+0000 7f1acb7fe640 1 -- 172.21.15.105:0/567746245 learned_addr learned my addr 172.21.15.105:0/567746245 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:04.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.938+0000 7f1ad0df7640 1 --2- 172.21.15.105:0/567746245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2aa24ebb0 0x55e2aa24bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:04.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.938+0000 7f1acb7fe640 1 -- 172.21.15.105:0/567746245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aa2439d0 msgr2=0x55e2aa24e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:04.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.938+0000 7f1acb7fe640 1 --2- 172.21.15.105:0/567746245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aa2439d0 0x55e2aa24e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:04.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.938+0000 7f1acb7fe640 1 -- 172.21.15.105:0/567746245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2aa24ebb0 msgr2=0x55e2aa24bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:04.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.938+0000 7f1acb7fe640 1 --2- 172.21.15.105:0/567746245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2aa24ebb0 0x55e2aa24bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:04.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.938+0000 7f1acb7fe640 1 -- 172.21.15.105:0/567746245 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2a9f1b2b0 con 0x55e2aa2442f0 2024-09-16T02:02:04.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.938+0000 7f1ad0df7640 1 --2- 172.21.15.105:0/567746245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2aa24ebb0 0x55e2aa24bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:02:04.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.938+0000 7f1acb7fe640 1 --2- 172.21.15.105:0/567746245 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2aa2442f0 0x55e2aa24b830 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000e2a0 tx=0x7f1ac000e760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f08eef7e2de27d4f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:04.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.938+0000 7f1aab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:02:04.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.938+0000 7f1aab7fe640 1 -- 172.21.15.105:0/567746245 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0018020 con 0x55e2aa2442f0 2024-09-16T02:02:04.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.938+0000 7f1aab7fe640 1 -- 172.21.15.105:0/567746245 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f1ac0013660 con 0x55e2aa2442f0 2024-09-16T02:02:04.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.938+0000 7f1ad4c33740 1 -- 172.21.15.105:0/567746245 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ac4036050 con 0x55e2aa2442f0 2024-09-16T02:02:04.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.938+0000 7f1aab7fe640 1 -- 172.21.15.105:0/567746245 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1ac0023050 con 0x55e2aa2442f0 2024-09-16T02:02:04.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.938+0000 7f1ad4c33740 10 client.15024 fetch_fsmap learned FSMap version 27 2024-09-16T02:02:04.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.938+0000 7f1ad4c33740 1 -- 172.21.15.105:0/567746245 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e2aa25d100 con 0x55e2aa2442f0 2024-09-16T02:02:04.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.938+0000 7f1aab7fe640 1 -- 172.21.15.105:0/567746245 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ac4036050 con 0x55e2aa2442f0 2024-09-16T02:02:04.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.939+0000 7f1aab7fe640 1 -- 172.21.15.105:0/567746245 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1ac0014a40 con 0x55e2aa2442f0 2024-09-16T02:02:04.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.939+0000 7f1ad4c33740 10 client.15024 fetch_fsmap finished waiting for FSMap version 27 2024-09-16T02:02:04.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.939+0000 7f1ad4c33740 10 client.15024 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T02:02:04.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.939+0000 7f1ad4c33740 20 client.15024 populate_metadata read hostname 'smithi105' 2024-09-16T02:02:04.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.939+0000 7f1ad4c33740 1 --2- 172.21.15.105:0/567746245 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x55e2aa25f810 0x55e2aa260c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:04.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.939+0000 7f1ad4c33740 4 client.15024 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:02:04.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.939+0000 7f1ad4c33740 1 -- 172.21.15.105:0/567746245 --> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2aa1b7600 con 0x55e2aa25f810 2024-09-16T02:02:04.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.939+0000 7f1acbfff640 1 --2- 172.21.15.105:0/567746245 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x55e2aa25f810 0x55e2aa260c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:04.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.939+0000 7f1acbfff640 1 --2- 172.21.15.105:0/567746245 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x55e2aa25f810 0x55e2aa260c80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7fcb76d10e9f5864 server_cookie=bf55de4fbf980494 in_seq=0 out_seq=0 2024-09-16T02:02:04.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.939+0000 7f1aab7fe640 10 client.15024 ms_handle_connect on v2:172.21.15.190:6836/3181257549 2024-09-16T02:02:04.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.940+0000 7f1aab7fe640 1 -- 172.21.15.105:0/567746245 <== mds.3 v2:172.21.15.190:6836/3181257549 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e2aa1b7600 con 0x55e2aa25f810 2024-09-16T02:02:04.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.940+0000 7f1aab7fe640 10 client.15024 handle_command_reply: tid=0 2024-09-16T02:02:04.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.940+0000 7f1ad4c33740 1 client.15024 shutdown 2024-09-16T02:02:04.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.941+0000 7f1ad4c33740 1 -- 172.21.15.105:0/567746245 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2aa2442f0 msgr2=0x55e2aa24b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:04.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.941+0000 7f1ad4c33740 1 --2- 172.21.15.105:0/567746245 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2aa2442f0 0x55e2aa24b830 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000e2a0 tx=0x7f1ac000e760 comp rx=0 tx=0).stop 2024-09-16T02:02:04.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.941+0000 7f1ad4c33740 1 -- 172.21.15.105:0/567746245 shutdown_connections 2024-09-16T02:02:04.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.941+0000 7f1ad4c33740 1 --2- 172.21.15.105:0/567746245 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x55e2aa25f810 0x55e2aa260c80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:04.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.941+0000 7f1ad4c33740 1 --2- 172.21.15.105:0/567746245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2aa24ebb0 0x55e2aa24bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:04.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.941+0000 7f1ad4c33740 1 --2- 172.21.15.105:0/567746245 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2aa2442f0 0x55e2aa24b830 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:04.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.941+0000 7f1aab7fe640 0 client.15024 ms_handle_reset on v2:172.21.15.190:6836/3181257549 2024-09-16T02:02:04.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.941+0000 7f1ad4c33740 1 --2- 172.21.15.105:0/567746245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aa2439d0 0x55e2aa24e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:04.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.941+0000 7f1ad4c33740 1 -- 172.21.15.105:0/567746245 >> 172.21.15.105:0/567746245 conn(0x55e2aa23ff60 msgr2=0x55e2aa245c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:04.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.941+0000 7f1ad4c33740 1 -- 172.21.15.105:0/567746245 shutdown_connections 2024-09-16T02:02:04.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.941+0000 7f1ad4c33740 1 -- 172.21.15.105:0/567746245 wait complete. 2024-09-16T02:02:04.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.941+0000 7f1ad4c33740 20 client.15024 trim_cache size 0 max 16384 2024-09-16T02:02:04.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.958+0000 7f1ad4c33740 1 Processor -- start 2024-09-16T02:02:04.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.958+0000 7f1ad4c33740 1 -- start start 2024-09-16T02:02:04.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.959+0000 7f1ad4c33740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2aa2439d0 0x55e2aa25fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:04.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.959+0000 7f1ad4c33740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2aa2442f0 0x55e2aa269350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:04.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.959+0000 7f1ad4c33740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aa24ebb0 0x55e2aa245420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:04.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.959+0000 7f1ad4c33740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e2aa25efc0 con 0x55e2aa2442f0 2024-09-16T02:02:04.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.959+0000 7f1ad4c33740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e2aa25f690 con 0x55e2aa2439d0 2024-09-16T02:02:04.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.959+0000 7f1ad4c33740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e2aa25fd60 con 0x55e2aa24ebb0 2024-09-16T02:02:04.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.959+0000 7f1acb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2aa2442f0 0x55e2aa269350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:04.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.959+0000 7f1acb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2aa2442f0 0x55e2aa269350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57790/0 (socket says 172.21.15.105:57790) 2024-09-16T02:02:04.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.959+0000 7f1acb7fe640 1 -- 172.21.15.105:0/1861942687 learned_addr learned my addr 172.21.15.105:0/1861942687 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:04.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.959+0000 7f1acbfff640 1 --2- 172.21.15.105:0/1861942687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2aa2439d0 0x55e2aa25fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:04.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.959+0000 7f1ad0df7640 1 --2- 172.21.15.105:0/1861942687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aa24ebb0 0x55e2aa245420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:04.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.959+0000 7f1acb7fe640 1 -- 172.21.15.105:0/1861942687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aa24ebb0 msgr2=0x55e2aa245420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:04.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.959+0000 7f1acb7fe640 1 --2- 172.21.15.105:0/1861942687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aa24ebb0 0x55e2aa245420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:04.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.959+0000 7f1acb7fe640 1 -- 172.21.15.105:0/1861942687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2aa2439d0 msgr2=0x55e2aa25fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:04.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.959+0000 7f1acb7fe640 1 --2- 172.21.15.105:0/1861942687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2aa2439d0 0x55e2aa25fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:04.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.959+0000 7f1acb7fe640 1 -- 172.21.15.105:0/1861942687 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2aa24c870 con 0x55e2aa2442f0 2024-09-16T02:02:04.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.959+0000 7f1acb7fe640 1 --2- 172.21.15.105:0/1861942687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2aa2442f0 0x55e2aa269350 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000e2a0 tx=0x7f1ac0004660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b623ffab1265a7c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:04.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.959+0000 7f1ab1ffb640 1 -- 172.21.15.105:0/1861942687 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac002c070 con 0x55e2aa2442f0 2024-09-16T02:02:04.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.959+0000 7f1ab1ffb640 1 -- 172.21.15.105:0/1861942687 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ac0013d10 con 0x55e2aa2442f0 2024-09-16T02:02:04.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.959+0000 7f1ab1ffb640 1 -- 172.21.15.105:0/1861942687 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0018070 con 0x55e2aa2442f0 2024-09-16T02:02:04.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.960+0000 7f1ad4c33740 1 -- 172.21.15.105:0/1861942687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2aa2442f0 msgr2=0x55e2aa269350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:04.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.960+0000 7f1ad4c33740 1 --2- 172.21.15.105:0/1861942687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2aa2442f0 0x55e2aa269350 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000e2a0 tx=0x7f1ac0004660 comp rx=0 tx=0).stop 2024-09-16T02:02:04.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.960+0000 7f1ad4c33740 1 -- 172.21.15.105:0/1861942687 shutdown_connections 2024-09-16T02:02:04.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.960+0000 7f1ad4c33740 1 --2- 172.21.15.105:0/1861942687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aa24ebb0 0x55e2aa245420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:04.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.960+0000 7f1ad4c33740 1 --2- 172.21.15.105:0/1861942687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2aa2442f0 0x55e2aa269350 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:04.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.960+0000 7f1ad4c33740 1 --2- 172.21.15.105:0/1861942687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2aa2439d0 0x55e2aa25fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:04.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.960+0000 7f1ad4c33740 1 -- 172.21.15.105:0/1861942687 >> 172.21.15.105:0/1861942687 conn(0x55e2aa23ff60 msgr2=0x55e2aa24a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:04.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.960+0000 7f1ad4c33740 1 -- 172.21.15.105:0/1861942687 shutdown_connections 2024-09-16T02:02:04.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.960+0000 7f1ad4c33740 1 -- 172.21.15.105:0/1861942687 wait complete. 2024-09-16T02:02:04.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.960+0000 7f1ad4c33740 1 -- start start 2024-09-16T02:02:04.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.961+0000 7f1ad4c33740 1 Processor -- start 2024-09-16T02:02:04.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.961+0000 7f1ad4c33740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aa2439d0 0x55e2aa248ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:04.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.961+0000 7f1ad4c33740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2aa2442f0 0x55e2aa249530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:04.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.961+0000 7f1ad4c33740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2aa24ebb0 0x55e2aa247dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:04.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.961+0000 7f1ad4c33740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e2aa260b00 con 0x55e2aa24ebb0 2024-09-16T02:02:04.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.961+0000 7f1ad4c33740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e2aa260430 con 0x55e2aa2442f0 2024-09-16T02:02:04.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.961+0000 7f1ad4c33740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e2aa2611d0 con 0x55e2aa2439d0 2024-09-16T02:02:04.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.961+0000 7f1ad4c33740 10 client.0 _ll_register_callbacks cb 0x7f1ac000c170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:02:04.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.961+0000 7f1ad0df7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2aa24ebb0 0x55e2aa247dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:04.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.961+0000 7f1acbfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aa2439d0 0x55e2aa248ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:04.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.961+0000 7f1acbfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aa2439d0 0x55e2aa248ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.105:36736/0 (socket says 172.21.15.105:36736) 2024-09-16T02:02:04.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.961+0000 7f1acbfff640 1 -- 172.21.15.105:0/876410121 learned_addr learned my addr 172.21.15.105:0/876410121 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:04.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.961+0000 7f1acb7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2aa2442f0 0x55e2aa249530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:04.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.961+0000 7f1ad0df7640 1 -- 172.21.15.105:0/876410121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aa2439d0 msgr2=0x55e2aa248ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:04.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.961+0000 7f1ad0df7640 1 --2- 172.21.15.105:0/876410121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aa2439d0 0x55e2aa248ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:04.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.961+0000 7f1ad0df7640 1 -- 172.21.15.105:0/876410121 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2aa2442f0 msgr2=0x55e2aa249530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:04.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.961+0000 7f1ad0df7640 1 --2- 172.21.15.105:0/876410121 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2aa2442f0 0x55e2aa249530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:04.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.961+0000 7f1ad0df7640 1 -- 172.21.15.105:0/876410121 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2aa2485d0 con 0x55e2aa24ebb0 2024-09-16T02:02:04.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.962+0000 7f1ad0df7640 1 --2- 172.21.15.105:0/876410121 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2aa24ebb0 0x55e2aa247dd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1ac402f3b0 tx=0x7f1ac4031c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b88355e28b02d4e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:04.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.962+0000 7f1aab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:02:04.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.962+0000 7f1aab7fe640 1 -- 172.21.15.105:0/876410121 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac404d070 con 0x55e2aa24ebb0 2024-09-16T02:02:04.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.962+0000 7f1ad4c33740 1 -- 172.21.15.105:0/876410121 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ac0013d10 con 0x55e2aa24ebb0 2024-09-16T02:02:04.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.962+0000 7f1aab7fe640 1 -- 172.21.15.105:0/876410121 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f1ac4004250 con 0x55e2aa24ebb0 2024-09-16T02:02:04.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.962+0000 7f1aab7fe640 1 -- 172.21.15.105:0/876410121 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1ac4061050 con 0x55e2aa24ebb0 2024-09-16T02:02:04.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.962+0000 7f1ad4c33740 10 client.15030 fetch_fsmap learned FSMap version 27 2024-09-16T02:02:04.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.962+0000 7f1ad4c33740 1 -- 172.21.15.105:0/876410121 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e2aa2cdbd0 con 0x55e2aa24ebb0 2024-09-16T02:02:04.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.962+0000 7f1aab7fe640 1 -- 172.21.15.105:0/876410121 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ac0013d10 con 0x55e2aa24ebb0 2024-09-16T02:02:04.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.962+0000 7f1aab7fe640 1 -- 172.21.15.105:0/876410121 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1ac4030a70 con 0x55e2aa24ebb0 2024-09-16T02:02:04.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.962+0000 7f1ad4c33740 10 client.15030 fetch_fsmap finished waiting for FSMap version 27 2024-09-16T02:02:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.962+0000 7f1ad4c33740 10 client.15030 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T02:02:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.962+0000 7f1ad4c33740 20 client.15030 populate_metadata read hostname 'smithi105' 2024-09-16T02:02:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.962+0000 7f1ad4c33740 1 --2- 172.21.15.105:0/876410121 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x55e2aa2d2310 0x55e2aa2d1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.962+0000 7f1ad4c33740 4 client.15030 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T02:02:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.962+0000 7f1ad4c33740 1 -- 172.21.15.105:0/876410121 --> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] -- command(tid 0: {"prefix": "status"}) -- 0x55e2aa1b1b10 con 0x55e2aa2d2310 2024-09-16T02:02:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.963+0000 7f1acbfff640 1 --2- 172.21.15.105:0/876410121 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x55e2aa2d2310 0x55e2aa2d1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.963+0000 7f1acbfff640 1 --2- 172.21.15.105:0/876410121 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x55e2aa2d2310 0x55e2aa2d1020 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ef18b78f431703b9 server_cookie=1fc8db16f82c7be7 in_seq=0 out_seq=0 2024-09-16T02:02:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.963+0000 7f1aab7fe640 10 client.15030 ms_handle_connect on v2:172.21.15.190:6836/3181257549 2024-09-16T02:02:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.963+0000 7f1aab7fe640 1 -- 172.21.15.105:0/876410121 <== mds.3 v2:172.21.15.190:6836/3181257549 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55e2aa1b1b10 con 0x55e2aa2d2310 2024-09-16T02:02:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.963+0000 7f1aab7fe640 10 client.15030 handle_command_reply: tid=0 2024-09-16T02:02:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.963+0000 7f1ad4c33740 1 client.15030 shutdown 2024-09-16T02:02:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.963+0000 7f1ad4c33740 1 -- 172.21.15.105:0/876410121 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2aa24ebb0 msgr2=0x55e2aa247dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.963+0000 7f1ad4c33740 1 --2- 172.21.15.105:0/876410121 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2aa24ebb0 0x55e2aa247dd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1ac402f3b0 tx=0x7f1ac4031c70 comp rx=0 tx=0).stop 2024-09-16T02:02:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.963+0000 7f1ad4c33740 1 -- 172.21.15.105:0/876410121 shutdown_connections 2024-09-16T02:02:04.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.963+0000 7f1ad4c33740 1 --2- 172.21.15.105:0/876410121 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x55e2aa2d2310 0x55e2aa2d1020 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:04.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.963+0000 7f1ad4c33740 1 --2- 172.21.15.105:0/876410121 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2aa24ebb0 0x55e2aa247dd0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:04.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.963+0000 7f1ad4c33740 1 --2- 172.21.15.105:0/876410121 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2aa2442f0 0x55e2aa249530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:04.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.963+0000 7f1ad4c33740 1 --2- 172.21.15.105:0/876410121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aa2439d0 0x55e2aa248ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:04.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.963+0000 7f1aab7fe640 0 client.15030 ms_handle_reset on v2:172.21.15.190:6836/3181257549 2024-09-16T02:02:04.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.963+0000 7f1ad4c33740 1 -- 172.21.15.105:0/876410121 >> 172.21.15.105:0/876410121 conn(0x55e2aa23ff60 msgr2=0x55e2aa245c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:04.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.963+0000 7f1ad4c33740 1 -- 172.21.15.105:0/876410121 shutdown_connections 2024-09-16T02:02:04.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.963+0000 7f1ad4c33740 1 -- 172.21.15.105:0/876410121 wait complete. 2024-09-16T02:02:04.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.964+0000 7f1ad4c33740 20 client.15030 trim_cache size 0 max 16384 2024-09-16T02:02:04.972 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-16T02:02:04.973 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "423b7a2a-73ce-11ef-bceb-c7b262605968", 2024-09-16T02:02:04.973 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 3, 2024-09-16T02:02:04.973 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 24457, 2024-09-16T02:02:04.973 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2024-09-16T02:02:04.973 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2024-09-16T02:02:04.973 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2024-09-16T02:02:04.973 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 79.454104513999994, 2024-09-16T02:02:04.973 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 27, 2024-09-16T02:02:04.973 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 81, 2024-09-16T02:02:04.973 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 79, 2024-09-16T02:02:04.973 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 115.098000251 2024-09-16T02:02:04.973 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-16T02:02:04.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.964+0000 7f1ad285a640 1 -- 172.21.15.105:0/3923023983 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f1ab40a3350 msgr2=0x7f1ab40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:04.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.964+0000 7f1ad285a640 1 --2- 172.21.15.105:0/3923023983 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f1ab40a3350 0x7f1ab40a5810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4032e30 tx=0x7f1ac4042000 comp rx=0 tx=0).stop 2024-09-16T02:02:04.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.964+0000 7f1ad285a640 1 -- 172.21.15.105:0/3923023983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1acc101df0 msgr2=0x7f1acc198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:04.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.964+0000 7f1ad285a640 1 --2- 172.21.15.105:0/3923023983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1acc101df0 0x7f1acc198430 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1abc00b4a0 tx=0x7f1abc00b960 comp rx=0 tx=0).stop 2024-09-16T02:02:04.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.964+0000 7f1ad285a640 1 -- 172.21.15.105:0/3923023983 shutdown_connections 2024-09-16T02:02:04.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.964+0000 7f1ad285a640 1 --2- 172.21.15.105:0/3923023983 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f1ab40a3350 0x7f1ab40a5810 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:04.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.964+0000 7f1ad285a640 1 --2- 172.21.15.105:0/3923023983 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1acc1027a0 0x7f1acc198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:04.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.964+0000 7f1ad285a640 1 --2- 172.21.15.105:0/3923023983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1acc106f20 0x7f1acc19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:04.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.964+0000 7f1ad285a640 1 --2- 172.21.15.105:0/3923023983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1acc101df0 0x7f1acc198430 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:04.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.964+0000 7f1ad285a640 1 -- 172.21.15.105:0/3923023983 >> 172.21.15.105:0/3923023983 conn(0x7f1acc0fd8b0 msgr2=0x7f1acc0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:04.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.964+0000 7f1ad285a640 1 -- 172.21.15.105:0/3923023983 shutdown_connections 2024-09-16T02:02:04.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:04.964+0000 7f1ad285a640 1 -- 172.21.15.105:0/3923023983 wait complete. 2024-09-16T02:02:05.743 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-16T02:02:05.997 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:02:06.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.510+0000 7f42d8113640 1 -- 172.21.15.105:0/4073838069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42d0102570 msgr2=0x7f42d01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:06.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.510+0000 7f42d8113640 1 --2- 172.21.15.105:0/4073838069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42d0102570 0x7f42d01029b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009ca0 tx=0x7f42c402a950 comp rx=0 tx=0).stop 2024-09-16T02:02:06.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.510+0000 7f42d8113640 1 -- 172.21.15.105:0/4073838069 shutdown_connections 2024-09-16T02:02:06.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.510+0000 7f42d8113640 1 --2- 172.21.15.105:0/4073838069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42d0102ef0 0x7f42d010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:06.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.510+0000 7f42d8113640 1 --2- 172.21.15.105:0/4073838069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42d0102570 0x7f42d01029b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:06.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.510+0000 7f42d8113640 1 --2- 172.21.15.105:0/4073838069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42d0100e20 0x7f42d0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:06.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.510+0000 7f42d8113640 1 -- 172.21.15.105:0/4073838069 >> 172.21.15.105:0/4073838069 conn(0x7f42d00fc7d0 msgr2=0x7f42d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:06.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.510+0000 7f42d8113640 1 -- 172.21.15.105:0/4073838069 shutdown_connections 2024-09-16T02:02:06.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.510+0000 7f42d8113640 1 -- 172.21.15.105:0/4073838069 wait complete. 2024-09-16T02:02:06.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.511+0000 7f42d8113640 1 Processor -- start 2024-09-16T02:02:06.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.511+0000 7f42d8113640 1 -- start start 2024-09-16T02:02:06.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.511+0000 7f42d8113640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42d0100e20 0x7f42d0197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:06.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.511+0000 7f42d8113640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42d0102570 0x7f42d0197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:06.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.511+0000 7f42d8113640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42d0102ef0 0x7f42d019bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:06.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.511+0000 7f42d8113640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f42d0110e00 con 0x7f42d0100e20 2024-09-16T02:02:06.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.511+0000 7f42d8113640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f42d0110c80 con 0x7f42d0102570 2024-09-16T02:02:06.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.511+0000 7f42d8113640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f42d0110f80 con 0x7f42d0102ef0 2024-09-16T02:02:06.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.511+0000 7f42d5eaf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42d0100e20 0x7f42d0197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:06.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.512+0000 7f42d5eaf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42d0100e20 0x7f42d0197320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57806/0 (socket says 172.21.15.105:57806) 2024-09-16T02:02:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.512+0000 7f42d5eaf640 1 -- 172.21.15.105:0/2927142991 learned_addr learned my addr 172.21.15.105:0/2927142991 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.512+0000 7f42d56ae640 1 --2- 172.21.15.105:0/2927142991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42d0102570 0x7f42d0197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.512+0000 7f42d66b0640 1 --2- 172.21.15.105:0/2927142991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42d0102ef0 0x7f42d019bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.512+0000 7f42d5eaf640 1 -- 172.21.15.105:0/2927142991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42d0102ef0 msgr2=0x7f42d019bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.512+0000 7f42d5eaf640 1 --2- 172.21.15.105:0/2927142991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42d0102ef0 0x7f42d019bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.512+0000 7f42d5eaf640 1 -- 172.21.15.105:0/2927142991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42d0102570 msgr2=0x7f42d0197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.512+0000 7f42d5eaf640 1 --2- 172.21.15.105:0/2927142991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42d0102570 0x7f42d0197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.512+0000 7f42d5eaf640 1 -- 172.21.15.105:0/2927142991 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d019c2d0 con 0x7f42d0100e20 2024-09-16T02:02:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.512+0000 7f42d5eaf640 1 --2- 172.21.15.105:0/2927142991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42d0100e20 0x7f42d0197320 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f42c8009970 tx=0x7f42c8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.512+0000 7f42ceffd640 1 -- 172.21.15.105:0/2927142991 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c800d020 con 0x7f42d0100e20 2024-09-16T02:02:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.512+0000 7f42d8113640 1 -- 172.21.15.105:0/2927142991 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42d019c5c0 con 0x7f42d0100e20 2024-09-16T02:02:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.512+0000 7f42ceffd640 1 -- 172.21.15.105:0/2927142991 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42c800d1c0 con 0x7f42d0100e20 2024-09-16T02:02:06.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.512+0000 7f42ceffd640 1 -- 172.21.15.105:0/2927142991 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c800cc50 con 0x7f42d0100e20 2024-09-16T02:02:06.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.512+0000 7f42d8113640 1 -- 172.21.15.105:0/2927142991 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42d0069f70 con 0x7f42d0100e20 2024-09-16T02:02:06.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.515+0000 7f42da4ec740 1 Processor -- start 2024-09-16T02:02:06.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.515+0000 7f42da4ec740 1 -- start start 2024-09-16T02:02:06.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.515+0000 7f42ceffd640 1 -- 172.21.15.105:0/2927142991 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f42c800c460 con 0x7f42d0100e20 2024-09-16T02:02:06.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.515+0000 7f42da4ec740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5624776719d0 0x562477671db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:06.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.515+0000 7f42da4ec740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624776722f0 0x56247767c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:06.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.515+0000 7f42da4ec740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56247767cbb0 0x56247767efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:06.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.515+0000 7f42da4ec740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5624775d8090 con 0x5624776719d0 2024-09-16T02:02:06.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.515+0000 7f42da4ec740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56247757fa70 con 0x5624776722f0 2024-09-16T02:02:06.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.515+0000 7f42da4ec740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5624775d4050 con 0x56247767cbb0 2024-09-16T02:02:06.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.515+0000 7f42d66b0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5624776719d0 0x562477671db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:06.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.515+0000 7f42d66b0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5624776719d0 0x562477671db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57820/0 (socket says 172.21.15.105:57820) 2024-09-16T02:02:06.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.515+0000 7f42d66b0640 1 -- 172.21.15.105:0/2054470633 learned_addr learned my addr 172.21.15.105:0/2054470633 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:06.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.515+0000 7f42d5eaf640 1 --2- 172.21.15.105:0/2054470633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624776722f0 0x56247767c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:06.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.515+0000 7f42d56ae640 1 --2- 172.21.15.105:0/2054470633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56247767cbb0 0x56247767efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:06.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.515+0000 7f42d66b0640 1 -- 172.21.15.105:0/2054470633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56247767cbb0 msgr2=0x56247767efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:06.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.516+0000 7f42d66b0640 1 --2- 172.21.15.105:0/2054470633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56247767cbb0 0x56247767efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:06.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.516+0000 7f42d66b0640 1 -- 172.21.15.105:0/2054470633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624776722f0 msgr2=0x56247767c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:06.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.516+0000 7f42d66b0640 1 --2- 172.21.15.105:0/2054470633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624776722f0 0x56247767c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:06.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.516+0000 7f42d66b0640 1 -- 172.21.15.105:0/2054470633 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56247758c130 con 0x5624776719d0 2024-09-16T02:02:06.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.516+0000 7f42d66b0640 1 --2- 172.21.15.105:0/2054470633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5624776719d0 0x562477671db0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f42c000cb40 tx=0x7f42c000e350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4538ea28a247b6cf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:06.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.516+0000 7f42ceffd640 1 --2- 172.21.15.105:0/2927142991 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f42b80a32d0 0x7f42b80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:06.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.516+0000 7f42b7fff640 1 -- 172.21.15.105:0/2054470633 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c0004020 con 0x5624776719d0 2024-09-16T02:02:06.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.516+0000 7f42d5eaf640 1 --2- 172.21.15.105:0/2927142991 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f42b80a32d0 0x7f42b80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:06.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.516+0000 7f42ceffd640 1 -- 172.21.15.105:0/2927142991 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f42c80d0ec0 con 0x7f42d0100e20 2024-09-16T02:02:06.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.516+0000 7f42b7fff640 1 -- 172.21.15.105:0/2054470633 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42c00041c0 con 0x5624776719d0 2024-09-16T02:02:06.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.516+0000 7f42b7fff640 1 -- 172.21.15.105:0/2054470633 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c0010690 con 0x5624776719d0 2024-09-16T02:02:06.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.517+0000 7f42d5eaf640 1 --2- 172.21.15.105:0/2927142991 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f42b80a32d0 0x7f42b80a5790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x56247766edc0 tx=0x7f42c800c2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:06.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.517+0000 7f42da4ec740 1 -- 172.21.15.105:0/2054470633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5624776719d0 msgr2=0x562477671db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:06.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.517+0000 7f42da4ec740 1 --2- 172.21.15.105:0/2054470633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5624776719d0 0x562477671db0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f42c000cb40 tx=0x7f42c000e350 comp rx=0 tx=0).stop 2024-09-16T02:02:06.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.517+0000 7f42da4ec740 1 -- 172.21.15.105:0/2054470633 shutdown_connections 2024-09-16T02:02:06.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.517+0000 7f42da4ec740 1 --2- 172.21.15.105:0/2054470633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56247767cbb0 0x56247767efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:06.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.517+0000 7f42da4ec740 1 --2- 172.21.15.105:0/2054470633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624776722f0 0x56247767c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:06.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.517+0000 7f42da4ec740 1 --2- 172.21.15.105:0/2054470633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5624776719d0 0x562477671db0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:06.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.517+0000 7f42da4ec740 1 -- 172.21.15.105:0/2054470633 >> 172.21.15.105:0/2054470633 conn(0x56247766df60 msgr2=0x56247766e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:06.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.517+0000 7f42da4ec740 1 -- 172.21.15.105:0/2054470633 shutdown_connections 2024-09-16T02:02:06.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.517+0000 7f42da4ec740 1 -- 172.21.15.105:0/2054470633 wait complete. 2024-09-16T02:02:06.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.517+0000 7f42da4ec740 1 -- start start 2024-09-16T02:02:06.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.517+0000 7f42da4ec740 1 Processor -- start 2024-09-16T02:02:06.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.518+0000 7f42da4ec740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5624776719d0 0x56247767c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:06.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.518+0000 7f42da4ec740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5624776722f0 0x562477679770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:06.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.518+0000 7f42da4ec740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56247767cbb0 0x562477679cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:06.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.518+0000 7f42da4ec740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562477683b90 con 0x5624776722f0 2024-09-16T02:02:06.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.518+0000 7f42da4ec740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562477592f20 con 0x56247767cbb0 2024-09-16T02:02:06.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.518+0000 7f42da4ec740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x562477683d10 con 0x5624776719d0 2024-09-16T02:02:06.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.518+0000 7f42da4ec740 10 client.0 _ll_register_callbacks cb 0x56247755c030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:02:06.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.518+0000 7f42d56ae640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5624776722f0 0x562477679770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:06.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.518+0000 7f42d56ae640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5624776722f0 0x562477679770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57824/0 (socket says 172.21.15.105:57824) 2024-09-16T02:02:06.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.518+0000 7f42d56ae640 1 -- 172.21.15.105:0/2741349464 learned_addr learned my addr 172.21.15.105:0/2741349464 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:06.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.518+0000 7f42d66b0640 1 --2- 172.21.15.105:0/2741349464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5624776719d0 0x56247767c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:06.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.518+0000 7f42d66b0640 1 --2- 172.21.15.105:0/2741349464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56247767cbb0 0x562477679cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:06.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.518+0000 7f42d56ae640 1 -- 172.21.15.105:0/2741349464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5624776719d0 msgr2=0x56247767c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:06.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.518+0000 7f42d56ae640 1 --2- 172.21.15.105:0/2741349464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5624776719d0 0x56247767c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:06.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.518+0000 7f42d56ae640 1 -- 172.21.15.105:0/2741349464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56247767cbb0 msgr2=0x562477679cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:06.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.518+0000 7f42d56ae640 1 --2- 172.21.15.105:0/2741349464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56247767cbb0 0x562477679cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:06.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.518+0000 7f42d56ae640 1 -- 172.21.15.105:0/2741349464 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624773492b0 con 0x5624776722f0 2024-09-16T02:02:06.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.518+0000 7f42d66b0640 1 --2- 172.21.15.105:0/2741349464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56247767cbb0 0x562477679cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:02:06.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.518+0000 7f42d56ae640 1 --2- 172.21.15.105:0/2741349464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5624776722f0 0x562477679770 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009ca0 tx=0x7f42c402f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d3d35c7063722a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:06.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.518+0000 7f42b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:02:06.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.518+0000 7f42b57fa640 1 -- 172.21.15.105:0/2741349464 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c4049060 con 0x5624776722f0 2024-09-16T02:02:06.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.518+0000 7f42b57fa640 1 -- 172.21.15.105:0/2741349464 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f42c4003d80 con 0x5624776722f0 2024-09-16T02:02:06.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.519+0000 7f42da4ec740 1 -- 172.21.15.105:0/2741349464 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42c00041c0 con 0x5624776722f0 2024-09-16T02:02:06.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.519+0000 7f42b57fa640 1 -- 172.21.15.105:0/2741349464 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f42c4034040 con 0x5624776722f0 2024-09-16T02:02:06.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.519+0000 7f42da4ec740 10 client.15042 fetch_fsmap learned FSMap version 27 2024-09-16T02:02:06.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.519+0000 7f42da4ec740 1 -- 172.21.15.105:0/2741349464 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56247768f480 con 0x5624776722f0 2024-09-16T02:02:06.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.519+0000 7f42b57fa640 1 -- 172.21.15.105:0/2741349464 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42c00041c0 con 0x5624776722f0 2024-09-16T02:02:06.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.519+0000 7f42b57fa640 1 -- 172.21.15.105:0/2741349464 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f42c4030440 con 0x5624776722f0 2024-09-16T02:02:06.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.519+0000 7f42da4ec740 10 client.15042 fetch_fsmap finished waiting for FSMap version 27 2024-09-16T02:02:06.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.519+0000 7f42da4ec740 10 client.15042 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-16T02:02:06.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.519+0000 7f42da4ec740 20 client.15042 populate_metadata read hostname 'smithi105' 2024-09-16T02:02:06.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.519+0000 7f42da4ec740 1 --2- 172.21.15.105:0/2741349464 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x562477691ab0 0x562477692f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:06.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.519+0000 7f42da4ec740 4 client.15042 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:02:06.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.519+0000 7f42da4ec740 1 -- 172.21.15.105:0/2741349464 --> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624775e5600 con 0x562477691ab0 2024-09-16T02:02:06.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.519+0000 7f42d66b0640 1 --2- 172.21.15.105:0/2741349464 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x562477691ab0 0x562477692f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:06.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.520+0000 7f42d66b0640 1 --2- 172.21.15.105:0/2741349464 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x562477691ab0 0x562477692f20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7e064a65a5e57d00 server_cookie=5c32ab6014595091 in_seq=0 out_seq=0 2024-09-16T02:02:06.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.520+0000 7f42b57fa640 10 client.15042 ms_handle_connect on v2:172.21.15.137:6836/2788541030 2024-09-16T02:02:06.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.521+0000 7f42b57fa640 1 -- 172.21.15.105:0/2741349464 <== mds.2 v2:172.21.15.137:6836/2788541030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5624775e5600 con 0x562477691ab0 2024-09-16T02:02:06.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.521+0000 7f42b57fa640 10 client.15042 handle_command_reply: tid=0 2024-09-16T02:02:06.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.521+0000 7f42da4ec740 1 client.15042 shutdown 2024-09-16T02:02:06.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.521+0000 7f42da4ec740 1 -- 172.21.15.105:0/2741349464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5624776722f0 msgr2=0x562477679770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:06.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.521+0000 7f42da4ec740 1 --2- 172.21.15.105:0/2741349464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5624776722f0 0x562477679770 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009ca0 tx=0x7f42c402f5d0 comp rx=0 tx=0).stop 2024-09-16T02:02:06.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.521+0000 7f42da4ec740 1 -- 172.21.15.105:0/2741349464 shutdown_connections 2024-09-16T02:02:06.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.521+0000 7f42da4ec740 1 --2- 172.21.15.105:0/2741349464 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x562477691ab0 0x562477692f20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:06.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.521+0000 7f42da4ec740 1 --2- 172.21.15.105:0/2741349464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56247767cbb0 0x562477679cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:06.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.521+0000 7f42da4ec740 1 --2- 172.21.15.105:0/2741349464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5624776722f0 0x562477679770 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:06.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.521+0000 7f42b57fa640 0 client.15042 ms_handle_reset on v2:172.21.15.137:6836/2788541030 2024-09-16T02:02:06.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.521+0000 7f42da4ec740 1 --2- 172.21.15.105:0/2741349464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5624776719d0 0x56247767c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:06.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.521+0000 7f42da4ec740 1 -- 172.21.15.105:0/2741349464 >> 172.21.15.105:0/2741349464 conn(0x56247766df60 msgr2=0x562477673c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:06.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.521+0000 7f42da4ec740 1 -- 172.21.15.105:0/2741349464 shutdown_connections 2024-09-16T02:02:06.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.521+0000 7f42da4ec740 1 -- 172.21.15.105:0/2741349464 wait complete. 2024-09-16T02:02:06.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.521+0000 7f42da4ec740 20 client.15042 trim_cache size 0 max 16384 2024-09-16T02:02:06.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.539+0000 7f42da4ec740 1 Processor -- start 2024-09-16T02:02:06.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.539+0000 7f42da4ec740 1 -- start start 2024-09-16T02:02:06.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.539+0000 7f42da4ec740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624776719d0 0x562477692f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:06.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.539+0000 7f42da4ec740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5624776722f0 0x5624776769d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:06.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.539+0000 7f42da4ec740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56247767cbb0 0x562477676f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:06.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.539+0000 7f42da4ec740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562477691260 con 0x56247767cbb0 2024-09-16T02:02:06.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.539+0000 7f42da4ec740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562477691930 con 0x5624776719d0 2024-09-16T02:02:06.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.539+0000 7f42da4ec740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x562477692000 con 0x5624776722f0 2024-09-16T02:02:06.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.539+0000 7f42d66b0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56247767cbb0 0x562477676f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:06.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.539+0000 7f42d66b0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56247767cbb0 0x562477676f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57832/0 (socket says 172.21.15.105:57832) 2024-09-16T02:02:06.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.539+0000 7f42d66b0640 1 -- 172.21.15.105:0/1563552089 learned_addr learned my addr 172.21.15.105:0/1563552089 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:06.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.540+0000 7f42d66b0640 1 --2- 172.21.15.105:0/1563552089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624776719d0 0x562477692f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:06.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.540+0000 7f42d56ae640 1 --2- 172.21.15.105:0/1563552089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5624776722f0 0x5624776769d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:06.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.540+0000 7f42d66b0640 1 -- 172.21.15.105:0/1563552089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5624776722f0 msgr2=0x5624776769d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:06.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.540+0000 7f42d66b0640 1 --2- 172.21.15.105:0/1563552089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5624776722f0 0x5624776769d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:06.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.540+0000 7f42d66b0640 1 -- 172.21.15.105:0/1563552089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624776719d0 msgr2=0x562477692f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:06.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.540+0000 7f42d66b0640 1 --2- 172.21.15.105:0/1563552089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624776719d0 0x562477692f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:06.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.540+0000 7f42d66b0640 1 -- 172.21.15.105:0/1563552089 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562477683ca0 con 0x56247767cbb0 2024-09-16T02:02:06.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.540+0000 7f42d66b0640 1 --2- 172.21.15.105:0/1563552089 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56247767cbb0 0x562477676f10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f42c000e850 tx=0x7f42c0003bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18523e9d9dd10ca0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:06.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.540+0000 7f42b77fe640 1 -- 172.21.15.105:0/1563552089 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c002d040 con 0x56247767cbb0 2024-09-16T02:02:06.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.540+0000 7f42b77fe640 1 -- 172.21.15.105:0/1563552089 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42c000ed90 con 0x56247767cbb0 2024-09-16T02:02:06.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.540+0000 7f42b77fe640 1 -- 172.21.15.105:0/1563552089 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c0029420 con 0x56247767cbb0 2024-09-16T02:02:06.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.541+0000 7f42da4ec740 1 -- 172.21.15.105:0/1563552089 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56247767cbb0 msgr2=0x562477676f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:06.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.541+0000 7f42da4ec740 1 --2- 172.21.15.105:0/1563552089 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56247767cbb0 0x562477676f10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f42c000e850 tx=0x7f42c0003bb0 comp rx=0 tx=0).stop 2024-09-16T02:02:06.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.541+0000 7f42da4ec740 1 -- 172.21.15.105:0/1563552089 shutdown_connections 2024-09-16T02:02:06.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.541+0000 7f42da4ec740 1 --2- 172.21.15.105:0/1563552089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5624776722f0 0x5624776769d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:06.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.541+0000 7f42da4ec740 1 --2- 172.21.15.105:0/1563552089 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56247767cbb0 0x562477676f10 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:06.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.541+0000 7f42da4ec740 1 --2- 172.21.15.105:0/1563552089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624776719d0 0x562477692f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:06.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.541+0000 7f42da4ec740 1 -- 172.21.15.105:0/1563552089 >> 172.21.15.105:0/1563552089 conn(0x56247766df60 msgr2=0x562477678040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:06.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.541+0000 7f42da4ec740 1 -- 172.21.15.105:0/1563552089 shutdown_connections 2024-09-16T02:02:06.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.541+0000 7f42da4ec740 1 -- 172.21.15.105:0/1563552089 wait complete. 2024-09-16T02:02:06.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.541+0000 7f42da4ec740 1 -- start start 2024-09-16T02:02:06.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.541+0000 7f42da4ec740 1 Processor -- start 2024-09-16T02:02:06.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.542+0000 7f42da4ec740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5624776719d0 0x56247769b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:06.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.542+0000 7f42da4ec740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624776722f0 0x562477694b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:06.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.542+0000 7f42da4ec740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56247767cbb0 0x562477695060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:06.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.542+0000 7f42da4ec740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562477692da0 con 0x5624776719d0 2024-09-16T02:02:06.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.542+0000 7f42da4ec740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5624776926d0 con 0x5624776722f0 2024-09-16T02:02:06.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.542+0000 7f42da4ec740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x562477693470 con 0x56247767cbb0 2024-09-16T02:02:06.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.542+0000 7f42da4ec740 10 client.0 _ll_register_callbacks cb 0x7f42c402fd90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:02:06.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.542+0000 7f42d66b0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5624776719d0 0x56247769b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:06.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.542+0000 7f42d66b0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5624776719d0 0x56247769b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57846/0 (socket says 172.21.15.105:57846) 2024-09-16T02:02:06.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.542+0000 7f42d66b0640 1 -- 172.21.15.105:0/905183797 learned_addr learned my addr 172.21.15.105:0/905183797 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:06.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.542+0000 7f42d66b0640 1 -- 172.21.15.105:0/905183797 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56247767cbb0 msgr2=0x562477695060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:06.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.542+0000 7f42d66b0640 1 --2- 172.21.15.105:0/905183797 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56247767cbb0 0x562477695060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:06.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.542+0000 7f42d66b0640 1 -- 172.21.15.105:0/905183797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624776722f0 msgr2=0x562477694b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:06.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.542+0000 7f42d56ae640 1 --2- 172.21.15.105:0/905183797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624776722f0 0x562477694b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:06.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.542+0000 7f42d66b0640 1 --2- 172.21.15.105:0/905183797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624776722f0 0x562477694b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:06.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.542+0000 7f42d66b0640 1 -- 172.21.15.105:0/905183797 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562477695720 con 0x5624776719d0 2024-09-16T02:02:06.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.542+0000 7f42d66b0640 1 --2- 172.21.15.105:0/905183797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5624776719d0 0x56247769b7a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f42c0003e50 tx=0x7f42c0003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e9ac28aea4bbb00 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:06.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.542+0000 7f42b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:02:06.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.542+0000 7f42b57fa640 1 -- 172.21.15.105:0/905183797 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c00035f0 con 0x5624776719d0 2024-09-16T02:02:06.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.542+0000 7f42b57fa640 1 -- 172.21.15.105:0/905183797 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f42c0003790 con 0x5624776719d0 2024-09-16T02:02:06.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.543+0000 7f42da4ec740 1 -- 172.21.15.105:0/905183797 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42c000ed90 con 0x5624776719d0 2024-09-16T02:02:06.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.543+0000 7f42b57fa640 1 -- 172.21.15.105:0/905183797 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f42c00039d0 con 0x5624776719d0 2024-09-16T02:02:06.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.543+0000 7f42da4ec740 10 client.15048 fetch_fsmap learned FSMap version 27 2024-09-16T02:02:06.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.543+0000 7f42da4ec740 1 -- 172.21.15.105:0/905183797 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5624776fc1c0 con 0x5624776719d0 2024-09-16T02:02:06.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.543+0000 7f42b57fa640 1 -- 172.21.15.105:0/905183797 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42c000ed90 con 0x5624776719d0 2024-09-16T02:02:06.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.543+0000 7f42b57fa640 1 -- 172.21.15.105:0/905183797 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f42c0004c90 con 0x5624776719d0 2024-09-16T02:02:06.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.543+0000 7f42da4ec740 10 client.15048 fetch_fsmap finished waiting for FSMap version 27 2024-09-16T02:02:06.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.543+0000 7f42da4ec740 10 client.15048 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-16T02:02:06.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.543+0000 7f42da4ec740 20 client.15048 populate_metadata read hostname 'smithi105' 2024-09-16T02:02:06.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.543+0000 7f42da4ec740 1 --2- 172.21.15.105:0/905183797 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x562477691ab0 0x5624776ff1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:06.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.543+0000 7f42da4ec740 4 client.15048 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T02:02:06.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.543+0000 7f42da4ec740 1 -- 172.21.15.105:0/905183797 --> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] -- command(tid 0: {"prefix": "status"}) -- 0x5624775dfb10 con 0x562477691ab0 2024-09-16T02:02:06.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.544+0000 7f42d56ae640 1 --2- 172.21.15.105:0/905183797 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x562477691ab0 0x5624776ff1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:06.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.544+0000 7f42d56ae640 1 --2- 172.21.15.105:0/905183797 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x562477691ab0 0x5624776ff1f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2942bf1215121ed3 server_cookie=f337ac1dcb2ee64a in_seq=0 out_seq=0 2024-09-16T02:02:06.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.544+0000 7f42b57fa640 10 client.15048 ms_handle_connect on v2:172.21.15.137:6836/2788541030 2024-09-16T02:02:06.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.544+0000 7f42b57fa640 1 -- 172.21.15.105:0/905183797 <== mds.2 v2:172.21.15.137:6836/2788541030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5624775dfb10 con 0x562477691ab0 2024-09-16T02:02:06.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.544+0000 7f42b57fa640 10 client.15048 handle_command_reply: tid=0 2024-09-16T02:02:06.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.544+0000 7f42da4ec740 1 client.15048 shutdown 2024-09-16T02:02:06.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.545+0000 7f42da4ec740 1 -- 172.21.15.105:0/905183797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5624776719d0 msgr2=0x56247769b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:06.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.545+0000 7f42da4ec740 1 --2- 172.21.15.105:0/905183797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5624776719d0 0x56247769b7a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f42c0003e50 tx=0x7f42c0003e80 comp rx=0 tx=0).stop 2024-09-16T02:02:06.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.545+0000 7f42da4ec740 1 -- 172.21.15.105:0/905183797 shutdown_connections 2024-09-16T02:02:06.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.545+0000 7f42da4ec740 1 --2- 172.21.15.105:0/905183797 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x562477691ab0 0x5624776ff1f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:06.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.545+0000 7f42da4ec740 1 --2- 172.21.15.105:0/905183797 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56247767cbb0 0x562477695060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:06.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.545+0000 7f42da4ec740 1 --2- 172.21.15.105:0/905183797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624776722f0 0x562477694b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:06.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.545+0000 7f42b57fa640 0 client.15048 ms_handle_reset on v2:172.21.15.137:6836/2788541030 2024-09-16T02:02:06.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.545+0000 7f42da4ec740 1 --2- 172.21.15.105:0/905183797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5624776719d0 0x56247769b7a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:06.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.545+0000 7f42da4ec740 1 -- 172.21.15.105:0/905183797 >> 172.21.15.105:0/905183797 conn(0x56247766df60 msgr2=0x56247767ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:06.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.545+0000 7f42da4ec740 1 -- 172.21.15.105:0/905183797 shutdown_connections 2024-09-16T02:02:06.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.545+0000 7f42da4ec740 1 -- 172.21.15.105:0/905183797 wait complete. 2024-09-16T02:02:06.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.545+0000 7f42da4ec740 20 client.15048 trim_cache size 0 max 16384 2024-09-16T02:02:06.558 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-16T02:02:06.558 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "423b7a2a-73ce-11ef-bceb-c7b262605968", 2024-09-16T02:02:06.558 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 2, 2024-09-16T02:02:06.558 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 24461, 2024-09-16T02:02:06.558 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2024-09-16T02:02:06.558 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2024-09-16T02:02:06.558 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2024-09-16T02:02:06.559 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 82.047459876999994, 2024-09-16T02:02:06.559 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 27, 2024-09-16T02:02:06.559 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 81, 2024-09-16T02:02:06.559 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 79, 2024-09-16T02:02:06.559 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 113.975782204 2024-09-16T02:02:06.559 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-16T02:02:06.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.545+0000 7f42d8113640 1 -- 172.21.15.105:0/2927142991 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f42b80a32d0 msgr2=0x7f42b80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:06.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.545+0000 7f42d8113640 1 --2- 172.21.15.105:0/2927142991 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f42b80a32d0 0x7f42b80a5790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x56247766edc0 tx=0x7f42c800c2d0 comp rx=0 tx=0).stop 2024-09-16T02:02:06.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.545+0000 7f42d8113640 1 -- 172.21.15.105:0/2927142991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42d0100e20 msgr2=0x7f42d0197320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:06.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.545+0000 7f42d8113640 1 --2- 172.21.15.105:0/2927142991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42d0100e20 0x7f42d0197320 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f42c8009970 tx=0x7f42c8009e30 comp rx=0 tx=0).stop 2024-09-16T02:02:06.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.545+0000 7f42d8113640 1 -- 172.21.15.105:0/2927142991 shutdown_connections 2024-09-16T02:02:06.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.545+0000 7f42d8113640 1 --2- 172.21.15.105:0/2927142991 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f42b80a32d0 0x7f42b80a5790 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:06.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.545+0000 7f42d8113640 1 --2- 172.21.15.105:0/2927142991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42d0102ef0 0x7f42d019bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:06.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.545+0000 7f42d8113640 1 --2- 172.21.15.105:0/2927142991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42d0102570 0x7f42d0197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:06.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.545+0000 7f42d8113640 1 --2- 172.21.15.105:0/2927142991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42d0100e20 0x7f42d0197320 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:06.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.545+0000 7f42d8113640 1 -- 172.21.15.105:0/2927142991 >> 172.21.15.105:0/2927142991 conn(0x7f42d00fc7d0 msgr2=0x7f42d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:06.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.546+0000 7f42d8113640 1 -- 172.21.15.105:0/2927142991 shutdown_connections 2024-09-16T02:02:06.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:06.546+0000 7f42d8113640 1 -- 172.21.15.105:0/2927142991 wait complete. 2024-09-16T02:02:06.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:06 smithi105 ceph-mon[31197]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T02:02:06.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:06 smithi190 ceph-mon[34762]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T02:02:06.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:06 smithi137 ceph-mon[34099]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T02:02:07.378 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-16T02:02:07.627 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:02:08.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.076+0000 7fa5f4d9a640 1 -- 172.21.15.105:0/60449230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5f0106d00 msgr2=0x7fa5f01070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:08.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.076+0000 7fa5f4d9a640 1 --2- 172.21.15.105:0/60449230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5f0106d00 0x7fa5f01070e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0009900 tx=0x7fa5e002a410 comp rx=0 tx=0).stop 2024-09-16T02:02:08.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.076+0000 7fa5f4d9a640 1 -- 172.21.15.105:0/60449230 shutdown_connections 2024-09-16T02:02:08.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.076+0000 7fa5f4d9a640 1 --2- 172.21.15.105:0/60449230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5f0101f80 0x7fa5f010d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:08.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.076+0000 7fa5f4d9a640 1 --2- 172.21.15.105:0/60449230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5f01015e0 0x7fa5f0101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:08.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.076+0000 7fa5f4d9a640 1 --2- 172.21.15.105:0/60449230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5f0106d00 0x7fa5f01070e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:08.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.076+0000 7fa5f4d9a640 1 -- 172.21.15.105:0/60449230 >> 172.21.15.105:0/60449230 conn(0x7fa5f00fba90 msgr2=0x7fa5f00fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:08.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.076+0000 7fa5f4d9a640 1 -- 172.21.15.105:0/60449230 shutdown_connections 2024-09-16T02:02:08.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.076+0000 7fa5f4d9a640 1 -- 172.21.15.105:0/60449230 wait complete. 2024-09-16T02:02:08.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.076+0000 7fa5f4d9a640 1 Processor -- start 2024-09-16T02:02:08.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.077+0000 7fa5f4d9a640 1 -- start start 2024-09-16T02:02:08.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.077+0000 7fa5f4d9a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5f01015e0 0x7fa5f0198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:08.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.077+0000 7fa5f4d9a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5f0101f80 0x7fa5f01988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:08.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.077+0000 7fa5f4d9a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5f0106d00 0x7fa5f019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:08.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.077+0000 7fa5f4d9a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f0110e00 con 0x7fa5f01015e0 2024-09-16T02:02:08.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.077+0000 7fa5f4d9a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f0110c80 con 0x7fa5f0101f80 2024-09-16T02:02:08.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.077+0000 7fa5f4d9a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f0110f80 con 0x7fa5f0106d00 2024-09-16T02:02:08.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.077+0000 7fa5ee59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5f01015e0 0x7fa5f0198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:08.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.077+0000 7fa5ee59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5f01015e0 0x7fa5f0198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57874/0 (socket says 172.21.15.105:57874) 2024-09-16T02:02:08.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.077+0000 7fa5ee59c640 1 -- 172.21.15.105:0/2480314295 learned_addr learned my addr 172.21.15.105:0/2480314295 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:08.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.077+0000 7fa5edd9b640 1 --2- 172.21.15.105:0/2480314295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5f0101f80 0x7fa5f01988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:08.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.077+0000 7fa5eed9d640 1 --2- 172.21.15.105:0/2480314295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5f0106d00 0x7fa5f019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:08.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.077+0000 7fa5ee59c640 1 -- 172.21.15.105:0/2480314295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5f0106d00 msgr2=0x7fa5f019cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:08.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.077+0000 7fa5ee59c640 1 --2- 172.21.15.105:0/2480314295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5f0106d00 0x7fa5f019cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:08.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.077+0000 7fa5ee59c640 1 -- 172.21.15.105:0/2480314295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5f0101f80 msgr2=0x7fa5f01988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:08.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.077+0000 7fa5ee59c640 1 --2- 172.21.15.105:0/2480314295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5f0101f80 0x7fa5f01988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:08.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.077+0000 7fa5ee59c640 1 -- 172.21.15.105:0/2480314295 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5f019d310 con 0x7fa5f01015e0 2024-09-16T02:02:08.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.077+0000 7fa5edd9b640 1 --2- 172.21.15.105:0/2480314295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5f0101f80 0x7fa5f01988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:02:08.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.077+0000 7fa5ee59c640 1 --2- 172.21.15.105:0/2480314295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5f01015e0 0x7fa5f0198360 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0005940 tx=0x7fa5e0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:08.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.077+0000 7fa5df7fe640 1 -- 172.21.15.105:0/2480314295 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e003f030 con 0x7fa5f01015e0 2024-09-16T02:02:08.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.077+0000 7fa5df7fe640 1 -- 172.21.15.105:0/2480314295 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5e0003aa0 con 0x7fa5f01015e0 2024-09-16T02:02:08.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.077+0000 7fa5f4d9a640 1 -- 172.21.15.105:0/2480314295 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5f019d5a0 con 0x7fa5f01015e0 2024-09-16T02:02:08.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.078+0000 7fa5df7fe640 1 -- 172.21.15.105:0/2480314295 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e0035070 con 0x7fa5f01015e0 2024-09-16T02:02:08.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.078+0000 7fa5f4d9a640 1 -- 172.21.15.105:0/2480314295 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5f01a4ee0 con 0x7fa5f01015e0 2024-09-16T02:02:08.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.080+0000 7fa5df7fe640 1 -- 172.21.15.105:0/2480314295 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fa5e0042030 con 0x7fa5f01015e0 2024-09-16T02:02:08.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.080+0000 7fa5f7173740 1 Processor -- start 2024-09-16T02:02:08.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.081+0000 7fa5f7173740 1 -- start start 2024-09-16T02:02:08.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.081+0000 7fa5f7173740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dea70408d0 0x55dea7040cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:08.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.081+0000 7fa5f7173740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dea70411f0 0x55dea704fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:08.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.081+0000 7fa5f7173740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dea704ffc0 0x55dea70523b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:08.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.081+0000 7fa5f7173740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55dea6fa7090 con 0x55dea70411f0 2024-09-16T02:02:08.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.081+0000 7fa5f7173740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dea6f4ea70 con 0x55dea70408d0 2024-09-16T02:02:08.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.081+0000 7fa5f7173740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55dea6fa3050 con 0x55dea704ffc0 2024-09-16T02:02:08.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.081+0000 7fa5ee59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dea70411f0 0x55dea704fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:08.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.081+0000 7fa5ee59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dea70411f0 0x55dea704fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57880/0 (socket says 172.21.15.105:57880) 2024-09-16T02:02:08.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.081+0000 7fa5ee59c640 1 -- 172.21.15.105:0/203345313 learned_addr learned my addr 172.21.15.105:0/203345313 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:08.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.081+0000 7fa5eed9d640 1 --2- 172.21.15.105:0/203345313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dea70408d0 0x55dea7040cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:08.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.081+0000 7fa5df7fe640 1 --2- 172.21.15.105:0/2480314295 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa5d80a32d0 0x7fa5d80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:08.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.081+0000 7fa5ee59c640 1 -- 172.21.15.105:0/203345313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dea704ffc0 msgr2=0x55dea70523b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:08.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.081+0000 7fa5ee59c640 1 --2- 172.21.15.105:0/203345313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dea704ffc0 0x55dea70523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:08.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.081+0000 7fa5ee59c640 1 -- 172.21.15.105:0/203345313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dea70408d0 msgr2=0x55dea7040cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:08.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.081+0000 7fa5ee59c640 1 --2- 172.21.15.105:0/203345313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dea70408d0 0x55dea7040cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:08.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.081+0000 7fa5ee59c640 1 -- 172.21.15.105:0/203345313 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dea6f5b130 con 0x55dea70411f0 2024-09-16T02:02:08.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.081+0000 7fa5eed9d640 1 --2- 172.21.15.105:0/2480314295 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa5d80a32d0 0x7fa5d80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:08.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.082+0000 7fa5ee59c640 1 --2- 172.21.15.105:0/203345313 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dea70411f0 0x55dea704fa80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa5e00edbf0 tx=0x7fa5e00b8510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc77840ba364d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:08.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.082+0000 7fa5df7fe640 1 -- 172.21.15.105:0/2480314295 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fa5e00f0320 con 0x7fa5f01015e0 2024-09-16T02:02:08.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.082+0000 7fa5dcff9640 1 -- 172.21.15.105:0/203345313 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e00bc590 con 0x55dea70411f0 2024-09-16T02:02:08.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.082+0000 7fa5dcff9640 1 -- 172.21.15.105:0/203345313 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5e00e8220 con 0x55dea70411f0 2024-09-16T02:02:08.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.082+0000 7fa5dcff9640 1 -- 172.21.15.105:0/203345313 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e00bc120 con 0x55dea70411f0 2024-09-16T02:02:08.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.082+0000 7fa5f7173740 1 -- 172.21.15.105:0/203345313 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dea70411f0 msgr2=0x55dea704fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:08.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.082+0000 7fa5f7173740 1 --2- 172.21.15.105:0/203345313 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dea70411f0 0x55dea704fa80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa5e00edbf0 tx=0x7fa5e00b8510 comp rx=0 tx=0).stop 2024-09-16T02:02:08.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.082+0000 7fa5eed9d640 1 --2- 172.21.15.105:0/2480314295 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa5d80a32d0 0x7fa5d80a5790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa5f01078a0 tx=0x7fa5e8004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:08.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.083+0000 7fa5f7173740 1 -- 172.21.15.105:0/203345313 shutdown_connections 2024-09-16T02:02:08.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.083+0000 7fa5f7173740 1 --2- 172.21.15.105:0/203345313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dea704ffc0 0x55dea70523b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:08.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.083+0000 7fa5f7173740 1 --2- 172.21.15.105:0/203345313 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dea70411f0 0x55dea704fa80 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:08.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.083+0000 7fa5f7173740 1 --2- 172.21.15.105:0/203345313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dea70408d0 0x55dea7040cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:08.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.083+0000 7fa5f7173740 1 -- 172.21.15.105:0/203345313 >> 172.21.15.105:0/203345313 conn(0x55dea703cf60 msgr2=0x55dea703d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:08.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.083+0000 7fa5f7173740 1 -- 172.21.15.105:0/203345313 shutdown_connections 2024-09-16T02:02:08.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.083+0000 7fa5f7173740 1 -- 172.21.15.105:0/203345313 wait complete. 2024-09-16T02:02:08.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.083+0000 7fa5f7173740 1 -- start start 2024-09-16T02:02:08.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.083+0000 7fa5f7173740 1 Processor -- start 2024-09-16T02:02:08.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.083+0000 7fa5f7173740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dea70408d0 0x55dea704f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:08.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.083+0000 7fa5f7173740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dea70411f0 0x55dea704cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:08.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.083+0000 7fa5f7173740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dea704ffc0 0x55dea704d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:08.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.083+0000 7fa5f7173740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55dea70529b0 con 0x55dea704ffc0 2024-09-16T02:02:08.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.083+0000 7fa5f7173740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dea6f61f20 con 0x55dea70411f0 2024-09-16T02:02:08.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.083+0000 7fa5f7173740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55dea7052b30 con 0x55dea70408d0 2024-09-16T02:02:08.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.083+0000 7fa5f7173740 10 client.0 _ll_register_callbacks cb 0x55dea6f2b030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:02:08.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.083+0000 7fa5eed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dea704ffc0 0x55dea704d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:08.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.083+0000 7fa5eed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dea704ffc0 0x55dea704d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57894/0 (socket says 172.21.15.105:57894) 2024-09-16T02:02:08.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.083+0000 7fa5eed9d640 1 -- 172.21.15.105:0/4274801474 learned_addr learned my addr 172.21.15.105:0/4274801474 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:08.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.084+0000 7fa5edd9b640 1 --2- 172.21.15.105:0/4274801474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dea70411f0 0x55dea704cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:08.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.084+0000 7fa5ee59c640 1 --2- 172.21.15.105:0/4274801474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dea70408d0 0x55dea704f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:08.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.084+0000 7fa5eed9d640 1 -- 172.21.15.105:0/4274801474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dea70408d0 msgr2=0x55dea704f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:08.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.084+0000 7fa5eed9d640 1 --2- 172.21.15.105:0/4274801474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dea70408d0 0x55dea704f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:08.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.084+0000 7fa5eed9d640 1 -- 172.21.15.105:0/4274801474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dea70411f0 msgr2=0x55dea704cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:08.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.084+0000 7fa5eed9d640 1 --2- 172.21.15.105:0/4274801474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dea70411f0 0x55dea704cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.084+0000 7fa5eed9d640 1 -- 172.21.15.105:0/4274801474 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dea6d182b0 con 0x55dea704ffc0 2024-09-16T02:02:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.084+0000 7fa5edd9b640 1 --2- 172.21.15.105:0/4274801474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dea70411f0 0x55dea704cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:02:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.084+0000 7fa5eed9d640 1 --2- 172.21.15.105:0/4274801474 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dea704ffc0 0x55dea704d0e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa5e80139e0 tx=0x7fa5e8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb321d1221606097 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.084+0000 7fa5d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:02:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.084+0000 7fa5d5ffb640 1 -- 172.21.15.105:0/4274801474 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e801ed70 con 0x55dea704ffc0 2024-09-16T02:02:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.084+0000 7fa5d5ffb640 1 -- 172.21.15.105:0/4274801474 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fa5e801f450 con 0x55dea704ffc0 2024-09-16T02:02:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.084+0000 7fa5f7173740 1 -- 172.21.15.105:0/4274801474 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dea6fb4600 con 0x55dea704ffc0 2024-09-16T02:02:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.084+0000 7fa5d5ffb640 1 -- 172.21.15.105:0/4274801474 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa5e8004b20 con 0x55dea704ffc0 2024-09-16T02:02:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.084+0000 7fa5f7173740 10 client.15060 fetch_fsmap learned FSMap version 27 2024-09-16T02:02:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.084+0000 7fa5f7173740 1 -- 172.21.15.105:0/4274801474 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dea705e250 con 0x55dea704ffc0 2024-09-16T02:02:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.084+0000 7fa5d5ffb640 1 -- 172.21.15.105:0/4274801474 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55dea6fb4600 con 0x55dea704ffc0 2024-09-16T02:02:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.085+0000 7fa5d5ffb640 1 -- 172.21.15.105:0/4274801474 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa5e800f040 con 0x55dea704ffc0 2024-09-16T02:02:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.085+0000 7fa5f7173740 10 client.15060 fetch_fsmap finished waiting for FSMap version 27 2024-09-16T02:02:08.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.085+0000 7fa5f7173740 10 client.15060 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-16T02:02:08.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.085+0000 7fa5f7173740 20 client.15060 populate_metadata read hostname 'smithi105' 2024-09-16T02:02:08.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.085+0000 7fa5f7173740 1 --2- 172.21.15.105:0/4274801474 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55dea70608b0 0x55dea7061d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:08.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.085+0000 7fa5f7173740 4 client.15060 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:02:08.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.085+0000 7fa5f7173740 1 -- 172.21.15.105:0/4274801474 --> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dea6faeb10 con 0x55dea70608b0 2024-09-16T02:02:08.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.085+0000 7fa5ee59c640 1 --2- 172.21.15.105:0/4274801474 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55dea70608b0 0x55dea7061d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:08.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.086+0000 7fa5d5ffb640 10 client.15060 ms_handle_connect on v2:172.21.15.190:6838/1222573537 2024-09-16T02:02:08.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.086+0000 7fa5ee59c640 1 --2- 172.21.15.105:0/4274801474 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55dea70608b0 0x55dea7061d20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=21e296e53eb77340 server_cookie=3c91bafc14d003eb in_seq=0 out_seq=0 2024-09-16T02:02:08.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.087+0000 7fa5d5ffb640 1 -- 172.21.15.105:0/4274801474 <== mds.1 v2:172.21.15.190:6838/1222573537 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dea6faeb10 con 0x55dea70608b0 2024-09-16T02:02:08.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.087+0000 7fa5d5ffb640 10 client.15060 handle_command_reply: tid=0 2024-09-16T02:02:08.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.087+0000 7fa5f7173740 1 client.15060 shutdown 2024-09-16T02:02:08.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.087+0000 7fa5f7173740 1 -- 172.21.15.105:0/4274801474 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dea704ffc0 msgr2=0x55dea704d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:08.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.087+0000 7fa5f7173740 1 --2- 172.21.15.105:0/4274801474 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dea704ffc0 0x55dea704d0e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa5e80139e0 tx=0x7fa5e8004ab0 comp rx=0 tx=0).stop 2024-09-16T02:02:08.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.087+0000 7fa5f7173740 1 -- 172.21.15.105:0/4274801474 shutdown_connections 2024-09-16T02:02:08.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.087+0000 7fa5f7173740 1 --2- 172.21.15.105:0/4274801474 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55dea70608b0 0x55dea7061d20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:08.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.087+0000 7fa5f7173740 1 --2- 172.21.15.105:0/4274801474 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dea704ffc0 0x55dea704d0e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:08.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.087+0000 7fa5f7173740 1 --2- 172.21.15.105:0/4274801474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dea70411f0 0x55dea704cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:08.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.087+0000 7fa5f7173740 1 --2- 172.21.15.105:0/4274801474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dea70408d0 0x55dea704f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:08.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.087+0000 7fa5f7173740 1 -- 172.21.15.105:0/4274801474 >> 172.21.15.105:0/4274801474 conn(0x55dea703cf60 msgr2=0x55dea7042cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:08.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.087+0000 7fa5d5ffb640 0 client.15060 ms_handle_reset on v2:172.21.15.190:6838/1222573537 2024-09-16T02:02:08.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.087+0000 7fa5f7173740 1 -- 172.21.15.105:0/4274801474 shutdown_connections 2024-09-16T02:02:08.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.087+0000 7fa5f7173740 1 -- 172.21.15.105:0/4274801474 wait complete. 2024-09-16T02:02:08.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.087+0000 7fa5f7173740 20 client.15060 trim_cache size 0 max 16384 2024-09-16T02:02:08.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.105+0000 7fa5f7173740 1 Processor -- start 2024-09-16T02:02:08.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.105+0000 7fa5f7173740 1 -- start start 2024-09-16T02:02:08.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.105+0000 7fa5f7173740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dea70408d0 0x55dea705b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:08.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.105+0000 7fa5f7173740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dea70411f0 0x55dea704c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:08.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.105+0000 7fa5f7173740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dea704ffc0 0x55dea704b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:08.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.105+0000 7fa5f7173740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55dea7060060 con 0x55dea70411f0 2024-09-16T02:02:08.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.105+0000 7fa5f7173740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dea7060730 con 0x55dea70408d0 2024-09-16T02:02:08.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.105+0000 7fa5f7173740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55dea7060e00 con 0x55dea704ffc0 2024-09-16T02:02:08.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.105+0000 7fa5edd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dea70411f0 0x55dea704c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:08.123 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-16T02:02:08.123 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "423b7a2a-73ce-11ef-bceb-c7b262605968", 2024-09-16T02:02:08.123 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 1, 2024-09-16T02:02:08.123 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 24469, 2024-09-16T02:02:08.123 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2024-09-16T02:02:08.123 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2024-09-16T02:02:08.123 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2024-09-16T02:02:08.124 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 84.622816979999996, 2024-09-16T02:02:08.124 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 27, 2024-09-16T02:02:08.124 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 81, 2024-09-16T02:02:08.124 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 79, 2024-09-16T02:02:08.124 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 112.94657850900001 2024-09-16T02:02:08.124 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-16T02:02:08.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.105+0000 7fa5edd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dea70411f0 0x55dea704c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57896/0 (socket says 172.21.15.105:57896) 2024-09-16T02:02:08.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.105+0000 7fa5edd9b640 1 -- 172.21.15.105:0/2274510885 learned_addr learned my addr 172.21.15.105:0/2274510885 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:08.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.105+0000 7fa5ee59c640 1 --2- 172.21.15.105:0/2274510885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dea70408d0 0x55dea705b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:08.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.105+0000 7fa5eed9d640 1 --2- 172.21.15.105:0/2274510885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dea704ffc0 0x55dea704b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:08.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.105+0000 7fa5edd9b640 1 -- 172.21.15.105:0/2274510885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dea704ffc0 msgr2=0x55dea704b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:08.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.105+0000 7fa5edd9b640 1 --2- 172.21.15.105:0/2274510885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dea704ffc0 0x55dea704b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:08.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.105+0000 7fa5edd9b640 1 -- 172.21.15.105:0/2274510885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dea70408d0 msgr2=0x55dea705b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:08.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.105+0000 7fa5edd9b640 1 --2- 172.21.15.105:0/2274510885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dea70408d0 0x55dea705b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:08.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.105+0000 7fa5edd9b640 1 -- 172.21.15.105:0/2274510885 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dea704f470 con 0x55dea70411f0 2024-09-16T02:02:08.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.106+0000 7fa5edd9b640 1 --2- 172.21.15.105:0/2274510885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dea70411f0 0x55dea704c460 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0104b40 tx=0x7fa5e400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5315f71841f61f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:08.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.106+0000 7fa5d7fff640 1 -- 172.21.15.105:0/2274510885 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4010d60 con 0x55dea70411f0 2024-09-16T02:02:08.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.106+0000 7fa5d7fff640 1 -- 172.21.15.105:0/2274510885 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5e40118f0 con 0x55dea70411f0 2024-09-16T02:02:08.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.106+0000 7fa5d7fff640 1 -- 172.21.15.105:0/2274510885 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e40198b0 con 0x55dea70411f0 2024-09-16T02:02:08.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.106+0000 7fa5f7173740 1 -- 172.21.15.105:0/2274510885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dea70411f0 msgr2=0x55dea704c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:08.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.106+0000 7fa5f7173740 1 --2- 172.21.15.105:0/2274510885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dea70411f0 0x55dea704c460 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0104b40 tx=0x7fa5e400cf60 comp rx=0 tx=0).stop 2024-09-16T02:02:08.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.106+0000 7fa5f7173740 1 -- 172.21.15.105:0/2274510885 shutdown_connections 2024-09-16T02:02:08.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.106+0000 7fa5f7173740 1 --2- 172.21.15.105:0/2274510885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dea704ffc0 0x55dea704b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:08.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.106+0000 7fa5f7173740 1 --2- 172.21.15.105:0/2274510885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dea70411f0 0x55dea704c460 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:08.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.106+0000 7fa5f7173740 1 --2- 172.21.15.105:0/2274510885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dea70408d0 0x55dea705b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:08.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.106+0000 7fa5f7173740 1 -- 172.21.15.105:0/2274510885 >> 172.21.15.105:0/2274510885 conn(0x55dea703cf60 msgr2=0x55dea704cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:08.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.106+0000 7fa5f7173740 1 -- 172.21.15.105:0/2274510885 shutdown_connections 2024-09-16T02:02:08.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.106+0000 7fa5f7173740 1 -- 172.21.15.105:0/2274510885 wait complete. 2024-09-16T02:02:08.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.107+0000 7fa5f7173740 1 -- start start 2024-09-16T02:02:08.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.107+0000 7fa5f7173740 1 Processor -- start 2024-09-16T02:02:08.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.107+0000 7fa5f7173740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dea70408d0 0x55dea7062da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:08.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.107+0000 7fa5f7173740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dea70411f0 0x55dea70632e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:08.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.107+0000 7fa5f7173740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dea704ffc0 0x55dea7063820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:08.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.107+0000 7fa5f7173740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55dea7061ba0 con 0x55dea70408d0 2024-09-16T02:02:08.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.107+0000 7fa5f7173740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dea70614d0 con 0x55dea704ffc0 2024-09-16T02:02:08.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.107+0000 7fa5f7173740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55dea7062270 con 0x55dea70411f0 2024-09-16T02:02:08.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.107+0000 7fa5f7173740 10 client.0 _ll_register_callbacks cb 0x7fa5e8009cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:02:08.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.107+0000 7fa5ee59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dea70408d0 0x55dea7062da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:08.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.107+0000 7fa5ee59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dea70408d0 0x55dea7062da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57900/0 (socket says 172.21.15.105:57900) 2024-09-16T02:02:08.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.107+0000 7fa5ee59c640 1 -- 172.21.15.105:0/1256974849 learned_addr learned my addr 172.21.15.105:0/1256974849 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:08.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.107+0000 7fa5ee59c640 1 -- 172.21.15.105:0/1256974849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dea70411f0 msgr2=0x55dea70632e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:08.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.107+0000 7fa5edd9b640 1 --2- 172.21.15.105:0/1256974849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dea70411f0 0x55dea70632e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:08.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.107+0000 7fa5ee59c640 1 --2- 172.21.15.105:0/1256974849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dea70411f0 0x55dea70632e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:08.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.107+0000 7fa5ee59c640 1 -- 172.21.15.105:0/1256974849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dea704ffc0 msgr2=0x55dea7063820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:08.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.107+0000 7fa5ee59c640 1 --2- 172.21.15.105:0/1256974849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dea704ffc0 0x55dea7063820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:08.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.108+0000 7fa5ee59c640 1 -- 172.21.15.105:0/1256974849 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dea70c9030 con 0x55dea70408d0 2024-09-16T02:02:08.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.108+0000 7fa5edd9b640 1 --2- 172.21.15.105:0/1256974849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dea70411f0 0x55dea70632e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:02:08.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.108+0000 7fa5ee59c640 1 --2- 172.21.15.105:0/1256974849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dea70408d0 0x55dea7062da0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa5e00edd30 tx=0x7fa5e003b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acdead986b35f618 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:08.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.108+0000 7fa5d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:02:08.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.108+0000 7fa5d5ffb640 1 -- 172.21.15.105:0/1256974849 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e0049040 con 0x55dea70408d0 2024-09-16T02:02:08.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.108+0000 7fa5d5ffb640 1 -- 172.21.15.105:0/1256974849 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fa5e00baac0 con 0x55dea70408d0 2024-09-16T02:02:08.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.108+0000 7fa5f7173740 1 -- 172.21.15.105:0/1256974849 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5e40118f0 con 0x55dea70408d0 2024-09-16T02:02:08.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.108+0000 7fa5d5ffb640 1 -- 172.21.15.105:0/1256974849 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa5e00bc300 con 0x55dea70408d0 2024-09-16T02:02:08.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.108+0000 7fa5f7173740 10 client.15066 fetch_fsmap learned FSMap version 27 2024-09-16T02:02:08.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.108+0000 7fa5f7173740 1 -- 172.21.15.105:0/1256974849 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dea70ca330 con 0x55dea70408d0 2024-09-16T02:02:08.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.108+0000 7fa5d5ffb640 1 -- 172.21.15.105:0/1256974849 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5e40118f0 con 0x55dea70408d0 2024-09-16T02:02:08.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.108+0000 7fa5d5ffb640 1 -- 172.21.15.105:0/1256974849 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa5e00ed0f0 con 0x55dea70408d0 2024-09-16T02:02:08.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.108+0000 7fa5f7173740 10 client.15066 fetch_fsmap finished waiting for FSMap version 27 2024-09-16T02:02:08.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.108+0000 7fa5f7173740 10 client.15066 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-16T02:02:08.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.108+0000 7fa5f7173740 20 client.15066 populate_metadata read hostname 'smithi105' 2024-09-16T02:02:08.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.108+0000 7fa5f7173740 1 --2- 172.21.15.105:0/1256974849 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55dea70cede0 0x55dea70cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:08.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.108+0000 7fa5f7173740 4 client.15066 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T02:02:08.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.108+0000 7fa5f7173740 1 -- 172.21.15.105:0/1256974849 --> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] -- command(tid 0: {"prefix": "status"}) -- 0x55dea6f2b0e0 con 0x55dea70cede0 2024-09-16T02:02:08.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.109+0000 7fa5edd9b640 1 --2- 172.21.15.105:0/1256974849 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55dea70cede0 0x55dea70cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:08.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.109+0000 7fa5d5ffb640 10 client.15066 ms_handle_connect on v2:172.21.15.190:6838/1222573537 2024-09-16T02:02:08.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.109+0000 7fa5edd9b640 1 --2- 172.21.15.105:0/1256974849 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55dea70cede0 0x55dea70cdaf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fccd6fd3b39fa4bb server_cookie=2f769d2feab9b3dc in_seq=0 out_seq=0 2024-09-16T02:02:08.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.109+0000 7fa5d5ffb640 1 -- 172.21.15.105:0/1256974849 <== mds.1 v2:172.21.15.190:6838/1222573537 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55dea6f2b0e0 con 0x55dea70cede0 2024-09-16T02:02:08.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.109+0000 7fa5d5ffb640 10 client.15066 handle_command_reply: tid=0 2024-09-16T02:02:08.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.109+0000 7fa5f7173740 1 client.15066 shutdown 2024-09-16T02:02:08.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.109+0000 7fa5f7173740 1 -- 172.21.15.105:0/1256974849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dea70408d0 msgr2=0x55dea7062da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:08.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.109+0000 7fa5f7173740 1 --2- 172.21.15.105:0/1256974849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dea70408d0 0x55dea7062da0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa5e00edd30 tx=0x7fa5e003b5b0 comp rx=0 tx=0).stop 2024-09-16T02:02:08.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.109+0000 7fa5f7173740 1 -- 172.21.15.105:0/1256974849 shutdown_connections 2024-09-16T02:02:08.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.109+0000 7fa5f7173740 1 --2- 172.21.15.105:0/1256974849 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55dea70cede0 0x55dea70cdaf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:08.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.109+0000 7fa5d5ffb640 0 client.15066 ms_handle_reset on v2:172.21.15.190:6838/1222573537 2024-09-16T02:02:08.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.109+0000 7fa5f7173740 1 --2- 172.21.15.105:0/1256974849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dea70411f0 0x55dea70632e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:08.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.109+0000 7fa5f7173740 1 --2- 172.21.15.105:0/1256974849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dea704ffc0 0x55dea7063820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:08.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.109+0000 7fa5f7173740 1 --2- 172.21.15.105:0/1256974849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dea70408d0 0x55dea7062da0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:08.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.109+0000 7fa5f7173740 1 -- 172.21.15.105:0/1256974849 >> 172.21.15.105:0/1256974849 conn(0x55dea703cf60 msgr2=0x55dea704cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:08.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.110+0000 7fa5f7173740 1 -- 172.21.15.105:0/1256974849 shutdown_connections 2024-09-16T02:02:08.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.110+0000 7fa5f7173740 1 -- 172.21.15.105:0/1256974849 wait complete. 2024-09-16T02:02:08.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.110+0000 7fa5f7173740 20 client.15066 trim_cache size 0 max 16384 2024-09-16T02:02:08.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.110+0000 7fa5f4d9a640 1 -- 172.21.15.105:0/2480314295 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa5d80a32d0 msgr2=0x7fa5d80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:08.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.110+0000 7fa5f4d9a640 1 --2- 172.21.15.105:0/2480314295 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa5d80a32d0 0x7fa5d80a5790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa5f01078a0 tx=0x7fa5e8004a40 comp rx=0 tx=0).stop 2024-09-16T02:02:08.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.110+0000 7fa5f4d9a640 1 -- 172.21.15.105:0/2480314295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5f01015e0 msgr2=0x7fa5f0198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:08.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.110+0000 7fa5f4d9a640 1 --2- 172.21.15.105:0/2480314295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5f01015e0 0x7fa5f0198360 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0005940 tx=0x7fa5e0004270 comp rx=0 tx=0).stop 2024-09-16T02:02:08.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.110+0000 7fa5f4d9a640 1 -- 172.21.15.105:0/2480314295 shutdown_connections 2024-09-16T02:02:08.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.110+0000 7fa5f4d9a640 1 --2- 172.21.15.105:0/2480314295 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa5d80a32d0 0x7fa5d80a5790 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:08.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.110+0000 7fa5f4d9a640 1 --2- 172.21.15.105:0/2480314295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5f0106d00 0x7fa5f019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:08.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.110+0000 7fa5f4d9a640 1 --2- 172.21.15.105:0/2480314295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5f0101f80 0x7fa5f01988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:08.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.110+0000 7fa5f4d9a640 1 --2- 172.21.15.105:0/2480314295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5f01015e0 0x7fa5f0198360 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:08.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.110+0000 7fa5f4d9a640 1 -- 172.21.15.105:0/2480314295 >> 172.21.15.105:0/2480314295 conn(0x7fa5f00fba90 msgr2=0x7fa5f01006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:08.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.110+0000 7fa5f4d9a640 1 -- 172.21.15.105:0/2480314295 shutdown_connections 2024-09-16T02:02:08.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:08.110+0000 7fa5f4d9a640 1 -- 172.21.15.105:0/2480314295 wait complete. 2024-09-16T02:02:08.793 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:08 smithi105 ceph-mon[31197]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T02:02:08.824 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-16T02:02:08.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:08 smithi190 ceph-mon[34762]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T02:02:08.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:08 smithi137 ceph-mon[34099]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T02:02:09.070 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:02:09.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.535+0000 7f6285398640 1 -- 172.21.15.105:0/3232489043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6280101de0 msgr2=0x7f62801021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:09.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.535+0000 7f6285398640 1 --2- 172.21.15.105:0/3232489043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6280101de0 0x7f62801021c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6270009900 tx=0x7f627002a410 comp rx=0 tx=0).stop 2024-09-16T02:02:09.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.535+0000 7f6285398640 1 -- 172.21.15.105:0/3232489043 shutdown_connections 2024-09-16T02:02:09.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.535+0000 7f6285398640 1 --2- 172.21.15.105:0/3232489043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6280102790 0x7f62801067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:09.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.535+0000 7f6285398640 1 --2- 172.21.15.105:0/3232489043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6280106f10 0x7f628010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:09.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.535+0000 7f6285398640 1 --2- 172.21.15.105:0/3232489043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6280101de0 0x7f62801021c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:09.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.535+0000 7f6285398640 1 -- 172.21.15.105:0/3232489043 >> 172.21.15.105:0/3232489043 conn(0x7f62800fd8c0 msgr2=0x7f62800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:09.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.535+0000 7f6285398640 1 -- 172.21.15.105:0/3232489043 shutdown_connections 2024-09-16T02:02:09.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.535+0000 7f6285398640 1 -- 172.21.15.105:0/3232489043 wait complete. 2024-09-16T02:02:09.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.535+0000 7f6285398640 1 Processor -- start 2024-09-16T02:02:09.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.536+0000 7f6285398640 1 -- start start 2024-09-16T02:02:09.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.536+0000 7f6285398640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6280101de0 0x7f62801982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:09.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.536+0000 7f6285398640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6280102790 0x7f6280198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:09.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.536+0000 7f6285398640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6280106f10 0x7f628019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:09.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.536+0000 7f6285398640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6280111e20 con 0x7f6280101de0 2024-09-16T02:02:09.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.536+0000 7f6285398640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6280111ca0 con 0x7f6280106f10 2024-09-16T02:02:09.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.536+0000 7f6285398640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6280111fa0 con 0x7f6280102790 2024-09-16T02:02:09.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.536+0000 7f627effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6280101de0 0x7f62801982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:09.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.536+0000 7f627effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6280101de0 0x7f62801982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35538/0 (socket says 172.21.15.105:35538) 2024-09-16T02:02:09.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.536+0000 7f627effd640 1 -- 172.21.15.105:0/2311861296 learned_addr learned my addr 172.21.15.105:0/2311861296 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:09.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.536+0000 7f627e7fc640 1 --2- 172.21.15.105:0/2311861296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6280102790 0x7f6280198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:09.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.536+0000 7f627f7fe640 1 --2- 172.21.15.105:0/2311861296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6280106f10 0x7f628019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:09.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.536+0000 7f627effd640 1 -- 172.21.15.105:0/2311861296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6280102790 msgr2=0x7f6280198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:09.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.536+0000 7f627effd640 1 --2- 172.21.15.105:0/2311861296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6280102790 0x7f6280198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:09.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.536+0000 7f627effd640 1 -- 172.21.15.105:0/2311861296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6280106f10 msgr2=0x7f628019cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:09.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.536+0000 7f627effd640 1 --2- 172.21.15.105:0/2311861296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6280106f10 0x7f628019cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:09.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.536+0000 7f627effd640 1 -- 172.21.15.105:0/2311861296 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f628019d2a0 con 0x7f6280101de0 2024-09-16T02:02:09.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.536+0000 7f627effd640 1 --2- 172.21.15.105:0/2311861296 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6280101de0 0x7f62801982f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6270005940 tx=0x7f6270004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:09.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.537+0000 7f6267fff640 1 -- 172.21.15.105:0/2311861296 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627003f030 con 0x7f6280101de0 2024-09-16T02:02:09.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.537+0000 7f6285398640 1 -- 172.21.15.105:0/2311861296 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f628019d530 con 0x7f6280101de0 2024-09-16T02:02:09.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.537+0000 7f6267fff640 1 -- 172.21.15.105:0/2311861296 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6270003aa0 con 0x7f6280101de0 2024-09-16T02:02:09.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.537+0000 7f6267fff640 1 -- 172.21.15.105:0/2311861296 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6270035070 con 0x7f6280101de0 2024-09-16T02:02:09.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.537+0000 7f6285398640 1 -- 172.21.15.105:0/2311861296 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62801a4e70 con 0x7f6280101de0 2024-09-16T02:02:09.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.539+0000 7f6267fff640 1 -- 172.21.15.105:0/2311861296 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f6270042030 con 0x7f6280101de0 2024-09-16T02:02:09.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.539+0000 7f6287771740 1 Processor -- start 2024-09-16T02:02:09.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.539+0000 7f6287771740 1 -- start start 2024-09-16T02:02:09.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.539+0000 7f6287771740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7804c4890 0x55a7804c4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:09.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.539+0000 7f6287771740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7804c51b0 0x55a7804d3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:09.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.539+0000 7f6287771740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7804d3fc0 0x55a7804d63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:09.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.539+0000 7f6287771740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a78042b090 con 0x55a7804d3fc0 2024-09-16T02:02:09.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.539+0000 7f6287771740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a7803d2a70 con 0x55a7804c4890 2024-09-16T02:02:09.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.539+0000 7f6287771740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a780427050 con 0x55a7804c51b0 2024-09-16T02:02:09.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.540+0000 7f627e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7804d3fc0 0x55a7804d63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:09.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.540+0000 7f627e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7804d3fc0 0x55a7804d63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35542/0 (socket says 172.21.15.105:35542) 2024-09-16T02:02:09.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.540+0000 7f627e7fc640 1 -- 172.21.15.105:0/418449851 learned_addr learned my addr 172.21.15.105:0/418449851 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:09.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.540+0000 7f627f7fe640 1 --2- 172.21.15.105:0/418449851 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7804c4890 0x55a7804c4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:09.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.540+0000 7f627effd640 1 --2- 172.21.15.105:0/418449851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7804c51b0 0x55a7804d3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:09.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.540+0000 7f6267fff640 1 --2- 172.21.15.105:0/2311861296 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f62680a32d0 0x7f62680a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:09.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.540+0000 7f627e7fc640 1 -- 172.21.15.105:0/418449851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7804c51b0 msgr2=0x55a7804d3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:09.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.540+0000 7f627e7fc640 1 --2- 172.21.15.105:0/418449851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7804c51b0 0x55a7804d3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:09.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.540+0000 7f627e7fc640 1 -- 172.21.15.105:0/418449851 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7804c4890 msgr2=0x55a7804c4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:09.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.540+0000 7f627e7fc640 1 --2- 172.21.15.105:0/418449851 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7804c4890 0x55a7804c4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:09.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.540+0000 7f627e7fc640 1 -- 172.21.15.105:0/418449851 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7803df130 con 0x55a7804d3fc0 2024-09-16T02:02:09.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.540+0000 7f627f7fe640 1 --2- 172.21.15.105:0/2311861296 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f62680a32d0 0x7f62680a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:09.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.540+0000 7f6267fff640 1 -- 172.21.15.105:0/2311861296 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f62700f0270 con 0x7f6280101de0 2024-09-16T02:02:09.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.540+0000 7f627e7fc640 1 --2- 172.21.15.105:0/418449851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7804d3fc0 0x55a7804d63b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f627400b740 tx=0x7f627400bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5956bc09b54cdb21 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:09.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.540+0000 7f62657fa640 1 -- 172.21.15.105:0/418449851 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6274003dd0 con 0x55a7804d3fc0 2024-09-16T02:02:09.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.540+0000 7f62657fa640 1 -- 172.21.15.105:0/418449851 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6274015070 con 0x55a7804d3fc0 2024-09-16T02:02:09.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.540+0000 7f62657fa640 1 -- 172.21.15.105:0/418449851 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6274004d80 con 0x55a7804d3fc0 2024-09-16T02:02:09.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.541+0000 7f6287771740 1 -- 172.21.15.105:0/418449851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7804d3fc0 msgr2=0x55a7804d63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:09.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.541+0000 7f627f7fe640 1 --2- 172.21.15.105:0/2311861296 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f62680a32d0 0x7f62680a5790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6278007f30 tx=0x7f6278004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:09.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.541+0000 7f6287771740 1 --2- 172.21.15.105:0/418449851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7804d3fc0 0x55a7804d63b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f627400b740 tx=0x7f627400bc00 comp rx=0 tx=0).stop 2024-09-16T02:02:09.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.541+0000 7f6287771740 1 -- 172.21.15.105:0/418449851 shutdown_connections 2024-09-16T02:02:09.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.541+0000 7f6287771740 1 --2- 172.21.15.105:0/418449851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7804c51b0 0x55a7804d3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:09.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.541+0000 7f6287771740 1 --2- 172.21.15.105:0/418449851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7804d3fc0 0x55a7804d63b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:09.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.541+0000 7f6287771740 1 --2- 172.21.15.105:0/418449851 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7804c4890 0x55a7804c4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:09.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.541+0000 7f6287771740 1 -- 172.21.15.105:0/418449851 >> 172.21.15.105:0/418449851 conn(0x55a7804c0f60 msgr2=0x55a7804c1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:09.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.541+0000 7f6287771740 1 -- 172.21.15.105:0/418449851 shutdown_connections 2024-09-16T02:02:09.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.541+0000 7f6287771740 1 -- 172.21.15.105:0/418449851 wait complete. 2024-09-16T02:02:09.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.541+0000 7f6287771740 1 -- start start 2024-09-16T02:02:09.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.541+0000 7f6287771740 1 Processor -- start 2024-09-16T02:02:09.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.542+0000 7f6287771740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7804c4890 0x55a7804d3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:09.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.542+0000 7f6287771740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7804c51b0 0x55a7804d0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:09.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.542+0000 7f6287771740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7804d3fc0 0x55a7804d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:09.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.542+0000 7f6287771740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a7804d6b90 con 0x55a7804c4890 2024-09-16T02:02:09.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.542+0000 7f6287771740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a7803e5f20 con 0x55a7804d3fc0 2024-09-16T02:02:09.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.542+0000 7f6287771740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a7804d6d10 con 0x55a7804c51b0 2024-09-16T02:02:09.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.542+0000 7f6287771740 10 client.0 _ll_register_callbacks cb 0x55a7803af030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:02:09.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.542+0000 7f627effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7804c4890 0x55a7804d3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:09.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.542+0000 7f627effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7804c4890 0x55a7804d3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35544/0 (socket says 172.21.15.105:35544) 2024-09-16T02:02:09.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.542+0000 7f627effd640 1 -- 172.21.15.105:0/3393102761 learned_addr learned my addr 172.21.15.105:0/3393102761 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:09.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.542+0000 7f627f7fe640 1 --2- 172.21.15.105:0/3393102761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7804d3fc0 0x55a7804d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:09.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.542+0000 7f627e7fc640 1 --2- 172.21.15.105:0/3393102761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7804c51b0 0x55a7804d0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:09.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.542+0000 7f627effd640 1 -- 172.21.15.105:0/3393102761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7804c51b0 msgr2=0x55a7804d0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:09.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.542+0000 7f627effd640 1 --2- 172.21.15.105:0/3393102761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7804c51b0 0x55a7804d0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:09.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.542+0000 7f627effd640 1 -- 172.21.15.105:0/3393102761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7804d3fc0 msgr2=0x55a7804d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:09.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.542+0000 7f627effd640 1 --2- 172.21.15.105:0/3393102761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7804d3fc0 0x55a7804d10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:09.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.542+0000 7f627effd640 1 -- 172.21.15.105:0/3393102761 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a78019c2b0 con 0x55a7804c4890 2024-09-16T02:02:09.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.542+0000 7f627f7fe640 1 --2- 172.21.15.105:0/3393102761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7804d3fc0 0x55a7804d10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:02:09.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.542+0000 7f627effd640 1 --2- 172.21.15.105:0/3393102761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7804c4890 0x55a7804d3860 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f62700c4e50 tx=0x7f62700b8350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1619cd081760b4a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:09.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.542+0000 7f6262ff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:02:09.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.542+0000 7f6262ff5640 1 -- 172.21.15.105:0/3393102761 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6270049070 con 0x55a7804c4890 2024-09-16T02:02:09.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.542+0000 7f6262ff5640 1 -- 172.21.15.105:0/3393102761 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f6270003c50 con 0x55a7804c4890 2024-09-16T02:02:09.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.542+0000 7f6287771740 1 -- 172.21.15.105:0/3393102761 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6274015070 con 0x55a7804c4890 2024-09-16T02:02:09.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.542+0000 7f6262ff5640 1 -- 172.21.15.105:0/3393102761 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f62700bc070 con 0x55a7804c4890 2024-09-16T02:02:09.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.542+0000 7f6287771740 10 client.15078 fetch_fsmap learned FSMap version 27 2024-09-16T02:02:09.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.542+0000 7f6287771740 1 -- 172.21.15.105:0/3393102761 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a7804e24f0 con 0x55a7804c4890 2024-09-16T02:02:09.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.543+0000 7f6262ff5640 1 -- 172.21.15.105:0/3393102761 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6274015070 con 0x55a7804c4890 2024-09-16T02:02:09.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.543+0000 7f6262ff5640 1 -- 172.21.15.105:0/3393102761 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f62700b8d80 con 0x55a7804c4890 2024-09-16T02:02:09.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.543+0000 7f6287771740 10 client.15078 fetch_fsmap finished waiting for FSMap version 27 2024-09-16T02:02:09.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.543+0000 7f6287771740 10 client.15078 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T02:02:09.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.543+0000 7f6287771740 20 client.15078 populate_metadata read hostname 'smithi105' 2024-09-16T02:02:09.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.543+0000 7f6287771740 1 --2- 172.21.15.105:0/3393102761 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55a7804e4b90 0x55a7804e6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:09.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.543+0000 7f6287771740 4 client.15078 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:02:09.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.543+0000 7f6287771740 1 -- 172.21.15.105:0/3393102761 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a780438600 con 0x55a7804e4b90 2024-09-16T02:02:09.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.543+0000 7f627e7fc640 1 --2- 172.21.15.105:0/3393102761 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55a7804e4b90 0x55a7804e6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:09.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.543+0000 7f627e7fc640 1 --2- 172.21.15.105:0/3393102761 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55a7804e4b90 0x55a7804e6000 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c5cd943f6066177 server_cookie=c964a5c037ec8891 in_seq=0 out_seq=0 2024-09-16T02:02:09.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.543+0000 7f6262ff5640 10 client.15078 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:02:09.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.544+0000 7f6262ff5640 1 -- 172.21.15.105:0/3393102761 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a780438600 con 0x55a7804e4b90 2024-09-16T02:02:09.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.544+0000 7f6262ff5640 10 client.15078 handle_command_reply: tid=0 2024-09-16T02:02:09.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.544+0000 7f6287771740 1 client.15078 shutdown 2024-09-16T02:02:09.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.544+0000 7f6287771740 1 -- 172.21.15.105:0/3393102761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7804c4890 msgr2=0x55a7804d3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:09.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.544+0000 7f6287771740 1 --2- 172.21.15.105:0/3393102761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7804c4890 0x55a7804d3860 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f62700c4e50 tx=0x7f62700b8350 comp rx=0 tx=0).stop 2024-09-16T02:02:09.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.544+0000 7f6287771740 1 -- 172.21.15.105:0/3393102761 shutdown_connections 2024-09-16T02:02:09.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.544+0000 7f6287771740 1 --2- 172.21.15.105:0/3393102761 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55a7804e4b90 0x55a7804e6000 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:09.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.544+0000 7f6287771740 1 --2- 172.21.15.105:0/3393102761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7804c51b0 0x55a7804d0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:09.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.544+0000 7f6287771740 1 --2- 172.21.15.105:0/3393102761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7804d3fc0 0x55a7804d10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:09.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.544+0000 7f6262ff5640 0 client.15078 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:02:09.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.544+0000 7f6287771740 1 --2- 172.21.15.105:0/3393102761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7804c4890 0x55a7804d3860 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:09.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.544+0000 7f6287771740 1 -- 172.21.15.105:0/3393102761 >> 172.21.15.105:0/3393102761 conn(0x55a7804c0f60 msgr2=0x55a7804c6cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:09.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.544+0000 7f6287771740 1 -- 172.21.15.105:0/3393102761 shutdown_connections 2024-09-16T02:02:09.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.544+0000 7f6287771740 1 -- 172.21.15.105:0/3393102761 wait complete. 2024-09-16T02:02:09.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.545+0000 7f6287771740 20 client.15078 trim_cache size 0 max 16384 2024-09-16T02:02:09.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.565+0000 7f6287771740 1 Processor -- start 2024-09-16T02:02:09.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.565+0000 7f6287771740 1 -- start start 2024-09-16T02:02:09.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.565+0000 7f6287771740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7804c4890 0x55a7804e6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:09.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.565+0000 7f6287771740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7804c51b0 0x55a7804e3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:09.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.565+0000 7f6287771740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7804d3fc0 0x55a7804e3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:09.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.565+0000 7f6287771740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a7804e4340 con 0x55a7804c4890 2024-09-16T02:02:09.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.565+0000 7f6287771740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a7804e4a10 con 0x55a7804d3fc0 2024-09-16T02:02:09.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.565+0000 7f6287771740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a7804e50e0 con 0x55a7804c51b0 2024-09-16T02:02:09.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.566+0000 7f627e7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7804c51b0 0x55a7804e3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:09.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.566+0000 7f627e7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7804c51b0 0x55a7804e3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.105:55104/0 (socket says 172.21.15.105:55104) 2024-09-16T02:02:09.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.566+0000 7f627e7fc640 1 -- 172.21.15.105:0/2026882125 learned_addr learned my addr 172.21.15.105:0/2026882125 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:09.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.566+0000 7f627f7fe640 1 --2- 172.21.15.105:0/2026882125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7804d3fc0 0x55a7804e3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:09.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.566+0000 7f627effd640 1 --2- 172.21.15.105:0/2026882125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7804c4890 0x55a7804e6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:09.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.567+0000 7f627effd640 1 -- 172.21.15.105:0/2026882125 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7804c51b0 msgr2=0x55a7804e3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:09.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.567+0000 7f627effd640 1 --2- 172.21.15.105:0/2026882125 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7804c51b0 0x55a7804e3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:09.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.567+0000 7f627effd640 1 -- 172.21.15.105:0/2026882125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7804d3fc0 msgr2=0x55a7804e3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:09.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.567+0000 7f627effd640 1 --2- 172.21.15.105:0/2026882125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7804d3fc0 0x55a7804e3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:09.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.567+0000 7f627effd640 1 -- 172.21.15.105:0/2026882125 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7804c4fe0 con 0x55a7804c4890 2024-09-16T02:02:09.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.568+0000 7f6267fff640 1 -- 172.21.15.105:0/2311861296 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f6270003c50 con 0x7f6280101de0 2024-09-16T02:02:09.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.568+0000 7f627effd640 1 --2- 172.21.15.105:0/2026882125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7804c4890 0x55a7804e6000 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f62700ee180 tx=0x7f62700a7050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e86a1c8bfb4ba501 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:09.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.569+0000 7f6264ff9640 1 -- 172.21.15.105:0/2026882125 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62700f8030 con 0x55a7804c4890 2024-09-16T02:02:09.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.569+0000 7f6264ff9640 1 -- 172.21.15.105:0/2026882125 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f627003b460 con 0x55a7804c4890 2024-09-16T02:02:09.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.569+0000 7f6264ff9640 1 -- 172.21.15.105:0/2026882125 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62700b0af0 con 0x55a7804c4890 2024-09-16T02:02:09.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.570+0000 7f6287771740 1 -- 172.21.15.105:0/2026882125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7804c4890 msgr2=0x55a7804e6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:09.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.570+0000 7f6287771740 1 --2- 172.21.15.105:0/2026882125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7804c4890 0x55a7804e6000 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f62700ee180 tx=0x7f62700a7050 comp rx=0 tx=0).stop 2024-09-16T02:02:09.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.570+0000 7f6287771740 1 -- 172.21.15.105:0/2026882125 shutdown_connections 2024-09-16T02:02:09.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.570+0000 7f6287771740 1 --2- 172.21.15.105:0/2026882125 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7804c51b0 0x55a7804e3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:09.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.570+0000 7f6287771740 1 --2- 172.21.15.105:0/2026882125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7804d3fc0 0x55a7804e3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:09.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.570+0000 7f6287771740 1 --2- 172.21.15.105:0/2026882125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7804c4890 0x55a7804e6000 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:09.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.570+0000 7f6287771740 1 -- 172.21.15.105:0/2026882125 >> 172.21.15.105:0/2026882125 conn(0x55a7804c0f60 msgr2=0x55a7804bbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:09.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.570+0000 7f6287771740 1 -- 172.21.15.105:0/2026882125 shutdown_connections 2024-09-16T02:02:09.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.570+0000 7f6287771740 1 -- 172.21.15.105:0/2026882125 wait complete. 2024-09-16T02:02:09.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.570+0000 7f6287771740 1 -- start start 2024-09-16T02:02:09.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.570+0000 7f6287771740 1 Processor -- start 2024-09-16T02:02:09.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.571+0000 7f6287771740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7804c4890 0x55a7804ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:09.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.571+0000 7f6287771740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7804c51b0 0x55a7804cce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:09.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.571+0000 7f6287771740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7804d3fc0 0x55a7804cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:09.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.571+0000 7f6287771740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a7804e5e80 con 0x55a7804c51b0 2024-09-16T02:02:09.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.571+0000 7f6287771740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a7804e57b0 con 0x55a7804d3fc0 2024-09-16T02:02:09.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.571+0000 7f6287771740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a7804e6550 con 0x55a7804c4890 2024-09-16T02:02:09.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.571+0000 7f6287771740 10 client.0 _ll_register_callbacks cb 0x7f62700f38f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:02:09.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.571+0000 7f627e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7804c51b0 0x55a7804cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:09.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.571+0000 7f627e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7804c51b0 0x55a7804cce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35560/0 (socket says 172.21.15.105:35560) 2024-09-16T02:02:09.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.571+0000 7f627e7fc640 1 -- 172.21.15.105:0/1718674260 learned_addr learned my addr 172.21.15.105:0/1718674260 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:09.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.571+0000 7f627f7fe640 1 --2- 172.21.15.105:0/1718674260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7804d3fc0 0x55a7804cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:09.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.571+0000 7f627effd640 1 --2- 172.21.15.105:0/1718674260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7804c4890 0x55a7804ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:09.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.571+0000 7f627e7fc640 1 -- 172.21.15.105:0/1718674260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7804c4890 msgr2=0x55a7804ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:09.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.571+0000 7f627e7fc640 1 --2- 172.21.15.105:0/1718674260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7804c4890 0x55a7804ce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:09.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.571+0000 7f627e7fc640 1 -- 172.21.15.105:0/1718674260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7804d3fc0 msgr2=0x55a7804cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:09.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.571+0000 7f627e7fc640 1 --2- 172.21.15.105:0/1718674260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7804d3fc0 0x55a7804cd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:09.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.571+0000 7f627e7fc640 1 -- 172.21.15.105:0/1718674260 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7804cdc20 con 0x55a7804c51b0 2024-09-16T02:02:09.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.571+0000 7f627f7fe640 1 --2- 172.21.15.105:0/1718674260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7804d3fc0 0x55a7804cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:02:09.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.571+0000 7f627effd640 1 --2- 172.21.15.105:0/1718674260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7804c4890 0x55a7804ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:02:09.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.571+0000 7f627e7fc640 1 --2- 172.21.15.105:0/1718674260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7804c51b0 0x55a7804cce40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f627400b740 tx=0x7f6274005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12635d4029710903 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:09.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.571+0000 7f6262ff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:02:09.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.571+0000 7f6262ff5640 1 -- 172.21.15.105:0/1718674260 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6274011440 con 0x55a7804c51b0 2024-09-16T02:02:09.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.571+0000 7f6262ff5640 1 -- 172.21.15.105:0/1718674260 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f62740115e0 con 0x55a7804c51b0 2024-09-16T02:02:09.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.571+0000 7f6287771740 1 -- 172.21.15.105:0/1718674260 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a780432b10 con 0x55a7804c51b0 2024-09-16T02:02:09.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.571+0000 7f6262ff5640 1 -- 172.21.15.105:0/1718674260 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f62740117e0 con 0x55a7804c51b0 2024-09-16T02:02:09.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.571+0000 7f6287771740 10 client.15084 fetch_fsmap learned FSMap version 27 2024-09-16T02:02:09.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.571+0000 7f6287771740 1 -- 172.21.15.105:0/1718674260 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a7804ee320 con 0x55a7804c51b0 2024-09-16T02:02:09.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.572+0000 7f6262ff5640 1 -- 172.21.15.105:0/1718674260 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55a780432b10 con 0x55a7804c51b0 2024-09-16T02:02:09.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.572+0000 7f6262ff5640 1 -- 172.21.15.105:0/1718674260 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f627402ea00 con 0x55a7804c51b0 2024-09-16T02:02:09.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.572+0000 7f6287771740 10 client.15084 fetch_fsmap finished waiting for FSMap version 27 2024-09-16T02:02:09.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.572+0000 7f6287771740 10 client.15084 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T02:02:09.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.572+0000 7f6287771740 20 client.15084 populate_metadata read hostname 'smithi105' 2024-09-16T02:02:09.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.572+0000 7f6287771740 1 --2- 172.21.15.105:0/1718674260 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55a780563d00 0x55a780562a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:09.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.572+0000 7f6287771740 4 client.15084 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T02:02:09.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.572+0000 7f6287771740 1 -- 172.21.15.105:0/1718674260 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "status"}) -- 0x55a7803af0e0 con 0x55a780563d00 2024-09-16T02:02:09.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.572+0000 7f627effd640 1 --2- 172.21.15.105:0/1718674260 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55a780563d00 0x55a780562a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:09.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.572+0000 7f6262ff5640 10 client.15084 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:02:09.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.572+0000 7f627effd640 1 --2- 172.21.15.105:0/1718674260 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55a780563d00 0x55a780562a10 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8c39049fb6164b2 server_cookie=8db05a27331e6127 in_seq=0 out_seq=0 2024-09-16T02:02:09.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.573+0000 7f6262ff5640 1 -- 172.21.15.105:0/1718674260 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a7803af0e0 con 0x55a780563d00 2024-09-16T02:02:09.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.573+0000 7f6262ff5640 10 client.15084 handle_command_reply: tid=0 2024-09-16T02:02:09.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.573+0000 7f6287771740 1 client.15084 shutdown 2024-09-16T02:02:09.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.573+0000 7f6287771740 1 -- 172.21.15.105:0/1718674260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7804c51b0 msgr2=0x55a7804cce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:09.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.573+0000 7f6287771740 1 --2- 172.21.15.105:0/1718674260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7804c51b0 0x55a7804cce40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f627400b740 tx=0x7f6274005d20 comp rx=0 tx=0).stop 2024-09-16T02:02:09.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.573+0000 7f6287771740 1 -- 172.21.15.105:0/1718674260 shutdown_connections 2024-09-16T02:02:09.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.573+0000 7f6287771740 1 --2- 172.21.15.105:0/1718674260 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55a780563d00 0x55a780562a10 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:09.606 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-16T02:02:09.606 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "423b7a2a-73ce-11ef-bceb-c7b262605968", 2024-09-16T02:02:09.607 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 0, 2024-09-16T02:02:09.607 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 24482, 2024-09-16T02:02:09.607 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2024-09-16T02:02:09.607 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2024-09-16T02:02:09.607 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2024-09-16T02:02:09.607 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 94.122749552000002, 2024-09-16T02:02:09.607 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 27, 2024-09-16T02:02:09.607 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 81, 2024-09-16T02:02:09.607 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 79, 2024-09-16T02:02:09.607 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 104.957249514 2024-09-16T02:02:09.607 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-16T02:02:09.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.573+0000 7f6287771740 1 --2- 172.21.15.105:0/1718674260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7804d3fc0 0x55a7804cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:09.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.573+0000 7f6287771740 1 --2- 172.21.15.105:0/1718674260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7804c51b0 0x55a7804cce40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:09.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.573+0000 7f6287771740 1 --2- 172.21.15.105:0/1718674260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7804c4890 0x55a7804ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:09.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.573+0000 7f6287771740 1 -- 172.21.15.105:0/1718674260 >> 172.21.15.105:0/1718674260 conn(0x55a7804c0f60 msgr2=0x55a7804e3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:09.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.573+0000 7f6262ff5640 0 client.15084 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:02:09.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.573+0000 7f6287771740 1 -- 172.21.15.105:0/1718674260 shutdown_connections 2024-09-16T02:02:09.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.573+0000 7f6287771740 1 -- 172.21.15.105:0/1718674260 wait complete. 2024-09-16T02:02:09.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.573+0000 7f6287771740 20 client.15084 trim_cache size 0 max 16384 2024-09-16T02:02:09.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.574+0000 7f6285398640 1 -- 172.21.15.105:0/2311861296 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f62680a32d0 msgr2=0x7f62680a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:09.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.574+0000 7f6285398640 1 --2- 172.21.15.105:0/2311861296 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f62680a32d0 0x7f62680a5790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6278007f30 tx=0x7f6278004a40 comp rx=0 tx=0).stop 2024-09-16T02:02:09.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.574+0000 7f6285398640 1 -- 172.21.15.105:0/2311861296 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6280101de0 msgr2=0x7f62801982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:09.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.574+0000 7f6285398640 1 --2- 172.21.15.105:0/2311861296 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6280101de0 0x7f62801982f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6270005940 tx=0x7f6270004270 comp rx=0 tx=0).stop 2024-09-16T02:02:09.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.574+0000 7f6285398640 1 -- 172.21.15.105:0/2311861296 shutdown_connections 2024-09-16T02:02:09.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.574+0000 7f6285398640 1 --2- 172.21.15.105:0/2311861296 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f62680a32d0 0x7f62680a5790 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:09.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.574+0000 7f6285398640 1 --2- 172.21.15.105:0/2311861296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6280102790 0x7f6280198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:09.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.574+0000 7f6285398640 1 --2- 172.21.15.105:0/2311861296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6280106f10 0x7f628019cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:09.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.574+0000 7f6285398640 1 --2- 172.21.15.105:0/2311861296 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6280101de0 0x7f62801982f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:09.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.574+0000 7f6285398640 1 -- 172.21.15.105:0/2311861296 >> 172.21.15.105:0/2311861296 conn(0x7f62800fd8c0 msgr2=0x7f62800fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:09.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.574+0000 7f6285398640 1 -- 172.21.15.105:0/2311861296 shutdown_connections 2024-09-16T02:02:09.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:09.574+0000 7f6285398640 1 -- 172.21.15.105:0/2311861296 wait complete. 2024-09-16T02:02:10.494 INFO:tasks.cephadm:Setup complete, yielding 2024-09-16T02:02:10.494 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T02:02:10.504 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi105.front.sepia.ceph.com 2024-09-16T02:02:10.505 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-16T02:02:10.754 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:02:10.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:10 smithi105 ceph-mon[31197]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-16T02:02:10.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:10 smithi105 ceph-mon[31197]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-16T02:02:10.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:10 smithi190 ceph-mon[34762]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-16T02:02:10.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:10 smithi190 ceph-mon[34762]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-16T02:02:10.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:10 smithi137 ceph-mon[34099]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-16T02:02:10.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:10 smithi137 ceph-mon[34099]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-16T02:02:11.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.218+0000 7fa6763f7640 1 -- 172.21.15.105:0/4163611811 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa670102e00 msgr2=0x7fa670103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:11.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.218+0000 7fa6763f7640 1 --2- 172.21.15.105:0/4163611811 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa670102e00 0x7fa670103280 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa658009900 tx=0x7fa65802a410 comp rx=0 tx=0).stop 2024-09-16T02:02:11.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.218+0000 7fa6763f7640 1 -- 172.21.15.105:0/4163611811 shutdown_connections 2024-09-16T02:02:11.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.218+0000 7fa6763f7640 1 --2- 172.21.15.105:0/4163611811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6701037c0 0x7fa67010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:11.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.218+0000 7fa6763f7640 1 --2- 172.21.15.105:0/4163611811 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa670102e00 0x7fa670103280 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:11.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.218+0000 7fa6763f7640 1 --2- 172.21.15.105:0/4163611811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa670101c00 0x7fa670102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:11.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.218+0000 7fa6763f7640 1 -- 172.21.15.105:0/4163611811 >> 172.21.15.105:0/4163611811 conn(0x7fa6700fd3b0 msgr2=0x7fa6700ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:11.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.218+0000 7fa6763f7640 1 -- 172.21.15.105:0/4163611811 shutdown_connections 2024-09-16T02:02:11.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.218+0000 7fa6763f7640 1 -- 172.21.15.105:0/4163611811 wait complete. 2024-09-16T02:02:11.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.219+0000 7fa6763f7640 1 Processor -- start 2024-09-16T02:02:11.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.219+0000 7fa6763f7640 1 -- start start 2024-09-16T02:02:11.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.220+0000 7fa6763f7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa670101c00 0x7fa670197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:11.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.220+0000 7fa6763f7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa670102e00 0x7fa670198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:11.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.220+0000 7fa6763f7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6701037c0 0x7fa67019f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:11.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.220+0000 7fa6763f7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa67010d830 con 0x7fa670101c00 2024-09-16T02:02:11.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.220+0000 7fa6763f7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa67010d6b0 con 0x7fa6701037c0 2024-09-16T02:02:11.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.220+0000 7fa6763f7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa67010d9b0 con 0x7fa670102e00 2024-09-16T02:02:11.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.220+0000 7fa66ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa670101c00 0x7fa670197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:11.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.220+0000 7fa66ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa670101c00 0x7fa670197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35574/0 (socket says 172.21.15.105:35574) 2024-09-16T02:02:11.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.220+0000 7fa66ffff640 1 -- 172.21.15.105:0/3557228356 learned_addr learned my addr 172.21.15.105:0/3557228356 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:11.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.220+0000 7fa66ffff640 1 -- 172.21.15.105:0/3557228356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa670102e00 msgr2=0x7fa670198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.220+0000 7fa66ffff640 1 --2- 172.21.15.105:0/3557228356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa670102e00 0x7fa670198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.220+0000 7fa66ffff640 1 -- 172.21.15.105:0/3557228356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6701037c0 msgr2=0x7fa67019f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.220+0000 7fa66ffff640 1 --2- 172.21.15.105:0/3557228356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6701037c0 0x7fa67019f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.220+0000 7fa66ffff640 1 -- 172.21.15.105:0/3557228356 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa67019fc00 con 0x7fa670101c00 2024-09-16T02:02:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.220+0000 7fa66ffff640 1 --2- 172.21.15.105:0/3557228356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa670101c00 0x7fa670197fd0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa66000b450 tx=0x7fa66000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.220+0000 7fa66d7fa640 1 -- 172.21.15.105:0/3557228356 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa660015020 con 0x7fa670101c00 2024-09-16T02:02:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.220+0000 7fa66d7fa640 1 -- 172.21.15.105:0/3557228356 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa660011d40 con 0x7fa670101c00 2024-09-16T02:02:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.220+0000 7fa6763f7640 1 -- 172.21.15.105:0/3557228356 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa67019fef0 con 0x7fa670101c00 2024-09-16T02:02:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.220+0000 7fa66d7fa640 1 -- 172.21.15.105:0/3557228356 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66000f340 con 0x7fa670101c00 2024-09-16T02:02:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.220+0000 7fa6763f7640 1 -- 172.21.15.105:0/3557228356 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6701a0420 con 0x7fa670101c00 2024-09-16T02:02:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.221+0000 7fa6763f7640 1 -- 172.21.15.105:0/3557228356 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6701090c0 con 0x7fa670101c00 2024-09-16T02:02:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.223+0000 7fa66d7fa640 1 -- 172.21.15.105:0/3557228356 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fa660018020 con 0x7fa670101c00 2024-09-16T02:02:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.223+0000 7fa66d7fa640 1 --2- 172.21.15.105:0/3557228356 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa65c0a3270 0x7fa65c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.223+0000 7fa66f7fe640 1 --2- 172.21.15.105:0/3557228356 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa65c0a3270 0x7fa65c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:11.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.223+0000 7fa66d7fa640 1 -- 172.21.15.105:0/3557228356 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fa6600cfa20 con 0x7fa670101c00 2024-09-16T02:02:11.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.224+0000 7fa66f7fe640 1 --2- 172.21.15.105:0/3557228356 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa65c0a3270 0x7fa65c0a5730 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa65802a930 tx=0x7fa6580314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:11.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.227+0000 7fa66d7fa640 1 -- 172.21.15.105:0/3557228356 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa66001f020 con 0x7fa670101c00 2024-09-16T02:02:11.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.408+0000 7fa6763f7640 1 -- 172.21.15.105:0/3557228356 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fa670062630 con 0x7fa65c0a3270 2024-09-16T02:02:11.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.409+0000 7fa66d7fa640 1 -- 172.21.15.105:0/3557228356 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fa670062630 con 0x7fa65c0a3270 2024-09-16T02:02:11.411 INFO:teuthology.orchestra.run.smithi105.stdout:Backend: cephadm 2024-09-16T02:02:11.411 INFO:teuthology.orchestra.run.smithi105.stdout:Available: Yes 2024-09-16T02:02:11.411 INFO:teuthology.orchestra.run.smithi105.stdout:Paused: No 2024-09-16T02:02:11.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.411+0000 7fa6763f7640 1 -- 172.21.15.105:0/3557228356 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa65c0a3270 msgr2=0x7fa65c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:11.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.411+0000 7fa6763f7640 1 --2- 172.21.15.105:0/3557228356 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa65c0a3270 0x7fa65c0a5730 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa65802a930 tx=0x7fa6580314a0 comp rx=0 tx=0).stop 2024-09-16T02:02:11.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.411+0000 7fa6763f7640 1 -- 172.21.15.105:0/3557228356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa670101c00 msgr2=0x7fa670197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:11.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.411+0000 7fa6763f7640 1 --2- 172.21.15.105:0/3557228356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa670101c00 0x7fa670197fd0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa66000b450 tx=0x7fa66000b910 comp rx=0 tx=0).stop 2024-09-16T02:02:11.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.411+0000 7fa6763f7640 1 -- 172.21.15.105:0/3557228356 shutdown_connections 2024-09-16T02:02:11.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.411+0000 7fa6763f7640 1 --2- 172.21.15.105:0/3557228356 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa65c0a3270 0x7fa65c0a5730 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:11.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.411+0000 7fa6763f7640 1 --2- 172.21.15.105:0/3557228356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa670102e00 0x7fa670198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:11.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.411+0000 7fa6763f7640 1 --2- 172.21.15.105:0/3557228356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6701037c0 0x7fa67019f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:11.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.411+0000 7fa6763f7640 1 --2- 172.21.15.105:0/3557228356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa670101c00 0x7fa670197fd0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:11.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.411+0000 7fa6763f7640 1 -- 172.21.15.105:0/3557228356 >> 172.21.15.105:0/3557228356 conn(0x7fa6700fd3b0 msgr2=0x7fa670106290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:11.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.411+0000 7fa6763f7640 1 -- 172.21.15.105:0/3557228356 shutdown_connections 2024-09-16T02:02:11.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:11.411+0000 7fa6763f7640 1 -- 172.21.15.105:0/3557228356 wait complete. 2024-09-16T02:02:12.107 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-16T02:02:12.350 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:02:12.765 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:12 smithi105 ceph-mon[31197]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:02:12.765 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:12 smithi105 ceph-mon[31197]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-16T02:02:12.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:12 smithi190 ceph-mon[34762]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:02:12.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:12 smithi190 ceph-mon[34762]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-16T02:02:12.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:12.843+0000 7fb6d4897640 1 -- 172.21.15.105:0/3323159560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6d0105010 msgr2=0x7fb6d0107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:12.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:12.843+0000 7fb6d4897640 1 --2- 172.21.15.105:0/3323159560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6d0105010 0x7fb6d0107400 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8009900 tx=0x7fb6b802a410 comp rx=0 tx=0).stop 2024-09-16T02:02:12.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:12.844+0000 7fb6d4897640 1 -- 172.21.15.105:0/3323159560 shutdown_connections 2024-09-16T02:02:12.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:12.844+0000 7fb6d4897640 1 --2- 172.21.15.105:0/3323159560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6d0107940 0x7fb6d0109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:12.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:12.844+0000 7fb6d4897640 1 --2- 172.21.15.105:0/3323159560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6d0105010 0x7fb6d0107400 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:12.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:12.844+0000 7fb6d4897640 1 --2- 172.21.15.105:0/3323159560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6d01026e0 0x7fb6d0104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:12.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:12.844+0000 7fb6d4897640 1 -- 172.21.15.105:0/3323159560 >> 172.21.15.105:0/3323159560 conn(0x7fb6d00fc280 msgr2=0x7fb6d00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:12.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:12.844+0000 7fb6d4897640 1 -- 172.21.15.105:0/3323159560 shutdown_connections 2024-09-16T02:02:12.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:12.844+0000 7fb6d4897640 1 -- 172.21.15.105:0/3323159560 wait complete. 2024-09-16T02:02:12.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:12.844+0000 7fb6d4897640 1 Processor -- start 2024-09-16T02:02:12.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:12.844+0000 7fb6d4897640 1 -- start start 2024-09-16T02:02:12.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:12.844+0000 7fb6d4897640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6d01026e0 0x7fb6d0197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:12.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:12.844+0000 7fb6d4897640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6d0105010 0x7fb6d0198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:12.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:12.845+0000 7fb6d4897640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6d0107940 0x7fb6d019f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:12.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:12.845+0000 7fb6d4897640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d010d760 con 0x7fb6d0107940 2024-09-16T02:02:12.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:12.845+0000 7fb6d4897640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d010d5e0 con 0x7fb6d0105010 2024-09-16T02:02:12.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:12.845+0000 7fb6d4897640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d010d8e0 con 0x7fb6d01026e0 2024-09-16T02:02:12.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:12.845+0000 7fb6ced9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6d0107940 0x7fb6d019f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:12.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:12.845+0000 7fb6ced9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6d0107940 0x7fb6d019f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35594/0 (socket says 172.21.15.105:35594) 2024-09-16T02:02:12.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:12.845+0000 7fb6ced9d640 1 -- 172.21.15.105:0/3710092209 learned_addr learned my addr 172.21.15.105:0/3710092209 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:12.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:12.845+0000 7fb6cdd9b640 1 --2- 172.21.15.105:0/3710092209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6d0105010 0x7fb6d0198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:12.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:12.845+0000 7fb6ced9d640 1 -- 172.21.15.105:0/3710092209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6d01026e0 msgr2=0x7fb6d0197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:12.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:12.845+0000 7fb6ced9d640 1 --2- 172.21.15.105:0/3710092209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6d01026e0 0x7fb6d0197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:12.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:12.845+0000 7fb6ced9d640 1 -- 172.21.15.105:0/3710092209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6d0105010 msgr2=0x7fb6d0198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:12.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:12.845+0000 7fb6ced9d640 1 --2- 172.21.15.105:0/3710092209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6d0105010 0x7fb6d0198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:12.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:12.845+0000 7fb6ced9d640 1 -- 172.21.15.105:0/3710092209 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d019fa20 con 0x7fb6d0107940 2024-09-16T02:02:12.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:12.845+0000 7fb6ced9d640 1 --2- 172.21.15.105:0/3710092209 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6d0107940 0x7fb6d019f3b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8009fd0 tx=0x7fb6c800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:12.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:12.845+0000 7fb6c77fe640 1 -- 172.21.15.105:0/3710092209 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8011830 con 0x7fb6d0107940 2024-09-16T02:02:12.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:12.845+0000 7fb6d4897640 1 -- 172.21.15.105:0/3710092209 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6d019fd10 con 0x7fb6d0107940 2024-09-16T02:02:12.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:12.845+0000 7fb6c77fe640 1 -- 172.21.15.105:0/3710092209 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6c8016070 con 0x7fb6d0107940 2024-09-16T02:02:12.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:12.845+0000 7fb6c77fe640 1 -- 172.21.15.105:0/3710092209 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8010de0 con 0x7fb6d0107940 2024-09-16T02:02:12.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:12.845+0000 7fb6d4897640 1 -- 172.21.15.105:0/3710092209 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6d01a01f0 con 0x7fb6d0107940 2024-09-16T02:02:12.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:12.846+0000 7fb6d4897640 1 -- 172.21.15.105:0/3710092209 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6d00fdd70 con 0x7fb6d0107940 2024-09-16T02:02:12.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:12.847+0000 7fb6c77fe640 1 -- 172.21.15.105:0/3710092209 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fb6c80119d0 con 0x7fb6d0107940 2024-09-16T02:02:12.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:12.849+0000 7fb6c77fe640 1 --2- 172.21.15.105:0/3710092209 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb6bc0a3270 0x7fb6bc0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:12.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:12.849+0000 7fb6ce59c640 1 --2- 172.21.15.105:0/3710092209 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb6bc0a3270 0x7fb6bc0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:12.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:12.849+0000 7fb6c77fe640 1 -- 172.21.15.105:0/3710092209 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fb6c80ceff0 con 0x7fb6d0107940 2024-09-16T02:02:12.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:12.849+0000 7fb6ce59c640 1 --2- 172.21.15.105:0/3710092209 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb6bc0a3270 0x7fb6bc0a5730 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0005fd0 tx=0x7fb6c00071c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:12.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:12.853+0000 7fb6c77fe640 1 -- 172.21.15.105:0/3710092209 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb6c809bd90 con 0x7fb6d0107940 2024-09-16T02:02:12.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:12 smithi137 ceph-mon[34099]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:02:12.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:12 smithi137 ceph-mon[34099]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-16T02:02:13.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:13.027+0000 7fb6d4897640 1 -- 172.21.15.105:0/3710092209 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fb6d0062630 con 0x7fb6bc0a3270 2024-09-16T02:02:13.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:13.040+0000 7fb6c77fe640 1 -- 172.21.15.105:0/3710092209 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7fb6d0062630 con 0x7fb6bc0a3270 2024-09-16T02:02:13.042 INFO:teuthology.orchestra.run.smithi105.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T02:02:13.042 INFO:teuthology.orchestra.run.smithi105.stdout:mds.a smithi105 running (111s) 31s ago 111s 17.2M - 19.1.1-326-g8921c8ef f1455bd0219c eeeb2888b1f4 2024-09-16T02:02:13.042 INFO:teuthology.orchestra.run.smithi105.stdout:mds.b smithi137 running (108s) 41s ago 108s 26.5M - 19.1.1-326-g8921c8ef f1455bd0219c d260d3616179 2024-09-16T02:02:13.042 INFO:teuthology.orchestra.run.smithi105.stdout:mds.c smithi190 running (2m) 41s ago 2m 17.0M - 19.1.1-326-g8921c8ef f1455bd0219c 9877b492490b 2024-09-16T02:02:13.042 INFO:teuthology.orchestra.run.smithi105.stdout:mds.d smithi105 running (2m) 31s ago 2m 14.5M - 19.1.1-326-g8921c8ef f1455bd0219c 66e3dc8a813a 2024-09-16T02:02:13.042 INFO:teuthology.orchestra.run.smithi105.stdout:mds.e smithi137 running (2m) 41s ago 2m 18.0M - 19.1.1-326-g8921c8ef f1455bd0219c c81646a1b225 2024-09-16T02:02:13.042 INFO:teuthology.orchestra.run.smithi105.stdout:mds.f smithi190 running (2m) 41s ago 2m 20.2M - 19.1.1-326-g8921c8ef f1455bd0219c f2747f302c47 2024-09-16T02:02:13.042 INFO:teuthology.orchestra.run.smithi105.stdout:mds.g smithi105 running (115s) 31s ago 114s 16.5M - 19.1.1-326-g8921c8ef f1455bd0219c f203576a25fd 2024-09-16T02:02:13.042 INFO:teuthology.orchestra.run.smithi105.stdout:mds.h smithi137 running (2m) 41s ago 2m 18.4M - 19.1.1-326-g8921c8ef f1455bd0219c 39d3393c8f3e 2024-09-16T02:02:13.043 INFO:teuthology.orchestra.run.smithi105.stdout:mds.i smithi190 running (118s) 41s ago 117s 18.0M - 19.1.1-326-g8921c8ef f1455bd0219c 6ac8de37e3cc 2024-09-16T02:02:13.043 INFO:teuthology.orchestra.run.smithi105.stdout:mds.j smithi105 running (2m) 31s ago 2m 14.5M - 19.1.1-326-g8921c8ef f1455bd0219c d83fe50c53e3 2024-09-16T02:02:13.043 INFO:teuthology.orchestra.run.smithi105.stdout:mds.k smithi137 running (2m) 41s ago 2m 19.3M - 19.1.1-326-g8921c8ef f1455bd0219c defb1dff59d0 2024-09-16T02:02:13.043 INFO:teuthology.orchestra.run.smithi105.stdout:mds.l smithi190 running (2m) 41s ago 2m 18.4M - 19.1.1-326-g8921c8ef f1455bd0219c 24708c1f2009 2024-09-16T02:02:13.043 INFO:teuthology.orchestra.run.smithi105.stdout:mgr.x smithi105 *:9283,8765 running (9m) 31s ago 9m 509M - 19.1.1-326-g8921c8ef f1455bd0219c 774d5ede24dc 2024-09-16T02:02:13.043 INFO:teuthology.orchestra.run.smithi105.stdout:mgr.y smithi137 *:8443,8765 running (7m) 41s ago 7m 442M - 19.1.1-326-g8921c8ef f1455bd0219c 19c45ee05332 2024-09-16T02:02:13.043 INFO:teuthology.orchestra.run.smithi105.stdout:mgr.z smithi190 *:8443,8765 running (7m) 41s ago 7m 442M - 19.1.1-326-g8921c8ef f1455bd0219c ccc24b8c20c8 2024-09-16T02:02:13.043 INFO:teuthology.orchestra.run.smithi105.stdout:mon.a smithi105 running (9m) 31s ago 9m 49.5M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 6554b6c61004 2024-09-16T02:02:13.043 INFO:teuthology.orchestra.run.smithi105.stdout:mon.b smithi137 running (7m) 41s ago 7m 42.2M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 9b210384a941 2024-09-16T02:02:13.043 INFO:teuthology.orchestra.run.smithi105.stdout:mon.c smithi190 running (8m) 41s ago 8m 42.0M 2048M 19.1.1-326-g8921c8ef f1455bd0219c ef4296c19219 2024-09-16T02:02:13.043 INFO:teuthology.orchestra.run.smithi105.stdout:osd.0 smithi105 running (7m) 31s ago 7m 44.9M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 4157ffd1183d 2024-09-16T02:02:13.043 INFO:teuthology.orchestra.run.smithi105.stdout:osd.1 smithi137 running (6m) 41s ago 6m 56.5M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 557105c5c9c5 2024-09-16T02:02:13.043 INFO:teuthology.orchestra.run.smithi105.stdout:osd.2 smithi190 running (6m) 41s ago 6m 56.6M 4272M 19.1.1-326-g8921c8ef f1455bd0219c b1e8df4b4ee3 2024-09-16T02:02:13.043 INFO:teuthology.orchestra.run.smithi105.stdout:osd.3 smithi105 running (6m) 31s ago 6m 52.7M 4272M 19.1.1-326-g8921c8ef f1455bd0219c d961aa8fca88 2024-09-16T02:02:13.044 INFO:teuthology.orchestra.run.smithi105.stdout:osd.4 smithi137 running (5m) 41s ago 5m 42.7M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 4f5f277921f2 2024-09-16T02:02:13.044 INFO:teuthology.orchestra.run.smithi105.stdout:osd.5 smithi190 running (5m) 41s ago 5m 46.3M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 5f4f7269ef89 2024-09-16T02:02:13.044 INFO:teuthology.orchestra.run.smithi105.stdout:osd.6 smithi105 running (4m) 31s ago 4m 51.9M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 01ed28b823f4 2024-09-16T02:02:13.044 INFO:teuthology.orchestra.run.smithi105.stdout:osd.7 smithi137 running (4m) 41s ago 4m 53.9M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 714626cf7745 2024-09-16T02:02:13.044 INFO:teuthology.orchestra.run.smithi105.stdout:osd.8 smithi190 running (3m) 41s ago 3m 42.7M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 0aa5ec2b6213 2024-09-16T02:02:13.044 INFO:teuthology.orchestra.run.smithi105.stdout:osd.9 smithi105 running (3m) 31s ago 3m 52.0M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 8e431fc270a4 2024-09-16T02:02:13.044 INFO:teuthology.orchestra.run.smithi105.stdout:osd.10 smithi137 running (3m) 41s ago 3m 53.6M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 1d8ba7b3745e 2024-09-16T02:02:13.044 INFO:teuthology.orchestra.run.smithi105.stdout:osd.11 smithi190 running (2m) 41s ago 2m 40.3M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 0710c35df955 2024-09-16T02:02:13.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:13.042+0000 7fb6d4897640 1 -- 172.21.15.105:0/3710092209 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb6bc0a3270 msgr2=0x7fb6bc0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:13.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:13.042+0000 7fb6d4897640 1 --2- 172.21.15.105:0/3710092209 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb6bc0a3270 0x7fb6bc0a5730 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0005fd0 tx=0x7fb6c00071c0 comp rx=0 tx=0).stop 2024-09-16T02:02:13.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:13.042+0000 7fb6d4897640 1 -- 172.21.15.105:0/3710092209 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6d0107940 msgr2=0x7fb6d019f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:13.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:13.042+0000 7fb6d4897640 1 --2- 172.21.15.105:0/3710092209 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6d0107940 0x7fb6d019f3b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8009fd0 tx=0x7fb6c800b8e0 comp rx=0 tx=0).stop 2024-09-16T02:02:13.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:13.042+0000 7fb6d4897640 1 -- 172.21.15.105:0/3710092209 shutdown_connections 2024-09-16T02:02:13.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:13.042+0000 7fb6d4897640 1 --2- 172.21.15.105:0/3710092209 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb6bc0a3270 0x7fb6bc0a5730 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:13.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:13.042+0000 7fb6d4897640 1 --2- 172.21.15.105:0/3710092209 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6d0107940 0x7fb6d019f3b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:13.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:13.042+0000 7fb6d4897640 1 --2- 172.21.15.105:0/3710092209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6d0105010 0x7fb6d0198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:13.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:13.042+0000 7fb6d4897640 1 --2- 172.21.15.105:0/3710092209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6d01026e0 0x7fb6d0197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:13.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:13.042+0000 7fb6d4897640 1 -- 172.21.15.105:0/3710092209 >> 172.21.15.105:0/3710092209 conn(0x7fb6d00fc280 msgr2=0x7fb6d0103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:13.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:13.043+0000 7fb6d4897640 1 -- 172.21.15.105:0/3710092209 shutdown_connections 2024-09-16T02:02:13.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:13.043+0000 7fb6d4897640 1 -- 172.21.15.105:0/3710092209 wait complete. 2024-09-16T02:02:13.858 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-16T02:02:14.105 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:02:14.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.612+0000 7f4fdf49d640 1 -- 172.21.15.105:0/504139699 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4fd8102700 msgr2=0x7f4fd8104af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:14.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.612+0000 7f4fdf49d640 1 --2- 172.21.15.105:0/504139699 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4fd8102700 0x7f4fd8104af0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0009900 tx=0x7f4fd002a410 comp rx=0 tx=0).stop 2024-09-16T02:02:14.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.612+0000 7f4fdf49d640 1 -- 172.21.15.105:0/504139699 shutdown_connections 2024-09-16T02:02:14.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.612+0000 7f4fdf49d640 1 --2- 172.21.15.105:0/504139699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fd8105030 0x7f4fd8107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:14.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.612+0000 7f4fdf49d640 1 --2- 172.21.15.105:0/504139699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fd8107960 0x7f4fd8109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:14.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.612+0000 7f4fdf49d640 1 --2- 172.21.15.105:0/504139699 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4fd8102700 0x7f4fd8104af0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:14.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.612+0000 7f4fdf49d640 1 -- 172.21.15.105:0/504139699 >> 172.21.15.105:0/504139699 conn(0x7f4fd80fc280 msgr2=0x7f4fd80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:14.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.613+0000 7f4fdf49d640 1 -- 172.21.15.105:0/504139699 shutdown_connections 2024-09-16T02:02:14.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.613+0000 7f4fdf49d640 1 -- 172.21.15.105:0/504139699 wait complete. 2024-09-16T02:02:14.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.613+0000 7f4fdf49d640 1 Processor -- start 2024-09-16T02:02:14.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.613+0000 7f4fdf49d640 1 -- start start 2024-09-16T02:02:14.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.613+0000 7f4fdf49d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fd8102700 0x7f4fd8197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:14.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.613+0000 7f4fdf49d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4fd8105030 0x7f4fd8198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:14.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.613+0000 7f4fdf49d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fd8107960 0x7f4fd819f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:14.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.613+0000 7f4fdf49d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd810d780 con 0x7f4fd8105030 2024-09-16T02:02:14.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.613+0000 7f4fdf49d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd810d600 con 0x7f4fd8102700 2024-09-16T02:02:14.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.613+0000 7f4fdf49d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd810d900 con 0x7f4fd8107960 2024-09-16T02:02:14.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.613+0000 7f4fdca38640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4fd8105030 0x7f4fd8198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:14.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.613+0000 7f4fdca38640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4fd8105030 0x7f4fd8198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35620/0 (socket says 172.21.15.105:35620) 2024-09-16T02:02:14.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.613+0000 7f4fdca38640 1 -- 172.21.15.105:0/392971109 learned_addr learned my addr 172.21.15.105:0/392971109 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:14.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.614+0000 7f4fdd239640 1 --2- 172.21.15.105:0/392971109 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fd8102700 0x7f4fd8197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:14.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.614+0000 7f4fdca38640 1 -- 172.21.15.105:0/392971109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fd8107960 msgr2=0x7f4fd819f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:14.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.614+0000 7f4fdca38640 1 --2- 172.21.15.105:0/392971109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fd8107960 0x7f4fd819f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:14.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.614+0000 7f4fdca38640 1 -- 172.21.15.105:0/392971109 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fd8102700 msgr2=0x7f4fd8197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:14.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.614+0000 7f4fdca38640 1 --2- 172.21.15.105:0/392971109 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fd8102700 0x7f4fd8197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:14.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.614+0000 7f4fdca38640 1 -- 172.21.15.105:0/392971109 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd819fb20 con 0x7f4fd8105030 2024-09-16T02:02:14.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.614+0000 7f4fdd239640 1 --2- 172.21.15.105:0/392971109 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fd8102700 0x7f4fd8197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:02:14.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.614+0000 7f4fdca38640 1 --2- 172.21.15.105:0/392971109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4fd8105030 0x7f4fd8198430 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc00b9c0 tx=0x7f4fcc00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:14.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.614+0000 7f4fd67fc640 1 -- 172.21.15.105:0/392971109 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc019070 con 0x7f4fd8105030 2024-09-16T02:02:14.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.614+0000 7f4fd67fc640 1 -- 172.21.15.105:0/392971109 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4fcc00cba0 con 0x7f4fd8105030 2024-09-16T02:02:14.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.614+0000 7f4fdf49d640 1 -- 172.21.15.105:0/392971109 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fd819fe10 con 0x7f4fd8105030 2024-09-16T02:02:14.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.614+0000 7f4fd67fc640 1 -- 172.21.15.105:0/392971109 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc012580 con 0x7f4fd8105030 2024-09-16T02:02:14.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.614+0000 7f4fdf49d640 1 -- 172.21.15.105:0/392971109 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fd81a0340 con 0x7f4fd8105030 2024-09-16T02:02:14.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.615+0000 7f4fdf49d640 1 -- 172.21.15.105:0/392971109 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fd80fdd70 con 0x7f4fd8105030 2024-09-16T02:02:14.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.616+0000 7f4fd67fc640 1 -- 172.21.15.105:0/392971109 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f4fcc01c070 con 0x7f4fd8105030 2024-09-16T02:02:14.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.617+0000 7f4fd67fc640 1 --2- 172.21.15.105:0/392971109 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f4fc00a32f0 0x7f4fc00a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:14.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.617+0000 7f4fdd239640 1 --2- 172.21.15.105:0/392971109 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f4fc00a32f0 0x7f4fc00a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:14.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.617+0000 7f4fd67fc640 1 -- 172.21.15.105:0/392971109 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f4fcc0cef20 con 0x7f4fd8105030 2024-09-16T02:02:14.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.617+0000 7f4fdd239640 1 --2- 172.21.15.105:0/392971109 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f4fc00a32f0 0x7f4fc00a57b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4fd00314e0 tx=0x7f4fd0031d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:14.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.621+0000 7f4fd67fc640 1 -- 172.21.15.105:0/392971109 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4fcc09bc20 con 0x7f4fd8105030 2024-09-16T02:02:14.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:14 smithi105 ceph-mon[31197]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:02:14.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:14 smithi105 ceph-mon[31197]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-16T02:02:14.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.802+0000 7f4fdf49d640 1 -- 172.21.15.105:0/392971109 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f4fd8062630 con 0x7f4fc00a32f0 2024-09-16T02:02:14.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.808+0000 7f4fd67fc640 1 -- 172.21.15.105:0/392971109 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f4fd8062630 con 0x7f4fc00a32f0 2024-09-16T02:02:14.810 INFO:teuthology.orchestra.run.smithi105.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-16T02:02:14.810 INFO:teuthology.orchestra.run.smithi105.stdout:mds.all 12/12 43s ago 2m smithi105=a;smithi105=d;smithi105=g;smithi105=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l;count:12 2024-09-16T02:02:14.810 INFO:teuthology.orchestra.run.smithi105.stdout:mgr 3/3 43s ago 7m smithi105=x;smithi137=y;smithi190=z;count:3 2024-09-16T02:02:14.810 INFO:teuthology.orchestra.run.smithi105.stdout:mon 3/3 43s ago 8m smithi105:172.21.15.105=a;smithi137:172.21.15.137=b;smithi190:172.21.15.190=c;count:3 2024-09-16T02:02:14.810 INFO:teuthology.orchestra.run.smithi105.stdout:osd 12 43s ago - 2024-09-16T02:02:14.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.810+0000 7f4fdf49d640 1 -- 172.21.15.105:0/392971109 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f4fc00a32f0 msgr2=0x7f4fc00a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:14.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.810+0000 7f4fdf49d640 1 --2- 172.21.15.105:0/392971109 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f4fc00a32f0 0x7f4fc00a57b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4fd00314e0 tx=0x7f4fd0031d30 comp rx=0 tx=0).stop 2024-09-16T02:02:14.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.810+0000 7f4fdf49d640 1 -- 172.21.15.105:0/392971109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4fd8105030 msgr2=0x7f4fd8198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:14.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.810+0000 7f4fdf49d640 1 --2- 172.21.15.105:0/392971109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4fd8105030 0x7f4fd8198430 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc00b9c0 tx=0x7f4fcc00be80 comp rx=0 tx=0).stop 2024-09-16T02:02:14.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.810+0000 7f4fdf49d640 1 -- 172.21.15.105:0/392971109 shutdown_connections 2024-09-16T02:02:14.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.810+0000 7f4fdf49d640 1 --2- 172.21.15.105:0/392971109 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f4fc00a32f0 0x7f4fc00a57b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:14.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.810+0000 7f4fdf49d640 1 --2- 172.21.15.105:0/392971109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fd8107960 0x7f4fd819f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:14.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.810+0000 7f4fdf49d640 1 --2- 172.21.15.105:0/392971109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4fd8105030 0x7f4fd8198430 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:14.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.810+0000 7f4fdf49d640 1 --2- 172.21.15.105:0/392971109 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fd8102700 0x7f4fd8197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:14.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.810+0000 7f4fdf49d640 1 -- 172.21.15.105:0/392971109 >> 172.21.15.105:0/392971109 conn(0x7f4fd80fc280 msgr2=0x7f4fd8103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:14.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.810+0000 7f4fdf49d640 1 -- 172.21.15.105:0/392971109 shutdown_connections 2024-09-16T02:02:14.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:14.810+0000 7f4fdf49d640 1 -- 172.21.15.105:0/392971109 wait complete. 2024-09-16T02:02:14.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:14 smithi190 ceph-mon[34762]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:02:14.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:14 smithi190 ceph-mon[34762]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-16T02:02:14.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:14 smithi137 ceph-mon[34099]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:02:14.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:14 smithi137 ceph-mon[34099]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-16T02:02:15.527 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-16T02:02:15.772 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:02:16.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.312+0000 7fe90e6d0640 1 -- 172.21.15.105:0/2743299439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe908102e00 msgr2=0x7fe908103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:16.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.312+0000 7fe90e6d0640 1 --2- 172.21.15.105:0/2743299439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe908102e00 0x7fe908103280 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc009900 tx=0x7fe8fc02a410 comp rx=0 tx=0).stop 2024-09-16T02:02:16.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.312+0000 7fe90e6d0640 1 -- 172.21.15.105:0/2743299439 shutdown_connections 2024-09-16T02:02:16.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.312+0000 7fe90e6d0640 1 --2- 172.21.15.105:0/2743299439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9081037c0 0x7fe90810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:16.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.312+0000 7fe90e6d0640 1 --2- 172.21.15.105:0/2743299439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe908102e00 0x7fe908103280 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:16.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.312+0000 7fe90e6d0640 1 --2- 172.21.15.105:0/2743299439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe908101c00 0x7fe908102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:16.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.312+0000 7fe90e6d0640 1 -- 172.21.15.105:0/2743299439 >> 172.21.15.105:0/2743299439 conn(0x7fe9080fd390 msgr2=0x7fe9080ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:16.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.312+0000 7fe90e6d0640 1 -- 172.21.15.105:0/2743299439 shutdown_connections 2024-09-16T02:02:16.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.312+0000 7fe90e6d0640 1 -- 172.21.15.105:0/2743299439 wait complete. 2024-09-16T02:02:16.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.313+0000 7fe90e6d0640 1 Processor -- start 2024-09-16T02:02:16.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.313+0000 7fe90e6d0640 1 -- start start 2024-09-16T02:02:16.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.313+0000 7fe90e6d0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe908101c00 0x7fe908197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:16.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.313+0000 7fe90e6d0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe908102e00 0x7fe9081983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:16.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.313+0000 7fe90e6d0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9081037c0 0x7fe90819f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:16.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.313+0000 7fe90e6d0640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe90810d7b0 con 0x7fe908102e00 2024-09-16T02:02:16.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.313+0000 7fe90e6d0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe90810d630 con 0x7fe9081037c0 2024-09-16T02:02:16.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.313+0000 7fe90e6d0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe90810d930 con 0x7fe908101c00 2024-09-16T02:02:16.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.313+0000 7fe9077fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe908102e00 0x7fe9081983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:16.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.313+0000 7fe9077fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe908102e00 0x7fe9081983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35628/0 (socket says 172.21.15.105:35628) 2024-09-16T02:02:16.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.313+0000 7fe9077fe640 1 -- 172.21.15.105:0/4106164675 learned_addr learned my addr 172.21.15.105:0/4106164675 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:16.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.313+0000 7fe907fff640 1 --2- 172.21.15.105:0/4106164675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe908101c00 0x7fe908197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:16.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.313+0000 7fe9077fe640 1 -- 172.21.15.105:0/4106164675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe908101c00 msgr2=0x7fe908197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:16.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.313+0000 7fe9077fe640 1 --2- 172.21.15.105:0/4106164675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe908101c00 0x7fe908197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:16.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.313+0000 7fe9077fe640 1 -- 172.21.15.105:0/4106164675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9081037c0 msgr2=0x7fe90819f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:16.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.313+0000 7fe9077fe640 1 --2- 172.21.15.105:0/4106164675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9081037c0 0x7fe90819f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:16.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.313+0000 7fe9077fe640 1 -- 172.21.15.105:0/4106164675 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe90819faa0 con 0x7fe908102e00 2024-09-16T02:02:16.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.313+0000 7fe907fff640 1 --2- 172.21.15.105:0/4106164675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe908101c00 0x7fe908197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:02:16.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.314+0000 7fe9077fe640 1 --2- 172.21.15.105:0/4106164675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe908102e00 0x7fe9081983b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc005940 tx=0x7fe8fc004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:16.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.314+0000 7fe9057fa640 1 -- 172.21.15.105:0/4106164675 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc03f030 con 0x7fe908102e00 2024-09-16T02:02:16.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.314+0000 7fe9057fa640 1 -- 172.21.15.105:0/4106164675 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8fc003cb0 con 0x7fe908102e00 2024-09-16T02:02:16.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.314+0000 7fe9057fa640 1 -- 172.21.15.105:0/4106164675 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc035070 con 0x7fe908102e00 2024-09-16T02:02:16.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.314+0000 7fe90e6d0640 1 -- 172.21.15.105:0/4106164675 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe90819fd30 con 0x7fe908102e00 2024-09-16T02:02:16.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.314+0000 7fe90e6d0640 1 -- 172.21.15.105:0/4106164675 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9081a0280 con 0x7fe908102e00 2024-09-16T02:02:16.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.315+0000 7fe90e6d0640 1 -- 172.21.15.105:0/4106164675 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9081090a0 con 0x7fe908102e00 2024-09-16T02:02:16.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.316+0000 7fe9057fa640 1 -- 172.21.15.105:0/4106164675 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fe8fc042030 con 0x7fe908102e00 2024-09-16T02:02:16.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.316+0000 7fe9057fa640 1 --2- 172.21.15.105:0/4106164675 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe8f00a32f0 0x7fe8f00a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:16.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.317+0000 7fe907fff640 1 --2- 172.21.15.105:0/4106164675 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe8f00a32f0 0x7fe8f00a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:16.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.317+0000 7fe9057fa640 1 -- 172.21.15.105:0/4106164675 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fe8fc0f0030 con 0x7fe908102e00 2024-09-16T02:02:16.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.317+0000 7fe907fff640 1 --2- 172.21.15.105:0/4106164675 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe8f00a32f0 0x7fe8f00a57b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8006fd0 tx=0x7fe8f8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:16.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.321+0000 7fe9057fa640 1 -- 172.21.15.105:0/4106164675 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe8fc0bcd30 con 0x7fe908102e00 2024-09-16T02:02:16.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.494+0000 7fe90e6d0640 1 -- 172.21.15.105:0/4106164675 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fe908062630 con 0x7fe8f00a32f0 2024-09-16T02:02:16.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.495+0000 7fe9057fa640 1 -- 172.21.15.105:0/4106164675 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fe908062630 con 0x7fe8f00a32f0 2024-09-16T02:02:16.497 INFO:teuthology.orchestra.run.smithi105.stdout:HOST ADDR LABELS STATUS 2024-09-16T02:02:16.497 INFO:teuthology.orchestra.run.smithi105.stdout:smithi105 172.21.15.105 2024-09-16T02:02:16.497 INFO:teuthology.orchestra.run.smithi105.stdout:smithi137 172.21.15.137 2024-09-16T02:02:16.497 INFO:teuthology.orchestra.run.smithi105.stdout:smithi190 172.21.15.190 2024-09-16T02:02:16.497 INFO:teuthology.orchestra.run.smithi105.stdout:3 hosts in cluster 2024-09-16T02:02:16.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.497+0000 7fe90e6d0640 1 -- 172.21.15.105:0/4106164675 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe8f00a32f0 msgr2=0x7fe8f00a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:16.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.497+0000 7fe90e6d0640 1 --2- 172.21.15.105:0/4106164675 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe8f00a32f0 0x7fe8f00a57b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8006fd0 tx=0x7fe8f8008040 comp rx=0 tx=0).stop 2024-09-16T02:02:16.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.497+0000 7fe90e6d0640 1 -- 172.21.15.105:0/4106164675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe908102e00 msgr2=0x7fe9081983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:16.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.497+0000 7fe90e6d0640 1 --2- 172.21.15.105:0/4106164675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe908102e00 0x7fe9081983b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc005940 tx=0x7fe8fc004430 comp rx=0 tx=0).stop 2024-09-16T02:02:16.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.497+0000 7fe90e6d0640 1 -- 172.21.15.105:0/4106164675 shutdown_connections 2024-09-16T02:02:16.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.497+0000 7fe90e6d0640 1 --2- 172.21.15.105:0/4106164675 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe8f00a32f0 0x7fe8f00a57b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:16.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.497+0000 7fe90e6d0640 1 --2- 172.21.15.105:0/4106164675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9081037c0 0x7fe90819f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:16.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.497+0000 7fe90e6d0640 1 --2- 172.21.15.105:0/4106164675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe908102e00 0x7fe9081983b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:16.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.497+0000 7fe90e6d0640 1 --2- 172.21.15.105:0/4106164675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe908101c00 0x7fe908197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:16.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.497+0000 7fe90e6d0640 1 -- 172.21.15.105:0/4106164675 >> 172.21.15.105:0/4106164675 conn(0x7fe9080fd390 msgr2=0x7fe908106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:16.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.497+0000 7fe90e6d0640 1 -- 172.21.15.105:0/4106164675 shutdown_connections 2024-09-16T02:02:16.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:16.497+0000 7fe90e6d0640 1 -- 172.21.15.105:0/4106164675 wait complete. 2024-09-16T02:02:16.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:16 smithi105 ceph-mon[31197]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:02:16.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:16 smithi105 ceph-mon[31197]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-16T02:02:16.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:16 smithi190 ceph-mon[34762]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:02:16.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:16 smithi190 ceph-mon[34762]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-16T02:02:16.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:16 smithi137 ceph-mon[34099]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:02:16.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:16 smithi137 ceph-mon[34099]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-16T02:02:17.349 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-16T02:02:17.599 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:02:17.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:17 smithi190 ceph-mon[34762]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:02:17.866 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:17 smithi105 ceph-mon[31197]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:02:17.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:17 smithi137 ceph-mon[34099]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:02:18.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.074+0000 7fb2d0376640 1 -- 172.21.15.105:0/2818047203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2c8102e00 msgr2=0x7fb2c8103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:18.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.074+0000 7fb2d0376640 1 --2- 172.21.15.105:0/2818047203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2c8102e00 0x7fb2c8103280 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4009900 tx=0x7fb2c402a410 comp rx=0 tx=0).stop 2024-09-16T02:02:18.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.074+0000 7fb2d0376640 1 -- 172.21.15.105:0/2818047203 shutdown_connections 2024-09-16T02:02:18.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.074+0000 7fb2d0376640 1 --2- 172.21.15.105:0/2818047203 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2c81037c0 0x7fb2c810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:18.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.074+0000 7fb2d0376640 1 --2- 172.21.15.105:0/2818047203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2c8102e00 0x7fb2c8103280 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:18.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.074+0000 7fb2d0376640 1 --2- 172.21.15.105:0/2818047203 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2c8101c00 0x7fb2c8102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:18.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.074+0000 7fb2d0376640 1 -- 172.21.15.105:0/2818047203 >> 172.21.15.105:0/2818047203 conn(0x7fb2c80fd390 msgr2=0x7fb2c80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:18.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.074+0000 7fb2d0376640 1 -- 172.21.15.105:0/2818047203 shutdown_connections 2024-09-16T02:02:18.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.074+0000 7fb2d0376640 1 -- 172.21.15.105:0/2818047203 wait complete. 2024-09-16T02:02:18.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.074+0000 7fb2d0376640 1 Processor -- start 2024-09-16T02:02:18.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.074+0000 7fb2d0376640 1 -- start start 2024-09-16T02:02:18.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.075+0000 7fb2d0376640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2c8101c00 0x7fb2c8075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:18.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.075+0000 7fb2d0376640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2c8102e00 0x7fb2c80745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:18.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.075+0000 7fb2d0376640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2c81037c0 0x7fb2c8074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:18.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.075+0000 7fb2d0376640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c810d830 con 0x7fb2c8102e00 2024-09-16T02:02:18.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.075+0000 7fb2d0376640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c810d6b0 con 0x7fb2c81037c0 2024-09-16T02:02:18.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.075+0000 7fb2d0376640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c810d9b0 con 0x7fb2c8101c00 2024-09-16T02:02:18.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.075+0000 7fb2cd911640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2c8102e00 0x7fb2c80745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:18.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.075+0000 7fb2cd911640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2c8102e00 0x7fb2c80745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35634/0 (socket says 172.21.15.105:35634) 2024-09-16T02:02:18.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.075+0000 7fb2cd911640 1 -- 172.21.15.105:0/1490229208 learned_addr learned my addr 172.21.15.105:0/1490229208 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:18.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.075+0000 7fb2ce913640 1 --2- 172.21.15.105:0/1490229208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2c81037c0 0x7fb2c8074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:18.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.075+0000 7fb2cd911640 1 -- 172.21.15.105:0/1490229208 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2c8101c00 msgr2=0x7fb2c8075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:18.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.075+0000 7fb2ce112640 1 --2- 172.21.15.105:0/1490229208 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2c8101c00 0x7fb2c8075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:18.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.075+0000 7fb2cd911640 1 --2- 172.21.15.105:0/1490229208 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2c8101c00 0x7fb2c8075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:18.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.075+0000 7fb2cd911640 1 -- 172.21.15.105:0/1490229208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2c81037c0 msgr2=0x7fb2c8074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:18.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.075+0000 7fb2cd911640 1 --2- 172.21.15.105:0/1490229208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2c81037c0 0x7fb2c8074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:18.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.075+0000 7fb2cd911640 1 -- 172.21.15.105:0/1490229208 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c80753e0 con 0x7fb2c8102e00 2024-09-16T02:02:18.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.075+0000 7fb2ce913640 1 --2- 172.21.15.105:0/1490229208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2c81037c0 0x7fb2c8074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:02:18.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.075+0000 7fb2ce112640 1 --2- 172.21.15.105:0/1490229208 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2c8101c00 0x7fb2c8075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:02:18.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.075+0000 7fb2cd911640 1 --2- 172.21.15.105:0/1490229208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2c8102e00 0x7fb2c80745e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4005940 tx=0x7fb2c4004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:18.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.075+0000 7fb2bf7fe640 1 -- 172.21.15.105:0/1490229208 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c403f030 con 0x7fb2c8102e00 2024-09-16T02:02:18.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.075+0000 7fb2bf7fe640 1 -- 172.21.15.105:0/1490229208 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2c4003cb0 con 0x7fb2c8102e00 2024-09-16T02:02:18.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.075+0000 7fb2d0376640 1 -- 172.21.15.105:0/1490229208 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2c81a20f0 con 0x7fb2c8102e00 2024-09-16T02:02:18.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.075+0000 7fb2bf7fe640 1 -- 172.21.15.105:0/1490229208 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4035070 con 0x7fb2c8102e00 2024-09-16T02:02:18.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.075+0000 7fb2d0376640 1 -- 172.21.15.105:0/1490229208 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2c81a2670 con 0x7fb2c8102e00 2024-09-16T02:02:18.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.076+0000 7fb2d0376640 1 -- 172.21.15.105:0/1490229208 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2c8109140 con 0x7fb2c8102e00 2024-09-16T02:02:18.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.077+0000 7fb2bf7fe640 1 -- 172.21.15.105:0/1490229208 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fb2c4042030 con 0x7fb2c8102e00 2024-09-16T02:02:18.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.078+0000 7fb2bf7fe640 1 --2- 172.21.15.105:0/1490229208 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb2b40a32c0 0x7fb2b40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:18.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.078+0000 7fb2ce112640 1 --2- 172.21.15.105:0/1490229208 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb2b40a32c0 0x7fb2b40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:18.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.079+0000 7fb2bf7fe640 1 -- 172.21.15.105:0/1490229208 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fb2c40f06d0 con 0x7fb2c8102e00 2024-09-16T02:02:18.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.079+0000 7fb2ce112640 1 --2- 172.21.15.105:0/1490229208 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb2b40a32c0 0x7fb2b40a5780 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8006fd0 tx=0x7fb2b8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:18.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.082+0000 7fb2bf7fe640 1 -- 172.21.15.105:0/1490229208 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb2c40bd370 con 0x7fb2c8102e00 2024-09-16T02:02:18.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.256+0000 7fb2d0376640 1 -- 172.21.15.105:0/1490229208 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fb2c8062630 con 0x7fb2b40a32c0 2024-09-16T02:02:18.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.258+0000 7fb2bf7fe640 1 -- 172.21.15.105:0/1490229208 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fb2c8062630 con 0x7fb2b40a32c0 2024-09-16T02:02:18.260 INFO:teuthology.orchestra.run.smithi105.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-16T02:02:18.260 INFO:teuthology.orchestra.run.smithi105.stdout:smithi105 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400TR400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-16T02:02:18.260 INFO:teuthology.orchestra.run.smithi105.stdout:smithi137 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233004A400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-16T02:02:18.260 INFO:teuthology.orchestra.run.smithi105.stdout:smithi190 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT63210012400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-16T02:02:18.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.261+0000 7fb2d0376640 1 -- 172.21.15.105:0/1490229208 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb2b40a32c0 msgr2=0x7fb2b40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:18.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.261+0000 7fb2d0376640 1 --2- 172.21.15.105:0/1490229208 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb2b40a32c0 0x7fb2b40a5780 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8006fd0 tx=0x7fb2b8008040 comp rx=0 tx=0).stop 2024-09-16T02:02:18.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.261+0000 7fb2d0376640 1 -- 172.21.15.105:0/1490229208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2c8102e00 msgr2=0x7fb2c80745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:18.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.261+0000 7fb2d0376640 1 --2- 172.21.15.105:0/1490229208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2c8102e00 0x7fb2c80745e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4005940 tx=0x7fb2c4004430 comp rx=0 tx=0).stop 2024-09-16T02:02:18.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.261+0000 7fb2d0376640 1 -- 172.21.15.105:0/1490229208 shutdown_connections 2024-09-16T02:02:18.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.261+0000 7fb2d0376640 1 --2- 172.21.15.105:0/1490229208 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb2b40a32c0 0x7fb2b40a5780 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:18.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.261+0000 7fb2d0376640 1 --2- 172.21.15.105:0/1490229208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2c81037c0 0x7fb2c8074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:18.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.261+0000 7fb2d0376640 1 --2- 172.21.15.105:0/1490229208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2c8102e00 0x7fb2c80745e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:18.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.261+0000 7fb2d0376640 1 --2- 172.21.15.105:0/1490229208 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2c8101c00 0x7fb2c8075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:18.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.261+0000 7fb2d0376640 1 -- 172.21.15.105:0/1490229208 >> 172.21.15.105:0/1490229208 conn(0x7fb2c80fd390 msgr2=0x7fb2c8106310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:18.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.261+0000 7fb2d0376640 1 -- 172.21.15.105:0/1490229208 shutdown_connections 2024-09-16T02:02:18.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:18.261+0000 7fb2d0376640 1 -- 172.21.15.105:0/1490229208 wait complete. 2024-09-16T02:02:18.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:18 smithi190 ceph-mon[34762]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-16T02:02:18.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:18 smithi137 ceph-mon[34099]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-16T02:02:19.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:18 smithi105 ceph-mon[31197]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-16T02:02:19.152 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T02:02:19.161 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi105.front.sepia.ceph.com 2024-09-16T02:02:19.162 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-16T02:02:19.405 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:02:19.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:19 smithi105 ceph-mon[31197]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:02:19.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:19 smithi190 ceph-mon[34762]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:02:19.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.886+0000 7fb3c5a24640 1 -- 172.21.15.105:0/1564652680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3c0101cd0 msgr2=0x7fb3c0102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:19.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.886+0000 7fb3c5a24640 1 --2- 172.21.15.105:0/1564652680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3c0101cd0 0x7fb3c0102150 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8009900 tx=0x7fb3a802a410 comp rx=0 tx=0).stop 2024-09-16T02:02:19.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.886+0000 7fb3c5a24640 1 -- 172.21.15.105:0/1564652680 shutdown_connections 2024-09-16T02:02:19.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.886+0000 7fb3c5a24640 1 --2- 172.21.15.105:0/1564652680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3c0102690 0x7fb3c0108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:19.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.886+0000 7fb3c5a24640 1 --2- 172.21.15.105:0/1564652680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3c0101cd0 0x7fb3c0102150 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:19.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.886+0000 7fb3c5a24640 1 --2- 172.21.15.105:0/1564652680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3c0100ad0 0x7fb3c0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:19.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.886+0000 7fb3c5a24640 1 -- 172.21.15.105:0/1564652680 >> 172.21.15.105:0/1564652680 conn(0x7fb3c00fc280 msgr2=0x7fb3c00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:19.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.886+0000 7fb3c5a24640 1 -- 172.21.15.105:0/1564652680 shutdown_connections 2024-09-16T02:02:19.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.886+0000 7fb3c5a24640 1 -- 172.21.15.105:0/1564652680 wait complete. 2024-09-16T02:02:19.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.887+0000 7fb3c5a24640 1 Processor -- start 2024-09-16T02:02:19.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.887+0000 7fb3c5a24640 1 -- start start 2024-09-16T02:02:19.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.887+0000 7fb3c5a24640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3c0100ad0 0x7fb3c0197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:19.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.887+0000 7fb3c5a24640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3c0101cd0 0x7fb3c0198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:19.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.887+0000 7fb3c5a24640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3c0102690 0x7fb3c019f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:19.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.887+0000 7fb3c5a24640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c010c700 con 0x7fb3c0101cd0 2024-09-16T02:02:19.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.887+0000 7fb3c5a24640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c010c580 con 0x7fb3c0102690 2024-09-16T02:02:19.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.887+0000 7fb3c5a24640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c010c880 con 0x7fb3c0100ad0 2024-09-16T02:02:19.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.887+0000 7fb3b7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3c0101cd0 0x7fb3c0198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:19.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.887+0000 7fb3b7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3c0101cd0 0x7fb3c0198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60480/0 (socket says 172.21.15.105:60480) 2024-09-16T02:02:19.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.887+0000 7fb3b7fff640 1 -- 172.21.15.105:0/691653802 learned_addr learned my addr 172.21.15.105:0/691653802 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:19.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.887+0000 7fb3beffd640 1 --2- 172.21.15.105:0/691653802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3c0100ad0 0x7fb3c0197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:19.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.887+0000 7fb3bf7fe640 1 --2- 172.21.15.105:0/691653802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3c0102690 0x7fb3c019f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:19.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.887+0000 7fb3b7fff640 1 -- 172.21.15.105:0/691653802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3c0100ad0 msgr2=0x7fb3c0197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:19.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.887+0000 7fb3b7fff640 1 --2- 172.21.15.105:0/691653802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3c0100ad0 0x7fb3c0197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:19.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.887+0000 7fb3b7fff640 1 -- 172.21.15.105:0/691653802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3c0102690 msgr2=0x7fb3c019f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:19.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.887+0000 7fb3b7fff640 1 --2- 172.21.15.105:0/691653802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3c0102690 0x7fb3c019f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:19.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.887+0000 7fb3b7fff640 1 -- 172.21.15.105:0/691653802 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3c019fc20 con 0x7fb3c0101cd0 2024-09-16T02:02:19.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.887+0000 7fb3bf7fe640 1 --2- 172.21.15.105:0/691653802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3c0102690 0x7fb3c019f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:02:19.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.888+0000 7fb3b7fff640 1 --2- 172.21.15.105:0/691653802 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3c0101cd0 0x7fb3c0198530 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8005940 tx=0x7fb3a8004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:19.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.888+0000 7fb3c4a22640 1 -- 172.21.15.105:0/691653802 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a803f030 con 0x7fb3c0101cd0 2024-09-16T02:02:19.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.888+0000 7fb3c4a22640 1 -- 172.21.15.105:0/691653802 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3a8003cb0 con 0x7fb3c0101cd0 2024-09-16T02:02:19.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.888+0000 7fb3c5a24640 1 -- 172.21.15.105:0/691653802 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3c019feb0 con 0x7fb3c0101cd0 2024-09-16T02:02:19.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.888+0000 7fb3c4a22640 1 -- 172.21.15.105:0/691653802 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8035070 con 0x7fb3c0101cd0 2024-09-16T02:02:19.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.888+0000 7fb3c5a24640 1 -- 172.21.15.105:0/691653802 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3c01a0380 con 0x7fb3c0101cd0 2024-09-16T02:02:19.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.889+0000 7fb3c5a24640 1 -- 172.21.15.105:0/691653802 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3c01a05b0 con 0x7fb3c0101cd0 2024-09-16T02:02:19.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.890+0000 7fb3c4a22640 1 -- 172.21.15.105:0/691653802 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fb3a8042030 con 0x7fb3c0101cd0 2024-09-16T02:02:19.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.891+0000 7fb3c4a22640 1 --2- 172.21.15.105:0/691653802 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb3ac0a3270 0x7fb3ac0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:19.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.891+0000 7fb3beffd640 1 --2- 172.21.15.105:0/691653802 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb3ac0a3270 0x7fb3ac0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:19.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.891+0000 7fb3c4a22640 1 -- 172.21.15.105:0/691653802 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fb3a80f0040 con 0x7fb3c0101cd0 2024-09-16T02:02:19.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.891+0000 7fb3beffd640 1 --2- 172.21.15.105:0/691653802 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb3ac0a3270 0x7fb3ac0a5730 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0101b30 tx=0x7fb3b0009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:19.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:19.895+0000 7fb3c4a22640 1 -- 172.21.15.105:0/691653802 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb3a80bcd40 con 0x7fb3c0101cd0 2024-09-16T02:02:19.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:19 smithi137 ceph-mon[34099]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:02:20.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:20.096+0000 7fb3c5a24640 1 -- 172.21.15.105:0/691653802 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fb3c01a0780 con 0x7fb3c0101cd0 2024-09-16T02:02:20.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:20.097+0000 7fb3c4a22640 1 -- 172.21.15.105:0/691653802 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3140 (secure 0 0 0) 0x7fb3a80b8070 con 0x7fb3c0101cd0 2024-09-16T02:02:20.099 INFO:teuthology.orchestra.run.smithi105.stdout:e27 2024-09-16T02:02:20.099 INFO:teuthology.orchestra.run.smithi105.stdout:btime 2024-09-16T02:00:47:526725+0000 2024-09-16T02:02:20.099 INFO:teuthology.orchestra.run.smithi105.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T02:02:20.099 INFO:teuthology.orchestra.run.smithi105.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-16T02:02:20.099 INFO:teuthology.orchestra.run.smithi105.stdout:legacy client fscid: 1 2024-09-16T02:02:20.099 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:02:20.099 INFO:teuthology.orchestra.run.smithi105.stdout:Filesystem 'cephfs' (1) 2024-09-16T02:02:20.100 INFO:teuthology.orchestra.run.smithi105.stdout:fs_name cephfs 2024-09-16T02:02:20.100 INFO:teuthology.orchestra.run.smithi105.stdout:epoch 27 2024-09-16T02:02:20.100 INFO:teuthology.orchestra.run.smithi105.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-16T02:02:20.100 INFO:teuthology.orchestra.run.smithi105.stdout:created 2024-09-16T02:00:35.112266+0000 2024-09-16T02:02:20.100 INFO:teuthology.orchestra.run.smithi105.stdout:modified 2024-09-16T02:00:47.526704+0000 2024-09-16T02:02:20.100 INFO:teuthology.orchestra.run.smithi105.stdout:tableserver 0 2024-09-16T02:02:20.100 INFO:teuthology.orchestra.run.smithi105.stdout:root 0 2024-09-16T02:02:20.100 INFO:teuthology.orchestra.run.smithi105.stdout:session_timeout 300 2024-09-16T02:02:20.100 INFO:teuthology.orchestra.run.smithi105.stdout:session_autoclose 300 2024-09-16T02:02:20.100 INFO:teuthology.orchestra.run.smithi105.stdout:max_file_size 1099511627776 2024-09-16T02:02:20.100 INFO:teuthology.orchestra.run.smithi105.stdout:max_xattr_size 65536 2024-09-16T02:02:20.101 INFO:teuthology.orchestra.run.smithi105.stdout:required_client_features {} 2024-09-16T02:02:20.101 INFO:teuthology.orchestra.run.smithi105.stdout:last_failure 0 2024-09-16T02:02:20.101 INFO:teuthology.orchestra.run.smithi105.stdout:last_failure_osd_epoch 0 2024-09-16T02:02:20.101 INFO:teuthology.orchestra.run.smithi105.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-16T02:02:20.101 INFO:teuthology.orchestra.run.smithi105.stdout:max_mds 5 2024-09-16T02:02:20.101 INFO:teuthology.orchestra.run.smithi105.stdout:in 0,1,2,3,4 2024-09-16T02:02:20.101 INFO:teuthology.orchestra.run.smithi105.stdout:up {0=24482,1=24469,2=24461,3=24457,4=24445} 2024-09-16T02:02:20.101 INFO:teuthology.orchestra.run.smithi105.stdout:failed 2024-09-16T02:02:20.101 INFO:teuthology.orchestra.run.smithi105.stdout:damaged 2024-09-16T02:02:20.101 INFO:teuthology.orchestra.run.smithi105.stdout:stopped 2024-09-16T02:02:20.101 INFO:teuthology.orchestra.run.smithi105.stdout:data_pools [3] 2024-09-16T02:02:20.101 INFO:teuthology.orchestra.run.smithi105.stdout:metadata_pool 2 2024-09-16T02:02:20.101 INFO:teuthology.orchestra.run.smithi105.stdout:inline_data disabled 2024-09-16T02:02:20.101 INFO:teuthology.orchestra.run.smithi105.stdout:balancer 2024-09-16T02:02:20.101 INFO:teuthology.orchestra.run.smithi105.stdout:bal_rank_mask -1 2024-09-16T02:02:20.101 INFO:teuthology.orchestra.run.smithi105.stdout:standby_count_wanted 0 2024-09-16T02:02:20.102 INFO:teuthology.orchestra.run.smithi105.stdout:qdb_cluster leader: 24482 members: 24482,24469,24461,24457,24445 2024-09-16T02:02:20.102 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.b{0:24482} state up:active seq 4 addr [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T02:02:20.102 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.e{0:24443} state up:standby-replay seq 1 addr [v2:172.21.15.137:6834/763783602,v1:172.21.15.137:6835/763783602] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T02:02:20.102 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.i{1:24469} state up:active seq 9 addr [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T02:02:20.102 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.k{1:24422} state up:standby-replay seq 1 addr [v2:172.21.15.137:6832/3025442862,v1:172.21.15.137:6833/3025442862] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T02:02:20.102 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.h{2:24461} state up:active seq 9 addr [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T02:02:20.102 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.l{2:24421} state up:standby-replay seq 1 addr [v2:172.21.15.190:6832/683519097,v1:172.21.15.190:6833/683519097] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T02:02:20.102 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.c{3:24457} state up:active seq 10 addr [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T02:02:20.102 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.a{3:14625} state up:standby-replay seq 1 addr [v2:172.21.15.105:6840/163156514,v1:172.21.15.105:6841/163156514] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T02:02:20.102 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.f{4:24445} state up:active seq 11 addr [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T02:02:20.102 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.g{4:14613} state up:standby-replay seq 1 addr [v2:172.21.15.105:6838/4112062361,v1:172.21.15.105:6839/4112062361] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T02:02:20.102 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:02:20.102 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:02:20.102 INFO:teuthology.orchestra.run.smithi105.stdout:Standby daemons: 2024-09-16T02:02:20.102 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:02:20.103 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.j{-1:14550} state up:standby seq 1 addr [v2:172.21.15.105:6834/2057699920,v1:172.21.15.105:6835/2057699920] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T02:02:20.103 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.d{-1:14562} state up:standby seq 1 addr [v2:172.21.15.105:6836/1764886948,v1:172.21.15.105:6837/1764886948] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T02:02:20.103 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 27 2024-09-16T02:02:20.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:20.099+0000 7fb3c5a24640 1 -- 172.21.15.105:0/691653802 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb3ac0a3270 msgr2=0x7fb3ac0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:20.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:20.099+0000 7fb3c5a24640 1 --2- 172.21.15.105:0/691653802 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb3ac0a3270 0x7fb3ac0a5730 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0101b30 tx=0x7fb3b0009110 comp rx=0 tx=0).stop 2024-09-16T02:02:20.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:20.099+0000 7fb3c5a24640 1 -- 172.21.15.105:0/691653802 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3c0101cd0 msgr2=0x7fb3c0198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:20.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:20.099+0000 7fb3c5a24640 1 --2- 172.21.15.105:0/691653802 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3c0101cd0 0x7fb3c0198530 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8005940 tx=0x7fb3a8004430 comp rx=0 tx=0).stop 2024-09-16T02:02:20.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:20.099+0000 7fb3c5a24640 1 -- 172.21.15.105:0/691653802 shutdown_connections 2024-09-16T02:02:20.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:20.099+0000 7fb3c5a24640 1 --2- 172.21.15.105:0/691653802 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb3ac0a3270 0x7fb3ac0a5730 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:20.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:20.099+0000 7fb3c5a24640 1 --2- 172.21.15.105:0/691653802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3c0102690 0x7fb3c019f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:20.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:20.099+0000 7fb3c5a24640 1 --2- 172.21.15.105:0/691653802 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3c0101cd0 0x7fb3c0198530 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:20.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:20.099+0000 7fb3c5a24640 1 --2- 172.21.15.105:0/691653802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3c0100ad0 0x7fb3c0197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:20.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:20.099+0000 7fb3c5a24640 1 -- 172.21.15.105:0/691653802 >> 172.21.15.105:0/691653802 conn(0x7fb3c00fc280 msgr2=0x7fb3c006a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:20.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:20.099+0000 7fb3c5a24640 1 -- 172.21.15.105:0/691653802 shutdown_connections 2024-09-16T02:02:20.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:20.099+0000 7fb3c5a24640 1 -- 172.21.15.105:0/691653802 wait complete. 2024-09-16T02:02:20.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:20 smithi105 ceph-mon[31197]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-16T02:02:20.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:20 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/691653802' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T02:02:20.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:20 smithi190 ceph-mon[34762]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-16T02:02:20.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:20 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/691653802' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T02:02:20.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:20 smithi137 ceph-mon[34099]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-16T02:02:20.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:20 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/691653802' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T02:02:20.937 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-16T02:02:21.180 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:02:21.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.658+0000 7f4160f0a640 1 -- 172.21.15.105:0/780436736 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f415c101d10 msgr2=0x7f415c102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:21.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.658+0000 7f4160f0a640 1 --2- 172.21.15.105:0/780436736 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f415c101d10 0x7f415c102190 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f414c009ca0 tx=0x7f414c02a8c0 comp rx=0 tx=0).stop 2024-09-16T02:02:21.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.658+0000 7f4160f0a640 1 -- 172.21.15.105:0/780436736 shutdown_connections 2024-09-16T02:02:21.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.658+0000 7f4160f0a640 1 --2- 172.21.15.105:0/780436736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f415c1026d0 0x7f415c108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:21.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.658+0000 7f4160f0a640 1 --2- 172.21.15.105:0/780436736 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f415c101d10 0x7f415c102190 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:21.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.658+0000 7f4160f0a640 1 --2- 172.21.15.105:0/780436736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f415c100b10 0x7f415c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:21.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.658+0000 7f4160f0a640 1 -- 172.21.15.105:0/780436736 >> 172.21.15.105:0/780436736 conn(0x7f415c0fc2a0 msgr2=0x7f415c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:21.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.658+0000 7f4160f0a640 1 -- 172.21.15.105:0/780436736 shutdown_connections 2024-09-16T02:02:21.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.658+0000 7f4160f0a640 1 -- 172.21.15.105:0/780436736 wait complete. 2024-09-16T02:02:21.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.659+0000 7f4160f0a640 1 Processor -- start 2024-09-16T02:02:21.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.659+0000 7f4160f0a640 1 -- start start 2024-09-16T02:02:21.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.659+0000 7f4160f0a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f415c100b10 0x7f415c197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:21.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.659+0000 7f4160f0a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f415c101d10 0x7f415c1982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:21.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.659+0000 7f4160f0a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f415c1026d0 0x7f415c19f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:21.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.659+0000 7f4160f0a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f415c10c640 con 0x7f415c101d10 2024-09-16T02:02:21.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.659+0000 7f4160f0a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f415c10c4c0 con 0x7f415c1026d0 2024-09-16T02:02:21.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.659+0000 7f4160f0a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f415c10c7c0 con 0x7f415c100b10 2024-09-16T02:02:21.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.659+0000 7f4159d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f415c101d10 0x7f415c1982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:21.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.659+0000 7f4159d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f415c101d10 0x7f415c1982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60500/0 (socket says 172.21.15.105:60500) 2024-09-16T02:02:21.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.659+0000 7f4159d9b640 1 -- 172.21.15.105:0/51626054 learned_addr learned my addr 172.21.15.105:0/51626054 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:21.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.660+0000 7f415ad9d640 1 --2- 172.21.15.105:0/51626054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f415c1026d0 0x7f415c19f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:21.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.660+0000 7f4159d9b640 1 -- 172.21.15.105:0/51626054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f415c100b10 msgr2=0x7f415c197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:21.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.660+0000 7f4159d9b640 1 --2- 172.21.15.105:0/51626054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f415c100b10 0x7f415c197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:21.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.660+0000 7f4159d9b640 1 -- 172.21.15.105:0/51626054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f415c1026d0 msgr2=0x7f415c19f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:21.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.660+0000 7f4159d9b640 1 --2- 172.21.15.105:0/51626054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f415c1026d0 0x7f415c19f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:21.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.660+0000 7f4159d9b640 1 -- 172.21.15.105:0/51626054 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f415c19f9e0 con 0x7f415c101d10 2024-09-16T02:02:21.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.660+0000 7f4159d9b640 1 --2- 172.21.15.105:0/51626054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f415c101d10 0x7f415c1982f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f414c009dc0 tx=0x7f414c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:21.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.660+0000 7f414b7fe640 1 -- 172.21.15.105:0/51626054 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c004040 con 0x7f415c101d10 2024-09-16T02:02:21.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.660+0000 7f414b7fe640 1 -- 172.21.15.105:0/51626054 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f414c004690 con 0x7f415c101d10 2024-09-16T02:02:21.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.660+0000 7f414b7fe640 1 -- 172.21.15.105:0/51626054 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c02f4c0 con 0x7f415c101d10 2024-09-16T02:02:21.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.660+0000 7f4160f0a640 1 -- 172.21.15.105:0/51626054 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f415c19fc70 con 0x7f415c101d10 2024-09-16T02:02:21.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.660+0000 7f4160f0a640 1 -- 172.21.15.105:0/51626054 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f415c1a0160 con 0x7f415c101d10 2024-09-16T02:02:21.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.661+0000 7f4160f0a640 1 -- 172.21.15.105:0/51626054 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f415c1a0330 con 0x7f415c101d10 2024-09-16T02:02:21.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.662+0000 7f414b7fe640 1 -- 172.21.15.105:0/51626054 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f414c0305c0 con 0x7f415c101d10 2024-09-16T02:02:21.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.663+0000 7f414b7fe640 1 --2- 172.21.15.105:0/51626054 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f41440a32f0 0x7f41440a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:21.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.663+0000 7f415a59c640 1 --2- 172.21.15.105:0/51626054 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f41440a32f0 0x7f41440a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:21.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.663+0000 7f414b7fe640 1 -- 172.21.15.105:0/51626054 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f414c0f0eb0 con 0x7f415c101d10 2024-09-16T02:02:21.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.663+0000 7f415a59c640 1 --2- 172.21.15.105:0/51626054 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f41440a32f0 0x7f41440a57b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f415c101b70 tx=0x7f4150005c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:21.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.667+0000 7f414b7fe640 1 -- 172.21.15.105:0/51626054 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f414c0bdbb0 con 0x7f415c101d10 2024-09-16T02:02:21.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.836+0000 7f4160f0a640 1 -- 172.21.15.105:0/51626054 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f415c1a0ae0 con 0x7f415c101d10 2024-09-16T02:02:21.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.837+0000 7f414b7fe640 1 -- 172.21.15.105:0/51626054 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v81) ==== 56+0+5351 (secure 0 0 0) 0x7f414c0bd4b0 con 0x7f415c101d10 2024-09-16T02:02:21.839 INFO:teuthology.orchestra.run.smithi105.stdout:epoch 81 2024-09-16T02:02:21.839 INFO:teuthology.orchestra.run.smithi105.stdout:fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 2024-09-16T02:02:21.839 INFO:teuthology.orchestra.run.smithi105.stdout:created 2024-09-16T01:52:44.388033+0000 2024-09-16T02:02:21.839 INFO:teuthology.orchestra.run.smithi105.stdout:modified 2024-09-16T02:01:40.556339+0000 2024-09-16T02:02:21.839 INFO:teuthology.orchestra.run.smithi105.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-16T02:02:21.839 INFO:teuthology.orchestra.run.smithi105.stdout:crush_version 26 2024-09-16T02:02:21.839 INFO:teuthology.orchestra.run.smithi105.stdout:full_ratio 0.95 2024-09-16T02:02:21.839 INFO:teuthology.orchestra.run.smithi105.stdout:backfillfull_ratio 0.9 2024-09-16T02:02:21.839 INFO:teuthology.orchestra.run.smithi105.stdout:nearfull_ratio 0.85 2024-09-16T02:02:21.839 INFO:teuthology.orchestra.run.smithi105.stdout:require_min_compat_client luminous 2024-09-16T02:02:21.840 INFO:teuthology.orchestra.run.smithi105.stdout:min_compat_client luminous 2024-09-16T02:02:21.840 INFO:teuthology.orchestra.run.smithi105.stdout:require_osd_release squid 2024-09-16T02:02:21.840 INFO:teuthology.orchestra.run.smithi105.stdout:stretch_mode_enabled false 2024-09-16T02:02:21.840 INFO:teuthology.orchestra.run.smithi105.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 20 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-16T02:02:21.840 INFO:teuthology.orchestra.run.smithi105.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-16T02:02:21.840 INFO:teuthology.orchestra.run.smithi105.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-16T02:02:21.840 INFO:teuthology.orchestra.run.smithi105.stdout:max_osd 12 2024-09-16T02:02:21.840 INFO:teuthology.orchestra.run.smithi105.stdout:osd.0 up in weight 1 up_from 8 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.105:6802/1226660290,v1:172.21.15.105:6803/1226660290] [v2:172.21.15.105:6804/1226660290,v1:172.21.15.105:6805/1226660290] exists,up a60d6c72-da8b-4b02-bb33-672be795e7f7 2024-09-16T02:02:21.840 INFO:teuthology.orchestra.run.smithi105.stdout:osd.1 up in weight 1 up_from 13 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6800/34753956,v1:172.21.15.137:6801/34753956] [v2:172.21.15.137:6802/34753956,v1:172.21.15.137:6803/34753956] exists,up b1d9ae84-b9a0-4a7e-b0fe-c402d46809fc 2024-09-16T02:02:21.840 INFO:teuthology.orchestra.run.smithi105.stdout:osd.2 up in weight 1 up_from 17 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.190:6800/1159531918,v1:172.21.15.190:6801/1159531918] [v2:172.21.15.190:6802/1159531918,v1:172.21.15.190:6803/1159531918] exists,up 137faae6-2e8b-44b5-8976-3d95a6eef1aa 2024-09-16T02:02:21.840 INFO:teuthology.orchestra.run.smithi105.stdout:osd.3 up in weight 1 up_from 23 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.105:6810/604905679,v1:172.21.15.105:6811/604905679] [v2:172.21.15.105:6812/604905679,v1:172.21.15.105:6813/604905679] exists,up 5743c37b-5cd8-42b6-81c8-5cf5afc71e9a 2024-09-16T02:02:21.840 INFO:teuthology.orchestra.run.smithi105.stdout:osd.4 up in weight 1 up_from 30 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6808/3758109169,v1:172.21.15.137:6809/3758109169] [v2:172.21.15.137:6810/3758109169,v1:172.21.15.137:6811/3758109169] exists,up e439d4a2-b54c-4de9-97f6-3510afe57f36 2024-09-16T02:02:21.840 INFO:teuthology.orchestra.run.smithi105.stdout:osd.5 up in weight 1 up_from 35 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.190:6808/2086773754,v1:172.21.15.190:6809/2086773754] [v2:172.21.15.190:6810/2086773754,v1:172.21.15.190:6811/2086773754] exists,up d1e2a7d5-99f8-40fa-af6e-0305cc0af677 2024-09-16T02:02:21.841 INFO:teuthology.orchestra.run.smithi105.stdout:osd.6 up in weight 1 up_from 42 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.105:6818/4012018725,v1:172.21.15.105:6819/4012018725] [v2:172.21.15.105:6820/4012018725,v1:172.21.15.105:6821/4012018725] exists,up cd418159-a0f6-44c7-9213-c010b132fb56 2024-09-16T02:02:21.841 INFO:teuthology.orchestra.run.smithi105.stdout:osd.7 up in weight 1 up_from 47 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6816/888678288,v1:172.21.15.137:6817/888678288] [v2:172.21.15.137:6818/888678288,v1:172.21.15.137:6819/888678288] exists,up 76c15863-801e-4a72-be73-ba9df46c15b3 2024-09-16T02:02:21.841 INFO:teuthology.orchestra.run.smithi105.stdout:osd.8 up in weight 1 up_from 54 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.190:6816/2132638341,v1:172.21.15.190:6817/2132638341] [v2:172.21.15.190:6818/2132638341,v1:172.21.15.190:6819/2132638341] exists,up 6718597c-6f42-48ee-9493-edb2396fe707 2024-09-16T02:02:21.841 INFO:teuthology.orchestra.run.smithi105.stdout:osd.9 up in weight 1 up_from 61 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.105:6826/3375034979,v1:172.21.15.105:6827/3375034979] [v2:172.21.15.105:6828/3375034979,v1:172.21.15.105:6829/3375034979] exists,up 5178edc0-a2e8-491c-821c-6f41ec5aeb2c 2024-09-16T02:02:21.841 INFO:teuthology.orchestra.run.smithi105.stdout:osd.10 up in weight 1 up_from 66 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6824/318562234,v1:172.21.15.137:6825/318562234] [v2:172.21.15.137:6826/318562234,v1:172.21.15.137:6827/318562234] exists,up ad145d3e-a6e2-4dd7-bea3-bcd01eb3413e 2024-09-16T02:02:21.841 INFO:teuthology.orchestra.run.smithi105.stdout:osd.11 up in weight 1 up_from 73 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.190:6824/3820460689,v1:172.21.15.190:6825/3820460689] [v2:172.21.15.190:6826/3820460689,v1:172.21.15.190:6827/3820460689] exists,up 6d44bced-3a8b-4763-a93e-0ea34acb4cb4 2024-09-16T02:02:21.841 INFO:teuthology.orchestra.run.smithi105.stdout:pg_upmap_items 2.b [2,4] 2024-09-16T02:02:21.841 INFO:teuthology.orchestra.run.smithi105.stdout:pg_upmap_items 2.e [2,4] 2024-09-16T02:02:21.841 INFO:teuthology.orchestra.run.smithi105.stdout:pg_upmap_items 2.1a [8,1] 2024-09-16T02:02:21.841 INFO:teuthology.orchestra.run.smithi105.stdout:pg_upmap_items 2.1c [8,11] 2024-09-16T02:02:21.841 INFO:teuthology.orchestra.run.smithi105.stdout:pg_upmap_items 3.0 [4,7] 2024-09-16T02:02:21.841 INFO:teuthology.orchestra.run.smithi105.stdout:pg_upmap_items 3.6 [4,6] 2024-09-16T02:02:21.841 INFO:teuthology.orchestra.run.smithi105.stdout:pg_upmap_items 3.c [3,9] 2024-09-16T02:02:21.841 INFO:teuthology.orchestra.run.smithi105.stdout:pg_upmap_items 3.e [5,9] 2024-09-16T02:02:21.842 INFO:teuthology.orchestra.run.smithi105.stdout:pg_upmap_items 3.1b [4,8] 2024-09-16T02:02:21.842 INFO:teuthology.orchestra.run.smithi105.stdout:pg_upmap_items 3.33 [5,9] 2024-09-16T02:02:21.842 INFO:teuthology.orchestra.run.smithi105.stdout:blocklist 172.21.15.105:0/2453739657 expires 2024-09-17T01:53:39.351314+0000 2024-09-16T02:02:21.842 INFO:teuthology.orchestra.run.smithi105.stdout:blocklist 172.21.15.105:6801/1469189484 expires 2024-09-17T01:53:39.351314+0000 2024-09-16T02:02:21.842 INFO:teuthology.orchestra.run.smithi105.stdout:blocklist 172.21.15.105:0/3466166879 expires 2024-09-17T01:53:39.351314+0000 2024-09-16T02:02:21.842 INFO:teuthology.orchestra.run.smithi105.stdout:blocklist 172.21.15.105:6801/1268348742 expires 2024-09-17T01:53:15.522227+0000 2024-09-16T02:02:21.842 INFO:teuthology.orchestra.run.smithi105.stdout:blocklist 172.21.15.105:0/2897248478 expires 2024-09-17T01:53:39.351314+0000 2024-09-16T02:02:21.842 INFO:teuthology.orchestra.run.smithi105.stdout:blocklist 172.21.15.105:0/3709363371 expires 2024-09-17T01:53:15.522227+0000 2024-09-16T02:02:21.842 INFO:teuthology.orchestra.run.smithi105.stdout:blocklist 172.21.15.105:6800/1268348742 expires 2024-09-17T01:53:15.522227+0000 2024-09-16T02:02:21.843 INFO:teuthology.orchestra.run.smithi105.stdout:blocklist 172.21.15.105:6800/1469189484 expires 2024-09-17T01:53:39.351314+0000 2024-09-16T02:02:21.843 INFO:teuthology.orchestra.run.smithi105.stdout:blocklist 172.21.15.105:0/1198176299 expires 2024-09-17T01:53:15.522227+0000 2024-09-16T02:02:21.843 INFO:teuthology.orchestra.run.smithi105.stdout:blocklist 172.21.15.105:0/3031917848 expires 2024-09-17T01:53:15.522227+0000 2024-09-16T02:02:21.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.839+0000 7f4160f0a640 1 -- 172.21.15.105:0/51626054 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f41440a32f0 msgr2=0x7f41440a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:21.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.839+0000 7f4160f0a640 1 --2- 172.21.15.105:0/51626054 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f41440a32f0 0x7f41440a57b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f415c101b70 tx=0x7f4150005c90 comp rx=0 tx=0).stop 2024-09-16T02:02:21.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.839+0000 7f4160f0a640 1 -- 172.21.15.105:0/51626054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f415c101d10 msgr2=0x7f415c1982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:21.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.839+0000 7f4160f0a640 1 --2- 172.21.15.105:0/51626054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f415c101d10 0x7f415c1982f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f414c009dc0 tx=0x7f414c003ba0 comp rx=0 tx=0).stop 2024-09-16T02:02:21.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.839+0000 7f4160f0a640 1 -- 172.21.15.105:0/51626054 shutdown_connections 2024-09-16T02:02:21.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.839+0000 7f4160f0a640 1 --2- 172.21.15.105:0/51626054 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f41440a32f0 0x7f41440a57b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:21.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.839+0000 7f4160f0a640 1 --2- 172.21.15.105:0/51626054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f415c1026d0 0x7f415c19f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:21.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.839+0000 7f4160f0a640 1 --2- 172.21.15.105:0/51626054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f415c101d10 0x7f415c1982f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:21.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.839+0000 7f4160f0a640 1 --2- 172.21.15.105:0/51626054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f415c100b10 0x7f415c197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:21.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.839+0000 7f4160f0a640 1 -- 172.21.15.105:0/51626054 >> 172.21.15.105:0/51626054 conn(0x7f415c0fc2a0 msgr2=0x7f415c06a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:21.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.839+0000 7f4160f0a640 1 -- 172.21.15.105:0/51626054 shutdown_connections 2024-09-16T02:02:21.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:21.839+0000 7f4160f0a640 1 -- 172.21.15.105:0/51626054 wait complete. 2024-09-16T02:02:22.657 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-16T02:02:22.668 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:02:22.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:22 smithi190 ceph-mon[34762]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-16T02:02:22.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:22 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/51626054' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-16T02:02:22.915 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:02:22.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:22 smithi137 ceph-mon[34099]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-16T02:02:22.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:22 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/51626054' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-16T02:02:23.035 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:22 smithi105 ceph-mon[31197]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-16T02:02:23.035 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:22 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/51626054' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-16T02:02:23.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.357+0000 7fe8ff4d4640 1 -- 172.21.15.105:0/3960817650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8f8102570 msgr2=0x7fe8f81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:23.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.357+0000 7fe8ff4d4640 1 --2- 172.21.15.105:0/3960817650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8f8102570 0x7fe8f81029b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8009900 tx=0x7fe8e802a480 comp rx=0 tx=0).stop 2024-09-16T02:02:23.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.357+0000 7fe8ff4d4640 1 -- 172.21.15.105:0/3960817650 shutdown_connections 2024-09-16T02:02:23.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.357+0000 7fe8ff4d4640 1 --2- 172.21.15.105:0/3960817650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8f8102ef0 0x7fe8f810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:23.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.357+0000 7fe8ff4d4640 1 --2- 172.21.15.105:0/3960817650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8f8102570 0x7fe8f81029b0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:23.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.357+0000 7fe8ff4d4640 1 --2- 172.21.15.105:0/3960817650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8f8100e20 0x7fe8f8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:23.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.357+0000 7fe8ff4d4640 1 -- 172.21.15.105:0/3960817650 >> 172.21.15.105:0/3960817650 conn(0x7fe8f80fc7d0 msgr2=0x7fe8f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:23.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.358+0000 7fe8ff4d4640 1 -- 172.21.15.105:0/3960817650 shutdown_connections 2024-09-16T02:02:23.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.358+0000 7fe8ff4d4640 1 -- 172.21.15.105:0/3960817650 wait complete. 2024-09-16T02:02:23.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.358+0000 7fe8ff4d4640 1 Processor -- start 2024-09-16T02:02:23.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.358+0000 7fe8ff4d4640 1 -- start start 2024-09-16T02:02:23.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.358+0000 7fe8ff4d4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8f8100e20 0x7fe8f8198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:23.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.358+0000 7fe8ff4d4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8f8102570 0x7fe8f8198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:23.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.358+0000 7fe8ff4d4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8f8102ef0 0x7fe8f819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:23.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.358+0000 7fe8ff4d4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f8110ed0 con 0x7fe8f8102570 2024-09-16T02:02:23.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.358+0000 7fe8ff4d4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f8110d50 con 0x7fe8f8100e20 2024-09-16T02:02:23.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.358+0000 7fe8ff4d4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f8111050 con 0x7fe8f8102ef0 2024-09-16T02:02:23.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.358+0000 7fe8fca6f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8f8102570 0x7fe8f8198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:23.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.358+0000 7fe8fca6f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8f8102570 0x7fe8f8198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60524/0 (socket says 172.21.15.105:60524) 2024-09-16T02:02:23.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.359+0000 7fe8fca6f640 1 -- 172.21.15.105:0/1109734695 learned_addr learned my addr 172.21.15.105:0/1109734695 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:23.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.359+0000 7fe8fda71640 1 --2- 172.21.15.105:0/1109734695 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8f8102ef0 0x7fe8f819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:23.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.359+0000 7fe8fd270640 1 --2- 172.21.15.105:0/1109734695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8f8100e20 0x7fe8f8198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:23.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.359+0000 7fe8fca6f640 1 -- 172.21.15.105:0/1109734695 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8f8102ef0 msgr2=0x7fe8f819cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:23.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.359+0000 7fe8fca6f640 1 --2- 172.21.15.105:0/1109734695 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8f8102ef0 0x7fe8f819cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:23.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.359+0000 7fe8fca6f640 1 -- 172.21.15.105:0/1109734695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8f8100e20 msgr2=0x7fe8f8198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:23.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.359+0000 7fe8fca6f640 1 --2- 172.21.15.105:0/1109734695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8f8100e20 0x7fe8f8198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:23.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.359+0000 7fe8fca6f640 1 -- 172.21.15.105:0/1109734695 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f819d4b0 con 0x7fe8f8102570 2024-09-16T02:02:23.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.359+0000 7fe8fd270640 1 --2- 172.21.15.105:0/1109734695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8f8100e20 0x7fe8f8198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:02:23.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.359+0000 7fe8fda71640 1 --2- 172.21.15.105:0/1109734695 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8f8102ef0 0x7fe8f819cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:02:23.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.359+0000 7fe8fca6f640 1 --2- 172.21.15.105:0/1109734695 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8f8102570 0x7fe8f8198a40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8006010 tx=0x7fe8e80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:23.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.359+0000 7fe8ee7fc640 1 -- 172.21.15.105:0/1109734695 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e803f070 con 0x7fe8f8102570 2024-09-16T02:02:23.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.359+0000 7fe8ff4d4640 1 -- 172.21.15.105:0/1109734695 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8f819d740 con 0x7fe8f8102570 2024-09-16T02:02:23.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.359+0000 7fe8ee7fc640 1 -- 172.21.15.105:0/1109734695 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8e8003aa0 con 0x7fe8f8102570 2024-09-16T02:02:23.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.359+0000 7fe8ee7fc640 1 -- 172.21.15.105:0/1109734695 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8035030 con 0x7fe8f8102570 2024-09-16T02:02:23.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.359+0000 7fe8ff4d4640 1 -- 172.21.15.105:0/1109734695 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8f81a5080 con 0x7fe8f8102570 2024-09-16T02:02:23.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.360+0000 7fe8ff4d4640 1 -- 172.21.15.105:0/1109734695 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8f8101200 con 0x7fe8f8102570 2024-09-16T02:02:23.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.361+0000 7fe8ee7fc640 1 -- 172.21.15.105:0/1109734695 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fe8e8042070 con 0x7fe8f8102570 2024-09-16T02:02:23.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.362+0000 7fe8ee7fc640 1 --2- 172.21.15.105:0/1109734695 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe8e40a3240 0x7fe8e40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:23.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.362+0000 7fe8fd270640 1 --2- 172.21.15.105:0/1109734695 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe8e40a3240 0x7fe8e40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:23.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.362+0000 7fe8ee7fc640 1 -- 172.21.15.105:0/1109734695 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fe8e80f0980 con 0x7fe8f8102570 2024-09-16T02:02:23.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.363+0000 7fe8fd270640 1 --2- 172.21.15.105:0/1109734695 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe8e40a3240 0x7fe8e40a5700 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4006fd0 tx=0x7fe8f4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:23.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.365+0000 7fe8ee7fc640 1 -- 172.21.15.105:0/1109734695 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe8e80bd680 con 0x7fe8f8102570 2024-09-16T02:02:23.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.565+0000 7fe8ff4d4640 1 -- 172.21.15.105:0/1109734695 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8f8199860 con 0x7fe8f8102570 2024-09-16T02:02:23.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.566+0000 7fe8ee7fc640 1 -- 172.21.15.105:0/1109734695 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11991 (secure 0 0 0) 0x7fe8f8199860 con 0x7fe8f8102570 2024-09-16T02:02:23.568 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:02:23.569 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":27,"btime":"2024-09-16T02:00:47:526725+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2057699920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2057699920},{"type":"v1","addr":"172.21.15.105:6835","nonce":2057699920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1764886948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1764886948},{"type":"v1","addr":"172.21.15.105:6837","nonce":1764886948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T02:00:35.112266+0000","modified":"2024-09-16T02:00:47.526704+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24469,"mds_2":24461,"mds_3":24457,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/4112062361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":4112062361},{"type":"v1","addr":"172.21.15.105:6839","nonce":4112062361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6841/163156514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":163156514},{"type":"v1","addr":"172.21.15.105:6841","nonce":163156514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/683519097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":683519097},{"type":"v1","addr":"172.21.15.190:6833","nonce":683519097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3025442862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3025442862},{"type":"v1","addr":"172.21.15.137:6833","nonce":3025442862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/763783602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":763783602},{"type":"v1","addr":"172.21.15.137:6835","nonce":763783602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.190:6835/1851042281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1851042281},{"type":"v1","addr":"172.21.15.190:6835","nonce":1851042281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/3181257549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3181257549},{"type":"v1","addr":"172.21.15.190:6837","nonce":3181257549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6837/2788541030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2788541030},{"type":"v1","addr":"172.21.15.137:6837","nonce":2788541030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.190:6839/1222573537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":1222573537},{"type":"v1","addr":"172.21.15.190:6839","nonce":1222573537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/3454924009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3454924009},{"type":"v1","addr":"172.21.15.137:6839","nonce":3454924009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24469,24461,24457,24445]},"id":1}]} 2024-09-16T02:02:23.569 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 27 2024-09-16T02:02:23.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.568+0000 7fe8ff4d4640 1 -- 172.21.15.105:0/1109734695 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe8e40a3240 msgr2=0x7fe8e40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:23.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.568+0000 7fe8ff4d4640 1 --2- 172.21.15.105:0/1109734695 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe8e40a3240 0x7fe8e40a5700 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4006fd0 tx=0x7fe8f4008040 comp rx=0 tx=0).stop 2024-09-16T02:02:23.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.568+0000 7fe8ff4d4640 1 -- 172.21.15.105:0/1109734695 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8f8102570 msgr2=0x7fe8f8198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:23.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.568+0000 7fe8ff4d4640 1 --2- 172.21.15.105:0/1109734695 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8f8102570 0x7fe8f8198a40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8006010 tx=0x7fe8e80042f0 comp rx=0 tx=0).stop 2024-09-16T02:02:23.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.568+0000 7fe8ff4d4640 1 -- 172.21.15.105:0/1109734695 shutdown_connections 2024-09-16T02:02:23.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.568+0000 7fe8ff4d4640 1 --2- 172.21.15.105:0/1109734695 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe8e40a3240 0x7fe8e40a5700 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:23.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.568+0000 7fe8ff4d4640 1 --2- 172.21.15.105:0/1109734695 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8f8102ef0 0x7fe8f819cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:23.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.568+0000 7fe8ff4d4640 1 --2- 172.21.15.105:0/1109734695 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8f8102570 0x7fe8f8198a40 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:23.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.568+0000 7fe8ff4d4640 1 --2- 172.21.15.105:0/1109734695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8f8100e20 0x7fe8f8198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:23.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.568+0000 7fe8ff4d4640 1 -- 172.21.15.105:0/1109734695 >> 172.21.15.105:0/1109734695 conn(0x7fe8f80fc7d0 msgr2=0x7fe8f806a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:23.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.568+0000 7fe8ff4d4640 1 -- 172.21.15.105:0/1109734695 shutdown_connections 2024-09-16T02:02:23.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:23.568+0000 7fe8ff4d4640 1 -- 172.21.15.105:0/1109734695 wait complete. 2024-09-16T02:02:23.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:23 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/1109734695' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:02:23.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:23 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/1109734695' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:02:23.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:23 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/1109734695' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:02:24.392 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:02:24.641 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:02:24.761 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:24 smithi105 ceph-mon[31197]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-16T02:02:24.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:24 smithi190 ceph-mon[34762]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-16T02:02:24.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:24 smithi137 ceph-mon[34099]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-16T02:02:25.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.118+0000 7f88e5559640 1 -- 172.21.15.105:0/3720053486 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88e0100a60 msgr2=0x7f88e0100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:25.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.118+0000 7f88e5559640 1 --2- 172.21.15.105:0/3720053486 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88e0100a60 0x7f88e0100ec0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f88d4009930 tx=0x7f88d402a420 comp rx=0 tx=0).stop 2024-09-16T02:02:25.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.118+0000 7f88e5559640 1 -- 172.21.15.105:0/3720053486 shutdown_connections 2024-09-16T02:02:25.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.118+0000 7f88e5559640 1 --2- 172.21.15.105:0/3720053486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88e0101400 0x7f88e010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:25.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.118+0000 7f88e5559640 1 --2- 172.21.15.105:0/3720053486 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88e0100a60 0x7f88e0100ec0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:25.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.118+0000 7f88e5559640 1 --2- 172.21.15.105:0/3720053486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88e0106a60 0x7f88e0106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:25.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.118+0000 7f88e5559640 1 -- 172.21.15.105:0/3720053486 >> 172.21.15.105:0/3720053486 conn(0x7f88e00fc780 msgr2=0x7f88e00feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:25.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.118+0000 7f88e5559640 1 -- 172.21.15.105:0/3720053486 shutdown_connections 2024-09-16T02:02:25.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.118+0000 7f88e5559640 1 -- 172.21.15.105:0/3720053486 wait complete. 2024-09-16T02:02:25.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.118+0000 7f88e5559640 1 Processor -- start 2024-09-16T02:02:25.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.119+0000 7f88e5559640 1 -- start start 2024-09-16T02:02:25.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.119+0000 7f88e5559640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88e0100a60 0x7f88e0198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:25.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.119+0000 7f88e5559640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88e0101400 0x7f88e0198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:25.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.119+0000 7f88e5559640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88e0106a60 0x7f88e019cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:25.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.119+0000 7f88e5559640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f88e0110d30 con 0x7f88e0100a60 2024-09-16T02:02:25.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.119+0000 7f88e5559640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f88e0110bb0 con 0x7f88e0106a60 2024-09-16T02:02:25.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.119+0000 7f88e5559640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f88e0110eb0 con 0x7f88e0101400 2024-09-16T02:02:25.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.119+0000 7f88deffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88e0100a60 0x7f88e0198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:25.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.119+0000 7f88deffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88e0100a60 0x7f88e0198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60552/0 (socket says 172.21.15.105:60552) 2024-09-16T02:02:25.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.119+0000 7f88deffd640 1 -- 172.21.15.105:0/3446648048 learned_addr learned my addr 172.21.15.105:0/3446648048 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:25.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.119+0000 7f88df7fe640 1 --2- 172.21.15.105:0/3446648048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88e0106a60 0x7f88e019cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:25.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.119+0000 7f88de7fc640 1 --2- 172.21.15.105:0/3446648048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88e0101400 0x7f88e0198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:25.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.119+0000 7f88deffd640 1 -- 172.21.15.105:0/3446648048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88e0101400 msgr2=0x7f88e0198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:25.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.119+0000 7f88deffd640 1 --2- 172.21.15.105:0/3446648048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88e0101400 0x7f88e0198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:25.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.119+0000 7f88deffd640 1 -- 172.21.15.105:0/3446648048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88e0106a60 msgr2=0x7f88e019cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:25.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.119+0000 7f88deffd640 1 --2- 172.21.15.105:0/3446648048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88e0106a60 0x7f88e019cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:25.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.119+0000 7f88deffd640 1 -- 172.21.15.105:0/3446648048 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e019d3f0 con 0x7f88e0100a60 2024-09-16T02:02:25.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.119+0000 7f88deffd640 1 --2- 172.21.15.105:0/3446648048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88e0100a60 0x7f88e0198440 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f88d000b4a0 tx=0x7f88d000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:25.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.119+0000 7f88c7fff640 1 -- 172.21.15.105:0/3446648048 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d0014070 con 0x7f88e0100a60 2024-09-16T02:02:25.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.119+0000 7f88c7fff640 1 -- 172.21.15.105:0/3446648048 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88d0010ac0 con 0x7f88e0100a60 2024-09-16T02:02:25.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.120+0000 7f88e5559640 1 -- 172.21.15.105:0/3446648048 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88e019d6e0 con 0x7f88e0100a60 2024-09-16T02:02:25.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.120+0000 7f88c7fff640 1 -- 172.21.15.105:0/3446648048 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d0017c30 con 0x7f88e0100a60 2024-09-16T02:02:25.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.120+0000 7f88e5559640 1 -- 172.21.15.105:0/3446648048 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88e01a4fc0 con 0x7f88e0100a60 2024-09-16T02:02:25.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.120+0000 7f88e5559640 1 -- 172.21.15.105:0/3446648048 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88e0100ec0 con 0x7f88e0100a60 2024-09-16T02:02:25.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.122+0000 7f88c7fff640 1 -- 172.21.15.105:0/3446648048 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f88d0017070 con 0x7f88e0100a60 2024-09-16T02:02:25.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.122+0000 7f88c7fff640 1 --2- 172.21.15.105:0/3446648048 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f88c80a32c0 0x7f88c80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:25.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.122+0000 7f88de7fc640 1 --2- 172.21.15.105:0/3446648048 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f88c80a32c0 0x7f88c80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:25.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.122+0000 7f88c7fff640 1 -- 172.21.15.105:0/3446648048 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f88d00cf010 con 0x7f88e0100a60 2024-09-16T02:02:25.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.123+0000 7f88de7fc640 1 --2- 172.21.15.105:0/3446648048 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f88c80a32c0 0x7f88c80a5780 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f88d4000c00 tx=0x7f88d4032c80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:25.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.125+0000 7f88c7fff640 1 -- 172.21.15.105:0/3446648048 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f88d009bdb0 con 0x7f88e0100a60 2024-09-16T02:02:25.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.326+0000 7f88e5559640 1 -- 172.21.15.105:0/3446648048 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88e010a060 con 0x7f88e0100a60 2024-09-16T02:02:25.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.327+0000 7f88c7fff640 1 -- 172.21.15.105:0/3446648048 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11991 (secure 0 0 0) 0x7f88d009b6b0 con 0x7f88e0100a60 2024-09-16T02:02:25.329 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:02:25.329 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":27,"btime":"2024-09-16T02:00:47:526725+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2057699920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2057699920},{"type":"v1","addr":"172.21.15.105:6835","nonce":2057699920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1764886948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1764886948},{"type":"v1","addr":"172.21.15.105:6837","nonce":1764886948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T02:00:35.112266+0000","modified":"2024-09-16T02:00:47.526704+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24469,"mds_2":24461,"mds_3":24457,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/4112062361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":4112062361},{"type":"v1","addr":"172.21.15.105:6839","nonce":4112062361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6841/163156514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":163156514},{"type":"v1","addr":"172.21.15.105:6841","nonce":163156514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/683519097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":683519097},{"type":"v1","addr":"172.21.15.190:6833","nonce":683519097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3025442862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3025442862},{"type":"v1","addr":"172.21.15.137:6833","nonce":3025442862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/763783602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":763783602},{"type":"v1","addr":"172.21.15.137:6835","nonce":763783602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.190:6835/1851042281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1851042281},{"type":"v1","addr":"172.21.15.190:6835","nonce":1851042281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/3181257549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3181257549},{"type":"v1","addr":"172.21.15.190:6837","nonce":3181257549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6837/2788541030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2788541030},{"type":"v1","addr":"172.21.15.137:6837","nonce":2788541030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.190:6839/1222573537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":1222573537},{"type":"v1","addr":"172.21.15.190:6839","nonce":1222573537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/3454924009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3454924009},{"type":"v1","addr":"172.21.15.137:6839","nonce":3454924009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24469,24461,24457,24445]},"id":1}]} 2024-09-16T02:02:25.329 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 27 2024-09-16T02:02:25.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.329+0000 7f88e5559640 1 -- 172.21.15.105:0/3446648048 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f88c80a32c0 msgr2=0x7f88c80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:25.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.329+0000 7f88e5559640 1 --2- 172.21.15.105:0/3446648048 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f88c80a32c0 0x7f88c80a5780 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f88d4000c00 tx=0x7f88d4032c80 comp rx=0 tx=0).stop 2024-09-16T02:02:25.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.329+0000 7f88e5559640 1 -- 172.21.15.105:0/3446648048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88e0100a60 msgr2=0x7f88e0198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:25.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.329+0000 7f88e5559640 1 --2- 172.21.15.105:0/3446648048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88e0100a60 0x7f88e0198440 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f88d000b4a0 tx=0x7f88d000b960 comp rx=0 tx=0).stop 2024-09-16T02:02:25.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.329+0000 7f88e5559640 1 -- 172.21.15.105:0/3446648048 shutdown_connections 2024-09-16T02:02:25.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.329+0000 7f88e5559640 1 --2- 172.21.15.105:0/3446648048 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f88c80a32c0 0x7f88c80a5780 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:25.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.329+0000 7f88e5559640 1 --2- 172.21.15.105:0/3446648048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88e0101400 0x7f88e0198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:25.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.329+0000 7f88e5559640 1 --2- 172.21.15.105:0/3446648048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88e0106a60 0x7f88e019cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:25.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.329+0000 7f88e5559640 1 --2- 172.21.15.105:0/3446648048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88e0100a60 0x7f88e0198440 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:25.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.329+0000 7f88e5559640 1 -- 172.21.15.105:0/3446648048 >> 172.21.15.105:0/3446648048 conn(0x7f88e00fc780 msgr2=0x7f88e006ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:25.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.329+0000 7f88e5559640 1 -- 172.21.15.105:0/3446648048 shutdown_connections 2024-09-16T02:02:25.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:25.329+0000 7f88e5559640 1 -- 172.21.15.105:0/3446648048 wait complete. 2024-09-16T02:02:25.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:25 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/3446648048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:02:25.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:25 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/3446648048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:02:26.033 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T02:02:26.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:25 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/3446648048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:02:26.277 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:02:26.683 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:26 smithi105 ceph-mon[31197]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-16T02:02:26.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.761+0000 7f07bab66640 1 -- 172.21.15.105:0/149073389 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07b41036e0 msgr2=0x7f07b4107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:26.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.761+0000 7f07bab66640 1 --2- 172.21.15.105:0/149073389 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07b41036e0 0x7f07b4107b90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f07a8009930 tx=0x7f07a802a420 comp rx=0 tx=0).stop 2024-09-16T02:02:26.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.761+0000 7f07bab66640 1 -- 172.21.15.105:0/149073389 shutdown_connections 2024-09-16T02:02:26.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.761+0000 7f07bab66640 1 --2- 172.21.15.105:0/149073389 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07b41082a0 0x7f07b410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:26.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.761+0000 7f07bab66640 1 --2- 172.21.15.105:0/149073389 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07b41036e0 0x7f07b4107b90 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:26.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.761+0000 7f07bab66640 1 --2- 172.21.15.105:0/149073389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07b4102d30 0x7f07b4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:26.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.761+0000 7f07bab66640 1 -- 172.21.15.105:0/149073389 >> 172.21.15.105:0/149073389 conn(0x7f07b40fc7d0 msgr2=0x7f07b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:26.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.761+0000 7f07bab66640 1 -- 172.21.15.105:0/149073389 shutdown_connections 2024-09-16T02:02:26.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.761+0000 7f07bab66640 1 -- 172.21.15.105:0/149073389 wait complete. 2024-09-16T02:02:26.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.761+0000 7f07bab66640 1 Processor -- start 2024-09-16T02:02:26.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.761+0000 7f07bab66640 1 -- start start 2024-09-16T02:02:26.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.762+0000 7f07bab66640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07b4102d30 0x7f07b41984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:26.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.762+0000 7f07bab66640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07b41036e0 0x7f07b4198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:26.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.762+0000 7f07bab66640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07b41082a0 0x7f07b419cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:26.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.762+0000 7f07bab66640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f07b4111f10 con 0x7f07b4102d30 2024-09-16T02:02:26.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.762+0000 7f07bab66640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f07b4111d90 con 0x7f07b41082a0 2024-09-16T02:02:26.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.762+0000 7f07bab66640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f07b4112090 con 0x7f07b41036e0 2024-09-16T02:02:26.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.762+0000 7f07b8902640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07b4102d30 0x7f07b41984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:26.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.762+0000 7f07b8902640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07b4102d30 0x7f07b41984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60564/0 (socket says 172.21.15.105:60564) 2024-09-16T02:02:26.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.762+0000 7f07b8902640 1 -- 172.21.15.105:0/3581205057 learned_addr learned my addr 172.21.15.105:0/3581205057 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:26.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.762+0000 7f07b9103640 1 --2- 172.21.15.105:0/3581205057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07b41082a0 0x7f07b419cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:26.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.762+0000 7f07b3fff640 1 --2- 172.21.15.105:0/3581205057 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07b41036e0 0x7f07b4198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:26.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.762+0000 7f07b8902640 1 -- 172.21.15.105:0/3581205057 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07b41036e0 msgr2=0x7f07b4198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:26.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.762+0000 7f07b8902640 1 --2- 172.21.15.105:0/3581205057 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07b41036e0 0x7f07b4198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:26.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.762+0000 7f07b8902640 1 -- 172.21.15.105:0/3581205057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07b41082a0 msgr2=0x7f07b419cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:26.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.762+0000 7f07b8902640 1 --2- 172.21.15.105:0/3581205057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07b41082a0 0x7f07b419cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:26.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.762+0000 7f07b8902640 1 -- 172.21.15.105:0/3581205057 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b419d4a0 con 0x7f07b4102d30 2024-09-16T02:02:26.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.762+0000 7f07b8902640 1 --2- 172.21.15.105:0/3581205057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07b4102d30 0x7f07b41984f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f07a400c920 tx=0x7f07a400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:26.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.762+0000 7f07b1ffb640 1 -- 172.21.15.105:0/3581205057 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a4007c70 con 0x7f07b4102d30 2024-09-16T02:02:26.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.762+0000 7f07bab66640 1 -- 172.21.15.105:0/3581205057 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07b419d790 con 0x7f07b4102d30 2024-09-16T02:02:26.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.762+0000 7f07b1ffb640 1 -- 172.21.15.105:0/3581205057 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07a4007e10 con 0x7f07b4102d30 2024-09-16T02:02:26.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.762+0000 7f07b1ffb640 1 -- 172.21.15.105:0/3581205057 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a4018820 con 0x7f07b4102d30 2024-09-16T02:02:26.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.763+0000 7f07bab66640 1 -- 172.21.15.105:0/3581205057 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07b41a5070 con 0x7f07b4102d30 2024-09-16T02:02:26.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.764+0000 7f07bab66640 1 -- 172.21.15.105:0/3581205057 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07b4103110 con 0x7f07b4102d30 2024-09-16T02:02:26.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.765+0000 7f07b1ffb640 1 -- 172.21.15.105:0/3581205057 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f07a40189c0 con 0x7f07b4102d30 2024-09-16T02:02:26.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.765+0000 7f07b1ffb640 1 --2- 172.21.15.105:0/3581205057 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f079c0a3240 0x7f079c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:26.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.765+0000 7f07b3fff640 1 --2- 172.21.15.105:0/3581205057 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f079c0a3240 0x7f079c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:26.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.766+0000 7f07b1ffb640 1 -- 172.21.15.105:0/3581205057 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f07a40cfac0 con 0x7f07b4102d30 2024-09-16T02:02:26.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.766+0000 7f07b3fff640 1 --2- 172.21.15.105:0/3581205057 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f079c0a3240 0x7f079c0a5700 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f07a8009900 tx=0x7f07a8031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:26.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.769+0000 7f07b1ffb640 1 -- 172.21.15.105:0/3581205057 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f07a4014030 con 0x7f07b4102d30 2024-09-16T02:02:26.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:26 smithi137 ceph-mon[34099]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-16T02:02:26.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.937+0000 7f07bab66640 1 -- 172.21.15.105:0/3581205057 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f07b410a2b0 con 0x7f07b4102d30 2024-09-16T02:02:26.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.939+0000 7f07b1ffb640 1 -- 172.21.15.105:0/3581205057 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22850 (secure 0 0 0) 0x7f07a409c760 con 0x7f07b4102d30 2024-09-16T02:02:26.941 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:02:26.941 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":81,"fsid":"423b7a2a-73ce-11ef-bceb-c7b262605968","created":"2024-09-16T01:52:44.388033+0000","modified":"2024-09-16T02:01:40.556339+0000","last_up_change":"2024-09-16T01:59:42.218034+0000","last_in_change":"2024-09-16T01:59:18.785252+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T01:55:53.680808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T02:00:31.286117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T02:00:33.310045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a60d6c72-da8b-4b02-bb33-672be795e7f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6803","nonce":1226660290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6805","nonce":1226660290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6809","nonce":1226660290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6807","nonce":1226660290}]},"public_addr":"172.21.15.105:6803/1226660290","cluster_addr":"172.21.15.105:6805/1226660290","heartbeat_back_addr":"172.21.15.105:6809/1226660290","heartbeat_front_addr":"172.21.15.105:6807/1226660290","state":["exists","up"]},{"osd":1,"uuid":"b1d9ae84-b9a0-4a7e-b0fe-c402d46809fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6801","nonce":34753956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6803","nonce":34753956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6807","nonce":34753956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6805","nonce":34753956}]},"public_addr":"172.21.15.137:6801/34753956","cluster_addr":"172.21.15.137:6803/34753956","heartbeat_back_addr":"172.21.15.137:6807/34753956","heartbeat_front_addr":"172.21.15.137:6805/34753956","state":["exists","up"]},{"osd":2,"uuid":"137faae6-2e8b-44b5-8976-3d95a6eef1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6801","nonce":1159531918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6803","nonce":1159531918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6807","nonce":1159531918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6805","nonce":1159531918}]},"public_addr":"172.21.15.190:6801/1159531918","cluster_addr":"172.21.15.190:6803/1159531918","heartbeat_back_addr":"172.21.15.190:6807/1159531918","heartbeat_front_addr":"172.21.15.190:6805/1159531918","state":["exists","up"]},{"osd":3,"uuid":"5743c37b-5cd8-42b6-81c8-5cf5afc71e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6811","nonce":604905679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6813","nonce":604905679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6817","nonce":604905679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6815","nonce":604905679}]},"public_addr":"172.21.15.105:6811/604905679","cluster_addr":"172.21.15.105:6813/604905679","heartbeat_back_addr":"172.21.15.105:6817/604905679","heartbeat_front_addr":"172.21.15.105:6815/604905679","state":["exists","up"]},{"osd":4,"uuid":"e439d4a2-b54c-4de9-97f6-3510afe57f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6809","nonce":3758109169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6811","nonce":3758109169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6815","nonce":3758109169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6813","nonce":3758109169}]},"public_addr":"172.21.15.137:6809/3758109169","cluster_addr":"172.21.15.137:6811/3758109169","heartbeat_back_addr":"172.21.15.137:6815/3758109169","heartbeat_front_addr":"172.21.15.137:6813/3758109169","state":["exists","up"]},{"osd":5,"uuid":"d1e2a7d5-99f8-40fa-af6e-0305cc0af677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6809","nonce":2086773754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6811","nonce":2086773754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6815","nonce":2086773754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6813","nonce":2086773754}]},"public_addr":"172.21.15.190:6809/2086773754","cluster_addr":"172.21.15.190:6811/2086773754","heartbeat_back_addr":"172.21.15.190:6815/2086773754","heartbeat_front_addr":"172.21.15.190:6813/2086773754","state":["exists","up"]},{"osd":6,"uuid":"cd418159-a0f6-44c7-9213-c010b132fb56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6819","nonce":4012018725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6821","nonce":4012018725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6825","nonce":4012018725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6823","nonce":4012018725}]},"public_addr":"172.21.15.105:6819/4012018725","cluster_addr":"172.21.15.105:6821/4012018725","heartbeat_back_addr":"172.21.15.105:6825/4012018725","heartbeat_front_addr":"172.21.15.105:6823/4012018725","state":["exists","up"]},{"osd":7,"uuid":"76c15863-801e-4a72-be73-ba9df46c15b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6817","nonce":888678288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6819","nonce":888678288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6823","nonce":888678288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6821","nonce":888678288}]},"public_addr":"172.21.15.137:6817/888678288","cluster_addr":"172.21.15.137:6819/888678288","heartbeat_back_addr":"172.21.15.137:6823/888678288","heartbeat_front_addr":"172.21.15.137:6821/888678288","state":["exists","up"]},{"osd":8,"uuid":"6718597c-6f42-48ee-9493-edb2396fe707","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6817","nonce":2132638341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6819","nonce":2132638341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6823","nonce":2132638341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6821","nonce":2132638341}]},"public_addr":"172.21.15.190:6817/2132638341","cluster_addr":"172.21.15.190:6819/2132638341","heartbeat_back_addr":"172.21.15.190:6823/2132638341","heartbeat_front_addr":"172.21.15.190:6821/2132638341","state":["exists","up"]},{"osd":9,"uuid":"5178edc0-a2e8-491c-821c-6f41ec5aeb2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6827","nonce":3375034979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6829","nonce":3375034979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6833","nonce":3375034979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6831","nonce":3375034979}]},"public_addr":"172.21.15.105:6827/3375034979","cluster_addr":"172.21.15.105:6829/3375034979","heartbeat_back_addr":"172.21.15.105:6833/3375034979","heartbeat_front_addr":"172.21.15.105:6831/3375034979","state":["exists","up"]},{"osd":10,"uuid":"ad145d3e-a6e2-4dd7-bea3-bcd01eb3413e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6825","nonce":318562234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6827","nonce":318562234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6831","nonce":318562234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6829","nonce":318562234}]},"public_addr":"172.21.15.137:6825/318562234","cluster_addr":"172.21.15.137:6827/318562234","heartbeat_back_addr":"172.21.15.137:6831/318562234","heartbeat_front_addr":"172.21.15.137:6829/318562234","state":["exists","up"]},{"osd":11,"uuid":"6d44bced-3a8b-4763-a93e-0ea34acb4cb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6825","nonce":3820460689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6827","nonce":3820460689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6831","nonce":3820460689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6829","nonce":3820460689}]},"public_addr":"172.21.15.190:6825/3820460689","cluster_addr":"172.21.15.190:6827/3820460689","heartbeat_back_addr":"172.21.15.190:6831/3820460689","heartbeat_front_addr":"172.21.15.190:6829/3820460689","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:55:02.759033+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:55:26.829142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:55:51.120200+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:56:17.277870+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:56:42.313701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:07.013416+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:32.477916+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:58.643764+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:23.454908+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:48.869082+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:14.921573+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:40.908993+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":7}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:0/2453739657":"2024-09-17T01:53:39.351314+0000","172.21.15.105:6801/1469189484":"2024-09-17T01:53:39.351314+0000","172.21.15.105:0/3466166879":"2024-09-17T01:53:39.351314+0000","172.21.15.105:6801/1268348742":"2024-09-17T01:53:15.522227+0000","172.21.15.105:0/2897248478":"2024-09-17T01:53:39.351314+0000","172.21.15.105:0/3709363371":"2024-09-17T01:53:15.522227+0000","172.21.15.105:6800/1268348742":"2024-09-17T01:53:15.522227+0000","172.21.15.105:6800/1469189484":"2024-09-17T01:53:39.351314+0000","172.21.15.105:0/1198176299":"2024-09-17T01:53:15.522227+0000","172.21.15.105:0/3031917848":"2024-09-17T01:53:15.522227+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T02:02:26.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.941+0000 7f07bab66640 1 -- 172.21.15.105:0/3581205057 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f079c0a3240 msgr2=0x7f079c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:26.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.941+0000 7f07bab66640 1 --2- 172.21.15.105:0/3581205057 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f079c0a3240 0x7f079c0a5700 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f07a8009900 tx=0x7f07a8031400 comp rx=0 tx=0).stop 2024-09-16T02:02:26.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.941+0000 7f07bab66640 1 -- 172.21.15.105:0/3581205057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07b4102d30 msgr2=0x7f07b41984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:26.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.941+0000 7f07bab66640 1 --2- 172.21.15.105:0/3581205057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07b4102d30 0x7f07b41984f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f07a400c920 tx=0x7f07a400cde0 comp rx=0 tx=0).stop 2024-09-16T02:02:26.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.942+0000 7f07bab66640 1 -- 172.21.15.105:0/3581205057 shutdown_connections 2024-09-16T02:02:26.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.942+0000 7f07bab66640 1 --2- 172.21.15.105:0/3581205057 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f079c0a3240 0x7f079c0a5700 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:26.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.942+0000 7f07bab66640 1 --2- 172.21.15.105:0/3581205057 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07b41036e0 0x7f07b4198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:26.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.942+0000 7f07bab66640 1 --2- 172.21.15.105:0/3581205057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07b41082a0 0x7f07b419cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:26.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.942+0000 7f07bab66640 1 --2- 172.21.15.105:0/3581205057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07b4102d30 0x7f07b41984f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:26.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.942+0000 7f07bab66640 1 -- 172.21.15.105:0/3581205057 >> 172.21.15.105:0/3581205057 conn(0x7f07b40fc7d0 msgr2=0x7f07b4105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:26.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.942+0000 7f07bab66640 1 -- 172.21.15.105:0/3581205057 shutdown_connections 2024-09-16T02:02:26.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:26.942+0000 7f07bab66640 1 -- 172.21.15.105:0/3581205057 wait complete. 2024-09-16T02:02:27.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:26 smithi190 ceph-mon[34762]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-16T02:02:27.761 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-16T02:00:35.112266+0000', 'modified': '2024-09-16T02:00:47.526704+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24482, 'mds_1': 24469, 'mds_2': 24461, 'mds_3': 24457, 'mds_4': 24445}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14613': {'gid': 14613, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.105:6839/4112062361', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6838', 'nonce': 4112062361}, {'type': 'v1', 'addr': '172.21.15.105:6839', 'nonce': 4112062361}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14625': {'gid': 14625, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.105:6841/163156514', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6840', 'nonce': 163156514}, {'type': 'v1', 'addr': '172.21.15.105:6841', 'nonce': 163156514}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24421': {'gid': 24421, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.190:6833/683519097', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6832', 'nonce': 683519097}, {'type': 'v1', 'addr': '172.21.15.190:6833', 'nonce': 683519097}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24422': {'gid': 24422, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.137:6833/3025442862', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6832', 'nonce': 3025442862}, {'type': 'v1', 'addr': '172.21.15.137:6833', 'nonce': 3025442862}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.137:6835/763783602', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6834', 'nonce': 763783602}, {'type': 'v1', 'addr': '172.21.15.137:6835', 'nonce': 763783602}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.190:6835/1851042281', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6834', 'nonce': 1851042281}, {'type': 'v1', 'addr': '172.21.15.190:6835', 'nonce': 1851042281}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.190:6837/3181257549', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6836', 'nonce': 3181257549}, {'type': 'v1', 'addr': '172.21.15.190:6837', 'nonce': 3181257549}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24461': {'gid': 24461, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.137:6837/2788541030', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6836', 'nonce': 2788541030}, {'type': 'v1', 'addr': '172.21.15.137:6837', 'nonce': 2788541030}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.190:6839/1222573537', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6838', 'nonce': 1222573537}, {'type': 'v1', 'addr': '172.21.15.190:6839', 'nonce': 1222573537}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.137:6839/3454924009', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6838', 'nonce': 3454924009}, {'type': 'v1', 'addr': '172.21.15.137:6839', 'nonce': 3454924009}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24482, 'qdb_cluster': [24482, 24469, 24461, 24457, 24445]} 2024-09-16T02:02:27.761 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-16T02:02:27.761 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.f status 2024-09-16T02:02:27.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:27 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/3581205057' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:02:27.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:27 smithi137 ceph-mon[34099]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T02:02:28.006 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:02:28.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:27 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/3581205057' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:02:28.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:27 smithi105 ceph-mon[31197]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T02:02:28.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:27 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/3581205057' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:02:28.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:27 smithi190 ceph-mon[34762]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T02:02:28.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.530+0000 7fc0d9765640 1 -- 172.21.15.105:0/3937001707 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0d4102d40 msgr2=0x7fc0d4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:28.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.530+0000 7fc0d9765640 1 --2- 172.21.15.105:0/3937001707 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0d4102d40 0x7fc0d4103120 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4009900 tx=0x7fc0c402a480 comp rx=0 tx=0).stop 2024-09-16T02:02:28.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.530+0000 7fc0d9765640 1 -- 172.21.15.105:0/3937001707 shutdown_connections 2024-09-16T02:02:28.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.530+0000 7fc0d9765640 1 --2- 172.21.15.105:0/3937001707 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0d4103660 0x7fc0d410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:28.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.530+0000 7fc0d9765640 1 --2- 172.21.15.105:0/3937001707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0d410c170 0x7fc0d410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:28.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.530+0000 7fc0d9765640 1 --2- 172.21.15.105:0/3937001707 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0d4102d40 0x7fc0d4103120 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:28.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.530+0000 7fc0d9765640 1 -- 172.21.15.105:0/3937001707 >> 172.21.15.105:0/3937001707 conn(0x7fc0d40fc7a0 msgr2=0x7fc0d40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:28.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.530+0000 7fc0d9765640 1 -- 172.21.15.105:0/3937001707 shutdown_connections 2024-09-16T02:02:28.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.530+0000 7fc0d9765640 1 -- 172.21.15.105:0/3937001707 wait complete. 2024-09-16T02:02:28.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.531+0000 7fc0d9765640 1 Processor -- start 2024-09-16T02:02:28.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.531+0000 7fc0d9765640 1 -- start start 2024-09-16T02:02:28.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.531+0000 7fc0d9765640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0d4102d40 0x7fc0d41983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:28.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.531+0000 7fc0d9765640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0d4103660 0x7fc0d4198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:28.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.531+0000 7fc0d9765640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0d410c170 0x7fc0d419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:28.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.531+0000 7fc0d9765640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d4111e60 con 0x7fc0d4102d40 2024-09-16T02:02:28.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.531+0000 7fc0d9765640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d4111ce0 con 0x7fc0d410c170 2024-09-16T02:02:28.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.531+0000 7fc0d9765640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d4111fe0 con 0x7fc0d4103660 2024-09-16T02:02:28.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.531+0000 7fc0d2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0d4102d40 0x7fc0d41983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:28.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.531+0000 7fc0d2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0d4102d40 0x7fc0d41983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37798/0 (socket says 172.21.15.105:37798) 2024-09-16T02:02:28.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.531+0000 7fc0d2ffd640 1 -- 172.21.15.105:0/1131901647 learned_addr learned my addr 172.21.15.105:0/1131901647 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:28.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.531+0000 7fc0ca7fc640 1 --2- 172.21.15.105:0/1131901647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0d4103660 0x7fc0d4198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:28.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.532+0000 7fc0d2ffd640 1 -- 172.21.15.105:0/1131901647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0d4103660 msgr2=0x7fc0d4198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:28.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.532+0000 7fc0d2ffd640 1 --2- 172.21.15.105:0/1131901647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0d4103660 0x7fc0d4198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:28.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.532+0000 7fc0d2ffd640 1 -- 172.21.15.105:0/1131901647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0d410c170 msgr2=0x7fc0d419ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:28.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.532+0000 7fc0d2ffd640 1 --2- 172.21.15.105:0/1131901647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0d410c170 0x7fc0d419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:28.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.532+0000 7fc0d2ffd640 1 -- 172.21.15.105:0/1131901647 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d419d3a0 con 0x7fc0d4102d40 2024-09-16T02:02:28.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.532+0000 7fc0ca7fc640 1 --2- 172.21.15.105:0/1131901647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0d4103660 0x7fc0d4198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:02:28.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.532+0000 7fc0d2ffd640 1 --2- 172.21.15.105:0/1131901647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0d4102d40 0x7fc0d41983f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4005940 tx=0x7fc0c4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:28.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.532+0000 7fc0d0ff9640 1 -- 172.21.15.105:0/1131901647 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c403f030 con 0x7fc0d4102d40 2024-09-16T02:02:28.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.532+0000 7fc0d9765640 1 -- 172.21.15.105:0/1131901647 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0d419d630 con 0x7fc0d4102d40 2024-09-16T02:02:28.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.532+0000 7fc0d0ff9640 1 -- 172.21.15.105:0/1131901647 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc0c4003aa0 con 0x7fc0d4102d40 2024-09-16T02:02:28.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.532+0000 7fc0d0ff9640 1 -- 172.21.15.105:0/1131901647 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4035070 con 0x7fc0d4102d40 2024-09-16T02:02:28.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.532+0000 7fc0d9765640 1 -- 172.21.15.105:0/1131901647 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0d41a4f70 con 0x7fc0d4102d40 2024-09-16T02:02:28.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.534+0000 7fc0d0ff9640 1 -- 172.21.15.105:0/1131901647 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fc0c4042030 con 0x7fc0d4102d40 2024-09-16T02:02:28.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.534+0000 7fc0dbb3e740 1 Processor -- start 2024-09-16T02:02:28.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.534+0000 7fc0dbb3e740 1 -- start start 2024-09-16T02:02:28.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.535+0000 7fc0dbb3e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564471b819d0 0x564471b81db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:28.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.535+0000 7fc0dbb3e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564471b822f0 0x564471b8c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:28.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.535+0000 7fc0dbb3e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564471b8cbb0 0x564471b8efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:28.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.535+0000 7fc0dbb3e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564471ae8090 con 0x564471b819d0 2024-09-16T02:02:28.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.535+0000 7fc0dbb3e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564471a8fa70 con 0x564471b8cbb0 2024-09-16T02:02:28.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.535+0000 7fc0dbb3e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564471ae4050 con 0x564471b822f0 2024-09-16T02:02:28.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.535+0000 7fc0d37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564471b819d0 0x564471b81db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:28.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.535+0000 7fc0d37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564471b819d0 0x564471b81db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37814/0 (socket says 172.21.15.105:37814) 2024-09-16T02:02:28.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.535+0000 7fc0d37fe640 1 -- 172.21.15.105:0/358204837 learned_addr learned my addr 172.21.15.105:0/358204837 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:28.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.535+0000 7fc0ca7fc640 1 --2- 172.21.15.105:0/358204837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564471b8cbb0 0x564471b8efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:28.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.535+0000 7fc0d0ff9640 1 --2- 172.21.15.105:0/1131901647 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc0c00a3240 0x7fc0c00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:28.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.535+0000 7fc0d2ffd640 1 --2- 172.21.15.105:0/358204837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564471b822f0 0x564471b8c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:28.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.535+0000 7fc0d37fe640 1 -- 172.21.15.105:0/358204837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564471b822f0 msgr2=0x564471b8c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:28.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.535+0000 7fc0d37fe640 1 --2- 172.21.15.105:0/358204837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564471b822f0 0x564471b8c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:28.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.535+0000 7fc0d37fe640 1 -- 172.21.15.105:0/358204837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564471b8cbb0 msgr2=0x564471b8efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:28.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.535+0000 7fc0d37fe640 1 --2- 172.21.15.105:0/358204837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564471b8cbb0 0x564471b8efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:28.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.535+0000 7fc0d37fe640 1 -- 172.21.15.105:0/358204837 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564471a9c130 con 0x564471b819d0 2024-09-16T02:02:28.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.535+0000 7fc0ca7fc640 1 --2- 172.21.15.105:0/358204837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564471b8cbb0 0x564471b8efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:02:28.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.535+0000 7fc0d2ffd640 1 --2- 172.21.15.105:0/358204837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564471b822f0 0x564471b8c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:02:28.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.535+0000 7fc0d0ff9640 1 -- 172.21.15.105:0/1131901647 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fc0c40f01d0 con 0x7fc0d4102d40 2024-09-16T02:02:28.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.535+0000 7fc0d37fe640 1 --2- 172.21.15.105:0/1131901647 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc0c00a3240 0x7fc0c00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:28.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.536+0000 7fc0d37fe640 1 --2- 172.21.15.105:0/358204837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564471b819d0 0x564471b81db0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc009f00 tx=0x7fc0cc00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d4600fde8fbfd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:28.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.536+0000 7fc0c97fa640 1 -- 172.21.15.105:0/358204837 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc01b550 con 0x564471b819d0 2024-09-16T02:02:28.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.536+0000 7fc0c97fa640 1 -- 172.21.15.105:0/358204837 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc0cc01bba0 con 0x564471b819d0 2024-09-16T02:02:28.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.536+0000 7fc0c97fa640 1 -- 172.21.15.105:0/358204837 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc0119f0 con 0x564471b819d0 2024-09-16T02:02:28.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.536+0000 7fc0d37fe640 1 --2- 172.21.15.105:0/1131901647 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc0c00a3240 0x7fc0c00a5700 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc01f000 tx=0x7fc0cc020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:28.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.536+0000 7fc0dbb3e740 1 -- 172.21.15.105:0/358204837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564471b819d0 msgr2=0x564471b81db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:28.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.536+0000 7fc0dbb3e740 1 --2- 172.21.15.105:0/358204837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564471b819d0 0x564471b81db0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc009f00 tx=0x7fc0cc00e6c0 comp rx=0 tx=0).stop 2024-09-16T02:02:28.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.537+0000 7fc0dbb3e740 1 -- 172.21.15.105:0/358204837 shutdown_connections 2024-09-16T02:02:28.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.537+0000 7fc0dbb3e740 1 --2- 172.21.15.105:0/358204837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564471b822f0 0x564471b8c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:28.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.537+0000 7fc0dbb3e740 1 --2- 172.21.15.105:0/358204837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564471b8cbb0 0x564471b8efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:28.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.537+0000 7fc0dbb3e740 1 --2- 172.21.15.105:0/358204837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564471b819d0 0x564471b81db0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:28.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.537+0000 7fc0dbb3e740 1 -- 172.21.15.105:0/358204837 >> 172.21.15.105:0/358204837 conn(0x564471b7df60 msgr2=0x564471b7e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:28.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.537+0000 7fc0dbb3e740 1 -- 172.21.15.105:0/358204837 shutdown_connections 2024-09-16T02:02:28.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.537+0000 7fc0dbb3e740 1 -- 172.21.15.105:0/358204837 wait complete. 2024-09-16T02:02:28.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.537+0000 7fc0dbb3e740 1 -- start start 2024-09-16T02:02:28.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.537+0000 7fc0dbb3e740 1 Processor -- start 2024-09-16T02:02:28.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.537+0000 7fc0dbb3e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564471b819d0 0x564471b8c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:28.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.537+0000 7fc0dbb3e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564471b822f0 0x564471b89790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:28.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.537+0000 7fc0dbb3e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564471b8cbb0 0x564471b89cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:28.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.537+0000 7fc0dbb3e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564471b8f780 con 0x564471b822f0 2024-09-16T02:02:28.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.537+0000 7fc0dbb3e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564471aa2f20 con 0x564471b819d0 2024-09-16T02:02:28.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.537+0000 7fc0dbb3e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564471b8f900 con 0x564471b8cbb0 2024-09-16T02:02:28.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.537+0000 7fc0dbb3e740 10 client.0 _ll_register_callbacks cb 0x564471a6c030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:02:28.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.537+0000 7fc0ca7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564471b822f0 0x564471b89790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:28.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.537+0000 7fc0ca7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564471b822f0 0x564471b89790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37818/0 (socket says 172.21.15.105:37818) 2024-09-16T02:02:28.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.537+0000 7fc0ca7fc640 1 -- 172.21.15.105:0/3259019127 learned_addr learned my addr 172.21.15.105:0/3259019127 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:28.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.538+0000 7fc0d37fe640 1 --2- 172.21.15.105:0/3259019127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564471b8cbb0 0x564471b89cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:28.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.538+0000 7fc0d2ffd640 1 --2- 172.21.15.105:0/3259019127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564471b819d0 0x564471b8c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:28.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.538+0000 7fc0ca7fc640 1 -- 172.21.15.105:0/3259019127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564471b8cbb0 msgr2=0x564471b89cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:28.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.538+0000 7fc0ca7fc640 1 --2- 172.21.15.105:0/3259019127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564471b8cbb0 0x564471b89cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:28.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.538+0000 7fc0ca7fc640 1 -- 172.21.15.105:0/3259019127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564471b819d0 msgr2=0x564471b8c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:28.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.538+0000 7fc0ca7fc640 1 --2- 172.21.15.105:0/3259019127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564471b819d0 0x564471b8c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:28.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.538+0000 7fc0ca7fc640 1 -- 172.21.15.105:0/3259019127 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644718592b0 con 0x564471b822f0 2024-09-16T02:02:28.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.538+0000 7fc0d2ffd640 1 --2- 172.21.15.105:0/3259019127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564471b819d0 0x564471b8c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:02:28.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.538+0000 7fc0ca7fc640 1 --2- 172.21.15.105:0/3259019127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564471b822f0 0x564471b89790 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc00f1e0 tx=0x7fc0bc00f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc53af1962ce0bf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:28.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.538+0000 7fc0ba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:02:28.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.538+0000 7fc0ba7fc640 1 -- 172.21.15.105:0/3259019127 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc018020 con 0x564471b822f0 2024-09-16T02:02:28.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.538+0000 7fc0ba7fc640 1 -- 172.21.15.105:0/3259019127 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fc0bc014870 con 0x564471b822f0 2024-09-16T02:02:28.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.538+0000 7fc0dbb3e740 1 -- 172.21.15.105:0/3259019127 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0cc01bba0 con 0x564471b822f0 2024-09-16T02:02:28.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.538+0000 7fc0ba7fc640 1 -- 172.21.15.105:0/3259019127 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc0bc023050 con 0x564471b822f0 2024-09-16T02:02:28.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.538+0000 7fc0dbb3e740 10 client.15156 fetch_fsmap learned FSMap version 27 2024-09-16T02:02:28.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.538+0000 7fc0dbb3e740 1 -- 172.21.15.105:0/3259019127 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564471b9b040 con 0x564471b822f0 2024-09-16T02:02:28.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.538+0000 7fc0ba7fc640 1 -- 172.21.15.105:0/3259019127 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc0cc01bba0 con 0x564471b822f0 2024-09-16T02:02:28.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.538+0000 7fc0ba7fc640 1 -- 172.21.15.105:0/3259019127 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc0bc013a10 con 0x564471b822f0 2024-09-16T02:02:28.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.539+0000 7fc0dbb3e740 10 client.15156 fetch_fsmap finished waiting for FSMap version 27 2024-09-16T02:02:28.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.539+0000 7fc0dbb3e740 10 client.15156 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-16T02:02:28.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.539+0000 7fc0dbb3e740 20 client.15156 populate_metadata read hostname 'smithi105' 2024-09-16T02:02:28.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.539+0000 7fc0dbb3e740 1 --2- 172.21.15.105:0/3259019127 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x564471b9d700 0x564471b9eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:28.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.539+0000 7fc0dbb3e740 4 client.15156 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:02:28.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.539+0000 7fc0dbb3e740 1 -- 172.21.15.105:0/3259019127 --> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564471af5600 con 0x564471b9d700 2024-09-16T02:02:28.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.539+0000 7fc0d2ffd640 1 --2- 172.21.15.105:0/3259019127 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x564471b9d700 0x564471b9eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:28.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.539+0000 7fc0ba7fc640 10 client.15156 ms_handle_connect on v2:172.21.15.190:6834/1851042281 2024-09-16T02:02:28.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.539+0000 7fc0d2ffd640 1 --2- 172.21.15.105:0/3259019127 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x564471b9d700 0x564471b9eb70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dc6d832e41cc2989 server_cookie=a3e007c702bfc9cf in_seq=0 out_seq=0 2024-09-16T02:02:28.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.540+0000 7fc0ba7fc640 1 -- 172.21.15.105:0/3259019127 <== mds.4 v2:172.21.15.190:6834/1851042281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564471af5600 con 0x564471b9d700 2024-09-16T02:02:28.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.540+0000 7fc0ba7fc640 10 client.15156 handle_command_reply: tid=0 2024-09-16T02:02:28.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.540+0000 7fc0dbb3e740 1 client.15156 shutdown 2024-09-16T02:02:28.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.540+0000 7fc0dbb3e740 1 -- 172.21.15.105:0/3259019127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564471b822f0 msgr2=0x564471b89790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:28.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.540+0000 7fc0dbb3e740 1 --2- 172.21.15.105:0/3259019127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564471b822f0 0x564471b89790 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc00f1e0 tx=0x7fc0bc00f6a0 comp rx=0 tx=0).stop 2024-09-16T02:02:28.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.540+0000 7fc0dbb3e740 1 -- 172.21.15.105:0/3259019127 shutdown_connections 2024-09-16T02:02:28.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.540+0000 7fc0dbb3e740 1 --2- 172.21.15.105:0/3259019127 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x564471b9d700 0x564471b9eb70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:28.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.540+0000 7fc0dbb3e740 1 --2- 172.21.15.105:0/3259019127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564471b8cbb0 0x564471b89cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:28.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.540+0000 7fc0dbb3e740 1 --2- 172.21.15.105:0/3259019127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564471b822f0 0x564471b89790 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:28.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.540+0000 7fc0ba7fc640 0 client.15156 ms_handle_reset on v2:172.21.15.190:6834/1851042281 2024-09-16T02:02:28.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.540+0000 7fc0dbb3e740 1 --2- 172.21.15.105:0/3259019127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564471b819d0 0x564471b8c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:28.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.540+0000 7fc0dbb3e740 1 -- 172.21.15.105:0/3259019127 >> 172.21.15.105:0/3259019127 conn(0x564471b7df60 msgr2=0x564471b83c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:28.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.540+0000 7fc0dbb3e740 1 -- 172.21.15.105:0/3259019127 shutdown_connections 2024-09-16T02:02:28.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.540+0000 7fc0dbb3e740 1 -- 172.21.15.105:0/3259019127 wait complete. 2024-09-16T02:02:28.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.540+0000 7fc0dbb3e740 20 client.15156 trim_cache size 0 max 16384 2024-09-16T02:02:28.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.558+0000 7fc0dbb3e740 1 Processor -- start 2024-09-16T02:02:28.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.558+0000 7fc0dbb3e740 1 -- start start 2024-09-16T02:02:28.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.558+0000 7fc0dbb3e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564471b819d0 0x564471b99060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:28.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.558+0000 7fc0dbb3e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564471b822f0 0x564471ba6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:28.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.558+0000 7fc0dbb3e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564471b8cbb0 0x564471ba7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:28.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.558+0000 7fc0dbb3e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564471b9ceb0 con 0x564471b8cbb0 2024-09-16T02:02:28.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.558+0000 7fc0dbb3e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564471b9d580 con 0x564471b819d0 2024-09-16T02:02:28.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.558+0000 7fc0dbb3e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564471b9dc50 con 0x564471b822f0 2024-09-16T02:02:28.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.558+0000 7fc0d37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564471b8cbb0 0x564471ba7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:28.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.558+0000 7fc0d37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564471b8cbb0 0x564471ba7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37826/0 (socket says 172.21.15.105:37826) 2024-09-16T02:02:28.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.558+0000 7fc0d37fe640 1 -- 172.21.15.105:0/922015760 learned_addr learned my addr 172.21.15.105:0/922015760 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:28.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.559+0000 7fc0d2ffd640 1 --2- 172.21.15.105:0/922015760 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564471b819d0 0x564471b99060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:28.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.559+0000 7fc0ca7fc640 1 --2- 172.21.15.105:0/922015760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564471b822f0 0x564471ba6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:28.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.559+0000 7fc0d37fe640 1 -- 172.21.15.105:0/922015760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564471b822f0 msgr2=0x564471ba6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:28.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.559+0000 7fc0d37fe640 1 --2- 172.21.15.105:0/922015760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564471b822f0 0x564471ba6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:28.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.559+0000 7fc0d37fe640 1 -- 172.21.15.105:0/922015760 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564471b819d0 msgr2=0x564471b99060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:28.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.559+0000 7fc0d37fe640 1 --2- 172.21.15.105:0/922015760 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564471b819d0 0x564471b99060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:28.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.559+0000 7fc0d37fe640 1 -- 172.21.15.105:0/922015760 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564471b7d810 con 0x564471b8cbb0 2024-09-16T02:02:28.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.559+0000 7fc0d37fe640 1 --2- 172.21.15.105:0/922015760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564471b8cbb0 0x564471ba7390 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc005cc0 tx=0x7fc0cc0123f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72cca47de7819e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:28.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.559+0000 7fc0c8ff9640 1 -- 172.21.15.105:0/922015760 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc034070 con 0x564471b8cbb0 2024-09-16T02:02:28.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.559+0000 7fc0c8ff9640 1 -- 172.21.15.105:0/922015760 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc0cc0119d0 con 0x564471b8cbb0 2024-09-16T02:02:28.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.559+0000 7fc0c8ff9640 1 -- 172.21.15.105:0/922015760 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc01b910 con 0x564471b8cbb0 2024-09-16T02:02:28.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.560+0000 7fc0dbb3e740 1 -- 172.21.15.105:0/922015760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564471b8cbb0 msgr2=0x564471ba7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:28.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.560+0000 7fc0dbb3e740 1 --2- 172.21.15.105:0/922015760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564471b8cbb0 0x564471ba7390 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc005cc0 tx=0x7fc0cc0123f0 comp rx=0 tx=0).stop 2024-09-16T02:02:28.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.560+0000 7fc0dbb3e740 1 -- 172.21.15.105:0/922015760 shutdown_connections 2024-09-16T02:02:28.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.560+0000 7fc0dbb3e740 1 --2- 172.21.15.105:0/922015760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564471b822f0 0x564471ba6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:28.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.560+0000 7fc0dbb3e740 1 --2- 172.21.15.105:0/922015760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564471b8cbb0 0x564471ba7390 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:28.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.560+0000 7fc0dbb3e740 1 --2- 172.21.15.105:0/922015760 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564471b819d0 0x564471b99060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:28.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.560+0000 7fc0dbb3e740 1 -- 172.21.15.105:0/922015760 >> 172.21.15.105:0/922015760 conn(0x564471b7df60 msgr2=0x564471b88060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:28.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.560+0000 7fc0dbb3e740 1 -- 172.21.15.105:0/922015760 shutdown_connections 2024-09-16T02:02:28.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.560+0000 7fc0dbb3e740 1 -- 172.21.15.105:0/922015760 wait complete. 2024-09-16T02:02:28.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.560+0000 7fc0dbb3e740 1 -- start start 2024-09-16T02:02:28.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.560+0000 7fc0dbb3e740 1 Processor -- start 2024-09-16T02:02:28.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.561+0000 7fc0dbb3e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564471b819d0 0x564471b86160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:28.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.561+0000 7fc0dbb3e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564471b822f0 0x564471b866a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:28.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.561+0000 7fc0dbb3e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564471b8cbb0 0x564471ba5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:28.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.561+0000 7fc0dbb3e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564471b9e9f0 con 0x564471b819d0 2024-09-16T02:02:28.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.561+0000 7fc0dbb3e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564471b9e320 con 0x564471b8cbb0 2024-09-16T02:02:28.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.561+0000 7fc0dbb3e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564471b9f0c0 con 0x564471b822f0 2024-09-16T02:02:28.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.561+0000 7fc0dbb3e740 10 client.0 _ll_register_callbacks cb 0x7fc0d406b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:02:28.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.561+0000 7fc0d2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564471b819d0 0x564471b86160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:28.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.561+0000 7fc0d2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564471b819d0 0x564471b86160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37828/0 (socket says 172.21.15.105:37828) 2024-09-16T02:02:28.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.561+0000 7fc0d2ffd640 1 -- 172.21.15.105:0/3539253007 learned_addr learned my addr 172.21.15.105:0/3539253007 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:28.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.561+0000 7fc0d37fe640 1 --2- 172.21.15.105:0/3539253007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564471b8cbb0 0x564471ba5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:28.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.561+0000 7fc0d2ffd640 1 -- 172.21.15.105:0/3539253007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564471b822f0 msgr2=0x564471b866a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:28.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.561+0000 7fc0ca7fc640 1 --2- 172.21.15.105:0/3539253007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564471b822f0 0x564471b866a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:28.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.561+0000 7fc0d2ffd640 1 --2- 172.21.15.105:0/3539253007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564471b822f0 0x564471b866a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:28.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.561+0000 7fc0d2ffd640 1 -- 172.21.15.105:0/3539253007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564471b8cbb0 msgr2=0x564471ba5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:28.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.561+0000 7fc0d2ffd640 1 --2- 172.21.15.105:0/3539253007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564471b8cbb0 0x564471ba5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:28.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.561+0000 7fc0d2ffd640 1 -- 172.21.15.105:0/3539253007 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564471ba63b0 con 0x564471b819d0 2024-09-16T02:02:28.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.561+0000 7fc0d2ffd640 1 --2- 172.21.15.105:0/3539253007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564471b819d0 0x564471b86160 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc0c40b8c30 tx=0x7fc0c408b510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d625065cd90d5df server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:28.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.561+0000 7fc0ba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:02:28.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.561+0000 7fc0ba7fc640 1 -- 172.21.15.105:0/3539253007 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c40ef070 con 0x564471b819d0 2024-09-16T02:02:28.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.561+0000 7fc0ba7fc640 1 -- 172.21.15.105:0/3539253007 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fc0c40baad0 con 0x564471b819d0 2024-09-16T02:02:28.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.561+0000 7fc0dbb3e740 1 -- 172.21.15.105:0/3539253007 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0cc0119d0 con 0x564471b819d0 2024-09-16T02:02:28.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.561+0000 7fc0ba7fc640 1 -- 172.21.15.105:0/3539253007 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc0c4049040 con 0x564471b819d0 2024-09-16T02:02:28.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.561+0000 7fc0dbb3e740 10 client.15162 fetch_fsmap learned FSMap version 27 2024-09-16T02:02:28.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.561+0000 7fc0dbb3e740 1 -- 172.21.15.105:0/3539253007 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564471ba7690 con 0x564471b819d0 2024-09-16T02:02:28.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.562+0000 7fc0ba7fc640 1 -- 172.21.15.105:0/3539253007 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc0cc0119d0 con 0x564471b819d0 2024-09-16T02:02:28.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.562+0000 7fc0ba7fc640 1 -- 172.21.15.105:0/3539253007 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc0c40b8e40 con 0x564471b819d0 2024-09-16T02:02:28.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.562+0000 7fc0dbb3e740 10 client.15162 fetch_fsmap finished waiting for FSMap version 27 2024-09-16T02:02:28.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.562+0000 7fc0dbb3e740 10 client.15162 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-16T02:02:28.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.562+0000 7fc0dbb3e740 20 client.15162 populate_metadata read hostname 'smithi105' 2024-09-16T02:02:28.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.562+0000 7fc0dbb3e740 1 --2- 172.21.15.105:0/3539253007 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x564471c10760 0x564471c0f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:28.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.562+0000 7fc0dbb3e740 4 client.15162 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T02:02:28.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.562+0000 7fc0dbb3e740 1 -- 172.21.15.105:0/3539253007 --> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] -- command(tid 0: {"prefix": "status"}) -- 0x564471aefb10 con 0x564471c10760 2024-09-16T02:02:28.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.562+0000 7fc0ca7fc640 1 --2- 172.21.15.105:0/3539253007 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x564471c10760 0x564471c0f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:28.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.562+0000 7fc0ba7fc640 10 client.15162 ms_handle_connect on v2:172.21.15.190:6834/1851042281 2024-09-16T02:02:28.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.562+0000 7fc0ca7fc640 1 --2- 172.21.15.105:0/3539253007 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x564471c10760 0x564471c0f470 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=20600cfd0269dca4 server_cookie=5c132c7f6d518066 in_seq=0 out_seq=0 2024-09-16T02:02:28.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.563+0000 7fc0ba7fc640 1 -- 172.21.15.105:0/3539253007 <== mds.4 v2:172.21.15.190:6834/1851042281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x564471aefb10 con 0x564471c10760 2024-09-16T02:02:28.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.563+0000 7fc0ba7fc640 10 client.15162 handle_command_reply: tid=0 2024-09-16T02:02:28.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.563+0000 7fc0dbb3e740 1 client.15162 shutdown 2024-09-16T02:02:28.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.563+0000 7fc0dbb3e740 1 -- 172.21.15.105:0/3539253007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564471b819d0 msgr2=0x564471b86160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:28.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.563+0000 7fc0dbb3e740 1 --2- 172.21.15.105:0/3539253007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564471b819d0 0x564471b86160 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc0c40b8c30 tx=0x7fc0c408b510 comp rx=0 tx=0).stop 2024-09-16T02:02:28.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.563+0000 7fc0dbb3e740 1 -- 172.21.15.105:0/3539253007 shutdown_connections 2024-09-16T02:02:28.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.563+0000 7fc0dbb3e740 1 --2- 172.21.15.105:0/3539253007 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x564471c10760 0x564471c0f470 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:28.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.563+0000 7fc0dbb3e740 1 --2- 172.21.15.105:0/3539253007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564471b822f0 0x564471b866a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:28.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.563+0000 7fc0dbb3e740 1 --2- 172.21.15.105:0/3539253007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564471b8cbb0 0x564471ba5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:28.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.563+0000 7fc0dbb3e740 1 --2- 172.21.15.105:0/3539253007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564471b819d0 0x564471b86160 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:28.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.563+0000 7fc0dbb3e740 1 -- 172.21.15.105:0/3539253007 >> 172.21.15.105:0/3539253007 conn(0x564471b7df60 msgr2=0x564471b83420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:28.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.563+0000 7fc0ba7fc640 0 client.15162 ms_handle_reset on v2:172.21.15.190:6834/1851042281 2024-09-16T02:02:28.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.563+0000 7fc0dbb3e740 1 -- 172.21.15.105:0/3539253007 shutdown_connections 2024-09-16T02:02:28.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.563+0000 7fc0dbb3e740 1 -- 172.21.15.105:0/3539253007 wait complete. 2024-09-16T02:02:28.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.563+0000 7fc0dbb3e740 20 client.15162 trim_cache size 0 max 16384 2024-09-16T02:02:28.583 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-16T02:02:28.583 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "423b7a2a-73ce-11ef-bceb-c7b262605968", 2024-09-16T02:02:28.583 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 4, 2024-09-16T02:02:28.583 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 24445, 2024-09-16T02:02:28.583 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2024-09-16T02:02:28.583 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2024-09-16T02:02:28.583 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2024-09-16T02:02:28.583 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 102.046182194, 2024-09-16T02:02:28.583 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 27, 2024-09-16T02:02:28.583 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 81, 2024-09-16T02:02:28.583 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 79, 2024-09-16T02:02:28.583 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 141.35230345299999 2024-09-16T02:02:28.583 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-16T02:02:28.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.564+0000 7fc0d9765640 1 -- 172.21.15.105:0/1131901647 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc0c00a3240 msgr2=0x7fc0c00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:28.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.564+0000 7fc0d9765640 1 --2- 172.21.15.105:0/1131901647 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc0c00a3240 0x7fc0c00a5700 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc01f000 tx=0x7fc0cc020000 comp rx=0 tx=0).stop 2024-09-16T02:02:28.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.564+0000 7fc0d9765640 1 -- 172.21.15.105:0/1131901647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0d4102d40 msgr2=0x7fc0d41983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:28.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.564+0000 7fc0d9765640 1 --2- 172.21.15.105:0/1131901647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0d4102d40 0x7fc0d41983f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4005940 tx=0x7fc0c4004270 comp rx=0 tx=0).stop 2024-09-16T02:02:28.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.564+0000 7fc0d9765640 1 -- 172.21.15.105:0/1131901647 shutdown_connections 2024-09-16T02:02:28.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.564+0000 7fc0d9765640 1 --2- 172.21.15.105:0/1131901647 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc0c00a3240 0x7fc0c00a5700 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:28.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.564+0000 7fc0d9765640 1 --2- 172.21.15.105:0/1131901647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0d4103660 0x7fc0d4198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:28.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.564+0000 7fc0d9765640 1 --2- 172.21.15.105:0/1131901647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0d410c170 0x7fc0d419ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:28.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.564+0000 7fc0d9765640 1 --2- 172.21.15.105:0/1131901647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0d4102d40 0x7fc0d41983f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:28.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.564+0000 7fc0d9765640 1 -- 172.21.15.105:0/1131901647 >> 172.21.15.105:0/1131901647 conn(0x7fc0d40fc7a0 msgr2=0x7fc0d40fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:28.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.564+0000 7fc0d9765640 1 -- 172.21.15.105:0/1131901647 shutdown_connections 2024-09-16T02:02:28.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:28.564+0000 7fc0d9765640 1 -- 172.21.15.105:0/1131901647 wait complete. 2024-09-16T02:02:29.479 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-16T02:02:29.728 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:02:30.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.238+0000 7f765d4f3640 1 -- 172.21.15.105:0/3466180546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7658102d30 msgr2=0x7f7658103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:30.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.238+0000 7f765d4f3640 1 --2- 172.21.15.105:0/3466180546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7658102d30 0x7f7658103110 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7648009cd0 tx=0x7f764802a910 comp rx=0 tx=0).stop 2024-09-16T02:02:30.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.238+0000 7f765d4f3640 1 -- 172.21.15.105:0/3466180546 shutdown_connections 2024-09-16T02:02:30.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.239+0000 7f765d4f3640 1 --2- 172.21.15.105:0/3466180546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76581036e0 0x7f7658107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:30.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.239+0000 7f765d4f3640 1 --2- 172.21.15.105:0/3466180546 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76581082a0 0x7f765810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:30.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.239+0000 7f765d4f3640 1 --2- 172.21.15.105:0/3466180546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7658102d30 0x7f7658103110 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:30.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.239+0000 7f765d4f3640 1 -- 172.21.15.105:0/3466180546 >> 172.21.15.105:0/3466180546 conn(0x7f76580fc7d0 msgr2=0x7f76580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:30.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.239+0000 7f765d4f3640 1 -- 172.21.15.105:0/3466180546 shutdown_connections 2024-09-16T02:02:30.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.239+0000 7f765d4f3640 1 -- 172.21.15.105:0/3466180546 wait complete. 2024-09-16T02:02:30.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.239+0000 7f765d4f3640 1 Processor -- start 2024-09-16T02:02:30.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.239+0000 7f765d4f3640 1 -- start start 2024-09-16T02:02:30.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.239+0000 7f765d4f3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7658102d30 0x7f7658198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:30.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.239+0000 7f765d4f3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76581036e0 0x7f7658198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:30.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.240+0000 7f765d4f3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76581082a0 0x7f765819ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:30.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.240+0000 7f765d4f3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7658111f90 con 0x7f76581036e0 2024-09-16T02:02:30.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.240+0000 7f765d4f3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7658111e10 con 0x7f76581082a0 2024-09-16T02:02:30.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.240+0000 7f765d4f3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7658112110 con 0x7f7658102d30 2024-09-16T02:02:30.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.240+0000 7f76567fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76581036e0 0x7f7658198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:30.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.240+0000 7f76567fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76581036e0 0x7f7658198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37844/0 (socket says 172.21.15.105:37844) 2024-09-16T02:02:30.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.240+0000 7f76567fc640 1 -- 172.21.15.105:0/3630177158 learned_addr learned my addr 172.21.15.105:0/3630177158 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:30.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.240+0000 7f7656ffd640 1 --2- 172.21.15.105:0/3630177158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7658102d30 0x7f7658198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:30.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.240+0000 7f76567fc640 1 -- 172.21.15.105:0/3630177158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7658102d30 msgr2=0x7f7658198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:30.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.240+0000 7f76567fc640 1 --2- 172.21.15.105:0/3630177158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7658102d30 0x7f7658198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:30.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.240+0000 7f76567fc640 1 -- 172.21.15.105:0/3630177158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76581082a0 msgr2=0x7f765819ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:30.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.240+0000 7f76577fe640 1 --2- 172.21.15.105:0/3630177158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76581082a0 0x7f765819ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:30.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.240+0000 7f76567fc640 1 --2- 172.21.15.105:0/3630177158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76581082a0 0x7f765819ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:30.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.240+0000 7f76567fc640 1 -- 172.21.15.105:0/3630177158 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f765819d540 con 0x7f76581036e0 2024-09-16T02:02:30.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.240+0000 7f7656ffd640 1 --2- 172.21.15.105:0/3630177158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7658102d30 0x7f7658198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:02:30.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.240+0000 7f76567fc640 1 --2- 172.21.15.105:0/3630177158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76581036e0 0x7f7658198ad0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7640006fd0 tx=0x7f764000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:30.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.240+0000 7f764ffff640 1 -- 172.21.15.105:0/3630177158 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7640011830 con 0x7f76581036e0 2024-09-16T02:02:30.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.240+0000 7f765d4f3640 1 -- 172.21.15.105:0/3630177158 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f765819d830 con 0x7f76581036e0 2024-09-16T02:02:30.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.240+0000 7f764ffff640 1 -- 172.21.15.105:0/3630177158 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7640016070 con 0x7f76581036e0 2024-09-16T02:02:30.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.240+0000 7f764ffff640 1 -- 172.21.15.105:0/3630177158 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7640010df0 con 0x7f76581036e0 2024-09-16T02:02:30.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.240+0000 7f765d4f3640 1 -- 172.21.15.105:0/3630177158 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76581a5110 con 0x7f76581036e0 2024-09-16T02:02:30.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.243+0000 7f764ffff640 1 -- 172.21.15.105:0/3630177158 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f76400119d0 con 0x7f76581036e0 2024-09-16T02:02:30.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.243+0000 7f764ffff640 1 --2- 172.21.15.105:0/3630177158 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f76440a3240 0x7f76440a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:30.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.243+0000 7f765f8cc740 1 Processor -- start 2024-09-16T02:02:30.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.243+0000 7f765f8cc740 1 -- start start 2024-09-16T02:02:30.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.244+0000 7f7656ffd640 1 --2- 172.21.15.105:0/3630177158 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f76440a3240 0x7f76440a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:30.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.244+0000 7f764ffff640 1 -- 172.21.15.105:0/3630177158 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f76400cf000 con 0x7f76581036e0 2024-09-16T02:02:30.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.244+0000 7f765f8cc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631d79df870 0x5631d79dfc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:30.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.244+0000 7f765f8cc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631d79e0190 0x5631d79eea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:30.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.244+0000 7f765f8cc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631d79eefc0 0x5631d79f13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:30.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.244+0000 7f765f8cc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5631d7946090 con 0x5631d79eefc0 2024-09-16T02:02:30.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.244+0000 7f765f8cc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5631d78eda70 con 0x5631d79df870 2024-09-16T02:02:30.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.244+0000 7f765f8cc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5631d7942050 con 0x5631d79e0190 2024-09-16T02:02:30.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.244+0000 7f76577fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631d79eefc0 0x5631d79f13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:30.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.244+0000 7f76577fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631d79eefc0 0x5631d79f13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37848/0 (socket says 172.21.15.105:37848) 2024-09-16T02:02:30.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.244+0000 7f76577fe640 1 -- 172.21.15.105:0/777231799 learned_addr learned my addr 172.21.15.105:0/777231799 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:30.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.244+0000 7f7656ffd640 1 --2- 172.21.15.105:0/777231799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631d79df870 0x5631d79dfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:30.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.244+0000 7f7656ffd640 1 --2- 172.21.15.105:0/3630177158 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f76440a3240 0x7f76440a5700 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7648009430 tx=0x7f7648005ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:30.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.244+0000 7f76567fc640 1 --2- 172.21.15.105:0/777231799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631d79e0190 0x5631d79eea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:30.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.244+0000 7f76577fe640 1 -- 172.21.15.105:0/777231799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631d79e0190 msgr2=0x5631d79eea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:30.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.244+0000 7f76577fe640 1 --2- 172.21.15.105:0/777231799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631d79e0190 0x5631d79eea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:30.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.244+0000 7f76577fe640 1 -- 172.21.15.105:0/777231799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631d79df870 msgr2=0x5631d79dfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:30.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.244+0000 7f76577fe640 1 --2- 172.21.15.105:0/777231799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631d79df870 0x5631d79dfc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:30.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.244+0000 7f76577fe640 1 -- 172.21.15.105:0/777231799 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631d78fa130 con 0x5631d79eefc0 2024-09-16T02:02:30.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.245+0000 7f76567fc640 1 --2- 172.21.15.105:0/777231799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631d79e0190 0x5631d79eea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:02:30.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.245+0000 7f76577fe640 1 --2- 172.21.15.105:0/777231799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631d79eefc0 0x5631d79f13b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f765000ba10 tx=0x7f765000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96804875a62bba08 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:30.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.245+0000 7f764d7fa640 1 -- 172.21.15.105:0/777231799 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765000de50 con 0x5631d79eefc0 2024-09-16T02:02:30.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.245+0000 7f764d7fa640 1 -- 172.21.15.105:0/777231799 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76500178d0 con 0x5631d79eefc0 2024-09-16T02:02:30.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.245+0000 7f764d7fa640 1 -- 172.21.15.105:0/777231799 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765000f970 con 0x5631d79eefc0 2024-09-16T02:02:30.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.246+0000 7f765f8cc740 1 -- 172.21.15.105:0/777231799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631d79eefc0 msgr2=0x5631d79f13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:30.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.246+0000 7f765f8cc740 1 --2- 172.21.15.105:0/777231799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631d79eefc0 0x5631d79f13b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f765000ba10 tx=0x7f765000bed0 comp rx=0 tx=0).stop 2024-09-16T02:02:30.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.246+0000 7f765f8cc740 1 -- 172.21.15.105:0/777231799 shutdown_connections 2024-09-16T02:02:30.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.246+0000 7f765f8cc740 1 --2- 172.21.15.105:0/777231799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631d79e0190 0x5631d79eea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:30.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.246+0000 7f765f8cc740 1 --2- 172.21.15.105:0/777231799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631d79eefc0 0x5631d79f13b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:30.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.246+0000 7f765f8cc740 1 --2- 172.21.15.105:0/777231799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631d79df870 0x5631d79dfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:30.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.246+0000 7f765f8cc740 1 -- 172.21.15.105:0/777231799 >> 172.21.15.105:0/777231799 conn(0x5631d79dbf60 msgr2=0x5631d79dc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:30.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.246+0000 7f765f8cc740 1 -- 172.21.15.105:0/777231799 shutdown_connections 2024-09-16T02:02:30.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.246+0000 7f765f8cc740 1 -- 172.21.15.105:0/777231799 wait complete. 2024-09-16T02:02:30.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.246+0000 7f765f8cc740 1 -- start start 2024-09-16T02:02:30.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.246+0000 7f765f8cc740 1 Processor -- start 2024-09-16T02:02:30.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.246+0000 7f765f8cc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631d79df870 0x5631d79ee7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:30.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.246+0000 7f765f8cc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631d79e0190 0x5631d79ebb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:30.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.246+0000 7f765f8cc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631d79eefc0 0x5631d79ec080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:30.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.246+0000 7f765f8cc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5631d79f1b90 con 0x5631d79eefc0 2024-09-16T02:02:30.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.246+0000 7f765f8cc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5631d7900f20 con 0x5631d79df870 2024-09-16T02:02:30.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.246+0000 7f765f8cc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5631d79f1d10 con 0x5631d79e0190 2024-09-16T02:02:30.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.247+0000 7f765f8cc740 10 client.0 _ll_register_callbacks cb 0x5631d78ca030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:02:30.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.247+0000 7f76577fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631d79eefc0 0x5631d79ec080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:30.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.247+0000 7f76577fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631d79eefc0 0x5631d79ec080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37862/0 (socket says 172.21.15.105:37862) 2024-09-16T02:02:30.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.247+0000 7f76577fe640 1 -- 172.21.15.105:0/2893878275 learned_addr learned my addr 172.21.15.105:0/2893878275 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:30.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.247+0000 7f76577fe640 1 -- 172.21.15.105:0/2893878275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631d79e0190 msgr2=0x5631d79ebb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:30.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.247+0000 7f76577fe640 1 --2- 172.21.15.105:0/2893878275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631d79e0190 0x5631d79ebb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:30.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.247+0000 7f76577fe640 1 -- 172.21.15.105:0/2893878275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631d79df870 msgr2=0x5631d79ee7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:30.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.247+0000 7f76577fe640 1 --2- 172.21.15.105:0/2893878275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631d79df870 0x5631d79ee7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:30.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.247+0000 7f76577fe640 1 -- 172.21.15.105:0/2893878275 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631d76b72b0 con 0x5631d79eefc0 2024-09-16T02:02:30.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.247+0000 7f76577fe640 1 --2- 172.21.15.105:0/2893878275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631d79eefc0 0x5631d79ec080 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f765000f590 tx=0x7f7650017650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fbd35049ed45ba9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:30.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.247+0000 7f763e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:02:30.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.247+0000 7f763e7fc640 1 -- 172.21.15.105:0/2893878275 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7650026020 con 0x5631d79eefc0 2024-09-16T02:02:30.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.247+0000 7f765f8cc740 1 -- 172.21.15.105:0/2893878275 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76500178d0 con 0x5631d79eefc0 2024-09-16T02:02:30.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.247+0000 7f763e7fc640 1 -- 172.21.15.105:0/2893878275 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f7650017c90 con 0x5631d79eefc0 2024-09-16T02:02:30.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.247+0000 7f763e7fc640 1 -- 172.21.15.105:0/2893878275 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7650031050 con 0x5631d79eefc0 2024-09-16T02:02:30.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.247+0000 7f765f8cc740 10 client.15174 fetch_fsmap learned FSMap version 27 2024-09-16T02:02:30.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.247+0000 7f765f8cc740 1 -- 172.21.15.105:0/2893878275 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5631d79fd3f0 con 0x5631d79eefc0 2024-09-16T02:02:30.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.247+0000 7f763e7fc640 1 -- 172.21.15.105:0/2893878275 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76500178d0 con 0x5631d79eefc0 2024-09-16T02:02:30.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.248+0000 7f763e7fc640 1 -- 172.21.15.105:0/2893878275 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f765000fb00 con 0x5631d79eefc0 2024-09-16T02:02:30.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.248+0000 7f765f8cc740 10 client.15174 fetch_fsmap finished waiting for FSMap version 27 2024-09-16T02:02:30.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.248+0000 7f765f8cc740 10 client.15174 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T02:02:30.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.248+0000 7f765f8cc740 20 client.15174 populate_metadata read hostname 'smithi105' 2024-09-16T02:02:30.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.248+0000 7f765f8cc740 1 --2- 172.21.15.105:0/2893878275 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x5631d79ff720 0x5631d7a00b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:30.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.248+0000 7f765f8cc740 4 client.15174 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:02:30.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.248+0000 7f765f8cc740 1 -- 172.21.15.105:0/2893878275 --> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631d7953600 con 0x5631d79ff720 2024-09-16T02:02:30.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.248+0000 7f7656ffd640 1 --2- 172.21.15.105:0/2893878275 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x5631d79ff720 0x5631d7a00b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:30.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.249+0000 7f763e7fc640 10 client.15174 ms_handle_connect on v2:172.21.15.190:6836/3181257549 2024-09-16T02:02:30.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.249+0000 7f7656ffd640 1 --2- 172.21.15.105:0/2893878275 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x5631d79ff720 0x5631d7a00b90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2000620a72481903 server_cookie=6a46414f2bbc0317 in_seq=0 out_seq=0 2024-09-16T02:02:30.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.249+0000 7f763e7fc640 1 -- 172.21.15.105:0/2893878275 <== mds.3 v2:172.21.15.190:6836/3181257549 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5631d7953600 con 0x5631d79ff720 2024-09-16T02:02:30.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.249+0000 7f763e7fc640 10 client.15174 handle_command_reply: tid=0 2024-09-16T02:02:30.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.249+0000 7f765f8cc740 1 client.15174 shutdown 2024-09-16T02:02:30.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.250+0000 7f765f8cc740 1 -- 172.21.15.105:0/2893878275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631d79eefc0 msgr2=0x5631d79ec080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:30.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.250+0000 7f765f8cc740 1 --2- 172.21.15.105:0/2893878275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631d79eefc0 0x5631d79ec080 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f765000f590 tx=0x7f7650017650 comp rx=0 tx=0).stop 2024-09-16T02:02:30.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.250+0000 7f765f8cc740 1 -- 172.21.15.105:0/2893878275 shutdown_connections 2024-09-16T02:02:30.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.250+0000 7f765f8cc740 1 --2- 172.21.15.105:0/2893878275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631d79e0190 0x5631d79ebb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:30.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.250+0000 7f765f8cc740 1 --2- 172.21.15.105:0/2893878275 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x5631d79ff720 0x5631d7a00b90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:30.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.250+0000 7f765f8cc740 1 --2- 172.21.15.105:0/2893878275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631d79eefc0 0x5631d79ec080 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:30.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.250+0000 7f765f8cc740 1 --2- 172.21.15.105:0/2893878275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631d79df870 0x5631d79ee7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:30.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.250+0000 7f763e7fc640 0 client.15174 ms_handle_reset on v2:172.21.15.190:6836/3181257549 2024-09-16T02:02:30.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.250+0000 7f765f8cc740 1 -- 172.21.15.105:0/2893878275 >> 172.21.15.105:0/2893878275 conn(0x5631d79dbf60 msgr2=0x5631d79e1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:30.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.250+0000 7f765f8cc740 1 -- 172.21.15.105:0/2893878275 shutdown_connections 2024-09-16T02:02:30.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.250+0000 7f765f8cc740 1 -- 172.21.15.105:0/2893878275 wait complete. 2024-09-16T02:02:30.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.250+0000 7f765f8cc740 20 client.15174 trim_cache size 0 max 16384 2024-09-16T02:02:30.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.269+0000 7f765f8cc740 1 Processor -- start 2024-09-16T02:02:30.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.269+0000 7f765f8cc740 1 -- start start 2024-09-16T02:02:30.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.269+0000 7f765f8cc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631d79df870 0x5631d7a00b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:30.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.269+0000 7f765f8cc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631d79e0190 0x5631d79ec090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:30.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.269+0000 7f765f8cc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631d79eefc0 0x5631d79e7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:30.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.269+0000 7f765f8cc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5631d79feed0 con 0x5631d79df870 2024-09-16T02:02:30.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.269+0000 7f765f8cc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5631d79ff5a0 con 0x5631d79e0190 2024-09-16T02:02:30.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.269+0000 7f765f8cc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5631d79ffc70 con 0x5631d79eefc0 2024-09-16T02:02:30.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.269+0000 7f7656ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631d79df870 0x5631d7a00b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:30.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.269+0000 7f7656ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631d79df870 0x5631d7a00b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37878/0 (socket says 172.21.15.105:37878) 2024-09-16T02:02:30.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.269+0000 7f7656ffd640 1 -- 172.21.15.105:0/2378909382 learned_addr learned my addr 172.21.15.105:0/2378909382 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:30.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.270+0000 7f7656ffd640 1 -- 172.21.15.105:0/2378909382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631d79eefc0 msgr2=0x5631d79e7240 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T02:02:30.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.270+0000 7f7656ffd640 1 --2- 172.21.15.105:0/2378909382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631d79eefc0 0x5631d79e7240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:30.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.270+0000 7f7656ffd640 1 -- 172.21.15.105:0/2378909382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631d79e0190 msgr2=0x5631d79ec090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T02:02:30.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.270+0000 7f7656ffd640 1 --2- 172.21.15.105:0/2378909382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631d79e0190 0x5631d79ec090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:30.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.270+0000 7f7656ffd640 1 -- 172.21.15.105:0/2378909382 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631d79eeb60 con 0x5631d79df870 2024-09-16T02:02:30.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.270+0000 7f7656ffd640 1 --2- 172.21.15.105:0/2378909382 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631d79df870 0x5631d7a00b90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7648009cd0 tx=0x7f764803d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0dc2c0dca36c31 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:30.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.270+0000 7f764cff9640 1 -- 172.21.15.105:0/2378909382 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7648004240 con 0x5631d79df870 2024-09-16T02:02:30.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.270+0000 7f764cff9640 1 -- 172.21.15.105:0/2378909382 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7648036050 con 0x5631d79df870 2024-09-16T02:02:30.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.270+0000 7f764cff9640 1 -- 172.21.15.105:0/2378909382 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7648030a70 con 0x5631d79df870 2024-09-16T02:02:30.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.271+0000 7f765f8cc740 1 -- 172.21.15.105:0/2378909382 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631d79df870 msgr2=0x5631d7a00b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:30.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.271+0000 7f765f8cc740 1 --2- 172.21.15.105:0/2378909382 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631d79df870 0x5631d7a00b90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7648009cd0 tx=0x7f764803d000 comp rx=0 tx=0).stop 2024-09-16T02:02:30.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.271+0000 7f765f8cc740 1 -- 172.21.15.105:0/2378909382 shutdown_connections 2024-09-16T02:02:30.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.271+0000 7f765f8cc740 1 --2- 172.21.15.105:0/2378909382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631d79eefc0 0x5631d79e7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:30.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.271+0000 7f765f8cc740 1 --2- 172.21.15.105:0/2378909382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631d79e0190 0x5631d79ec090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:30.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.271+0000 7f765f8cc740 1 --2- 172.21.15.105:0/2378909382 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631d79df870 0x5631d7a00b90 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:30.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.271+0000 7f765f8cc740 1 -- 172.21.15.105:0/2378909382 >> 172.21.15.105:0/2378909382 conn(0x5631d79dbf60 msgr2=0x5631d79da7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:30.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.271+0000 7f765f8cc740 1 -- 172.21.15.105:0/2378909382 shutdown_connections 2024-09-16T02:02:30.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.271+0000 7f765f8cc740 1 -- 172.21.15.105:0/2378909382 wait complete. 2024-09-16T02:02:30.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.271+0000 7f765f8cc740 1 -- start start 2024-09-16T02:02:30.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.271+0000 7f765f8cc740 1 Processor -- start 2024-09-16T02:02:30.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.271+0000 7f765f8cc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631d79df870 0x5631d7a09400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:30.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.271+0000 7f765f8cc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631d79e0190 0x5631d79e8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:30.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.271+0000 7f765f8cc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631d79eefc0 0x5631d79e8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:30.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.271+0000 7f765f8cc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5631d7a00a10 con 0x5631d79df870 2024-09-16T02:02:30.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.271+0000 7f765f8cc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5631d7a00340 con 0x5631d79eefc0 2024-09-16T02:02:30.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.271+0000 7f765f8cc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5631d7a010e0 con 0x5631d79e0190 2024-09-16T02:02:30.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.271+0000 7f765f8cc740 10 client.0 _ll_register_callbacks cb 0x7f7648009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:02:30.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.272+0000 7f7656ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631d79df870 0x5631d7a09400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:30.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.272+0000 7f7656ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631d79df870 0x5631d7a09400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37894/0 (socket says 172.21.15.105:37894) 2024-09-16T02:02:30.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.272+0000 7f7656ffd640 1 -- 172.21.15.105:0/1652832542 learned_addr learned my addr 172.21.15.105:0/1652832542 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:30.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.272+0000 7f76567fc640 1 --2- 172.21.15.105:0/1652832542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631d79e0190 0x5631d79e8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:30.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.272+0000 7f7656ffd640 1 -- 172.21.15.105:0/1652832542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631d79e0190 msgr2=0x5631d79e8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:30.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.272+0000 7f7656ffd640 1 --2- 172.21.15.105:0/1652832542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631d79e0190 0x5631d79e8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:30.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.272+0000 7f76577fe640 1 --2- 172.21.15.105:0/1652832542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631d79eefc0 0x5631d79e8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:30.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.272+0000 7f7656ffd640 1 -- 172.21.15.105:0/1652832542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631d79eefc0 msgr2=0x5631d79e8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:30.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.272+0000 7f7656ffd640 1 --2- 172.21.15.105:0/1652832542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631d79eefc0 0x5631d79e8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:30.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.272+0000 7f7656ffd640 1 -- 172.21.15.105:0/1652832542 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631d79e9480 con 0x5631d79df870 2024-09-16T02:02:30.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.272+0000 7f76567fc640 1 --2- 172.21.15.105:0/1652832542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631d79e0190 0x5631d79e8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:02:30.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.272+0000 7f7656ffd640 1 --2- 172.21.15.105:0/1652832542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631d79df870 0x5631d7a09400 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f764802b0a0 tx=0x7f7648003e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df5310f20fc83f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:30.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.272+0000 7f763e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:02:30.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.272+0000 7f763e7fc640 1 -- 172.21.15.105:0/1652832542 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764805e020 con 0x5631d79df870 2024-09-16T02:02:30.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.272+0000 7f763e7fc640 1 -- 172.21.15.105:0/1652832542 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f7648030b20 con 0x5631d79df870 2024-09-16T02:02:30.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.272+0000 7f765f8cc740 1 -- 172.21.15.105:0/1652832542 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7648036050 con 0x5631d79df870 2024-09-16T02:02:30.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.272+0000 7f763e7fc640 1 -- 172.21.15.105:0/1652832542 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7648031ea0 con 0x5631d79df870 2024-09-16T02:02:30.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.272+0000 7f765f8cc740 10 client.15180 fetch_fsmap learned FSMap version 27 2024-09-16T02:02:30.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.272+0000 7f765f8cc740 1 -- 172.21.15.105:0/1652832542 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5631d7a69e30 con 0x5631d79df870 2024-09-16T02:02:30.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.273+0000 7f763e7fc640 1 -- 172.21.15.105:0/1652832542 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7648036050 con 0x5631d79df870 2024-09-16T02:02:30.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.273+0000 7f763e7fc640 1 -- 172.21.15.105:0/1652832542 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f764805a5c0 con 0x5631d79df870 2024-09-16T02:02:30.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.273+0000 7f765f8cc740 10 client.15180 fetch_fsmap finished waiting for FSMap version 27 2024-09-16T02:02:30.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.273+0000 7f765f8cc740 10 client.15180 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T02:02:30.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.273+0000 7f765f8cc740 20 client.15180 populate_metadata read hostname 'smithi105' 2024-09-16T02:02:30.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.273+0000 7f765f8cc740 1 --2- 172.21.15.105:0/1652832542 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x5631d7a6e440 0x5631d7a6d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:30.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.273+0000 7f765f8cc740 4 client.15180 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T02:02:30.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.273+0000 7f765f8cc740 1 -- 172.21.15.105:0/1652832542 --> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] -- command(tid 0: {"prefix": "status"}) -- 0x5631d794db10 con 0x5631d7a6e440 2024-09-16T02:02:30.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.273+0000 7f76567fc640 1 --2- 172.21.15.105:0/1652832542 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x5631d7a6e440 0x5631d7a6d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:30.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.273+0000 7f763e7fc640 10 client.15180 ms_handle_connect on v2:172.21.15.190:6836/3181257549 2024-09-16T02:02:30.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.273+0000 7f76567fc640 1 --2- 172.21.15.105:0/1652832542 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x5631d7a6e440 0x5631d7a6d150 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=55ad017bcbc64053 server_cookie=4f9687d58906dc73 in_seq=0 out_seq=0 2024-09-16T02:02:30.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.274+0000 7f763e7fc640 1 -- 172.21.15.105:0/1652832542 <== mds.3 v2:172.21.15.190:6836/3181257549 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5631d794db10 con 0x5631d7a6e440 2024-09-16T02:02:30.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.274+0000 7f763e7fc640 10 client.15180 handle_command_reply: tid=0 2024-09-16T02:02:30.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.274+0000 7f765f8cc740 1 client.15180 shutdown 2024-09-16T02:02:30.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.274+0000 7f765f8cc740 1 -- 172.21.15.105:0/1652832542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631d79df870 msgr2=0x5631d7a09400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:30.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.274+0000 7f765f8cc740 1 --2- 172.21.15.105:0/1652832542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631d79df870 0x5631d7a09400 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f764802b0a0 tx=0x7f7648003e40 comp rx=0 tx=0).stop 2024-09-16T02:02:30.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.274+0000 7f765f8cc740 1 -- 172.21.15.105:0/1652832542 shutdown_connections 2024-09-16T02:02:30.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.274+0000 7f765f8cc740 1 --2- 172.21.15.105:0/1652832542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631d79e0190 0x5631d79e8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:30.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.274+0000 7f765f8cc740 1 --2- 172.21.15.105:0/1652832542 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x5631d7a6e440 0x5631d7a6d150 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:30.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.274+0000 7f765f8cc740 1 --2- 172.21.15.105:0/1652832542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631d79eefc0 0x5631d79e8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:30.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.274+0000 7f765f8cc740 1 --2- 172.21.15.105:0/1652832542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631d79df870 0x5631d7a09400 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:30.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.274+0000 7f765f8cc740 1 -- 172.21.15.105:0/1652832542 >> 172.21.15.105:0/1652832542 conn(0x5631d79dbf60 msgr2=0x5631d79db000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:30.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.274+0000 7f763e7fc640 0 client.15180 ms_handle_reset on v2:172.21.15.190:6836/3181257549 2024-09-16T02:02:30.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.274+0000 7f765f8cc740 1 -- 172.21.15.105:0/1652832542 shutdown_connections 2024-09-16T02:02:30.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.274+0000 7f765f8cc740 1 -- 172.21.15.105:0/1652832542 wait complete. 2024-09-16T02:02:30.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.274+0000 7f765f8cc740 20 client.15180 trim_cache size 0 max 16384 2024-09-16T02:02:30.285 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-16T02:02:30.285 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "423b7a2a-73ce-11ef-bceb-c7b262605968", 2024-09-16T02:02:30.286 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 3, 2024-09-16T02:02:30.286 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 24457, 2024-09-16T02:02:30.286 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2024-09-16T02:02:30.286 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2024-09-16T02:02:30.286 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2024-09-16T02:02:30.286 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 104.76556961599999, 2024-09-16T02:02:30.286 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 27, 2024-09-16T02:02:30.286 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 81, 2024-09-16T02:02:30.286 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 79, 2024-09-16T02:02:30.286 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 140.40946516899999 2024-09-16T02:02:30.286 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-16T02:02:30.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.275+0000 7f765d4f3640 1 -- 172.21.15.105:0/3630177158 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f76440a3240 msgr2=0x7f76440a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:30.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.275+0000 7f765d4f3640 1 --2- 172.21.15.105:0/3630177158 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f76440a3240 0x7f76440a5700 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7648009430 tx=0x7f7648005ef0 comp rx=0 tx=0).stop 2024-09-16T02:02:30.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.275+0000 7f765d4f3640 1 -- 172.21.15.105:0/3630177158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76581036e0 msgr2=0x7f7658198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:30.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.275+0000 7f765d4f3640 1 --2- 172.21.15.105:0/3630177158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76581036e0 0x7f7658198ad0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7640006fd0 tx=0x7f764000b8e0 comp rx=0 tx=0).stop 2024-09-16T02:02:30.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.275+0000 7f765d4f3640 1 -- 172.21.15.105:0/3630177158 shutdown_connections 2024-09-16T02:02:30.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.275+0000 7f765d4f3640 1 --2- 172.21.15.105:0/3630177158 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f76440a3240 0x7f76440a5700 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:30.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.275+0000 7f765d4f3640 1 --2- 172.21.15.105:0/3630177158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76581082a0 0x7f765819ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:30.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.275+0000 7f765d4f3640 1 --2- 172.21.15.105:0/3630177158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76581036e0 0x7f7658198ad0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:30.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.275+0000 7f765d4f3640 1 --2- 172.21.15.105:0/3630177158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7658102d30 0x7f7658198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:30.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.275+0000 7f765d4f3640 1 -- 172.21.15.105:0/3630177158 >> 172.21.15.105:0/3630177158 conn(0x7f76580fc7d0 msgr2=0x7f7658105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:30.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.275+0000 7f765d4f3640 1 -- 172.21.15.105:0/3630177158 shutdown_connections 2024-09-16T02:02:30.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:30.275+0000 7f765d4f3640 1 -- 172.21.15.105:0/3630177158 wait complete. 2024-09-16T02:02:30.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:30 smithi105 ceph-mon[31197]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-16T02:02:30.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:30 smithi190 ceph-mon[34762]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-16T02:02:30.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:30 smithi137 ceph-mon[34099]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-16T02:02:31.054 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-16T02:02:31.298 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:02:31.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.790+0000 7fe60164f640 1 -- 172.21.15.105:0/3691542163 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5fc101420 msgr2=0x7fe5fc10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:31.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.790+0000 7fe60164f640 1 --2- 172.21.15.105:0/3691542163 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5fc101420 0x7fe5fc10d950 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4009cd0 tx=0x7fe5f402a8e0 comp rx=0 tx=0).stop 2024-09-16T02:02:31.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.790+0000 7fe60164f640 1 -- 172.21.15.105:0/3691542163 shutdown_connections 2024-09-16T02:02:31.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.790+0000 7fe60164f640 1 --2- 172.21.15.105:0/3691542163 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5fc101420 0x7fe5fc10d950 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:31.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.790+0000 7fe60164f640 1 --2- 172.21.15.105:0/3691542163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5fc100a80 0x7fe5fc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:31.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.790+0000 7fe60164f640 1 --2- 172.21.15.105:0/3691542163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5fc106a80 0x7fe5fc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:31.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.790+0000 7fe60164f640 1 -- 172.21.15.105:0/3691542163 >> 172.21.15.105:0/3691542163 conn(0x7fe5fc0fc7a0 msgr2=0x7fe5fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:31.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.790+0000 7fe60164f640 1 -- 172.21.15.105:0/3691542163 shutdown_connections 2024-09-16T02:02:31.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.790+0000 7fe60164f640 1 -- 172.21.15.105:0/3691542163 wait complete. 2024-09-16T02:02:31.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.791+0000 7fe60164f640 1 Processor -- start 2024-09-16T02:02:31.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.791+0000 7fe60164f640 1 -- start start 2024-09-16T02:02:31.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.791+0000 7fe60164f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5fc100a80 0x7fe5fc199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:31.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.791+0000 7fe60164f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5fc101420 0x7fe5fc1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:31.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.791+0000 7fe60164f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5fc106a80 0x7fe5fc1a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:31.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.791+0000 7fe60164f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe5fc110d70 con 0x7fe5fc106a80 2024-09-16T02:02:31.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.791+0000 7fe60164f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe5fc110bf0 con 0x7fe5fc100a80 2024-09-16T02:02:31.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.791+0000 7fe60164f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe5fc110ef0 con 0x7fe5fc101420 2024-09-16T02:02:31.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.791+0000 7fe5fb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5fc106a80 0x7fe5fc1a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:31.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.791+0000 7fe5fb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5fc106a80 0x7fe5fc1a42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37904/0 (socket says 172.21.15.105:37904) 2024-09-16T02:02:31.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.791+0000 7fe5fb7fe640 1 -- 172.21.15.105:0/865223313 learned_addr learned my addr 172.21.15.105:0/865223313 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:31.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.792+0000 7fe5fb7fe640 1 -- 172.21.15.105:0/865223313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5fc101420 msgr2=0x7fe5fc1997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:31.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.792+0000 7fe5fb7fe640 1 --2- 172.21.15.105:0/865223313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5fc101420 0x7fe5fc1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:31.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.792+0000 7fe5fb7fe640 1 -- 172.21.15.105:0/865223313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5fc100a80 msgr2=0x7fe5fc199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:31.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.792+0000 7fe5fb7fe640 1 --2- 172.21.15.105:0/865223313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5fc100a80 0x7fe5fc199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:31.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.792+0000 7fe5fb7fe640 1 -- 172.21.15.105:0/865223313 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5fc1a4820 con 0x7fe5fc106a80 2024-09-16T02:02:31.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.792+0000 7fe5fb7fe640 1 --2- 172.21.15.105:0/865223313 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5fc106a80 0x7fe5fc1a42e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4005cc0 tx=0x7fe5f4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:31.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.792+0000 7fe5e3fff640 1 -- 172.21.15.105:0/865223313 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f4004240 con 0x7fe5fc106a80 2024-09-16T02:02:31.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.792+0000 7fe5e3fff640 1 -- 172.21.15.105:0/865223313 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5f40305a0 con 0x7fe5fc106a80 2024-09-16T02:02:31.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.792+0000 7fe5e3fff640 1 -- 172.21.15.105:0/865223313 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f40466c0 con 0x7fe5fc106a80 2024-09-16T02:02:31.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.792+0000 7fe60164f640 1 -- 172.21.15.105:0/865223313 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5fc1a49f0 con 0x7fe5fc106a80 2024-09-16T02:02:31.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.792+0000 7fe60164f640 1 -- 172.21.15.105:0/865223313 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5fc1a4e50 con 0x7fe5fc106a80 2024-09-16T02:02:31.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.795+0000 7fe5e3fff640 1 -- 172.21.15.105:0/865223313 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fe5f4030ac0 con 0x7fe5fc106a80 2024-09-16T02:02:31.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.795+0000 7fe603a28740 1 Processor -- start 2024-09-16T02:02:31.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.795+0000 7fe603a28740 1 -- start start 2024-09-16T02:02:31.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.795+0000 7fe603a28740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d95e19d0 0x55a9d95e1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:31.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.795+0000 7fe603a28740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9d95e22f0 0x55a9d95ec670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:31.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.795+0000 7fe603a28740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9d95ecbb0 0x55a9d95eefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:31.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.795+0000 7fe603a28740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a9d9548090 con 0x55a9d95e22f0 2024-09-16T02:02:31.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.795+0000 7fe603a28740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a9d94efa70 con 0x55a9d95ecbb0 2024-09-16T02:02:31.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.795+0000 7fe603a28740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a9d9544050 con 0x55a9d95e19d0 2024-09-16T02:02:31.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.795+0000 7fe5faffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9d95e22f0 0x55a9d95ec670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:31.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.795+0000 7fe5faffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9d95e22f0 0x55a9d95ec670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37916/0 (socket says 172.21.15.105:37916) 2024-09-16T02:02:31.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.795+0000 7fe5faffd640 1 -- 172.21.15.105:0/832135733 learned_addr learned my addr 172.21.15.105:0/832135733 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:31.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.796+0000 7fe5fa7fc640 1 --2- 172.21.15.105:0/832135733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d95e19d0 0x55a9d95e1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:31.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.796+0000 7fe5faffd640 1 -- 172.21.15.105:0/832135733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d95e19d0 msgr2=0x55a9d95e1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:31.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.796+0000 7fe5e3fff640 1 --2- 172.21.15.105:0/865223313 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe5e40a32c0 0x7fe5e40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:31.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.796+0000 7fe5faffd640 1 --2- 172.21.15.105:0/832135733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d95e19d0 0x55a9d95e1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:31.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.796+0000 7fe5faffd640 1 -- 172.21.15.105:0/832135733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9d95ecbb0 msgr2=0x55a9d95eefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:31.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.796+0000 7fe5faffd640 1 --2- 172.21.15.105:0/832135733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9d95ecbb0 0x55a9d95eefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:31.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.796+0000 7fe5faffd640 1 -- 172.21.15.105:0/832135733 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9d94fc130 con 0x55a9d95e22f0 2024-09-16T02:02:31.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.796+0000 7fe5fa7fc640 1 --2- 172.21.15.105:0/832135733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d95e19d0 0x55a9d95e1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:02:31.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.796+0000 7fe5fb7fe640 1 --2- 172.21.15.105:0/865223313 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe5e40a32c0 0x7fe5e40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:31.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.796+0000 7fe5faffd640 1 --2- 172.21.15.105:0/832135733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9d95e22f0 0x55a9d95ec670 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec00a9f0 tx=0x7fe5ec00aeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f47d9812f1440cd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:31.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.796+0000 7fe5e3fff640 1 -- 172.21.15.105:0/865223313 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fe5f402e030 con 0x7fe5fc106a80 2024-09-16T02:02:31.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.796+0000 7fe5e17fa640 1 -- 172.21.15.105:0/832135733 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec00cde0 con 0x55a9d95e22f0 2024-09-16T02:02:31.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.796+0000 7fe5e17fa640 1 -- 172.21.15.105:0/832135733 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5ec00f900 con 0x55a9d95e22f0 2024-09-16T02:02:31.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.796+0000 7fe5e17fa640 1 -- 172.21.15.105:0/832135733 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec00e8b0 con 0x55a9d95e22f0 2024-09-16T02:02:31.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.797+0000 7fe603a28740 1 -- 172.21.15.105:0/832135733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9d95e22f0 msgr2=0x55a9d95ec670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:31.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.797+0000 7fe5fb7fe640 1 --2- 172.21.15.105:0/865223313 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe5e40a32c0 0x7fe5e40a5780 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe5f40b8000 tx=0x7fe5f40903e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:31.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.797+0000 7fe603a28740 1 --2- 172.21.15.105:0/832135733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9d95e22f0 0x55a9d95ec670 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec00a9f0 tx=0x7fe5ec00aeb0 comp rx=0 tx=0).stop 2024-09-16T02:02:31.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.797+0000 7fe603a28740 1 -- 172.21.15.105:0/832135733 shutdown_connections 2024-09-16T02:02:31.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.797+0000 7fe603a28740 1 --2- 172.21.15.105:0/832135733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9d95ecbb0 0x55a9d95eefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:31.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.797+0000 7fe603a28740 1 --2- 172.21.15.105:0/832135733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9d95e22f0 0x55a9d95ec670 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:31.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.797+0000 7fe603a28740 1 --2- 172.21.15.105:0/832135733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d95e19d0 0x55a9d95e1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:31.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.797+0000 7fe603a28740 1 -- 172.21.15.105:0/832135733 >> 172.21.15.105:0/832135733 conn(0x55a9d95ddf60 msgr2=0x55a9d95de340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:31.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.797+0000 7fe603a28740 1 -- 172.21.15.105:0/832135733 shutdown_connections 2024-09-16T02:02:31.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.797+0000 7fe603a28740 1 -- 172.21.15.105:0/832135733 wait complete. 2024-09-16T02:02:31.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.797+0000 7fe603a28740 1 -- start start 2024-09-16T02:02:31.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.797+0000 7fe603a28740 1 Processor -- start 2024-09-16T02:02:31.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.798+0000 7fe603a28740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9d95e19d0 0x55a9d95ec360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:31.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.798+0000 7fe603a28740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9d95e22f0 0x55a9d95e96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:31.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.798+0000 7fe603a28740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d95ecbb0 0x55a9d95e9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:31.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.798+0000 7fe603a28740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a9d95ef780 con 0x55a9d95e22f0 2024-09-16T02:02:31.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.798+0000 7fe603a28740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a9d9502f20 con 0x55a9d95e19d0 2024-09-16T02:02:31.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.798+0000 7fe603a28740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a9d95ef900 con 0x55a9d95ecbb0 2024-09-16T02:02:31.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.798+0000 7fe603a28740 10 client.0 _ll_register_callbacks cb 0x55a9d94cc030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:02:31.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.798+0000 7fe5faffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9d95e22f0 0x55a9d95e96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:31.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.798+0000 7fe5faffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9d95e22f0 0x55a9d95e96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37926/0 (socket says 172.21.15.105:37926) 2024-09-16T02:02:31.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.798+0000 7fe5faffd640 1 -- 172.21.15.105:0/3909700052 learned_addr learned my addr 172.21.15.105:0/3909700052 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:31.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.798+0000 7fe5fa7fc640 1 --2- 172.21.15.105:0/3909700052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d95ecbb0 0x55a9d95e9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:31.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.798+0000 7fe5fa7fc640 1 --2- 172.21.15.105:0/3909700052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9d95e19d0 0x55a9d95ec360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:31.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.798+0000 7fe5faffd640 1 -- 172.21.15.105:0/3909700052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d95ecbb0 msgr2=0x55a9d95e9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:31.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.798+0000 7fe5faffd640 1 --2- 172.21.15.105:0/3909700052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d95ecbb0 0x55a9d95e9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:31.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.798+0000 7fe5faffd640 1 -- 172.21.15.105:0/3909700052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9d95e19d0 msgr2=0x55a9d95ec360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:31.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.798+0000 7fe5faffd640 1 --2- 172.21.15.105:0/3909700052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9d95e19d0 0x55a9d95ec360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:31.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.798+0000 7fe5faffd640 1 -- 172.21.15.105:0/3909700052 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9d92b92b0 con 0x55a9d95e22f0 2024-09-16T02:02:31.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.798+0000 7fe5faffd640 1 --2- 172.21.15.105:0/3909700052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9d95e22f0 0x55a9d95e96b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec00f680 tx=0x7fe5ec00f6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7baa1d1fcc3f5f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:31.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.798+0000 7fe5deff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:02:31.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.798+0000 7fe5deff5640 1 -- 172.21.15.105:0/3909700052 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec028060 con 0x55a9d95e22f0 2024-09-16T02:02:31.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.798+0000 7fe5deff5640 1 -- 172.21.15.105:0/3909700052 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fe5ec00fcc0 con 0x55a9d95e22f0 2024-09-16T02:02:31.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.798+0000 7fe603a28740 1 -- 172.21.15.105:0/3909700052 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5ec00f900 con 0x55a9d95e22f0 2024-09-16T02:02:31.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.799+0000 7fe5deff5640 1 -- 172.21.15.105:0/3909700052 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe5ec00e5f0 con 0x55a9d95e22f0 2024-09-16T02:02:31.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.799+0000 7fe603a28740 10 client.15192 fetch_fsmap learned FSMap version 27 2024-09-16T02:02:31.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.799+0000 7fe603a28740 1 -- 172.21.15.105:0/3909700052 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a9d95fafc0 con 0x55a9d95e22f0 2024-09-16T02:02:31.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.799+0000 7fe5deff5640 1 -- 172.21.15.105:0/3909700052 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5ec00f900 con 0x55a9d95e22f0 2024-09-16T02:02:31.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.799+0000 7fe5deff5640 1 -- 172.21.15.105:0/3909700052 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe5ec02e700 con 0x55a9d95e22f0 2024-09-16T02:02:31.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.799+0000 7fe603a28740 10 client.15192 fetch_fsmap finished waiting for FSMap version 27 2024-09-16T02:02:31.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.799+0000 7fe603a28740 10 client.15192 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-16T02:02:31.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.799+0000 7fe603a28740 20 client.15192 populate_metadata read hostname 'smithi105' 2024-09-16T02:02:31.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.799+0000 7fe603a28740 1 --2- 172.21.15.105:0/3909700052 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x55a9d95fd290 0x55a9d95fe700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:31.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.799+0000 7fe603a28740 4 client.15192 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:02:31.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.799+0000 7fe603a28740 1 -- 172.21.15.105:0/3909700052 --> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9d9555600 con 0x55a9d95fd290 2024-09-16T02:02:31.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.799+0000 7fe5fa7fc640 1 --2- 172.21.15.105:0/3909700052 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x55a9d95fd290 0x55a9d95fe700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:31.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.800+0000 7fe5deff5640 10 client.15192 ms_handle_connect on v2:172.21.15.137:6836/2788541030 2024-09-16T02:02:31.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.800+0000 7fe5fa7fc640 1 --2- 172.21.15.105:0/3909700052 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x55a9d95fd290 0x55a9d95fe700 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=72a5300cab5806f4 server_cookie=9a077b58da4a9c44 in_seq=0 out_seq=0 2024-09-16T02:02:31.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.801+0000 7fe5deff5640 1 -- 172.21.15.105:0/3909700052 <== mds.2 v2:172.21.15.137:6836/2788541030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a9d9555600 con 0x55a9d95fd290 2024-09-16T02:02:31.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.801+0000 7fe5deff5640 10 client.15192 handle_command_reply: tid=0 2024-09-16T02:02:31.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.801+0000 7fe603a28740 1 client.15192 shutdown 2024-09-16T02:02:31.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.801+0000 7fe603a28740 1 -- 172.21.15.105:0/3909700052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9d95e22f0 msgr2=0x55a9d95e96b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:31.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.801+0000 7fe603a28740 1 --2- 172.21.15.105:0/3909700052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9d95e22f0 0x55a9d95e96b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec00f680 tx=0x7fe5ec00f6b0 comp rx=0 tx=0).stop 2024-09-16T02:02:31.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.801+0000 7fe603a28740 1 -- 172.21.15.105:0/3909700052 shutdown_connections 2024-09-16T02:02:31.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.801+0000 7fe603a28740 1 --2- 172.21.15.105:0/3909700052 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x55a9d95fd290 0x55a9d95fe700 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:31.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.801+0000 7fe603a28740 1 --2- 172.21.15.105:0/3909700052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d95ecbb0 0x55a9d95e9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:31.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.801+0000 7fe5deff5640 0 client.15192 ms_handle_reset on v2:172.21.15.137:6836/2788541030 2024-09-16T02:02:31.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.801+0000 7fe603a28740 1 --2- 172.21.15.105:0/3909700052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9d95e22f0 0x55a9d95e96b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:31.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.801+0000 7fe603a28740 1 --2- 172.21.15.105:0/3909700052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9d95e19d0 0x55a9d95ec360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:31.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.801+0000 7fe603a28740 1 -- 172.21.15.105:0/3909700052 >> 172.21.15.105:0/3909700052 conn(0x55a9d95ddf60 msgr2=0x55a9d95e3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:31.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.801+0000 7fe603a28740 1 -- 172.21.15.105:0/3909700052 shutdown_connections 2024-09-16T02:02:31.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.801+0000 7fe603a28740 1 -- 172.21.15.105:0/3909700052 wait complete. 2024-09-16T02:02:31.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.801+0000 7fe603a28740 20 client.15192 trim_cache size 0 max 16384 2024-09-16T02:02:31.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.818+0000 7fe603a28740 1 Processor -- start 2024-09-16T02:02:31.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.818+0000 7fe603a28740 1 -- start start 2024-09-16T02:02:31.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.819+0000 7fe603a28740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9d95e19d0 0x55a9d95fcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:31.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.819+0000 7fe603a28740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9d95e22f0 0x55a9d95eb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:31.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.819+0000 7fe603a28740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d95ecbb0 0x55a9d96069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:31.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.819+0000 7fe603a28740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a9d95fca40 con 0x55a9d95e22f0 2024-09-16T02:02:31.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.819+0000 7fe603a28740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a9d95fd110 con 0x55a9d95e19d0 2024-09-16T02:02:31.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.819+0000 7fe603a28740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a9d95fd7e0 con 0x55a9d95ecbb0 2024-09-16T02:02:31.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.819+0000 7fe5faffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9d95e22f0 0x55a9d95eb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:31.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.819+0000 7fe5faffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9d95e22f0 0x55a9d95eb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37942/0 (socket says 172.21.15.105:37942) 2024-09-16T02:02:31.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.819+0000 7fe5faffd640 1 -- 172.21.15.105:0/4195785206 learned_addr learned my addr 172.21.15.105:0/4195785206 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:31.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.819+0000 7fe5fa7fc640 1 --2- 172.21.15.105:0/4195785206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9d95e19d0 0x55a9d95fcbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:31.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.819+0000 7fe5faffd640 1 -- 172.21.15.105:0/4195785206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d95ecbb0 msgr2=0x55a9d96069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:31.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.819+0000 7fe5faffd640 1 --2- 172.21.15.105:0/4195785206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d95ecbb0 0x55a9d96069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:31.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.819+0000 7fe5faffd640 1 -- 172.21.15.105:0/4195785206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9d95e19d0 msgr2=0x55a9d95fcbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:31.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.819+0000 7fe5faffd640 1 --2- 172.21.15.105:0/4195785206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9d95e19d0 0x55a9d95fcbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:31.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.819+0000 7fe5faffd640 1 -- 172.21.15.105:0/4195785206 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9d95e2120 con 0x55a9d95e22f0 2024-09-16T02:02:31.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.819+0000 7fe5fa7fc640 1 --2- 172.21.15.105:0/4195785206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9d95e19d0 0x55a9d95fcbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:02:31.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.819+0000 7fe5faffd640 1 --2- 172.21.15.105:0/4195785206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9d95e22f0 0x55a9d95eb620 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec018420 tx=0x7fe5ec00a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5c7a05c32ca925 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:31.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.819+0000 7fe5e0ff9640 1 -- 172.21.15.105:0/4195785206 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec00a410 con 0x55a9d95e22f0 2024-09-16T02:02:31.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.819+0000 7fe5e0ff9640 1 -- 172.21.15.105:0/4195785206 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5ec00f720 con 0x55a9d95e22f0 2024-09-16T02:02:31.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.819+0000 7fe5e0ff9640 1 -- 172.21.15.105:0/4195785206 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec00e790 con 0x55a9d95e22f0 2024-09-16T02:02:31.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.820+0000 7fe603a28740 1 -- 172.21.15.105:0/4195785206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9d95e22f0 msgr2=0x55a9d95eb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:31.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.820+0000 7fe603a28740 1 --2- 172.21.15.105:0/4195785206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9d95e22f0 0x55a9d95eb620 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec018420 tx=0x7fe5ec00a300 comp rx=0 tx=0).stop 2024-09-16T02:02:31.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.820+0000 7fe603a28740 1 -- 172.21.15.105:0/4195785206 shutdown_connections 2024-09-16T02:02:31.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.820+0000 7fe603a28740 1 --2- 172.21.15.105:0/4195785206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d95ecbb0 0x55a9d96069e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:31.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.820+0000 7fe603a28740 1 --2- 172.21.15.105:0/4195785206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9d95e22f0 0x55a9d95eb620 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:31.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.820+0000 7fe603a28740 1 --2- 172.21.15.105:0/4195785206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9d95e19d0 0x55a9d95fcbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:31.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.820+0000 7fe603a28740 1 -- 172.21.15.105:0/4195785206 >> 172.21.15.105:0/4195785206 conn(0x55a9d95ddf60 msgr2=0x55a9d95de340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:31.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.820+0000 7fe603a28740 1 -- 172.21.15.105:0/4195785206 shutdown_connections 2024-09-16T02:02:31.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.820+0000 7fe603a28740 1 -- 172.21.15.105:0/4195785206 wait complete. 2024-09-16T02:02:31.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.820+0000 7fe603a28740 1 -- start start 2024-09-16T02:02:31.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.820+0000 7fe603a28740 1 Processor -- start 2024-09-16T02:02:31.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.820+0000 7fe603a28740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9d95e19d0 0x55a9d9606ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:31.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.820+0000 7fe603a28740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d95e22f0 0x55a9d9607220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:31.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.820+0000 7fe603a28740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9d95ecbb0 0x55a9d96001a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:31.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.820+0000 7fe603a28740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a9d95fe580 con 0x55a9d95e19d0 2024-09-16T02:02:31.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.820+0000 7fe603a28740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a9d95fdeb0 con 0x55a9d95ecbb0 2024-09-16T02:02:31.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.820+0000 7fe603a28740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a9d95fec50 con 0x55a9d95e22f0 2024-09-16T02:02:31.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.821+0000 7fe603a28740 10 client.0 _ll_register_callbacks cb 0x7fe5ec00a840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:02:31.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.821+0000 7fe5fa7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9d95e19d0 0x55a9d9606ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:31.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.821+0000 7fe5fa7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9d95e19d0 0x55a9d9606ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37950/0 (socket says 172.21.15.105:37950) 2024-09-16T02:02:31.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.821+0000 7fe5fa7fc640 1 -- 172.21.15.105:0/3397882829 learned_addr learned my addr 172.21.15.105:0/3397882829 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:31.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.821+0000 7fe5faffd640 1 --2- 172.21.15.105:0/3397882829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d95e22f0 0x55a9d9607220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:31.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.821+0000 7fe5fa7fc640 1 -- 172.21.15.105:0/3397882829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d95e22f0 msgr2=0x55a9d9607220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:31.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.821+0000 7fe5fa7fc640 1 --2- 172.21.15.105:0/3397882829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d95e22f0 0x55a9d9607220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:31.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.821+0000 7fe5fa7fc640 1 -- 172.21.15.105:0/3397882829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9d95ecbb0 msgr2=0x55a9d96001a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:31.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.821+0000 7fe5fa7fc640 1 --2- 172.21.15.105:0/3397882829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9d95ecbb0 0x55a9d96001a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:31.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.821+0000 7fe5fa7fc640 1 -- 172.21.15.105:0/3397882829 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9d96009d0 con 0x55a9d95e19d0 2024-09-16T02:02:31.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.821+0000 7fe5fa7fc640 1 --2- 172.21.15.105:0/3397882829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9d95e19d0 0x55a9d9606ce0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0004bd0 tx=0x7fe5f0001860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d556a08702e7bb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:31.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.821+0000 7fe5deff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:02:31.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.821+0000 7fe5deff5640 1 -- 172.21.15.105:0/3397882829 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f0025060 con 0x55a9d95e19d0 2024-09-16T02:02:31.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.821+0000 7fe5deff5640 1 -- 172.21.15.105:0/3397882829 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fe5f0012040 con 0x55a9d95e19d0 2024-09-16T02:02:31.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.821+0000 7fe603a28740 1 -- 172.21.15.105:0/3397882829 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5ec00f720 con 0x55a9d95e19d0 2024-09-16T02:02:31.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.821+0000 7fe5deff5640 1 -- 172.21.15.105:0/3397882829 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe5f0020690 con 0x55a9d95e19d0 2024-09-16T02:02:31.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.821+0000 7fe603a28740 10 client.15198 fetch_fsmap learned FSMap version 27 2024-09-16T02:02:31.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.821+0000 7fe603a28740 1 -- 172.21.15.105:0/3397882829 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a9d9605430 con 0x55a9d95e19d0 2024-09-16T02:02:31.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.822+0000 7fe5deff5640 1 -- 172.21.15.105:0/3397882829 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5ec00f720 con 0x55a9d95e19d0 2024-09-16T02:02:31.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.822+0000 7fe5deff5640 1 -- 172.21.15.105:0/3397882829 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe5f002a4f0 con 0x55a9d95e19d0 2024-09-16T02:02:31.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.822+0000 7fe603a28740 10 client.15198 fetch_fsmap finished waiting for FSMap version 27 2024-09-16T02:02:31.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.822+0000 7fe603a28740 10 client.15198 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-16T02:02:31.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.822+0000 7fe603a28740 20 client.15198 populate_metadata read hostname 'smithi105' 2024-09-16T02:02:31.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.822+0000 7fe603a28740 1 --2- 172.21.15.105:0/3397882829 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x55a9d966c100 0x55a9d966ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:31.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.822+0000 7fe603a28740 4 client.15198 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T02:02:31.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.822+0000 7fe603a28740 1 -- 172.21.15.105:0/3397882829 --> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] -- command(tid 0: {"prefix": "status"}) -- 0x55a9d954fb10 con 0x55a9d966c100 2024-09-16T02:02:31.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.822+0000 7fe5faffd640 1 --2- 172.21.15.105:0/3397882829 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x55a9d966c100 0x55a9d966ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:31.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.823+0000 7fe5deff5640 10 client.15198 ms_handle_connect on v2:172.21.15.137:6836/2788541030 2024-09-16T02:02:31.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.823+0000 7fe5faffd640 1 --2- 172.21.15.105:0/3397882829 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x55a9d966c100 0x55a9d966ae10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5da0b3f94c8e1942 server_cookie=eabbda98fb2d3e24 in_seq=0 out_seq=0 2024-09-16T02:02:31.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.823+0000 7fe5deff5640 1 -- 172.21.15.105:0/3397882829 <== mds.2 v2:172.21.15.137:6836/2788541030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a9d954fb10 con 0x55a9d966c100 2024-09-16T02:02:31.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.823+0000 7fe5deff5640 10 client.15198 handle_command_reply: tid=0 2024-09-16T02:02:31.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.823+0000 7fe603a28740 1 client.15198 shutdown 2024-09-16T02:02:31.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.823+0000 7fe603a28740 1 -- 172.21.15.105:0/3397882829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9d95e19d0 msgr2=0x55a9d9606ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:31.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.823+0000 7fe603a28740 1 --2- 172.21.15.105:0/3397882829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9d95e19d0 0x55a9d9606ce0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0004bd0 tx=0x7fe5f0001860 comp rx=0 tx=0).stop 2024-09-16T02:02:31.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.823+0000 7fe603a28740 1 -- 172.21.15.105:0/3397882829 shutdown_connections 2024-09-16T02:02:31.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.823+0000 7fe603a28740 1 --2- 172.21.15.105:0/3397882829 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x55a9d966c100 0x55a9d966ae10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:31.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.823+0000 7fe603a28740 1 --2- 172.21.15.105:0/3397882829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d95e22f0 0x55a9d9607220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:31.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.823+0000 7fe603a28740 1 --2- 172.21.15.105:0/3397882829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9d95ecbb0 0x55a9d96001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:31.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.823+0000 7fe603a28740 1 --2- 172.21.15.105:0/3397882829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9d95e19d0 0x55a9d9606ce0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:31.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.823+0000 7fe5deff5640 0 client.15198 ms_handle_reset on v2:172.21.15.137:6836/2788541030 2024-09-16T02:02:31.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.823+0000 7fe603a28740 1 -- 172.21.15.105:0/3397882829 >> 172.21.15.105:0/3397882829 conn(0x55a9d95ddf60 msgr2=0x55a9d95dd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:31.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.823+0000 7fe603a28740 1 -- 172.21.15.105:0/3397882829 shutdown_connections 2024-09-16T02:02:31.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.823+0000 7fe603a28740 1 -- 172.21.15.105:0/3397882829 wait complete. 2024-09-16T02:02:31.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.823+0000 7fe603a28740 20 client.15198 trim_cache size 0 max 16384 2024-09-16T02:02:31.831 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-16T02:02:31.831 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "423b7a2a-73ce-11ef-bceb-c7b262605968", 2024-09-16T02:02:31.832 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 2, 2024-09-16T02:02:31.832 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 24461, 2024-09-16T02:02:31.832 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2024-09-16T02:02:31.832 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2024-09-16T02:02:31.832 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2024-09-16T02:02:31.832 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 107.32681296299999, 2024-09-16T02:02:31.832 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 27, 2024-09-16T02:02:31.832 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 81, 2024-09-16T02:02:31.832 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 79, 2024-09-16T02:02:31.832 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 139.25513721900001 2024-09-16T02:02:31.832 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-16T02:02:31.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.824+0000 7fe60164f640 1 -- 172.21.15.105:0/865223313 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe5e40a32c0 msgr2=0x7fe5e40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:31.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.824+0000 7fe60164f640 1 --2- 172.21.15.105:0/865223313 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe5e40a32c0 0x7fe5e40a5780 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe5f40b8000 tx=0x7fe5f40903e0 comp rx=0 tx=0).stop 2024-09-16T02:02:31.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.824+0000 7fe60164f640 1 -- 172.21.15.105:0/865223313 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5fc106a80 msgr2=0x7fe5fc1a42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:31.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.824+0000 7fe60164f640 1 --2- 172.21.15.105:0/865223313 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5fc106a80 0x7fe5fc1a42e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4005cc0 tx=0x7fe5f4003ba0 comp rx=0 tx=0).stop 2024-09-16T02:02:31.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.824+0000 7fe60164f640 1 -- 172.21.15.105:0/865223313 shutdown_connections 2024-09-16T02:02:31.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.824+0000 7fe60164f640 1 --2- 172.21.15.105:0/865223313 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe5e40a32c0 0x7fe5e40a5780 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:31.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.824+0000 7fe60164f640 1 --2- 172.21.15.105:0/865223313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5fc101420 0x7fe5fc1997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:31.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.824+0000 7fe60164f640 1 --2- 172.21.15.105:0/865223313 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5fc106a80 0x7fe5fc1a42e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:31.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.824+0000 7fe60164f640 1 --2- 172.21.15.105:0/865223313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5fc100a80 0x7fe5fc199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:31.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.824+0000 7fe60164f640 1 -- 172.21.15.105:0/865223313 >> 172.21.15.105:0/865223313 conn(0x7fe5fc0fc7a0 msgr2=0x7fe5fc0fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:31.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.824+0000 7fe60164f640 1 -- 172.21.15.105:0/865223313 shutdown_connections 2024-09-16T02:02:31.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:31.824+0000 7fe60164f640 1 -- 172.21.15.105:0/865223313 wait complete. 2024-09-16T02:02:32.629 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-16T02:02:32.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:32 smithi105 ceph-mon[31197]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T02:02:32.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:32 smithi190 ceph-mon[34762]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T02:02:32.875 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:02:32.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:32 smithi137 ceph-mon[34099]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T02:02:33.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.339+0000 7fbf59b31640 1 -- 172.21.15.105:0/3643243833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf54106ba0 msgr2=0x7fbf54106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:33.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.339+0000 7fbf59b31640 1 --2- 172.21.15.105:0/3643243833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf54106ba0 0x7fbf54106f80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fbf44009900 tx=0x7fbf4402a410 comp rx=0 tx=0).stop 2024-09-16T02:02:33.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.339+0000 7fbf59b31640 1 -- 172.21.15.105:0/3643243833 shutdown_connections 2024-09-16T02:02:33.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.339+0000 7fbf59b31640 1 --2- 172.21.15.105:0/3643243833 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf54100a70 0x7fbf5410ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:33.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.339+0000 7fbf59b31640 1 --2- 172.21.15.105:0/3643243833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf541000d0 0x7fbf54100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:33.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.339+0000 7fbf59b31640 1 --2- 172.21.15.105:0/3643243833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf54106ba0 0x7fbf54106f80 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:33.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.339+0000 7fbf59b31640 1 -- 172.21.15.105:0/3643243833 >> 172.21.15.105:0/3643243833 conn(0x7fbf54077210 msgr2=0x7fbf540fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:33.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.339+0000 7fbf59b31640 1 -- 172.21.15.105:0/3643243833 shutdown_connections 2024-09-16T02:02:33.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.339+0000 7fbf59b31640 1 -- 172.21.15.105:0/3643243833 wait complete. 2024-09-16T02:02:33.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.340+0000 7fbf59b31640 1 Processor -- start 2024-09-16T02:02:33.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.340+0000 7fbf59b31640 1 -- start start 2024-09-16T02:02:33.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.340+0000 7fbf59b31640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf541000d0 0x7fbf541012b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:33.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.340+0000 7fbf59b31640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf54100a70 0x7fbf541017f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:33.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.340+0000 7fbf59b31640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf54106ba0 0x7fbf54101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:33.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.340+0000 7fbf59b31640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbf54111f80 con 0x7fbf54100a70 2024-09-16T02:02:33.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.340+0000 7fbf59b31640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbf54111e00 con 0x7fbf541000d0 2024-09-16T02:02:33.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.340+0000 7fbf59b31640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbf54112100 con 0x7fbf54106ba0 2024-09-16T02:02:33.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.340+0000 7fbf52ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf54100a70 0x7fbf541017f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:33.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.340+0000 7fbf52ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf54100a70 0x7fbf541017f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37972/0 (socket says 172.21.15.105:37972) 2024-09-16T02:02:33.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.340+0000 7fbf52ffd640 1 -- 172.21.15.105:0/3281420907 learned_addr learned my addr 172.21.15.105:0/3281420907 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:33.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.340+0000 7fbf537fe640 1 --2- 172.21.15.105:0/3281420907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf541000d0 0x7fbf541012b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:33.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.340+0000 7fbf53fff640 1 --2- 172.21.15.105:0/3281420907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf54106ba0 0x7fbf54101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:33.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.340+0000 7fbf52ffd640 1 -- 172.21.15.105:0/3281420907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf54106ba0 msgr2=0x7fbf54101d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:33.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.340+0000 7fbf52ffd640 1 --2- 172.21.15.105:0/3281420907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf54106ba0 0x7fbf54101d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:33.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.340+0000 7fbf52ffd640 1 -- 172.21.15.105:0/3281420907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf541000d0 msgr2=0x7fbf541012b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:33.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.340+0000 7fbf52ffd640 1 --2- 172.21.15.105:0/3281420907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf541000d0 0x7fbf541012b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:33.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.340+0000 7fbf52ffd640 1 -- 172.21.15.105:0/3281420907 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf541a6960 con 0x7fbf54100a70 2024-09-16T02:02:33.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.341+0000 7fbf52ffd640 1 --2- 172.21.15.105:0/3281420907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf54100a70 0x7fbf541017f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fbf4800b4a0 tx=0x7fbf4800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:33.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.341+0000 7fbf50ff9640 1 -- 172.21.15.105:0/3281420907 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf48014070 con 0x7fbf54100a70 2024-09-16T02:02:33.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.341+0000 7fbf59b31640 1 -- 172.21.15.105:0/3281420907 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf541a6c50 con 0x7fbf54100a70 2024-09-16T02:02:33.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.341+0000 7fbf50ff9640 1 -- 172.21.15.105:0/3281420907 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf48010ac0 con 0x7fbf54100a70 2024-09-16T02:02:33.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.341+0000 7fbf50ff9640 1 -- 172.21.15.105:0/3281420907 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf48017c30 con 0x7fbf54100a70 2024-09-16T02:02:33.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.341+0000 7fbf59b31640 1 -- 172.21.15.105:0/3281420907 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf541a7190 con 0x7fbf54100a70 2024-09-16T02:02:33.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.344+0000 7fbf50ff9640 1 -- 172.21.15.105:0/3281420907 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fbf48017070 con 0x7fbf54100a70 2024-09-16T02:02:33.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.344+0000 7fbf5bf0a740 1 Processor -- start 2024-09-16T02:02:33.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.344+0000 7fbf5bf0a740 1 -- start start 2024-09-16T02:02:33.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.344+0000 7fbf5bf0a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca586749d0 0x55ca58674db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:33.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.344+0000 7fbf5bf0a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca586752f0 0x55ca5867f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:33.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.344+0000 7fbf5bf0a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca5867fbb0 0x55ca58681fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:33.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.344+0000 7fbf5bf0a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ca585db090 con 0x55ca586749d0 2024-09-16T02:02:33.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.344+0000 7fbf5bf0a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ca58582a70 con 0x55ca586752f0 2024-09-16T02:02:33.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.344+0000 7fbf5bf0a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ca585d7050 con 0x55ca5867fbb0 2024-09-16T02:02:33.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.344+0000 7fbf53fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca586749d0 0x55ca58674db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:33.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.344+0000 7fbf53fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca586749d0 0x55ca58674db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37974/0 (socket says 172.21.15.105:37974) 2024-09-16T02:02:33.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.344+0000 7fbf53fff640 1 -- 172.21.15.105:0/3624555298 learned_addr learned my addr 172.21.15.105:0/3624555298 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:33.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.344+0000 7fbf52ffd640 1 --2- 172.21.15.105:0/3624555298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca5867fbb0 0x55ca58681fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:33.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.344+0000 7fbf537fe640 1 --2- 172.21.15.105:0/3624555298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca586752f0 0x55ca5867f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:33.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.344+0000 7fbf50ff9640 1 --2- 172.21.15.105:0/3281420907 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fbf3c0a3240 0x7fbf3c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:33.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.344+0000 7fbf53fff640 1 -- 172.21.15.105:0/3624555298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca5867fbb0 msgr2=0x55ca58681fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:33.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.344+0000 7fbf53fff640 1 --2- 172.21.15.105:0/3624555298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca5867fbb0 0x55ca58681fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:33.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.344+0000 7fbf53fff640 1 -- 172.21.15.105:0/3624555298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca586752f0 msgr2=0x55ca5867f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:33.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.344+0000 7fbf53fff640 1 --2- 172.21.15.105:0/3624555298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca586752f0 0x55ca5867f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:33.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.345+0000 7fbf53fff640 1 -- 172.21.15.105:0/3624555298 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca5858f130 con 0x55ca586749d0 2024-09-16T02:02:33.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.345+0000 7fbf50ff9640 1 -- 172.21.15.105:0/3281420907 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fbf480cf000 con 0x7fbf54100a70 2024-09-16T02:02:33.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.345+0000 7fbf53fff640 1 --2- 172.21.15.105:0/3281420907 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fbf3c0a3240 0x7fbf3c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:33.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.345+0000 7fbf53fff640 1 --2- 172.21.15.105:0/3624555298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca586749d0 0x55ca58674db0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c009f30 tx=0x7fbf4c00f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c202521d4756de server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:33.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.345+0000 7fbf39ffb640 1 -- 172.21.15.105:0/3624555298 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c01a610 con 0x55ca586749d0 2024-09-16T02:02:33.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.345+0000 7fbf39ffb640 1 -- 172.21.15.105:0/3624555298 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf4c01ac60 con 0x55ca586749d0 2024-09-16T02:02:33.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.345+0000 7fbf39ffb640 1 -- 172.21.15.105:0/3624555298 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c00d9b0 con 0x55ca586749d0 2024-09-16T02:02:33.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.346+0000 7fbf5bf0a740 1 -- 172.21.15.105:0/3624555298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca586749d0 msgr2=0x55ca58674db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:33.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.346+0000 7fbf53fff640 1 --2- 172.21.15.105:0/3281420907 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fbf3c0a3240 0x7fbf3c0a5700 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c020000 tx=0x7fbf4c01d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:33.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.346+0000 7fbf5bf0a740 1 --2- 172.21.15.105:0/3624555298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca586749d0 0x55ca58674db0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c009f30 tx=0x7fbf4c00f7f0 comp rx=0 tx=0).stop 2024-09-16T02:02:33.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.346+0000 7fbf5bf0a740 1 -- 172.21.15.105:0/3624555298 shutdown_connections 2024-09-16T02:02:33.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.346+0000 7fbf5bf0a740 1 --2- 172.21.15.105:0/3624555298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca5867fbb0 0x55ca58681fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:33.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.346+0000 7fbf5bf0a740 1 --2- 172.21.15.105:0/3624555298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca586752f0 0x55ca5867f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:33.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.346+0000 7fbf5bf0a740 1 --2- 172.21.15.105:0/3624555298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca586749d0 0x55ca58674db0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:33.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.346+0000 7fbf5bf0a740 1 -- 172.21.15.105:0/3624555298 >> 172.21.15.105:0/3624555298 conn(0x55ca58670f60 msgr2=0x55ca58671340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:33.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.346+0000 7fbf5bf0a740 1 -- 172.21.15.105:0/3624555298 shutdown_connections 2024-09-16T02:02:33.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.346+0000 7fbf5bf0a740 1 -- 172.21.15.105:0/3624555298 wait complete. 2024-09-16T02:02:33.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.346+0000 7fbf5bf0a740 1 -- start start 2024-09-16T02:02:33.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.346+0000 7fbf5bf0a740 1 Processor -- start 2024-09-16T02:02:33.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.347+0000 7fbf5bf0a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca586749d0 0x55ca5867f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:33.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.347+0000 7fbf5bf0a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca586752f0 0x55ca5867c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.347+0000 7fbf5bf0a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca5867fbb0 0x55ca5867ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.347+0000 7fbf5bf0a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ca58682780 con 0x55ca586749d0 2024-09-16T02:02:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.347+0000 7fbf5bf0a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ca58595f20 con 0x55ca586752f0 2024-09-16T02:02:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.347+0000 7fbf5bf0a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ca58682900 con 0x55ca5867fbb0 2024-09-16T02:02:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.347+0000 7fbf5bf0a740 10 client.0 _ll_register_callbacks cb 0x55ca5855f030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:02:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.347+0000 7fbf537fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca586749d0 0x55ca5867f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.347+0000 7fbf537fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca586749d0 0x55ca5867f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37984/0 (socket says 172.21.15.105:37984) 2024-09-16T02:02:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.347+0000 7fbf537fe640 1 -- 172.21.15.105:0/912519151 learned_addr learned my addr 172.21.15.105:0/912519151 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.347+0000 7fbf52ffd640 1 --2- 172.21.15.105:0/912519151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca586752f0 0x55ca5867c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.347+0000 7fbf537fe640 1 -- 172.21.15.105:0/912519151 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca5867fbb0 msgr2=0x55ca5867ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.347+0000 7fbf537fe640 1 --2- 172.21.15.105:0/912519151 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca5867fbb0 0x55ca5867ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.347+0000 7fbf537fe640 1 -- 172.21.15.105:0/912519151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca586752f0 msgr2=0x55ca5867c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:33.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.347+0000 7fbf537fe640 1 --2- 172.21.15.105:0/912519151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca586752f0 0x55ca5867c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:33.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.347+0000 7fbf537fe640 1 -- 172.21.15.105:0/912519151 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca5834c2b0 con 0x55ca586749d0 2024-09-16T02:02:33.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.347+0000 7fbf537fe640 1 --2- 172.21.15.105:0/912519151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca586749d0 0x55ca5867f440 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbf44009900 tx=0x7fbf440318e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c4e4c609ff3422 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:33.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.347+0000 7fbf32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:02:33.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.347+0000 7fbf32ffd640 1 -- 172.21.15.105:0/912519151 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf44035040 con 0x55ca586749d0 2024-09-16T02:02:33.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.347+0000 7fbf32ffd640 1 -- 172.21.15.105:0/912519151 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fbf44004040 con 0x55ca586749d0 2024-09-16T02:02:33.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.347+0000 7fbf5bf0a740 1 -- 172.21.15.105:0/912519151 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf4c01ac60 con 0x55ca586749d0 2024-09-16T02:02:33.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.348+0000 7fbf32ffd640 1 -- 172.21.15.105:0/912519151 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbf4404a070 con 0x55ca586749d0 2024-09-16T02:02:33.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.348+0000 7fbf5bf0a740 10 client.15210 fetch_fsmap learned FSMap version 27 2024-09-16T02:02:33.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.348+0000 7fbf5bf0a740 1 -- 172.21.15.105:0/912519151 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ca5868e090 con 0x55ca586749d0 2024-09-16T02:02:33.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.348+0000 7fbf32ffd640 1 -- 172.21.15.105:0/912519151 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf4c01ac60 con 0x55ca586749d0 2024-09-16T02:02:33.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.348+0000 7fbf32ffd640 1 -- 172.21.15.105:0/912519151 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbf44004750 con 0x55ca586749d0 2024-09-16T02:02:33.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.348+0000 7fbf5bf0a740 10 client.15210 fetch_fsmap finished waiting for FSMap version 27 2024-09-16T02:02:33.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.348+0000 7fbf5bf0a740 10 client.15210 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-16T02:02:33.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.348+0000 7fbf5bf0a740 20 client.15210 populate_metadata read hostname 'smithi105' 2024-09-16T02:02:33.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.348+0000 7fbf5bf0a740 1 --2- 172.21.15.105:0/912519151 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55ca58690730 0x55ca58691ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:33.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.348+0000 7fbf5bf0a740 4 client.15210 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:02:33.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.348+0000 7fbf5bf0a740 1 -- 172.21.15.105:0/912519151 --> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca585e8600 con 0x55ca58690730 2024-09-16T02:02:33.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.349+0000 7fbf52ffd640 1 --2- 172.21.15.105:0/912519151 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55ca58690730 0x55ca58691ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:33.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.350+0000 7fbf52ffd640 1 --2- 172.21.15.105:0/912519151 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55ca58690730 0x55ca58691ba0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=902f1097431ab1bc server_cookie=f0c3c1a1c4afe9a in_seq=0 out_seq=0 2024-09-16T02:02:33.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.350+0000 7fbf32ffd640 10 client.15210 ms_handle_connect on v2:172.21.15.190:6838/1222573537 2024-09-16T02:02:33.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.352+0000 7fbf32ffd640 1 -- 172.21.15.105:0/912519151 <== mds.1 v2:172.21.15.190:6838/1222573537 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ca585e8600 con 0x55ca58690730 2024-09-16T02:02:33.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.352+0000 7fbf32ffd640 10 client.15210 handle_command_reply: tid=0 2024-09-16T02:02:33.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.352+0000 7fbf5bf0a740 1 client.15210 shutdown 2024-09-16T02:02:33.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.352+0000 7fbf5bf0a740 1 -- 172.21.15.105:0/912519151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca586749d0 msgr2=0x55ca5867f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:33.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.352+0000 7fbf5bf0a740 1 --2- 172.21.15.105:0/912519151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca586749d0 0x55ca5867f440 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbf44009900 tx=0x7fbf440318e0 comp rx=0 tx=0).stop 2024-09-16T02:02:33.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.352+0000 7fbf5bf0a740 1 -- 172.21.15.105:0/912519151 shutdown_connections 2024-09-16T02:02:33.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.352+0000 7fbf5bf0a740 1 --2- 172.21.15.105:0/912519151 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55ca58690730 0x55ca58691ba0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:33.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.352+0000 7fbf5bf0a740 1 --2- 172.21.15.105:0/912519151 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca5867fbb0 0x55ca5867ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:33.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.352+0000 7fbf5bf0a740 1 --2- 172.21.15.105:0/912519151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca586752f0 0x55ca5867c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:33.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.352+0000 7fbf5bf0a740 1 --2- 172.21.15.105:0/912519151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca586749d0 0x55ca5867f440 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.352+0000 7fbf5bf0a740 1 -- 172.21.15.105:0/912519151 >> 172.21.15.105:0/912519151 conn(0x55ca58670f60 msgr2=0x55ca58676c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.352+0000 7fbf32ffd640 0 client.15210 ms_handle_reset on v2:172.21.15.190:6838/1222573537 2024-09-16T02:02:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.352+0000 7fbf5bf0a740 1 -- 172.21.15.105:0/912519151 shutdown_connections 2024-09-16T02:02:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.352+0000 7fbf5bf0a740 1 -- 172.21.15.105:0/912519151 wait complete. 2024-09-16T02:02:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.352+0000 7fbf5bf0a740 20 client.15210 trim_cache size 0 max 16384 2024-09-16T02:02:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.370+0000 7fbf5bf0a740 1 Processor -- start 2024-09-16T02:02:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.370+0000 7fbf5bf0a740 1 -- start start 2024-09-16T02:02:33.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.370+0000 7fbf5bf0a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca586749d0 0x55ca58690e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:33.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.370+0000 7fbf5bf0a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca586752f0 0x55ca5866fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:33.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.370+0000 7fbf5bf0a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca5867fbb0 0x55ca58670430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:33.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.370+0000 7fbf5bf0a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ca5868fee0 con 0x55ca586752f0 2024-09-16T02:02:33.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.370+0000 7fbf5bf0a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ca586905b0 con 0x55ca5867fbb0 2024-09-16T02:02:33.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.370+0000 7fbf5bf0a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ca58690c80 con 0x55ca586749d0 2024-09-16T02:02:33.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.370+0000 7fbf52ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca586752f0 0x55ca5866fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:33.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.370+0000 7fbf52ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca586752f0 0x55ca5866fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37986/0 (socket says 172.21.15.105:37986) 2024-09-16T02:02:33.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.370+0000 7fbf52ffd640 1 -- 172.21.15.105:0/978390024 learned_addr learned my addr 172.21.15.105:0/978390024 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:33.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.370+0000 7fbf53fff640 1 --2- 172.21.15.105:0/978390024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca5867fbb0 0x55ca58670430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:33.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.370+0000 7fbf52ffd640 1 -- 172.21.15.105:0/978390024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca586749d0 msgr2=0x55ca58690e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:33.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.370+0000 7fbf52ffd640 1 --2- 172.21.15.105:0/978390024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca586749d0 0x55ca58690e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:33.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.370+0000 7fbf52ffd640 1 -- 172.21.15.105:0/978390024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca5867fbb0 msgr2=0x55ca58670430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:33.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.370+0000 7fbf52ffd640 1 --2- 172.21.15.105:0/978390024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca5867fbb0 0x55ca58670430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:33.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.370+0000 7fbf52ffd640 1 -- 172.21.15.105:0/978390024 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca5867d550 con 0x55ca586752f0 2024-09-16T02:02:33.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.370+0000 7fbf53fff640 1 --2- 172.21.15.105:0/978390024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca5867fbb0 0x55ca58670430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:02:33.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.371+0000 7fbf52ffd640 1 --2- 172.21.15.105:0/978390024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca586752f0 0x55ca5866fef0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fbf48004700 tx=0x7fbf48097a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=301cfb5f3d02fb1c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:33.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.371+0000 7fbf397fa640 1 -- 172.21.15.105:0/978390024 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4801e040 con 0x55ca586752f0 2024-09-16T02:02:33.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.371+0000 7fbf397fa640 1 -- 172.21.15.105:0/978390024 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf4809fa50 con 0x55ca586752f0 2024-09-16T02:02:33.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.371+0000 7fbf397fa640 1 -- 172.21.15.105:0/978390024 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf48097d10 con 0x55ca586752f0 2024-09-16T02:02:33.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.371+0000 7fbf5bf0a740 1 -- 172.21.15.105:0/978390024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca586752f0 msgr2=0x55ca5866fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:33.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.371+0000 7fbf5bf0a740 1 --2- 172.21.15.105:0/978390024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca586752f0 0x55ca5866fef0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fbf48004700 tx=0x7fbf48097a70 comp rx=0 tx=0).stop 2024-09-16T02:02:33.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.371+0000 7fbf5bf0a740 1 -- 172.21.15.105:0/978390024 shutdown_connections 2024-09-16T02:02:33.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.371+0000 7fbf5bf0a740 1 --2- 172.21.15.105:0/978390024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca5867fbb0 0x55ca58670430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:33.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.371+0000 7fbf5bf0a740 1 --2- 172.21.15.105:0/978390024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca586752f0 0x55ca5866fef0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:33.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.371+0000 7fbf5bf0a740 1 --2- 172.21.15.105:0/978390024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca586749d0 0x55ca58690e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:33.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.371+0000 7fbf5bf0a740 1 -- 172.21.15.105:0/978390024 >> 172.21.15.105:0/978390024 conn(0x55ca58670f60 msgr2=0x55ca5866f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:33.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.371+0000 7fbf5bf0a740 1 -- 172.21.15.105:0/978390024 shutdown_connections 2024-09-16T02:02:33.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.371+0000 7fbf5bf0a740 1 -- 172.21.15.105:0/978390024 wait complete. 2024-09-16T02:02:33.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.372+0000 7fbf5bf0a740 1 -- start start 2024-09-16T02:02:33.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.372+0000 7fbf5bf0a740 1 Processor -- start 2024-09-16T02:02:33.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.372+0000 7fbf5bf0a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca586749d0 0x55ca58696b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:33.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.372+0000 7fbf5bf0a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca586752f0 0x55ca586970a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:33.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.372+0000 7fbf5bf0a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca5867fbb0 0x55ca586975e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:33.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.372+0000 7fbf5bf0a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ca58691a20 con 0x55ca5867fbb0 2024-09-16T02:02:33.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.372+0000 7fbf5bf0a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ca58691350 con 0x55ca586749d0 2024-09-16T02:02:33.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.372+0000 7fbf5bf0a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ca586920f0 con 0x55ca586752f0 2024-09-16T02:02:33.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.372+0000 7fbf5bf0a740 10 client.0 _ll_register_callbacks cb 0x7fbf44031570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:02:33.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.372+0000 7fbf53fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca5867fbb0 0x55ca586975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:33.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.372+0000 7fbf53fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca5867fbb0 0x55ca586975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37990/0 (socket says 172.21.15.105:37990) 2024-09-16T02:02:33.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.372+0000 7fbf53fff640 1 -- 172.21.15.105:0/3733296786 learned_addr learned my addr 172.21.15.105:0/3733296786 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:33.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.372+0000 7fbf52ffd640 1 --2- 172.21.15.105:0/3733296786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca586752f0 0x55ca586970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:33.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.373+0000 7fbf537fe640 1 --2- 172.21.15.105:0/3733296786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca586749d0 0x55ca58696b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:33.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.373+0000 7fbf53fff640 1 -- 172.21.15.105:0/3733296786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca586752f0 msgr2=0x55ca586970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:33.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.373+0000 7fbf53fff640 1 --2- 172.21.15.105:0/3733296786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca586752f0 0x55ca586970a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:33.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.373+0000 7fbf53fff640 1 -- 172.21.15.105:0/3733296786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca586749d0 msgr2=0x55ca58696b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:33.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.373+0000 7fbf53fff640 1 --2- 172.21.15.105:0/3733296786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca586749d0 0x55ca58696b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:33.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.373+0000 7fbf53fff640 1 -- 172.21.15.105:0/3733296786 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca586f8e40 con 0x55ca5867fbb0 2024-09-16T02:02:33.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.373+0000 7fbf537fe640 1 --2- 172.21.15.105:0/3733296786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca586749d0 0x55ca58696b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:02:33.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.373+0000 7fbf52ffd640 1 --2- 172.21.15.105:0/3733296786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca586752f0 0x55ca586970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:02:33.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.373+0000 7fbf53fff640 1 --2- 172.21.15.105:0/3733296786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca5867fbb0 0x55ca586975e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c00df40 tx=0x7fbf4c00df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6acb6c1af74e2a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:33.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.373+0000 7fbf32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:02:33.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.373+0000 7fbf32ffd640 1 -- 172.21.15.105:0/3733296786 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c02c040 con 0x55ca5867fbb0 2024-09-16T02:02:33.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.373+0000 7fbf32ffd640 1 -- 172.21.15.105:0/3733296786 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fbf4c004b80 con 0x55ca5867fbb0 2024-09-16T02:02:33.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.373+0000 7fbf5bf0a740 1 -- 172.21.15.105:0/3733296786 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf4809fa50 con 0x55ca5867fbb0 2024-09-16T02:02:33.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.373+0000 7fbf32ffd640 1 -- 172.21.15.105:0/3733296786 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbf4c004d80 con 0x55ca5867fbb0 2024-09-16T02:02:33.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.373+0000 7fbf5bf0a740 10 client.15216 fetch_fsmap learned FSMap version 27 2024-09-16T02:02:33.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.373+0000 7fbf5bf0a740 1 -- 172.21.15.105:0/3733296786 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ca586fa0c0 con 0x55ca5867fbb0 2024-09-16T02:02:33.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.373+0000 7fbf32ffd640 1 -- 172.21.15.105:0/3733296786 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf4809fa50 con 0x55ca5867fbb0 2024-09-16T02:02:33.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.373+0000 7fbf32ffd640 1 -- 172.21.15.105:0/3733296786 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbf4c0050e0 con 0x55ca5867fbb0 2024-09-16T02:02:33.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.373+0000 7fbf5bf0a740 10 client.15216 fetch_fsmap finished waiting for FSMap version 27 2024-09-16T02:02:33.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.373+0000 7fbf5bf0a740 10 client.15216 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-16T02:02:33.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.373+0000 7fbf5bf0a740 20 client.15216 populate_metadata read hostname 'smithi105' 2024-09-16T02:02:33.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.374+0000 7fbf5bf0a740 1 --2- 172.21.15.105:0/3733296786 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55ca586fe9e0 0x55ca586fd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:33.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.374+0000 7fbf5bf0a740 4 client.15216 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T02:02:33.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.374+0000 7fbf5bf0a740 1 -- 172.21.15.105:0/3733296786 --> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] -- command(tid 0: {"prefix": "status"}) -- 0x55ca585e2b10 con 0x55ca586fe9e0 2024-09-16T02:02:33.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.374+0000 7fbf537fe640 1 --2- 172.21.15.105:0/3733296786 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55ca586fe9e0 0x55ca586fd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:33.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.374+0000 7fbf537fe640 1 --2- 172.21.15.105:0/3733296786 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55ca586fe9e0 0x55ca586fd6f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=65f8d9e8d172a78c server_cookie=6c110556f2a8c73e in_seq=0 out_seq=0 2024-09-16T02:02:33.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.374+0000 7fbf32ffd640 10 client.15216 ms_handle_connect on v2:172.21.15.190:6838/1222573537 2024-09-16T02:02:33.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.374+0000 7fbf32ffd640 1 -- 172.21.15.105:0/3733296786 <== mds.1 v2:172.21.15.190:6838/1222573537 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ca585e2b10 con 0x55ca586fe9e0 2024-09-16T02:02:33.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.374+0000 7fbf32ffd640 10 client.15216 handle_command_reply: tid=0 2024-09-16T02:02:33.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.374+0000 7fbf5bf0a740 1 client.15216 shutdown 2024-09-16T02:02:33.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.375+0000 7fbf5bf0a740 1 -- 172.21.15.105:0/3733296786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca5867fbb0 msgr2=0x55ca586975e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:33.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.375+0000 7fbf5bf0a740 1 --2- 172.21.15.105:0/3733296786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca5867fbb0 0x55ca586975e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c00df40 tx=0x7fbf4c00df70 comp rx=0 tx=0).stop 2024-09-16T02:02:33.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.375+0000 7fbf5bf0a740 1 -- 172.21.15.105:0/3733296786 shutdown_connections 2024-09-16T02:02:33.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.375+0000 7fbf5bf0a740 1 --2- 172.21.15.105:0/3733296786 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55ca586fe9e0 0x55ca586fd6f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:33.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.375+0000 7fbf5bf0a740 1 --2- 172.21.15.105:0/3733296786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ca586752f0 0x55ca586970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:33.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.375+0000 7fbf5bf0a740 1 --2- 172.21.15.105:0/3733296786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca5867fbb0 0x55ca586975e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:33.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.375+0000 7fbf32ffd640 0 client.15216 ms_handle_reset on v2:172.21.15.190:6838/1222573537 2024-09-16T02:02:33.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.375+0000 7fbf5bf0a740 1 --2- 172.21.15.105:0/3733296786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca586749d0 0x55ca58696b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:33.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.375+0000 7fbf5bf0a740 1 -- 172.21.15.105:0/3733296786 >> 172.21.15.105:0/3733296786 conn(0x55ca58670f60 msgr2=0x55ca58670000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:33.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.375+0000 7fbf5bf0a740 1 -- 172.21.15.105:0/3733296786 shutdown_connections 2024-09-16T02:02:33.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.375+0000 7fbf5bf0a740 1 -- 172.21.15.105:0/3733296786 wait complete. 2024-09-16T02:02:33.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.375+0000 7fbf5bf0a740 20 client.15216 trim_cache size 0 max 16384 2024-09-16T02:02:33.390 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-16T02:02:33.390 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "423b7a2a-73ce-11ef-bceb-c7b262605968", 2024-09-16T02:02:33.390 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 1, 2024-09-16T02:02:33.391 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 24469, 2024-09-16T02:02:33.391 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2024-09-16T02:02:33.391 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2024-09-16T02:02:33.391 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2024-09-16T02:02:33.391 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 109.88907547300001, 2024-09-16T02:02:33.391 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 27, 2024-09-16T02:02:33.391 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 81, 2024-09-16T02:02:33.391 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 79, 2024-09-16T02:02:33.391 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 138.212836971 2024-09-16T02:02:33.391 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-16T02:02:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.375+0000 7fbf59b31640 1 -- 172.21.15.105:0/3281420907 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fbf3c0a3240 msgr2=0x7fbf3c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.375+0000 7fbf59b31640 1 --2- 172.21.15.105:0/3281420907 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fbf3c0a3240 0x7fbf3c0a5700 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c020000 tx=0x7fbf4c01d000 comp rx=0 tx=0).stop 2024-09-16T02:02:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.375+0000 7fbf59b31640 1 -- 172.21.15.105:0/3281420907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf54100a70 msgr2=0x7fbf541017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.375+0000 7fbf59b31640 1 --2- 172.21.15.105:0/3281420907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf54100a70 0x7fbf541017f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fbf4800b4a0 tx=0x7fbf4800b960 comp rx=0 tx=0).stop 2024-09-16T02:02:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.375+0000 7fbf59b31640 1 -- 172.21.15.105:0/3281420907 shutdown_connections 2024-09-16T02:02:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.375+0000 7fbf59b31640 1 --2- 172.21.15.105:0/3281420907 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fbf3c0a3240 0x7fbf3c0a5700 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.375+0000 7fbf59b31640 1 --2- 172.21.15.105:0/3281420907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf54106ba0 0x7fbf54101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.375+0000 7fbf59b31640 1 --2- 172.21.15.105:0/3281420907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf54100a70 0x7fbf541017f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.375+0000 7fbf59b31640 1 --2- 172.21.15.105:0/3281420907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf541000d0 0x7fbf541012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.375+0000 7fbf59b31640 1 -- 172.21.15.105:0/3281420907 >> 172.21.15.105:0/3281420907 conn(0x7fbf54077210 msgr2=0x7fbf540fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.376+0000 7fbf59b31640 1 -- 172.21.15.105:0/3281420907 shutdown_connections 2024-09-16T02:02:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:33.376+0000 7fbf59b31640 1 -- 172.21.15.105:0/3281420907 wait complete. 2024-09-16T02:02:34.072 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-16T02:02:34.314 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:02:34.588 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:34 smithi105 ceph-mon[31197]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-16T02:02:34.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:34 smithi190 ceph-mon[34762]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-16T02:02:34.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.807+0000 7fcdfefa1640 1 -- 172.21.15.105:0/4045913439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcdf8102b90 msgr2=0x7fcdf8102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:34.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.807+0000 7fcdfefa1640 1 --2- 172.21.15.105:0/4045913439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcdf8102b90 0x7fcdf8102f70 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0009900 tx=0x7fcdf002a480 comp rx=0 tx=0).stop 2024-09-16T02:02:34.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.807+0000 7fcdfefa1640 1 -- 172.21.15.105:0/4045913439 shutdown_connections 2024-09-16T02:02:34.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.807+0000 7fcdfefa1640 1 --2- 172.21.15.105:0/4045913439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdf810bfc0 0x7fcdf810e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:34.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.807+0000 7fcdfefa1640 1 --2- 172.21.15.105:0/4045913439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcdf81034b0 0x7fcdf810b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:34.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.807+0000 7fcdfefa1640 1 --2- 172.21.15.105:0/4045913439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcdf8102b90 0x7fcdf8102f70 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:34.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.807+0000 7fcdfefa1640 1 -- 172.21.15.105:0/4045913439 >> 172.21.15.105:0/4045913439 conn(0x7fcdf80fc610 msgr2=0x7fcdf80fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:34.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.808+0000 7fcdfefa1640 1 -- 172.21.15.105:0/4045913439 shutdown_connections 2024-09-16T02:02:34.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.808+0000 7fcdfefa1640 1 -- 172.21.15.105:0/4045913439 wait complete. 2024-09-16T02:02:34.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.808+0000 7fcdfefa1640 1 Processor -- start 2024-09-16T02:02:34.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.808+0000 7fcdfefa1640 1 -- start start 2024-09-16T02:02:34.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.808+0000 7fcdfefa1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcdf8102b90 0x7fcdf81982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:34.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.808+0000 7fcdfefa1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdf81034b0 0x7fcdf8198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:34.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.808+0000 7fcdfefa1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcdf810bfc0 0x7fcdf819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:34.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.808+0000 7fcdfefa1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf8111d30 con 0x7fcdf810bfc0 2024-09-16T02:02:34.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.808+0000 7fcdfefa1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf8111bb0 con 0x7fcdf8102b90 2024-09-16T02:02:34.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.808+0000 7fcdfefa1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf8111eb0 con 0x7fcdf81034b0 2024-09-16T02:02:34.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.808+0000 7fcdfe7a0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcdf810bfc0 0x7fcdf819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:34.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.808+0000 7fcdfe7a0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcdf810bfc0 0x7fcdf819cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38022/0 (socket says 172.21.15.105:38022) 2024-09-16T02:02:34.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.808+0000 7fcdfe7a0640 1 -- 172.21.15.105:0/675877705 learned_addr learned my addr 172.21.15.105:0/675877705 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:34.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.809+0000 7fcdfe7a0640 1 -- 172.21.15.105:0/675877705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdf81034b0 msgr2=0x7fcdf8198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:34.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.809+0000 7fcdfe7a0640 1 --2- 172.21.15.105:0/675877705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdf81034b0 0x7fcdf8198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:34.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.809+0000 7fcdfe7a0640 1 -- 172.21.15.105:0/675877705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcdf8102b90 msgr2=0x7fcdf81982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:34.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.809+0000 7fcdfdf9f640 1 --2- 172.21.15.105:0/675877705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcdf8102b90 0x7fcdf81982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:34.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.809+0000 7fcdfe7a0640 1 --2- 172.21.15.105:0/675877705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcdf8102b90 0x7fcdf81982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:34.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.809+0000 7fcdfe7a0640 1 -- 172.21.15.105:0/675877705 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdf819d2a0 con 0x7fcdf810bfc0 2024-09-16T02:02:34.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.809+0000 7fcdfdf9f640 1 --2- 172.21.15.105:0/675877705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcdf8102b90 0x7fcdf81982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:02:34.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.809+0000 7fcdfe7a0640 1 --2- 172.21.15.105:0/675877705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcdf810bfc0 0x7fcdf819cbc0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fcde800b970 tx=0x7fcde800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:34.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.809+0000 7fcdf6ffd640 1 -- 172.21.15.105:0/675877705 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde8019070 con 0x7fcdf810bfc0 2024-09-16T02:02:34.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.809+0000 7fcdf6ffd640 1 -- 172.21.15.105:0/675877705 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcde8014440 con 0x7fcdf810bfc0 2024-09-16T02:02:34.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.809+0000 7fcdfefa1640 1 -- 172.21.15.105:0/675877705 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdf819d590 con 0x7fcdf810bfc0 2024-09-16T02:02:34.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.809+0000 7fcdf6ffd640 1 -- 172.21.15.105:0/675877705 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde80075d0 con 0x7fcdf810bfc0 2024-09-16T02:02:34.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.809+0000 7fcdfefa1640 1 -- 172.21.15.105:0/675877705 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdf81a4e70 con 0x7fcdf810bfc0 2024-09-16T02:02:34.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.811+0000 7fcdf6ffd640 1 -- 172.21.15.105:0/675877705 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fcde801c070 con 0x7fcdf810bfc0 2024-09-16T02:02:34.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.812+0000 7fce025dc740 1 Processor -- start 2024-09-16T02:02:34.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.812+0000 7fcdf6ffd640 1 --2- 172.21.15.105:0/675877705 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fcde00a32c0 0x7fcde00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:34.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.812+0000 7fcdfdf9f640 1 --2- 172.21.15.105:0/675877705 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fcde00a32c0 0x7fcde00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:34.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.812+0000 7fcdf6ffd640 1 -- 172.21.15.105:0/675877705 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fcde80ceeb0 con 0x7fcdf810bfc0 2024-09-16T02:02:34.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.812+0000 7fce025dc740 1 -- start start 2024-09-16T02:02:34.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.812+0000 7fce025dc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebf6815bd0 0x55ebf6815fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:34.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.812+0000 7fce025dc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebf68164f0 0x55ebf6820870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:34.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.812+0000 7fce025dc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ebf6820db0 0x55ebf68231a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:34.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.812+0000 7fce025dc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ebf677c090 con 0x55ebf6815bd0 2024-09-16T02:02:34.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.812+0000 7fce025dc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ebf6723a70 con 0x55ebf68164f0 2024-09-16T02:02:34.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.812+0000 7fce025dc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ebf6778050 con 0x55ebf6820db0 2024-09-16T02:02:34.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.812+0000 7fcdfdf9f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebf6815bd0 0x55ebf6815fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:34.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.812+0000 7fcdfdf9f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebf6815bd0 0x55ebf6815fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38038/0 (socket says 172.21.15.105:38038) 2024-09-16T02:02:34.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.812+0000 7fcdfdf9f640 1 -- 172.21.15.105:0/3344535813 learned_addr learned my addr 172.21.15.105:0/3344535813 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:34.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.812+0000 7fcdfdf9f640 1 --2- 172.21.15.105:0/675877705 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fcde00a32c0 0x7fcde00a5780 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0000c00 tx=0x7fcdf0031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:34.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.812+0000 7fcdfd79e640 1 --2- 172.21.15.105:0/3344535813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebf68164f0 0x55ebf6820870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:34.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.813+0000 7fcdfdf9f640 1 -- 172.21.15.105:0/3344535813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ebf6820db0 msgr2=0x55ebf68231a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:34.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.813+0000 7fcdfdf9f640 1 --2- 172.21.15.105:0/3344535813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ebf6820db0 0x55ebf68231a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:34.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.813+0000 7fcdfdf9f640 1 -- 172.21.15.105:0/3344535813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebf68164f0 msgr2=0x55ebf6820870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:34.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.813+0000 7fcdfdf9f640 1 --2- 172.21.15.105:0/3344535813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebf68164f0 0x55ebf6820870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:34.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.813+0000 7fcdfdf9f640 1 -- 172.21.15.105:0/3344535813 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebf6730130 con 0x55ebf6815bd0 2024-09-16T02:02:34.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.813+0000 7fcdfdf9f640 1 --2- 172.21.15.105:0/3344535813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebf6815bd0 0x55ebf6815fb0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fcdf00317f0 tx=0x7fcdf003b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94c13fdde3b7c45 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:34.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.813+0000 7fcddffff640 1 -- 172.21.15.105:0/3344535813 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0035050 con 0x55ebf6815bd0 2024-09-16T02:02:34.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.813+0000 7fcddffff640 1 -- 172.21.15.105:0/3344535813 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcdf004d070 con 0x55ebf6815bd0 2024-09-16T02:02:34.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.813+0000 7fcddffff640 1 -- 172.21.15.105:0/3344535813 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf00035f0 con 0x55ebf6815bd0 2024-09-16T02:02:34.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.814+0000 7fce025dc740 1 -- 172.21.15.105:0/3344535813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebf6815bd0 msgr2=0x55ebf6815fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:34.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.814+0000 7fce025dc740 1 --2- 172.21.15.105:0/3344535813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebf6815bd0 0x55ebf6815fb0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fcdf00317f0 tx=0x7fcdf003b000 comp rx=0 tx=0).stop 2024-09-16T02:02:34.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.814+0000 7fce025dc740 1 -- 172.21.15.105:0/3344535813 shutdown_connections 2024-09-16T02:02:34.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.814+0000 7fce025dc740 1 --2- 172.21.15.105:0/3344535813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ebf6820db0 0x55ebf68231a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:34.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.814+0000 7fce025dc740 1 --2- 172.21.15.105:0/3344535813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebf68164f0 0x55ebf6820870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:34.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.814+0000 7fce025dc740 1 --2- 172.21.15.105:0/3344535813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebf6815bd0 0x55ebf6815fb0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:34.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.814+0000 7fce025dc740 1 -- 172.21.15.105:0/3344535813 >> 172.21.15.105:0/3344535813 conn(0x55ebf6812160 msgr2=0x55ebf6812540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:34.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.814+0000 7fce025dc740 1 -- 172.21.15.105:0/3344535813 shutdown_connections 2024-09-16T02:02:34.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.814+0000 7fce025dc740 1 -- 172.21.15.105:0/3344535813 wait complete. 2024-09-16T02:02:34.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.814+0000 7fce025dc740 1 -- start start 2024-09-16T02:02:34.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.814+0000 7fce025dc740 1 Processor -- start 2024-09-16T02:02:34.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.815+0000 7fce025dc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebf6815bd0 0x55ebf68206f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:34.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.815+0000 7fce025dc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebf68164f0 0x55ebf681da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:34.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.815+0000 7fce025dc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ebf6820db0 0x55ebf681df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:34.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.815+0000 7fce025dc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ebf6823980 con 0x55ebf68164f0 2024-09-16T02:02:34.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.815+0000 7fce025dc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ebf6736f20 con 0x55ebf6815bd0 2024-09-16T02:02:34.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.815+0000 7fce025dc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ebf6823b00 con 0x55ebf6820db0 2024-09-16T02:02:34.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.815+0000 7fce025dc740 10 client.0 _ll_register_callbacks cb 0x55ebf6700030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:02:34.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.815+0000 7fcdfd79e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebf68164f0 0x55ebf681da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:34.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.815+0000 7fcdfd79e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebf68164f0 0x55ebf681da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38050/0 (socket says 172.21.15.105:38050) 2024-09-16T02:02:34.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.815+0000 7fcdfd79e640 1 -- 172.21.15.105:0/516711558 learned_addr learned my addr 172.21.15.105:0/516711558 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:34.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.815+0000 7fcdfdf9f640 1 --2- 172.21.15.105:0/516711558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebf6815bd0 0x55ebf68206f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:34.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.815+0000 7fcdfd79e640 1 -- 172.21.15.105:0/516711558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ebf6820db0 msgr2=0x55ebf681df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:34.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.815+0000 7fcdfe7a0640 1 --2- 172.21.15.105:0/516711558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ebf6820db0 0x55ebf681df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:34.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.815+0000 7fcdfd79e640 1 --2- 172.21.15.105:0/516711558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ebf6820db0 0x55ebf681df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:34.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.815+0000 7fcdfd79e640 1 -- 172.21.15.105:0/516711558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebf6815bd0 msgr2=0x55ebf68206f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:34.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.815+0000 7fcdfd79e640 1 --2- 172.21.15.105:0/516711558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebf6815bd0 0x55ebf68206f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:34.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.815+0000 7fcdfd79e640 1 -- 172.21.15.105:0/516711558 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebf64ed2b0 con 0x55ebf68164f0 2024-09-16T02:02:34.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.815+0000 7fcdfd79e640 1 --2- 172.21.15.105:0/516711558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebf68164f0 0x55ebf681da40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fcdec00d160 tx=0x7fcdec00d620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d0355c397dc835 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:34.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.815+0000 7fcddd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:02:34.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.815+0000 7fcddd7fa640 1 -- 172.21.15.105:0/516711558 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdec00de50 con 0x55ebf68164f0 2024-09-16T02:02:34.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.815+0000 7fcddd7fa640 1 -- 172.21.15.105:0/516711558 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fcdec0125f0 con 0x55ebf68164f0 2024-09-16T02:02:34.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.815+0000 7fce025dc740 1 -- 172.21.15.105:0/516711558 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdf004d070 con 0x55ebf68164f0 2024-09-16T02:02:34.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.816+0000 7fcddd7fa640 1 -- 172.21.15.105:0/516711558 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcdec0127f0 con 0x55ebf68164f0 2024-09-16T02:02:34.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.816+0000 7fce025dc740 10 client.15228 fetch_fsmap learned FSMap version 27 2024-09-16T02:02:34.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.816+0000 7fce025dc740 1 -- 172.21.15.105:0/516711558 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ebf682f3c0 con 0x55ebf68164f0 2024-09-16T02:02:34.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.816+0000 7fcddd7fa640 1 -- 172.21.15.105:0/516711558 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcdf004d070 con 0x55ebf68164f0 2024-09-16T02:02:34.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.816+0000 7fcddd7fa640 1 -- 172.21.15.105:0/516711558 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fcdec013070 con 0x55ebf68164f0 2024-09-16T02:02:34.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.816+0000 7fce025dc740 10 client.15228 fetch_fsmap finished waiting for FSMap version 27 2024-09-16T02:02:34.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.816+0000 7fce025dc740 10 client.15228 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T02:02:34.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.816+0000 7fce025dc740 20 client.15228 populate_metadata read hostname 'smithi105' 2024-09-16T02:02:34.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.816+0000 7fce025dc740 1 --2- 172.21.15.105:0/516711558 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55ebf6831a60 0x55ebf6832ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:34.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.816+0000 7fce025dc740 4 client.15228 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:02:34.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.816+0000 7fce025dc740 1 -- 172.21.15.105:0/516711558 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ebf6789600 con 0x55ebf6831a60 2024-09-16T02:02:34.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.816+0000 7fcdfdf9f640 1 --2- 172.21.15.105:0/516711558 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55ebf6831a60 0x55ebf6832ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:34.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.817+0000 7fcdfdf9f640 1 --2- 172.21.15.105:0/516711558 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55ebf6831a60 0x55ebf6832ed0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61970adc8039a596 server_cookie=e6821116305af0f0 in_seq=0 out_seq=0 2024-09-16T02:02:34.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.817+0000 7fcddd7fa640 10 client.15228 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:02:34.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.817+0000 7fcddd7fa640 1 -- 172.21.15.105:0/516711558 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ebf6789600 con 0x55ebf6831a60 2024-09-16T02:02:34.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.817+0000 7fcddd7fa640 10 client.15228 handle_command_reply: tid=0 2024-09-16T02:02:34.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.817+0000 7fce025dc740 1 client.15228 shutdown 2024-09-16T02:02:34.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.818+0000 7fce025dc740 1 -- 172.21.15.105:0/516711558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebf68164f0 msgr2=0x55ebf681da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:34.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.818+0000 7fce025dc740 1 --2- 172.21.15.105:0/516711558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebf68164f0 0x55ebf681da40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fcdec00d160 tx=0x7fcdec00d620 comp rx=0 tx=0).stop 2024-09-16T02:02:34.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.818+0000 7fce025dc740 1 -- 172.21.15.105:0/516711558 shutdown_connections 2024-09-16T02:02:34.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.818+0000 7fce025dc740 1 --2- 172.21.15.105:0/516711558 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55ebf6831a60 0x55ebf6832ed0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:34.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.818+0000 7fce025dc740 1 --2- 172.21.15.105:0/516711558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ebf6820db0 0x55ebf681df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:34.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.818+0000 7fcddd7fa640 0 client.15228 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:02:34.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.818+0000 7fce025dc740 1 --2- 172.21.15.105:0/516711558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebf68164f0 0x55ebf681da40 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:34.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.818+0000 7fce025dc740 1 --2- 172.21.15.105:0/516711558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebf6815bd0 0x55ebf68206f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:34.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.818+0000 7fce025dc740 1 -- 172.21.15.105:0/516711558 >> 172.21.15.105:0/516711558 conn(0x55ebf6812160 msgr2=0x55ebf6817e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:34.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.818+0000 7fce025dc740 1 -- 172.21.15.105:0/516711558 shutdown_connections 2024-09-16T02:02:34.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.818+0000 7fce025dc740 1 -- 172.21.15.105:0/516711558 wait complete. 2024-09-16T02:02:34.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.818+0000 7fce025dc740 20 client.15228 trim_cache size 0 max 16384 2024-09-16T02:02:34.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.835+0000 7fce025dc740 1 Processor -- start 2024-09-16T02:02:34.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.835+0000 7fce025dc740 1 -- start start 2024-09-16T02:02:34.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.835+0000 7fce025dc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebf6815bd0 0x55ebf6832ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:34.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.835+0000 7fce025dc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebf68164f0 0x55ebf68187c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:34.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.835+0000 7fce025dc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ebf6820db0 0x55ebf6818d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:34.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.835+0000 7fce025dc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ebf6831210 con 0x55ebf6815bd0 2024-09-16T02:02:34.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.835+0000 7fce025dc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ebf68318e0 con 0x55ebf68164f0 2024-09-16T02:02:34.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.835+0000 7fce025dc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ebf6831fb0 con 0x55ebf6820db0 2024-09-16T02:02:34.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.835+0000 7fcdfdf9f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebf6815bd0 0x55ebf6832ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:34.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.836+0000 7fcdfdf9f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebf6815bd0 0x55ebf6832ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38062/0 (socket says 172.21.15.105:38062) 2024-09-16T02:02:34.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.836+0000 7fcdfdf9f640 1 -- 172.21.15.105:0/3458279486 learned_addr learned my addr 172.21.15.105:0/3458279486 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:34.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.836+0000 7fcdfdf9f640 1 -- 172.21.15.105:0/3458279486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ebf6820db0 msgr2=0x55ebf6818d00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T02:02:34.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.836+0000 7fcdfdf9f640 1 --2- 172.21.15.105:0/3458279486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ebf6820db0 0x55ebf6818d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:34.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.836+0000 7fcdfdf9f640 1 -- 172.21.15.105:0/3458279486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebf68164f0 msgr2=0x55ebf68187c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T02:02:34.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.836+0000 7fcdfdf9f640 1 --2- 172.21.15.105:0/3458279486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebf68164f0 0x55ebf68187c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:34.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.836+0000 7fcdfdf9f640 1 -- 172.21.15.105:0/3458279486 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebf6816320 con 0x55ebf6815bd0 2024-09-16T02:02:34.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.836+0000 7fcdfdf9f640 1 --2- 172.21.15.105:0/3458279486 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebf6815bd0 0x55ebf6832ed0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fcdf00317f0 tx=0x7fcdf003b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17daa87a496bbd08 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:34.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.836+0000 7fcddf7fe640 1 -- 172.21.15.105:0/3458279486 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0035040 con 0x55ebf6815bd0 2024-09-16T02:02:34.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.836+0000 7fcddf7fe640 1 -- 172.21.15.105:0/3458279486 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcdf004d070 con 0x55ebf6815bd0 2024-09-16T02:02:34.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.836+0000 7fcddf7fe640 1 -- 172.21.15.105:0/3458279486 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf003b750 con 0x55ebf6815bd0 2024-09-16T02:02:34.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.837+0000 7fce025dc740 1 -- 172.21.15.105:0/3458279486 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebf6815bd0 msgr2=0x55ebf6832ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:34.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.837+0000 7fce025dc740 1 --2- 172.21.15.105:0/3458279486 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebf6815bd0 0x55ebf6832ed0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fcdf00317f0 tx=0x7fcdf003b000 comp rx=0 tx=0).stop 2024-09-16T02:02:34.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.837+0000 7fce025dc740 1 -- 172.21.15.105:0/3458279486 shutdown_connections 2024-09-16T02:02:34.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.837+0000 7fce025dc740 1 --2- 172.21.15.105:0/3458279486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ebf6820db0 0x55ebf6818d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:34.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.837+0000 7fce025dc740 1 --2- 172.21.15.105:0/3458279486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebf68164f0 0x55ebf68187c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:34.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.837+0000 7fce025dc740 1 --2- 172.21.15.105:0/3458279486 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebf6815bd0 0x55ebf6832ed0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:34.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.837+0000 7fce025dc740 1 -- 172.21.15.105:0/3458279486 >> 172.21.15.105:0/3458279486 conn(0x55ebf6812160 msgr2=0x55ebf681cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:34.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.837+0000 7fce025dc740 1 -- 172.21.15.105:0/3458279486 shutdown_connections 2024-09-16T02:02:34.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.837+0000 7fce025dc740 1 -- 172.21.15.105:0/3458279486 wait complete. 2024-09-16T02:02:34.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.837+0000 7fce025dc740 1 -- start start 2024-09-16T02:02:34.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.837+0000 7fce025dc740 1 Processor -- start 2024-09-16T02:02:34.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.837+0000 7fce025dc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebf6815bd0 0x55ebf681a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:34.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.837+0000 7fce025dc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebf68164f0 0x55ebf681a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:34.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.837+0000 7fce025dc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ebf6820db0 0x55ebf683a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:34.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.838+0000 7fce025dc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ebf6832d50 con 0x55ebf6815bd0 2024-09-16T02:02:34.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.838+0000 7fce025dc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ebf6832680 con 0x55ebf68164f0 2024-09-16T02:02:34.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.838+0000 7fce025dc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ebf6833420 con 0x55ebf6820db0 2024-09-16T02:02:34.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.838+0000 7fce025dc740 10 client.0 _ll_register_callbacks cb 0x7fcdf806afd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:02:34.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.838+0000 7fcdfdf9f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebf6815bd0 0x55ebf681a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:34.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.838+0000 7fcdfdf9f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebf6815bd0 0x55ebf681a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38070/0 (socket says 172.21.15.105:38070) 2024-09-16T02:02:34.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.838+0000 7fcdfdf9f640 1 -- 172.21.15.105:0/4231115405 learned_addr learned my addr 172.21.15.105:0/4231115405 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:34.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.838+0000 7fcdfdf9f640 1 -- 172.21.15.105:0/4231115405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ebf6820db0 msgr2=0x55ebf683a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:34.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.838+0000 7fcdfdf9f640 1 --2- 172.21.15.105:0/4231115405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ebf6820db0 0x55ebf683a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:34.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.838+0000 7fcdfdf9f640 1 -- 172.21.15.105:0/4231115405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebf68164f0 msgr2=0x55ebf681a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:34.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.838+0000 7fcdfdf9f640 1 --2- 172.21.15.105:0/4231115405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebf68164f0 0x55ebf681a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:34.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.838+0000 7fcdfdf9f640 1 -- 172.21.15.105:0/4231115405 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebf683a640 con 0x55ebf6815bd0 2024-09-16T02:02:34.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.838+0000 7fcdfdf9f640 1 --2- 172.21.15.105:0/4231115405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebf6815bd0 0x55ebf681a3d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fcdf00317c0 tx=0x7fcdf0003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=826779f425536dd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:34.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.838+0000 7fcddd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:02:34.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.838+0000 7fcddd7fa640 1 -- 172.21.15.105:0/4231115405 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0068060 con 0x55ebf6815bd0 2024-09-16T02:02:34.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.838+0000 7fcddd7fa640 1 -- 172.21.15.105:0/4231115405 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fcdf0003ab0 con 0x55ebf6815bd0 2024-09-16T02:02:34.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.838+0000 7fce025dc740 1 -- 172.21.15.105:0/4231115405 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdf004d070 con 0x55ebf6815bd0 2024-09-16T02:02:34.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.838+0000 7fcddd7fa640 1 -- 172.21.15.105:0/4231115405 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcdf0035040 con 0x55ebf6815bd0 2024-09-16T02:02:34.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.838+0000 7fce025dc740 10 client.15234 fetch_fsmap learned FSMap version 27 2024-09-16T02:02:34.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.838+0000 7fce025dc740 1 -- 172.21.15.105:0/4231115405 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ebf683b940 con 0x55ebf6815bd0 2024-09-16T02:02:34.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.839+0000 7fcddd7fa640 1 -- 172.21.15.105:0/4231115405 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcdf004d070 con 0x55ebf6815bd0 2024-09-16T02:02:34.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.839+0000 7fcddd7fa640 1 -- 172.21.15.105:0/4231115405 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fcdf0048440 con 0x55ebf6815bd0 2024-09-16T02:02:34.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.839+0000 7fce025dc740 10 client.15234 fetch_fsmap finished waiting for FSMap version 27 2024-09-16T02:02:34.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.839+0000 7fce025dc740 10 client.15234 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T02:02:34.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.839+0000 7fce025dc740 20 client.15234 populate_metadata read hostname 'smithi105' 2024-09-16T02:02:34.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.839+0000 7fce025dc740 1 --2- 172.21.15.105:0/4231115405 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55ebf68a0030 0x55ebf689ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:34.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.839+0000 7fce025dc740 4 client.15234 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T02:02:34.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.839+0000 7fce025dc740 1 -- 172.21.15.105:0/4231115405 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "status"}) -- 0x55ebf6783b10 con 0x55ebf68a0030 2024-09-16T02:02:34.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.839+0000 7fcdfd79e640 1 --2- 172.21.15.105:0/4231115405 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55ebf68a0030 0x55ebf689ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:34.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.840+0000 7fcdfd79e640 1 --2- 172.21.15.105:0/4231115405 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55ebf68a0030 0x55ebf689ed40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2adb60f2a606ccdb server_cookie=8093cc5b4c4df2bc in_seq=0 out_seq=0 2024-09-16T02:02:34.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.840+0000 7fcddd7fa640 10 client.15234 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:02:34.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.840+0000 7fcddd7fa640 1 -- 172.21.15.105:0/4231115405 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ebf6783b10 con 0x55ebf68a0030 2024-09-16T02:02:34.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.840+0000 7fcddd7fa640 10 client.15234 handle_command_reply: tid=0 2024-09-16T02:02:34.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.840+0000 7fce025dc740 1 client.15234 shutdown 2024-09-16T02:02:34.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.840+0000 7fce025dc740 1 -- 172.21.15.105:0/4231115405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebf6815bd0 msgr2=0x55ebf681a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:34.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.840+0000 7fce025dc740 1 --2- 172.21.15.105:0/4231115405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebf6815bd0 0x55ebf681a3d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fcdf00317c0 tx=0x7fcdf0003680 comp rx=0 tx=0).stop 2024-09-16T02:02:34.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.840+0000 7fce025dc740 1 -- 172.21.15.105:0/4231115405 shutdown_connections 2024-09-16T02:02:34.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.840+0000 7fce025dc740 1 --2- 172.21.15.105:0/4231115405 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55ebf68a0030 0x55ebf689ed40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:34.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.840+0000 7fce025dc740 1 --2- 172.21.15.105:0/4231115405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ebf6820db0 0x55ebf683a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:34.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.840+0000 7fce025dc740 1 --2- 172.21.15.105:0/4231115405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebf68164f0 0x55ebf681a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:34.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.840+0000 7fcddd7fa640 0 client.15234 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:02:34.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.840+0000 7fce025dc740 1 --2- 172.21.15.105:0/4231115405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebf6815bd0 0x55ebf681a3d0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:34.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.840+0000 7fce025dc740 1 -- 172.21.15.105:0/4231115405 >> 172.21.15.105:0/4231115405 conn(0x55ebf6812160 msgr2=0x55ebf68187c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:34.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.840+0000 7fce025dc740 1 -- 172.21.15.105:0/4231115405 shutdown_connections 2024-09-16T02:02:34.915 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-16T02:02:34.915 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "423b7a2a-73ce-11ef-bceb-c7b262605968", 2024-09-16T02:02:34.915 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 0, 2024-09-16T02:02:34.915 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 24482, 2024-09-16T02:02:34.915 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2024-09-16T02:02:34.915 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2024-09-16T02:02:34.915 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2024-09-16T02:02:34.915 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 119.391301336, 2024-09-16T02:02:34.916 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 27, 2024-09-16T02:02:34.916 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 81, 2024-09-16T02:02:34.916 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 79, 2024-09-16T02:02:34.916 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 130.22580232799999 2024-09-16T02:02:34.916 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-16T02:02:34.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.840+0000 7fce025dc740 1 -- 172.21.15.105:0/4231115405 wait complete. 2024-09-16T02:02:34.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.841+0000 7fce025dc740 20 client.15234 trim_cache size 0 max 16384 2024-09-16T02:02:34.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.841+0000 7fcdfefa1640 1 -- 172.21.15.105:0/675877705 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fcde00a32c0 msgr2=0x7fcde00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:34.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.841+0000 7fcdfefa1640 1 --2- 172.21.15.105:0/675877705 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fcde00a32c0 0x7fcde00a5780 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0000c00 tx=0x7fcdf0031400 comp rx=0 tx=0).stop 2024-09-16T02:02:34.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.841+0000 7fcdfefa1640 1 -- 172.21.15.105:0/675877705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcdf810bfc0 msgr2=0x7fcdf819cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:34.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.841+0000 7fcdfefa1640 1 --2- 172.21.15.105:0/675877705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcdf810bfc0 0x7fcdf819cbc0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fcde800b970 tx=0x7fcde800be30 comp rx=0 tx=0).stop 2024-09-16T02:02:34.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.841+0000 7fcdfefa1640 1 -- 172.21.15.105:0/675877705 shutdown_connections 2024-09-16T02:02:34.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.841+0000 7fcdfefa1640 1 --2- 172.21.15.105:0/675877705 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fcde00a32c0 0x7fcde00a5780 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:34.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.841+0000 7fcdfefa1640 1 --2- 172.21.15.105:0/675877705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdf81034b0 0x7fcdf8198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:34.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.841+0000 7fcdfefa1640 1 --2- 172.21.15.105:0/675877705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcdf810bfc0 0x7fcdf819cbc0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:34.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.841+0000 7fcdfefa1640 1 --2- 172.21.15.105:0/675877705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcdf8102b90 0x7fcdf81982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:34.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.841+0000 7fcdfefa1640 1 -- 172.21.15.105:0/675877705 >> 172.21.15.105:0/675877705 conn(0x7fcdf80fc610 msgr2=0x7fcdf80fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:34.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.841+0000 7fcdfefa1640 1 -- 172.21.15.105:0/675877705 shutdown_connections 2024-09-16T02:02:34.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:34.841+0000 7fcdfefa1640 1 -- 172.21.15.105:0/675877705 wait complete. 2024-09-16T02:02:34.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:34 smithi137 ceph-mon[34099]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-16T02:02:35.646 INFO:teuthology.run_tasks:Running task sequential... 2024-09-16T02:02:35.655 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-16T02:02:35.655 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-16T02:02:35.656 INFO:teuthology.task.print:Enabling mgr modules 2024-09-16T02:02:35.656 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-16T02:02:35.714 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-16T02:02:35.715 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-09-16T02:02:35.715 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-09-16T02:02:35.715 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-16T02:02:35.715 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T02:02:35.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T02:02:35.715 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-09-16T02:02:35.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T02:02:35.732 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link delete ceph-brx 2024-09-16T02:02:35.799 INFO:teuthology.orchestra.run.smithi105.stderr:Cannot find device "ceph-brx" 2024-09-16T02:02:35.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T02:02:35.801 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-16T02:02:35.801 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-16T02:02:35.801 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-16T02:02:36.103 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:02:36.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.599+0000 7f81b173b640 1 -- 172.21.15.105:0/3754748607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81ac100e20 msgr2=0x7f81ac101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:36.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.599+0000 7f81b173b640 1 --2- 172.21.15.105:0/3754748607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81ac100e20 0x7f81ac101200 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f819c009900 tx=0x7f819c02a480 comp rx=0 tx=0).stop 2024-09-16T02:02:36.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.600+0000 7f81b173b640 1 -- 172.21.15.105:0/3754748607 shutdown_connections 2024-09-16T02:02:36.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.600+0000 7f81b173b640 1 --2- 172.21.15.105:0/3754748607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81ac102570 0x7f81ac1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:36.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.600+0000 7f81b173b640 1 --2- 172.21.15.105:0/3754748607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81ac102ef0 0x7f81ac10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:36.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.600+0000 7f81b173b640 1 --2- 172.21.15.105:0/3754748607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81ac100e20 0x7f81ac101200 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:36.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.600+0000 7f81b173b640 1 -- 172.21.15.105:0/3754748607 >> 172.21.15.105:0/3754748607 conn(0x7f81ac0fc7d0 msgr2=0x7f81ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:36.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.600+0000 7f81b173b640 1 -- 172.21.15.105:0/3754748607 shutdown_connections 2024-09-16T02:02:36.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.600+0000 7f81b173b640 1 -- 172.21.15.105:0/3754748607 wait complete. 2024-09-16T02:02:36.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.600+0000 7f81b173b640 1 Processor -- start 2024-09-16T02:02:36.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.600+0000 7f81b173b640 1 -- start start 2024-09-16T02:02:36.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.601+0000 7f81b173b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81ac100e20 0x7f81ac198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:36.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.601+0000 7f81b173b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81ac102570 0x7f81ac198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:36.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.601+0000 7f81b173b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81ac102ef0 0x7f81ac19ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:36.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.601+0000 7f81b173b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f81ac110ed0 con 0x7f81ac102ef0 2024-09-16T02:02:36.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.601+0000 7f81b173b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f81ac110d50 con 0x7f81ac100e20 2024-09-16T02:02:36.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.601+0000 7f81b173b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f81ac111050 con 0x7f81ac102570 2024-09-16T02:02:36.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.601+0000 7f81ab7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81ac102ef0 0x7f81ac19ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:36.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.601+0000 7f81ab7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81ac102ef0 0x7f81ac19ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38090/0 (socket says 172.21.15.105:38090) 2024-09-16T02:02:36.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.601+0000 7f81ab7fe640 1 -- 172.21.15.105:0/1426680631 learned_addr learned my addr 172.21.15.105:0/1426680631 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:36.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.601+0000 7f81ab7fe640 1 -- 172.21.15.105:0/1426680631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81ac102570 msgr2=0x7f81ac198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:36.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.601+0000 7f81ab7fe640 1 --2- 172.21.15.105:0/1426680631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81ac102570 0x7f81ac198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:36.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.601+0000 7f81ab7fe640 1 -- 172.21.15.105:0/1426680631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81ac100e20 msgr2=0x7f81ac198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:36.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.601+0000 7f81ab7fe640 1 --2- 172.21.15.105:0/1426680631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81ac100e20 0x7f81ac198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:36.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.601+0000 7f81ab7fe640 1 -- 172.21.15.105:0/1426680631 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81ac19d500 con 0x7f81ac102ef0 2024-09-16T02:02:36.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.601+0000 7f81ab7fe640 1 --2- 172.21.15.105:0/1426680631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81ac102ef0 0x7f81ac19ce20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f81a400b970 tx=0x7f81a400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:36.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.601+0000 7f8193fff640 1 -- 172.21.15.105:0/1426680631 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a4019070 con 0x7f81ac102ef0 2024-09-16T02:02:36.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.601+0000 7f8193fff640 1 -- 172.21.15.105:0/1426680631 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81a4014440 con 0x7f81ac102ef0 2024-09-16T02:02:36.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.602+0000 7f81b173b640 1 -- 172.21.15.105:0/1426680631 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81ac19d7f0 con 0x7f81ac102ef0 2024-09-16T02:02:36.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.602+0000 7f8193fff640 1 -- 172.21.15.105:0/1426680631 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a40075d0 con 0x7f81ac102ef0 2024-09-16T02:02:36.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.602+0000 7f81b173b640 1 -- 172.21.15.105:0/1426680631 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81ac1a50d0 con 0x7f81ac102ef0 2024-09-16T02:02:36.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.603+0000 7f81b173b640 1 -- 172.21.15.105:0/1426680631 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81ac101200 con 0x7f81ac102ef0 2024-09-16T02:02:36.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.605+0000 7f8193fff640 1 -- 172.21.15.105:0/1426680631 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f81a401c070 con 0x7f81ac102ef0 2024-09-16T02:02:36.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.605+0000 7f8193fff640 1 --2- 172.21.15.105:0/1426680631 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f81940a32c0 0x7f81940a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:36.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.605+0000 7f81aaffd640 1 --2- 172.21.15.105:0/1426680631 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f81940a32c0 0x7f81940a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:36.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.605+0000 7f8193fff640 1 -- 172.21.15.105:0/1426680631 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f81a40cf410 con 0x7f81ac102ef0 2024-09-16T02:02:36.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.606+0000 7f81aaffd640 1 --2- 172.21.15.105:0/1426680631 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f81940a32c0 0x7f81940a5780 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f819c02a9a0 tx=0x7f819c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:36.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.609+0000 7f8193fff640 1 -- 172.21.15.105:0/1426680631 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f81a409c110 con 0x7f81ac102ef0 2024-09-16T02:02:36.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:36 smithi105 ceph-mon[31197]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T02:02:36.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.817+0000 7f81b173b640 1 -- 172.21.15.105:0/1426680631 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f81ac1998b0 con 0x7f81ac102ef0 2024-09-16T02:02:36.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.817+0000 7f8193fff640 1 -- 172.21.15.105:0/1426680631 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7f81ac1998b0 con 0x7f81ac102ef0 2024-09-16T02:02:36.819 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T02:02:36.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.819+0000 7f81b173b640 1 -- 172.21.15.105:0/1426680631 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f81940a32c0 msgr2=0x7f81940a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:36.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.819+0000 7f81b173b640 1 --2- 172.21.15.105:0/1426680631 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f81940a32c0 0x7f81940a5780 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f819c02a9a0 tx=0x7f819c000bc0 comp rx=0 tx=0).stop 2024-09-16T02:02:36.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.819+0000 7f81b173b640 1 -- 172.21.15.105:0/1426680631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81ac102ef0 msgr2=0x7f81ac19ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:36.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.819+0000 7f81b173b640 1 --2- 172.21.15.105:0/1426680631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81ac102ef0 0x7f81ac19ce20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f81a400b970 tx=0x7f81a400be30 comp rx=0 tx=0).stop 2024-09-16T02:02:36.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.819+0000 7f81b173b640 1 -- 172.21.15.105:0/1426680631 shutdown_connections 2024-09-16T02:02:36.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.819+0000 7f81b173b640 1 --2- 172.21.15.105:0/1426680631 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f81940a32c0 0x7f81940a5780 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:36.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.819+0000 7f81b173b640 1 --2- 172.21.15.105:0/1426680631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81ac102570 0x7f81ac198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:36.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.819+0000 7f81b173b640 1 --2- 172.21.15.105:0/1426680631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81ac102ef0 0x7f81ac19ce20 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:36.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.819+0000 7f81b173b640 1 --2- 172.21.15.105:0/1426680631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81ac100e20 0x7f81ac198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:36.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.819+0000 7f81b173b640 1 -- 172.21.15.105:0/1426680631 >> 172.21.15.105:0/1426680631 conn(0x7f81ac0fc7d0 msgr2=0x7f81ac06a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:36.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.820+0000 7f81b173b640 1 -- 172.21.15.105:0/1426680631 shutdown_connections 2024-09-16T02:02:36.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:36.820+0000 7f81b173b640 1 -- 172.21.15.105:0/1426680631 wait complete. 2024-09-16T02:02:36.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:36 smithi190 ceph-mon[34762]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T02:02:36.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:36 smithi137 ceph-mon[34099]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T02:02:37.526 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-16T02:02:37.526 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T02:02:37.527 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2024-09-16T02:02:37.527 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T02:02:37.527 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T02:02:37.527 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-16T02:02:37.527 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-16T02:02:37.527 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T02:02:37.527 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T02:02:37.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T02:02:37.528 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2024-09-16T02:02:37.545 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T02:02:37.546 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T02:02:37.546 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T02:02:37.546 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-09-16T02:02:37.546 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2024-09-16T02:02:37.546 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-09-16T02:02:37.546 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T02:02:37.546 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2024-09-16T02:02:37.546 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2024-09-16T02:02:37.546 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T02:02:37.546 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2024-09-16T02:02:37.546 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2024-09-16T02:02:37.546 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T02:02:37.547 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2024-09-16T02:02:37.547 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T02:02:37.547 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2024-09-16T02:02:37.547 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T02:02:37.547 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 42194sec preferred_lft 42194sec 2024-09-16T02:02:37.547 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link noprefixroute 2024-09-16T02:02:37.547 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-09-16T02:02:37.547 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-16T02:02:37.547 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T02:02:37.548 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-09-16T02:02:37.548 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link add name ceph-brx type bridge 2024-09-16T02:02:37.548 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip addr flush dev ceph-brx 2024-09-16T02:02:37.548 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set ceph-brx up 2024-09-16T02:02:37.548 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-16T02:02:37.548 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-09-16T02:02:37.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T02:02:37.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T02:02:37.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T02:02:37.753 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T02:02:37.766 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:37 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/1426680631' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T02:02:37.777 INFO:teuthology.orchestra.run.smithi105.stdout:1 2024-09-16T02:02:37.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T02:02:37.779 DEBUG:teuthology.orchestra.run.smithi105:> route 2024-09-16T02:02:37.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:37 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/1426680631' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T02:02:37.850 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2024-09-16T02:02:37.850 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T02:02:37.850 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T02:02:37.850 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T02:02:37.850 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T02:02:37.851 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T02:02:37.851 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-09-16T02:02:37.851 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T02:02:37.851 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T02:02:37.851 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T02:02:37.851 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-09-16T02:02:37.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:37 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/1426680631' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T02:02:37.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T02:02:38.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T02:02:38.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T02:02:38.117 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-09-16T02:02:38.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T02:02:38.132 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list-id 2024-09-16T02:02:38.187 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T02:02:38.187 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-09-16T02:02:38.187 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T02:02:38.187 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-16T02:02:38.188 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-09-16T02:02:38.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T02:02:38.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T02:02:38.305 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-16T02:02:38.306 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T02:02:38.306 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-09-16T02:02:38.306 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-16T02:02:38.306 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-16T02:02:38.306 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-16T02:02:38.306 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-16T02:02:38.306 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-16T02:02:38.306 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-09-16T02:02:38.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T02:02:38.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T02:02:38.451 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T02:02:38.451 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-09-16T02:02:38.451 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.0 up 2024-09-16T02:02:38.451 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set dev brx.0 master ceph-brx 2024-09-16T02:02:38.451 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-09-16T02:02:38.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T02:02:38.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T02:02:38.527 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-09-16T02:02:38.528 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T02:02:38.528 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T02:02:38.543 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T02:02:38.543 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T02:02:38.543 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T02:02:38.598 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2024-09-16T02:02:38.663 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2024-09-16T02:02:38.719 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2024-09-16T02:02:38.719 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2024-09-16T02:02:38.719 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2024-09-16T02:02:38.719 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2024-09-16T02:02:38.719 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2024-09-16T02:02:38.719 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2024-09-16T02:02:38.719 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2024-09-16T02:02:38.719 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2024-09-16T02:02:38.719 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2024-09-16T02:02:38.720 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2024-09-16T02:02:38.720 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2024-09-16T02:02:38.720 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/selinux 2024-09-16T02:02:38.720 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T02:02:38.720 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2024-09-16T02:02:38.720 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2024-09-16T02:02:38.720 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2024-09-16T02:02:38.720 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2024-09-16T02:02:38.720 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-16T02:02:38.720 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2024-09-16T02:02:38.720 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2024-09-16T02:02:38.720 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysctl.service 2024-09-16T02:02:38.720 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-16T02:02:38.721 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/nfs/rpc_pipefs 2024-09-16T02:02:38.721 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2024-09-16T02:02:38.721 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2024-09-16T02:02:38.721 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay 2024-09-16T02:02:38.721 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/f8e25acf528545476b19439e1e9ce6c1682cf6cb43e75e471a832161b24f6c03/merged 2024-09-16T02:02:38.721 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/79a23f9406bdef4dec1bbef5101ad22e9e0515e5d606e02e415f632fc38961ec/merged 2024-09-16T02:02:38.721 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/0 2024-09-16T02:02:38.721 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T02:02:38.721 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/3ed71bb2a385617fe36b24a1b7d1b73677d8edf74c5101efe1d4ead7d10f262d/merged 2024-09-16T02:02:38.721 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/f62cbd0ee28f2b881512d53ecbee7e6b5542ef8544d13cfc9f57f6927225a70a/merged 2024-09-16T02:02:38.721 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/3fd8f383cde748f25a9749aa12b35e6683b5a84d52cd9c5764a0314644afdc7a/merged 2024-09-16T02:02:38.722 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/8611c43c89f9fdb358775326ef6c2207e5ed0c60553c3a23838fb2b7428f7d78/merged 2024-09-16T02:02:38.722 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/16b1bc9f363600dae0a706674099181576c9eea2a239db4753d7951b9804e24c/merged 2024-09-16T02:02:38.722 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/b2bc631b6dc01ef6dca62c9f4817ef18f553e68c6e790a200150839028548f48/merged 2024-09-16T02:02:38.722 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/0d228658795e31555507b2ed93ba4ec97a98bf6caa9b42f783ff0f97140eee15/merged 2024-09-16T02:02:38.722 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/0a24ea473087d90f82d27b88ae4e46a1c69cfe4189ff475cfccc28a28610751f/merged 2024-09-16T02:02:38.722 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2024-09-16T02:02:38.722 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T02:02:38.722 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T02:02:38.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T02:02:38.722 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2024-09-16T02:02:38.774 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-16T02:02:38.774 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/d0aa8553-5106-4f12-b555-f9d49c86e9ca --client_fs=cephfs) 2024-09-16T02:02:38.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:38 smithi105 ceph-mon[31197]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T02:02:38.796 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2024-09-16T02:02:38.820 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2024-09-16T02:02:38.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:38 smithi190 ceph-mon[34762]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T02:02:38.876 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2024-09-16T02:02:38.877 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2024-09-16T02:02:38.877 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2024-09-16T02:02:38.877 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2024-09-16T02:02:38.877 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2024-09-16T02:02:38.877 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2024-09-16T02:02:38.877 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2024-09-16T02:02:38.877 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2024-09-16T02:02:38.877 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2024-09-16T02:02:38.877 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2024-09-16T02:02:38.877 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2024-09-16T02:02:38.878 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/selinux 2024-09-16T02:02:38.878 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T02:02:38.878 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2024-09-16T02:02:38.878 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2024-09-16T02:02:38.878 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2024-09-16T02:02:38.878 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2024-09-16T02:02:38.878 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-16T02:02:38.878 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2024-09-16T02:02:38.878 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2024-09-16T02:02:38.878 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysctl.service 2024-09-16T02:02:38.878 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-16T02:02:38.878 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/nfs/rpc_pipefs 2024-09-16T02:02:38.878 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2024-09-16T02:02:38.879 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2024-09-16T02:02:38.879 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay 2024-09-16T02:02:38.879 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/f8e25acf528545476b19439e1e9ce6c1682cf6cb43e75e471a832161b24f6c03/merged 2024-09-16T02:02:38.879 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/79a23f9406bdef4dec1bbef5101ad22e9e0515e5d606e02e415f632fc38961ec/merged 2024-09-16T02:02:38.879 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/0 2024-09-16T02:02:38.879 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T02:02:38.879 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/3ed71bb2a385617fe36b24a1b7d1b73677d8edf74c5101efe1d4ead7d10f262d/merged 2024-09-16T02:02:38.879 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/f62cbd0ee28f2b881512d53ecbee7e6b5542ef8544d13cfc9f57f6927225a70a/merged 2024-09-16T02:02:38.879 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/3fd8f383cde748f25a9749aa12b35e6683b5a84d52cd9c5764a0314644afdc7a/merged 2024-09-16T02:02:38.879 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/8611c43c89f9fdb358775326ef6c2207e5ed0c60553c3a23838fb2b7428f7d78/merged 2024-09-16T02:02:38.879 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/16b1bc9f363600dae0a706674099181576c9eea2a239db4753d7951b9804e24c/merged 2024-09-16T02:02:38.879 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/b2bc631b6dc01ef6dca62c9f4817ef18f553e68c6e790a200150839028548f48/merged 2024-09-16T02:02:38.879 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/0d228658795e31555507b2ed93ba4ec97a98bf6caa9b42f783ff0f97140eee15/merged 2024-09-16T02:02:38.879 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/0a24ea473087d90f82d27b88ae4e46a1c69cfe4189ff475cfccc28a28610751f/merged 2024-09-16T02:02:38.880 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2024-09-16T02:02:38.880 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T02:02:38.880 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T02:02:38.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T02:02:38.880 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2024-09-16T02:02:38.900 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:2024-09-16T02:02:38.899+0000 7fefaee0c480 -1 init, newargv = 0x56175ded6a30 newargc=15 2024-09-16T02:02:38.900 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:ceph-fuse[83307]: starting ceph client 2024-09-16T02:02:38.920 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:38 smithi137 ceph-mon[34099]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T02:02:38.923 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:ceph-fuse[83307]: starting fuse 2024-09-16T02:02:40.017 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2024-09-16T02:02:40.050 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2024-09-16T02:02:40.109 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2024-09-16T02:02:40.109 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2024-09-16T02:02:40.109 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2024-09-16T02:02:40.110 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2024-09-16T02:02:40.110 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2024-09-16T02:02:40.110 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2024-09-16T02:02:40.110 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2024-09-16T02:02:40.110 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2024-09-16T02:02:40.110 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2024-09-16T02:02:40.110 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2024-09-16T02:02:40.110 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2024-09-16T02:02:40.110 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/selinux 2024-09-16T02:02:40.110 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T02:02:40.110 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2024-09-16T02:02:40.110 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2024-09-16T02:02:40.110 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2024-09-16T02:02:40.110 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2024-09-16T02:02:40.111 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-16T02:02:40.111 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2024-09-16T02:02:40.111 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2024-09-16T02:02:40.111 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysctl.service 2024-09-16T02:02:40.111 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-16T02:02:40.111 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/nfs/rpc_pipefs 2024-09-16T02:02:40.111 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2024-09-16T02:02:40.111 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2024-09-16T02:02:40.111 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay 2024-09-16T02:02:40.111 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/f8e25acf528545476b19439e1e9ce6c1682cf6cb43e75e471a832161b24f6c03/merged 2024-09-16T02:02:40.111 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/79a23f9406bdef4dec1bbef5101ad22e9e0515e5d606e02e415f632fc38961ec/merged 2024-09-16T02:02:40.111 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/0 2024-09-16T02:02:40.111 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T02:02:40.111 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/3ed71bb2a385617fe36b24a1b7d1b73677d8edf74c5101efe1d4ead7d10f262d/merged 2024-09-16T02:02:40.112 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/f62cbd0ee28f2b881512d53ecbee7e6b5542ef8544d13cfc9f57f6927225a70a/merged 2024-09-16T02:02:40.112 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/3fd8f383cde748f25a9749aa12b35e6683b5a84d52cd9c5764a0314644afdc7a/merged 2024-09-16T02:02:40.112 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/8611c43c89f9fdb358775326ef6c2207e5ed0c60553c3a23838fb2b7428f7d78/merged 2024-09-16T02:02:40.112 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/16b1bc9f363600dae0a706674099181576c9eea2a239db4753d7951b9804e24c/merged 2024-09-16T02:02:40.112 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/b2bc631b6dc01ef6dca62c9f4817ef18f553e68c6e790a200150839028548f48/merged 2024-09-16T02:02:40.112 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/0d228658795e31555507b2ed93ba4ec97a98bf6caa9b42f783ff0f97140eee15/merged 2024-09-16T02:02:40.112 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/0a24ea473087d90f82d27b88ae4e46a1c69cfe4189ff475cfccc28a28610751f/merged 2024-09-16T02:02:40.112 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2024-09-16T02:02:40.112 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T02:02:40.112 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T02:02:40.112 INFO:teuthology.orchestra.run.smithi105.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-16T02:02:40.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T02:02:40.113 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2024-09-16T02:02:40.167 INFO:teuthology.orchestra.run.smithi105.stdout:120 2024-09-16T02:02:40.167 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-16T02:02:40.168 DEBUG:teuthology.orchestra.run.smithi105:> sudo stdin-killer -- python3 -c ' 2024-09-16T02:02:40.168 DEBUG:teuthology.orchestra.run.smithi105:> import glob 2024-09-16T02:02:40.168 DEBUG:teuthology.orchestra.run.smithi105:> import re 2024-09-16T02:02:40.168 DEBUG:teuthology.orchestra.run.smithi105:> import os 2024-09-16T02:02:40.168 DEBUG:teuthology.orchestra.run.smithi105:> import subprocess 2024-09-16T02:02:40.168 DEBUG:teuthology.orchestra.run.smithi105:> 2024-09-16T02:02:40.168 DEBUG:teuthology.orchestra.run.smithi105:> def _find_admin_socket(client_name): 2024-09-16T02:02:40.168 DEBUG:teuthology.orchestra.run.smithi105:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-16T02:02:40.168 DEBUG:teuthology.orchestra.run.smithi105:> files = glob.glob(asok_path) 2024-09-16T02:02:40.168 DEBUG:teuthology.orchestra.run.smithi105:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-16T02:02:40.169 DEBUG:teuthology.orchestra.run.smithi105:> 2024-09-16T02:02:40.169 DEBUG:teuthology.orchestra.run.smithi105:> # Given a non-glob path, it better be there 2024-09-16T02:02:40.169 DEBUG:teuthology.orchestra.run.smithi105:> if "*" not in asok_path: 2024-09-16T02:02:40.169 DEBUG:teuthology.orchestra.run.smithi105:> assert(len(files) == 1) 2024-09-16T02:02:40.169 DEBUG:teuthology.orchestra.run.smithi105:> return files[0] 2024-09-16T02:02:40.169 DEBUG:teuthology.orchestra.run.smithi105:> 2024-09-16T02:02:40.169 DEBUG:teuthology.orchestra.run.smithi105:> for f in files: 2024-09-16T02:02:40.169 DEBUG:teuthology.orchestra.run.smithi105:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-16T02:02:40.169 DEBUG:teuthology.orchestra.run.smithi105:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-16T02:02:40.169 DEBUG:teuthology.orchestra.run.smithi105:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-16T02:02:40.169 DEBUG:teuthology.orchestra.run.smithi105:> contents = proc_f.read() 2024-09-16T02:02:40.169 DEBUG:teuthology.orchestra.run.smithi105:> if mountpoint in contents: 2024-09-16T02:02:40.169 DEBUG:teuthology.orchestra.run.smithi105:> return f 2024-09-16T02:02:40.169 DEBUG:teuthology.orchestra.run.smithi105:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-16T02:02:40.169 DEBUG:teuthology.orchestra.run.smithi105:> 2024-09-16T02:02:40.169 DEBUG:teuthology.orchestra.run.smithi105:> print(_find_admin_socket("client.0")) 2024-09-16T02:02:40.170 DEBUG:teuthology.orchestra.run.smithi105:> ' 2024-09-16T02:02:40.292 INFO:teuthology.orchestra.run.smithi105.stdout:/var/run/ceph/ceph-client.0.83307.asok 2024-09-16T02:02:40.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T02:02:40.303 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.83307.asok 2024-09-16T02:02:40.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T02:02:40.303 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.83307.asok status 2024-09-16T02:02:40.437 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-16T02:02:40.437 INFO:teuthology.orchestra.run.smithi105.stdout: "metadata": { 2024-09-16T02:02:40.437 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_sha1": "8921c8ef947807b717db041d85f0761b7914d5fc", 2024-09-16T02:02:40.437 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-16T02:02:40.437 INFO:teuthology.orchestra.run.smithi105.stdout: "entity_id": "0", 2024-09-16T02:02:40.437 INFO:teuthology.orchestra.run.smithi105.stdout: "hostname": "smithi105", 2024-09-16T02:02:40.437 INFO:teuthology.orchestra.run.smithi105.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-16T02:02:40.438 INFO:teuthology.orchestra.run.smithi105.stdout: "pid": "83307", 2024-09-16T02:02:40.438 INFO:teuthology.orchestra.run.smithi105.stdout: "root": "/volumes/qa/sv_0/d0aa8553-5106-4f12-b555-f9d49c86e9ca" 2024-09-16T02:02:40.438 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:02:40.438 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_count": 0, 2024-09-16T02:02:40.438 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_pinned_count": 0, 2024-09-16T02:02:40.438 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 15246, 2024-09-16T02:02:40.438 INFO:teuthology.orchestra.run.smithi105.stdout: "inst": { 2024-09-16T02:02:40.438 INFO:teuthology.orchestra.run.smithi105.stdout: "name": { 2024-09-16T02:02:40.438 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "client", 2024-09-16T02:02:40.438 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 15246 2024-09-16T02:02:40.438 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:02:40.438 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2024-09-16T02:02:40.438 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-16T02:02:40.438 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2024-09-16T02:02:40.439 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 1900637802 2024-09-16T02:02:40.439 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:02:40.439 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:02:40.439 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2024-09-16T02:02:40.439 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-16T02:02:40.439 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2024-09-16T02:02:40.439 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 1900637802 2024-09-16T02:02:40.439 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:02:40.439 INFO:teuthology.orchestra.run.smithi105.stdout: "inst_str": "client.15246 192.168.0.1:0/1900637802", 2024-09-16T02:02:40.439 INFO:teuthology.orchestra.run.smithi105.stdout: "addr_str": "192.168.0.1:0/1900637802", 2024-09-16T02:02:40.439 INFO:teuthology.orchestra.run.smithi105.stdout: "inode_count": 5, 2024-09-16T02:02:40.439 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_epoch": 27, 2024-09-16T02:02:40.439 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch": 81, 2024-09-16T02:02:40.439 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch_barrier": 0, 2024-09-16T02:02:40.440 INFO:teuthology.orchestra.run.smithi105.stdout: "blocklisted": false, 2024-09-16T02:02:40.440 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs" 2024-09-16T02:02:40.440 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-16T02:02:40.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T02:02:40.446 DEBUG:teuthology.orchestra.run.smithi105:> stat --file-system '--printf=%T 2024-09-16T02:02:40.446 DEBUG:teuthology.orchestra.run.smithi105:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-16T02:02:40.462 INFO:teuthology.orchestra.run.smithi105.stdout:fuseblk 2024-09-16T02:02:40.462 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-16T02:02:40.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T02:02:40.462 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T02:02:40.529 INFO:teuthology.run_tasks:Running task exec... 2024-09-16T02:02:40.539 INFO:teuthology.task.exec:Executing custom commands... 2024-09-16T02:02:40.539 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi105.front.sepia.ceph.com 2024-09-16T02:02:40.539 DEBUG:teuthology.orchestra.run.smithi105:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-16T02:02:40.592 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:40 smithi105 ceph-mon[31197]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-16T02:02:40.592 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:40 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-09-16T02:02:40.593 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:40 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 7, 6]}]: dispatch 2024-09-16T02:02:40.593 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:40 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.17", "id": [5, 1]}]: dispatch 2024-09-16T02:02:40.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.671+0000 7efe174c9640 1 -- 172.21.15.105:0/1043102554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe10103170 msgr2=0x7efe10103570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:40.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.671+0000 7efe174c9640 1 --2- 172.21.15.105:0/1043102554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe10103170 0x7efe10103570 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7efe08009930 tx=0x7efe0802a420 comp rx=0 tx=0).stop 2024-09-16T02:02:40.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.671+0000 7efe174c9640 1 -- 172.21.15.105:0/1043102554 shutdown_connections 2024-09-16T02:02:40.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.671+0000 7efe174c9640 1 --2- 172.21.15.105:0/1043102554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe100ffbe0 0x7efe10100060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:40.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.671+0000 7efe174c9640 1 --2- 172.21.15.105:0/1043102554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe101005a0 0x7efe1010a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:40.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.671+0000 7efe174c9640 1 --2- 172.21.15.105:0/1043102554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe10103170 0x7efe10103570 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:40.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.671+0000 7efe174c9640 1 -- 172.21.15.105:0/1043102554 >> 172.21.15.105:0/1043102554 conn(0x7efe100fb970 msgr2=0x7efe100fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:40.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.671+0000 7efe174c9640 1 -- 172.21.15.105:0/1043102554 shutdown_connections 2024-09-16T02:02:40.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.671+0000 7efe174c9640 1 -- 172.21.15.105:0/1043102554 wait complete. 2024-09-16T02:02:40.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.671+0000 7efe174c9640 1 Processor -- start 2024-09-16T02:02:40.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.671+0000 7efe174c9640 1 -- start start 2024-09-16T02:02:40.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.671+0000 7efe174c9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe100ffbe0 0x7efe10197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:40.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.671+0000 7efe174c9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe101005a0 0x7efe10198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:40.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.671+0000 7efe174c9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe10103170 0x7efe1019f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:40.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.671+0000 7efe174c9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efe1010d6a0 con 0x7efe101005a0 2024-09-16T02:02:40.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.671+0000 7efe174c9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efe1010d520 con 0x7efe100ffbe0 2024-09-16T02:02:40.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.671+0000 7efe174c9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7efe1010d820 con 0x7efe10103170 2024-09-16T02:02:40.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.672+0000 7efe14a64640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe101005a0 0x7efe10198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:40.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.672+0000 7efe14a64640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe101005a0 0x7efe10198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60494/0 (socket says 172.21.15.105:60494) 2024-09-16T02:02:40.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.672+0000 7efe14a64640 1 -- 172.21.15.105:0/2380259697 learned_addr learned my addr 172.21.15.105:0/2380259697 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:40.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.672+0000 7efe15265640 1 --2- 172.21.15.105:0/2380259697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe100ffbe0 0x7efe10197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:40.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.672+0000 7efe15a66640 1 --2- 172.21.15.105:0/2380259697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe10103170 0x7efe1019f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:40.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.672+0000 7efe14a64640 1 -- 172.21.15.105:0/2380259697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe10103170 msgr2=0x7efe1019f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:40.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.672+0000 7efe14a64640 1 --2- 172.21.15.105:0/2380259697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe10103170 0x7efe1019f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:40.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.672+0000 7efe14a64640 1 -- 172.21.15.105:0/2380259697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe100ffbe0 msgr2=0x7efe10197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:40.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.672+0000 7efe14a64640 1 --2- 172.21.15.105:0/2380259697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe100ffbe0 0x7efe10197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:40.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.672+0000 7efe14a64640 1 -- 172.21.15.105:0/2380259697 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe1019fa00 con 0x7efe101005a0 2024-09-16T02:02:40.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.672+0000 7efe14a64640 1 --2- 172.21.15.105:0/2380259697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe101005a0 0x7efe10198320 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7efe0400d950 tx=0x7efe0400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:40.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.672+0000 7efe0e7fc640 1 -- 172.21.15.105:0/2380259697 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe04017070 con 0x7efe101005a0 2024-09-16T02:02:40.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.672+0000 7efe174c9640 1 -- 172.21.15.105:0/2380259697 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe1019fcf0 con 0x7efe101005a0 2024-09-16T02:02:40.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.672+0000 7efe0e7fc640 1 -- 172.21.15.105:0/2380259697 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe04009c30 con 0x7efe101005a0 2024-09-16T02:02:40.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.672+0000 7efe0e7fc640 1 -- 172.21.15.105:0/2380259697 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0400f620 con 0x7efe101005a0 2024-09-16T02:02:40.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.672+0000 7efe174c9640 1 -- 172.21.15.105:0/2380259697 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe101a01e0 con 0x7efe101005a0 2024-09-16T02:02:40.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.673+0000 7efe174c9640 1 -- 172.21.15.105:0/2380259697 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe101091a0 con 0x7efe101005a0 2024-09-16T02:02:40.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.674+0000 7efe0e7fc640 1 -- 172.21.15.105:0/2380259697 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7efe040109a0 con 0x7efe101005a0 2024-09-16T02:02:40.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.675+0000 7efe0e7fc640 1 --2- 172.21.15.105:0/2380259697 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7efdf80a32f0 0x7efdf80a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:40.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.675+0000 7efe0e7fc640 1 -- 172.21.15.105:0/2380259697 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8335+0+0 (secure 0 0 0) 0x7efe040cf5a0 con 0x7efe101005a0 2024-09-16T02:02:40.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.675+0000 7efe15265640 1 --2- 172.21.15.105:0/2380259697 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7efdf80a32f0 0x7efdf80a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:40.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.676+0000 7efe15265640 1 --2- 172.21.15.105:0/2380259697 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7efdf80a32f0 0x7efdf80a57b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efe08004770 tx=0x7efe08000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:40.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.679+0000 7efe0e7fc640 1 -- 172.21.15.105:0/2380259697 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efe04009de0 con 0x7efe101005a0 2024-09-16T02:02:40.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:40 smithi190 ceph-mon[34762]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-16T02:02:40.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:40 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-09-16T02:02:40.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:40 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 7, 6]}]: dispatch 2024-09-16T02:02:40.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:40 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.17", "id": [5, 1]}]: dispatch 2024-09-16T02:02:40.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:40.884+0000 7efe174c9640 1 -- 172.21.15.105:0/2380259697 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7efe10062e90 con 0x7efe101005a0 2024-09-16T02:02:40.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:40 smithi137 ceph-mon[34099]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-16T02:02:40.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:40 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-09-16T02:02:40.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:40 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 7, 6]}]: dispatch 2024-09-16T02:02:40.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:40 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.17", "id": [5, 1]}]: dispatch 2024-09-16T02:02:41.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.560+0000 7efe0e7fc640 1 -- 172.21.15.105:0/2380259697 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v28) ==== 117+0+0 (secure 0 0 0) 0x7efe0409c260 con 0x7efe101005a0 2024-09-16T02:02:41.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.562+0000 7efe174c9640 1 -- 172.21.15.105:0/2380259697 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7efdf80a32f0 msgr2=0x7efdf80a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:41.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.562+0000 7efe174c9640 1 --2- 172.21.15.105:0/2380259697 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7efdf80a32f0 0x7efdf80a57b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efe08004770 tx=0x7efe08000bc0 comp rx=0 tx=0).stop 2024-09-16T02:02:41.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.562+0000 7efe174c9640 1 -- 172.21.15.105:0/2380259697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe101005a0 msgr2=0x7efe10198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:41.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.562+0000 7efe174c9640 1 --2- 172.21.15.105:0/2380259697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe101005a0 0x7efe10198320 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7efe0400d950 tx=0x7efe0400de10 comp rx=0 tx=0).stop 2024-09-16T02:02:41.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.562+0000 7efe174c9640 1 -- 172.21.15.105:0/2380259697 shutdown_connections 2024-09-16T02:02:41.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.562+0000 7efe174c9640 1 --2- 172.21.15.105:0/2380259697 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7efdf80a32f0 0x7efdf80a57b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:41.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.562+0000 7efe174c9640 1 --2- 172.21.15.105:0/2380259697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe10103170 0x7efe1019f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:41.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.562+0000 7efe174c9640 1 --2- 172.21.15.105:0/2380259697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe101005a0 0x7efe10198320 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:41.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.562+0000 7efe174c9640 1 --2- 172.21.15.105:0/2380259697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe100ffbe0 0x7efe10197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:41.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.562+0000 7efe174c9640 1 -- 172.21.15.105:0/2380259697 >> 172.21.15.105:0/2380259697 conn(0x7efe100fb970 msgr2=0x7efe10106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:41.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.563+0000 7efe174c9640 1 -- 172.21.15.105:0/2380259697 shutdown_connections 2024-09-16T02:02:41.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.563+0000 7efe174c9640 1 -- 172.21.15.105:0/2380259697 wait complete. 2024-09-16T02:02:41.577 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-16T02:02:41.588 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-16T02:02:41.631 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T02:02:41.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.732+0000 7f91b8665640 1 -- 172.21.15.105:0/399364542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91b0100a10 msgr2=0x7f91b0100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:41.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.732+0000 7f91b8665640 1 --2- 172.21.15.105:0/399364542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91b0100a10 0x7f91b0100e10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f91ac009900 tx=0x7f91ac02a410 comp rx=0 tx=0).stop 2024-09-16T02:02:41.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.732+0000 7f91b8665640 1 -- 172.21.15.105:0/399364542 shutdown_connections 2024-09-16T02:02:41.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.732+0000 7f91b8665640 1 --2- 172.21.15.105:0/399364542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91b01025c0 0x7f91b0108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:41.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.732+0000 7f91b8665640 1 --2- 172.21.15.105:0/399364542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91b0101c00 0x7f91b0102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:41.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.732+0000 7f91b8665640 1 --2- 172.21.15.105:0/399364542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91b0100a10 0x7f91b0100e10 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:41.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.732+0000 7f91b8665640 1 -- 172.21.15.105:0/399364542 >> 172.21.15.105:0/399364542 conn(0x7f91b00fc170 msgr2=0x7f91b00fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:41.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.732+0000 7f91b8665640 1 -- 172.21.15.105:0/399364542 shutdown_connections 2024-09-16T02:02:41.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.732+0000 7f91b8665640 1 -- 172.21.15.105:0/399364542 wait complete. 2024-09-16T02:02:41.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.733+0000 7f91b8665640 1 Processor -- start 2024-09-16T02:02:41.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.733+0000 7f91b8665640 1 -- start start 2024-09-16T02:02:41.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.733+0000 7f91b8665640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91b0100a10 0x7f91b0197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:41.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.733+0000 7f91b8665640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91b0101c00 0x7f91b01982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:41.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.733+0000 7f91b8665640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91b01025c0 0x7f91b019f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:41.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.733+0000 7f91b8665640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f91b010c540 con 0x7f91b0101c00 2024-09-16T02:02:41.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.733+0000 7f91b8665640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f91b010c3c0 con 0x7f91b01025c0 2024-09-16T02:02:41.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.733+0000 7f91b8665640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f91b010c6c0 con 0x7f91b0100a10 2024-09-16T02:02:41.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.733+0000 7f91b5c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91b0101c00 0x7f91b01982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:41.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.733+0000 7f91b5c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91b0101c00 0x7f91b01982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60516/0 (socket says 172.21.15.105:60516) 2024-09-16T02:02:41.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.733+0000 7f91b5c00640 1 -- 172.21.15.105:0/3978658784 learned_addr learned my addr 172.21.15.105:0/3978658784 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:41.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.734+0000 7f91b5c00640 1 -- 172.21.15.105:0/3978658784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91b0100a10 msgr2=0x7f91b0197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:41.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.734+0000 7f91b5c00640 1 --2- 172.21.15.105:0/3978658784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91b0100a10 0x7f91b0197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:41.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.734+0000 7f91b5c00640 1 -- 172.21.15.105:0/3978658784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91b01025c0 msgr2=0x7f91b019f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:41.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.734+0000 7f91b5c00640 1 --2- 172.21.15.105:0/3978658784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91b01025c0 0x7f91b019f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:41.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.734+0000 7f91b5c00640 1 -- 172.21.15.105:0/3978658784 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b019f9b0 con 0x7f91b0101c00 2024-09-16T02:02:41.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.734+0000 7f91b5c00640 1 --2- 172.21.15.105:0/3978658784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91b0101c00 0x7f91b01982e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f91a400e9f0 tx=0x7f91a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:41.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.734+0000 7f91a37fe640 1 -- 172.21.15.105:0/3978658784 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a400cd80 con 0x7f91b0101c00 2024-09-16T02:02:41.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.734+0000 7f91b8665640 1 -- 172.21.15.105:0/3978658784 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91b019fca0 con 0x7f91b0101c00 2024-09-16T02:02:41.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.734+0000 7f91a37fe640 1 -- 172.21.15.105:0/3978658784 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f91a40108e0 con 0x7f91b0101c00 2024-09-16T02:02:41.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.734+0000 7f91a37fe640 1 -- 172.21.15.105:0/3978658784 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a4018920 con 0x7f91b0101c00 2024-09-16T02:02:41.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.734+0000 7f91b8665640 1 -- 172.21.15.105:0/3978658784 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91b01a01e0 con 0x7f91b0101c00 2024-09-16T02:02:41.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.735+0000 7f91b8665640 1 -- 172.21.15.105:0/3978658784 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91b01a0410 con 0x7f91b0101c00 2024-09-16T02:02:41.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.736+0000 7f91a37fe640 1 -- 172.21.15.105:0/3978658784 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f91a4010a90 con 0x7f91b0101c00 2024-09-16T02:02:41.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.737+0000 7f91a37fe640 1 --2- 172.21.15.105:0/3978658784 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f919c0a3250 0x7f919c0a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:41.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.737+0000 7f91a37fe640 1 -- 172.21.15.105:0/3978658784 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f91a40cfe30 con 0x7f91b0101c00 2024-09-16T02:02:41.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.737+0000 7f91b6401640 1 --2- 172.21.15.105:0/3978658784 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f919c0a3250 0x7f919c0a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:41.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.737+0000 7f91b6401640 1 --2- 172.21.15.105:0/3978658784 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f919c0a3250 0x7f919c0a5710 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f91ac02a930 tx=0x7f91ac0314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:41.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.741+0000 7f91a37fe640 1 -- 172.21.15.105:0/3978658784 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f91a409caf0 con 0x7f91b0101c00 2024-09-16T02:02:41.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:41 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-09-16T02:02:41.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:41 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 7, 6]}]': finished 2024-09-16T02:02:41.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:41 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.17", "id": [5, 1]}]': finished 2024-09-16T02:02:41.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:41 smithi190 ceph-mon[34762]: osdmap e82: 12 total, 12 up, 12 in 2024-09-16T02:02:41.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:41 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/2380259697' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-16T02:02:41.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.911+0000 7f91b8665640 1 -- 172.21.15.105:0/3978658784 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f91b01a0d00 con 0x7f91b0101c00 2024-09-16T02:02:41.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.912+0000 7f91a37fe640 1 -- 172.21.15.105:0/3978658784 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22962 (secure 0 0 0) 0x7f91a409c3f0 con 0x7f91b0101c00 2024-09-16T02:02:41.917 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:02:41.917 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":83,"fsid":"423b7a2a-73ce-11ef-bceb-c7b262605968","created":"2024-09-16T01:52:44.388033+0000","modified":"2024-09-16T02:02:41.553985+0000","last_up_change":"2024-09-16T01:59:42.218034+0000","last_in_change":"2024-09-16T01:59:18.785252+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T01:55:53.680808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T02:00:31.286117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T02:00:33.310045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a60d6c72-da8b-4b02-bb33-672be795e7f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6803","nonce":1226660290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6805","nonce":1226660290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6809","nonce":1226660290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6807","nonce":1226660290}]},"public_addr":"172.21.15.105:6803/1226660290","cluster_addr":"172.21.15.105:6805/1226660290","heartbeat_back_addr":"172.21.15.105:6809/1226660290","heartbeat_front_addr":"172.21.15.105:6807/1226660290","state":["exists","up"]},{"osd":1,"uuid":"b1d9ae84-b9a0-4a7e-b0fe-c402d46809fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6801","nonce":34753956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6803","nonce":34753956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6807","nonce":34753956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6805","nonce":34753956}]},"public_addr":"172.21.15.137:6801/34753956","cluster_addr":"172.21.15.137:6803/34753956","heartbeat_back_addr":"172.21.15.137:6807/34753956","heartbeat_front_addr":"172.21.15.137:6805/34753956","state":["exists","up"]},{"osd":2,"uuid":"137faae6-2e8b-44b5-8976-3d95a6eef1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6801","nonce":1159531918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6803","nonce":1159531918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6807","nonce":1159531918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6805","nonce":1159531918}]},"public_addr":"172.21.15.190:6801/1159531918","cluster_addr":"172.21.15.190:6803/1159531918","heartbeat_back_addr":"172.21.15.190:6807/1159531918","heartbeat_front_addr":"172.21.15.190:6805/1159531918","state":["exists","up"]},{"osd":3,"uuid":"5743c37b-5cd8-42b6-81c8-5cf5afc71e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6811","nonce":604905679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6813","nonce":604905679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6817","nonce":604905679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6815","nonce":604905679}]},"public_addr":"172.21.15.105:6811/604905679","cluster_addr":"172.21.15.105:6813/604905679","heartbeat_back_addr":"172.21.15.105:6817/604905679","heartbeat_front_addr":"172.21.15.105:6815/604905679","state":["exists","up"]},{"osd":4,"uuid":"e439d4a2-b54c-4de9-97f6-3510afe57f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6809","nonce":3758109169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6811","nonce":3758109169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6815","nonce":3758109169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6813","nonce":3758109169}]},"public_addr":"172.21.15.137:6809/3758109169","cluster_addr":"172.21.15.137:6811/3758109169","heartbeat_back_addr":"172.21.15.137:6815/3758109169","heartbeat_front_addr":"172.21.15.137:6813/3758109169","state":["exists","up"]},{"osd":5,"uuid":"d1e2a7d5-99f8-40fa-af6e-0305cc0af677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6809","nonce":2086773754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6811","nonce":2086773754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6815","nonce":2086773754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6813","nonce":2086773754}]},"public_addr":"172.21.15.190:6809/2086773754","cluster_addr":"172.21.15.190:6811/2086773754","heartbeat_back_addr":"172.21.15.190:6815/2086773754","heartbeat_front_addr":"172.21.15.190:6813/2086773754","state":["exists","up"]},{"osd":6,"uuid":"cd418159-a0f6-44c7-9213-c010b132fb56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6819","nonce":4012018725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6821","nonce":4012018725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6825","nonce":4012018725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6823","nonce":4012018725}]},"public_addr":"172.21.15.105:6819/4012018725","cluster_addr":"172.21.15.105:6821/4012018725","heartbeat_back_addr":"172.21.15.105:6825/4012018725","heartbeat_front_addr":"172.21.15.105:6823/4012018725","state":["exists","up"]},{"osd":7,"uuid":"76c15863-801e-4a72-be73-ba9df46c15b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6817","nonce":888678288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6819","nonce":888678288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6823","nonce":888678288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6821","nonce":888678288}]},"public_addr":"172.21.15.137:6817/888678288","cluster_addr":"172.21.15.137:6819/888678288","heartbeat_back_addr":"172.21.15.137:6823/888678288","heartbeat_front_addr":"172.21.15.137:6821/888678288","state":["exists","up"]},{"osd":8,"uuid":"6718597c-6f42-48ee-9493-edb2396fe707","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6817","nonce":2132638341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6819","nonce":2132638341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6823","nonce":2132638341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6821","nonce":2132638341}]},"public_addr":"172.21.15.190:6817/2132638341","cluster_addr":"172.21.15.190:6819/2132638341","heartbeat_back_addr":"172.21.15.190:6823/2132638341","heartbeat_front_addr":"172.21.15.190:6821/2132638341","state":["exists","up"]},{"osd":9,"uuid":"5178edc0-a2e8-491c-821c-6f41ec5aeb2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6827","nonce":3375034979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6829","nonce":3375034979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6833","nonce":3375034979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6831","nonce":3375034979}]},"public_addr":"172.21.15.105:6827/3375034979","cluster_addr":"172.21.15.105:6829/3375034979","heartbeat_back_addr":"172.21.15.105:6833/3375034979","heartbeat_front_addr":"172.21.15.105:6831/3375034979","state":["exists","up"]},{"osd":10,"uuid":"ad145d3e-a6e2-4dd7-bea3-bcd01eb3413e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6825","nonce":318562234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6827","nonce":318562234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6831","nonce":318562234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6829","nonce":318562234}]},"public_addr":"172.21.15.137:6825/318562234","cluster_addr":"172.21.15.137:6827/318562234","heartbeat_back_addr":"172.21.15.137:6831/318562234","heartbeat_front_addr":"172.21.15.137:6829/318562234","state":["exists","up"]},{"osd":11,"uuid":"6d44bced-3a8b-4763-a93e-0ea34acb4cb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6825","nonce":3820460689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6827","nonce":3820460689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6831","nonce":3820460689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6829","nonce":3820460689}]},"public_addr":"172.21.15.190:6825/3820460689","cluster_addr":"172.21.15.190:6827/3820460689","heartbeat_back_addr":"172.21.15.190:6831/3820460689","heartbeat_front_addr":"172.21.15.190:6829/3820460689","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:55:02.759033+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:55:26.829142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:55:51.120200+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:56:17.277870+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:56:42.313701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:07.013416+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:32.477916+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:58.643764+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:23.454908+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:48.869082+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:14.921573+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:40.908993+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":7,"to":6}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.17","mappings":[{"from":5,"to":1}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":7}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:0/2453739657":"2024-09-17T01:53:39.351314+0000","172.21.15.105:6801/1469189484":"2024-09-17T01:53:39.351314+0000","172.21.15.105:0/3466166879":"2024-09-17T01:53:39.351314+0000","172.21.15.105:6801/1268348742":"2024-09-17T01:53:15.522227+0000","172.21.15.105:0/2897248478":"2024-09-17T01:53:39.351314+0000","172.21.15.105:0/3709363371":"2024-09-17T01:53:15.522227+0000","172.21.15.105:6800/1268348742":"2024-09-17T01:53:15.522227+0000","172.21.15.105:6800/1469189484":"2024-09-17T01:53:39.351314+0000","172.21.15.105:0/1198176299":"2024-09-17T01:53:15.522227+0000","172.21.15.105:0/3031917848":"2024-09-17T01:53:15.522227+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T02:02:41.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.914+0000 7f91b8665640 1 -- 172.21.15.105:0/3978658784 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f919c0a3250 msgr2=0x7f919c0a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:41.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.914+0000 7f91b8665640 1 --2- 172.21.15.105:0/3978658784 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f919c0a3250 0x7f919c0a5710 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f91ac02a930 tx=0x7f91ac0314a0 comp rx=0 tx=0).stop 2024-09-16T02:02:41.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.914+0000 7f91b8665640 1 -- 172.21.15.105:0/3978658784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91b0101c00 msgr2=0x7f91b01982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:41.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.914+0000 7f91b8665640 1 --2- 172.21.15.105:0/3978658784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91b0101c00 0x7f91b01982e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f91a400e9f0 tx=0x7f91a400eeb0 comp rx=0 tx=0).stop 2024-09-16T02:02:41.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.914+0000 7f91b8665640 1 -- 172.21.15.105:0/3978658784 shutdown_connections 2024-09-16T02:02:41.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.914+0000 7f91b8665640 1 --2- 172.21.15.105:0/3978658784 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f919c0a3250 0x7f919c0a5710 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:41.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.914+0000 7f91b8665640 1 --2- 172.21.15.105:0/3978658784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91b01025c0 0x7f91b019f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:41.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.914+0000 7f91b8665640 1 --2- 172.21.15.105:0/3978658784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91b0101c00 0x7f91b01982e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:41.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.914+0000 7f91b8665640 1 --2- 172.21.15.105:0/3978658784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91b0100a10 0x7f91b0197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:41.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.914+0000 7f91b8665640 1 -- 172.21.15.105:0/3978658784 >> 172.21.15.105:0/3978658784 conn(0x7f91b00fc170 msgr2=0x7f91b00693b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:41.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.914+0000 7f91b8665640 1 -- 172.21.15.105:0/3978658784 shutdown_connections 2024-09-16T02:02:41.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:41.914+0000 7f91b8665640 1 -- 172.21.15.105:0/3978658784 wait complete. 2024-09-16T02:02:41.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:41 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-09-16T02:02:41.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:41 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 7, 6]}]': finished 2024-09-16T02:02:41.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:41 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.17", "id": [5, 1]}]': finished 2024-09-16T02:02:41.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:41 smithi137 ceph-mon[34099]: osdmap e82: 12 total, 12 up, 12 in 2024-09-16T02:02:41.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:41 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/2380259697' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-16T02:02:41.925 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:41 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-09-16T02:02:41.925 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:41 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 7, 6]}]': finished 2024-09-16T02:02:41.925 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:41 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.17", "id": [5, 1]}]': finished 2024-09-16T02:02:41.925 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:41 smithi105 ceph-mon[31197]: osdmap e82: 12 total, 12 up, 12 in 2024-09-16T02:02:41.925 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:41 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2380259697' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-16T02:02:41.927 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T01:55:53.680808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-16T02:00:31.286117+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-16T02:00:33.310045+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T02:02:41.927 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T02:02:42.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.029+0000 7f009d8b7640 1 -- 172.21.15.105:0/1421475605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0098101c00 msgr2=0x7f0098102080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:42.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.029+0000 7f009d8b7640 1 --2- 172.21.15.105:0/1421475605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0098101c00 0x7f0098102080 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0080009900 tx=0x7f008002a410 comp rx=0 tx=0).stop 2024-09-16T02:02:42.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.029+0000 7f009d8b7640 1 -- 172.21.15.105:0/1421475605 shutdown_connections 2024-09-16T02:02:42.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.029+0000 7f009d8b7640 1 --2- 172.21.15.105:0/1421475605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00981025c0 0x7f0098108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.029+0000 7f009d8b7640 1 --2- 172.21.15.105:0/1421475605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0098101c00 0x7f0098102080 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.029+0000 7f009d8b7640 1 --2- 172.21.15.105:0/1421475605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0098100a10 0x7f0098100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.029+0000 7f009d8b7640 1 -- 172.21.15.105:0/1421475605 >> 172.21.15.105:0/1421475605 conn(0x7f00980fc190 msgr2=0x7f00980fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:42.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.029+0000 7f009d8b7640 1 -- 172.21.15.105:0/1421475605 shutdown_connections 2024-09-16T02:02:42.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.029+0000 7f009d8b7640 1 -- 172.21.15.105:0/1421475605 wait complete. 2024-09-16T02:02:42.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.029+0000 7f009d8b7640 1 Processor -- start 2024-09-16T02:02:42.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.029+0000 7f009d8b7640 1 -- start start 2024-09-16T02:02:42.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.030+0000 7f009d8b7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0098100a10 0x7f0098196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:42.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.030+0000 7f009d8b7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0098101c00 0x7f0098197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:42.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.030+0000 7f009d8b7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00981025c0 0x7f009819e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:42.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.030+0000 7f009d8b7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f009810c520 con 0x7f00981025c0 2024-09-16T02:02:42.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.030+0000 7f009d8b7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f009810c3a0 con 0x7f0098101c00 2024-09-16T02:02:42.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.030+0000 7f009d8b7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f009810c6a0 con 0x7f0098100a10 2024-09-16T02:02:42.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.030+0000 7f00977fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00981025c0 0x7f009819e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:42.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.030+0000 7f00977fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00981025c0 0x7f009819e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60538/0 (socket says 172.21.15.105:60538) 2024-09-16T02:02:42.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.030+0000 7f00977fe640 1 -- 172.21.15.105:0/2141458111 learned_addr learned my addr 172.21.15.105:0/2141458111 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:42.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.030+0000 7f00967fc640 1 --2- 172.21.15.105:0/2141458111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0098101c00 0x7f0098197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:42.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.030+0000 7f00977fe640 1 -- 172.21.15.105:0/2141458111 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0098100a10 msgr2=0x7f0098196d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:42.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.030+0000 7f0096ffd640 1 --2- 172.21.15.105:0/2141458111 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0098100a10 0x7f0098196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:42.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.030+0000 7f00977fe640 1 --2- 172.21.15.105:0/2141458111 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0098100a10 0x7f0098196d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.030+0000 7f00977fe640 1 -- 172.21.15.105:0/2141458111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0098101c00 msgr2=0x7f0098197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:42.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.030+0000 7f00977fe640 1 --2- 172.21.15.105:0/2141458111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0098101c00 0x7f0098197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.030+0000 7f00977fe640 1 -- 172.21.15.105:0/2141458111 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f009819e910 con 0x7f00981025c0 2024-09-16T02:02:42.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.030+0000 7f0096ffd640 1 --2- 172.21.15.105:0/2141458111 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0098100a10 0x7f0098196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:02:42.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.030+0000 7f00977fe640 1 --2- 172.21.15.105:0/2141458111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00981025c0 0x7f009819e270 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f009000cad0 tx=0x7f009000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:42.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.030+0000 7f009c8b5640 1 -- 172.21.15.105:0/2141458111 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009000ae20 con 0x7f00981025c0 2024-09-16T02:02:42.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.030+0000 7f009c8b5640 1 -- 172.21.15.105:0/2141458111 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0090005020 con 0x7f00981025c0 2024-09-16T02:02:42.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.030+0000 7f009d8b7640 1 -- 172.21.15.105:0/2141458111 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f009819ec00 con 0x7f00981025c0 2024-09-16T02:02:42.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.030+0000 7f009c8b5640 1 -- 172.21.15.105:0/2141458111 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009000f6b0 con 0x7f00981025c0 2024-09-16T02:02:42.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.030+0000 7f009d8b7640 1 -- 172.21.15.105:0/2141458111 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f009819f100 con 0x7f00981025c0 2024-09-16T02:02:42.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.031+0000 7f009d8b7640 1 -- 172.21.15.105:0/2141458111 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0098107f40 con 0x7f00981025c0 2024-09-16T02:02:42.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.032+0000 7f009c8b5640 1 -- 172.21.15.105:0/2141458111 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f00900051d0 con 0x7f00981025c0 2024-09-16T02:02:42.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.033+0000 7f009c8b5640 1 --2- 172.21.15.105:0/2141458111 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f00840a3250 0x7f00840a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:42.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.033+0000 7f0096ffd640 1 --2- 172.21.15.105:0/2141458111 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f00840a3250 0x7f00840a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:42.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.033+0000 7f009c8b5640 1 -- 172.21.15.105:0/2141458111 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f00900cfb90 con 0x7f00981025c0 2024-09-16T02:02:42.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.034+0000 7f0096ffd640 1 --2- 172.21.15.105:0/2141458111 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f00840a3250 0x7f00840a5710 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f00981981d0 tx=0x7f0088005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:42.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.037+0000 7f009c8b5640 1 -- 172.21.15.105:0/2141458111 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0090014030 con 0x7f00981025c0 2024-09-16T02:02:42.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.205+0000 7f009d8b7640 1 -- 172.21.15.105:0/2141458111 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f009802d7c0 con 0x7f00981025c0 2024-09-16T02:02:42.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.205+0000 7f009c8b5640 1 -- 172.21.15.105:0/2141458111 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7f009802d7c0 con 0x7f00981025c0 2024-09-16T02:02:42.208 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-09-16T02:02:42.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.207+0000 7f009d8b7640 1 -- 172.21.15.105:0/2141458111 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f00840a3250 msgr2=0x7f00840a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:42.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.207+0000 7f009d8b7640 1 --2- 172.21.15.105:0/2141458111 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f00840a3250 0x7f00840a5710 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f00981981d0 tx=0x7f0088005d50 comp rx=0 tx=0).stop 2024-09-16T02:02:42.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.207+0000 7f009d8b7640 1 -- 172.21.15.105:0/2141458111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00981025c0 msgr2=0x7f009819e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:42.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.207+0000 7f009d8b7640 1 --2- 172.21.15.105:0/2141458111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00981025c0 0x7f009819e270 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f009000cad0 tx=0x7f009000cf90 comp rx=0 tx=0).stop 2024-09-16T02:02:42.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.207+0000 7f009d8b7640 1 -- 172.21.15.105:0/2141458111 shutdown_connections 2024-09-16T02:02:42.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.207+0000 7f009d8b7640 1 --2- 172.21.15.105:0/2141458111 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f00840a3250 0x7f00840a5710 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.207+0000 7f009d8b7640 1 --2- 172.21.15.105:0/2141458111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00981025c0 0x7f009819e270 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.207+0000 7f009d8b7640 1 --2- 172.21.15.105:0/2141458111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0098101c00 0x7f0098197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.207+0000 7f009d8b7640 1 --2- 172.21.15.105:0/2141458111 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0098100a10 0x7f0098196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.207+0000 7f009d8b7640 1 -- 172.21.15.105:0/2141458111 >> 172.21.15.105:0/2141458111 conn(0x7f00980fc190 msgr2=0x7f0098105110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:42.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.207+0000 7f009d8b7640 1 -- 172.21.15.105:0/2141458111 shutdown_connections 2024-09-16T02:02:42.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.208+0000 7f009d8b7640 1 -- 172.21.15.105:0/2141458111 wait complete. 2024-09-16T02:02:42.220 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-16T02:02:42.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.323+0000 7f2e5923c640 1 -- 172.21.15.105:0/3322383397 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e541077e0 msgr2=0x7f2e54109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:42.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.323+0000 7f2e5923c640 1 --2- 172.21.15.105:0/3322383397 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e541077e0 0x7f2e54109c50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c009ca0 tx=0x7f2e4c02a8c0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.323+0000 7f2e5923c640 1 -- 172.21.15.105:0/3322383397 shutdown_connections 2024-09-16T02:02:42.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.323+0000 7f2e5923c640 1 --2- 172.21.15.105:0/3322383397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e54104eb0 0x7f2e541072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.323+0000 7f2e5923c640 1 --2- 172.21.15.105:0/3322383397 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e541077e0 0x7f2e54109c50 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.323+0000 7f2e5923c640 1 --2- 172.21.15.105:0/3322383397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e54102580 0x7f2e54104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.323+0000 7f2e5923c640 1 -- 172.21.15.105:0/3322383397 >> 172.21.15.105:0/3322383397 conn(0x7f2e540fc140 msgr2=0x7f2e540fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:42.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.323+0000 7f2e5923c640 1 -- 172.21.15.105:0/3322383397 shutdown_connections 2024-09-16T02:02:42.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.323+0000 7f2e5923c640 1 -- 172.21.15.105:0/3322383397 wait complete. 2024-09-16T02:02:42.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.324+0000 7f2e5923c640 1 Processor -- start 2024-09-16T02:02:42.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.324+0000 7f2e5923c640 1 -- start start 2024-09-16T02:02:42.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.324+0000 7f2e5923c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e54102580 0x7f2e54197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:42.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.324+0000 7f2e5923c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e54104eb0 0x7f2e541982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:42.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.324+0000 7f2e5923c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e541077e0 0x7f2e5419f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:42.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.324+0000 7f2e5923c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5410d670 con 0x7f2e541077e0 2024-09-16T02:02:42.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.324+0000 7f2e5923c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5410d4f0 con 0x7f2e54102580 2024-09-16T02:02:42.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.324+0000 7f2e5923c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5410d7f0 con 0x7f2e54104eb0 2024-09-16T02:02:42.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.324+0000 7f2e5359e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e541077e0 0x7f2e5419f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:42.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.324+0000 7f2e5359e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e541077e0 0x7f2e5419f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60554/0 (socket says 172.21.15.105:60554) 2024-09-16T02:02:42.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.324+0000 7f2e5359e640 1 -- 172.21.15.105:0/3365406418 learned_addr learned my addr 172.21.15.105:0/3365406418 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:42.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.324+0000 7f2e4bfff640 1 --2- 172.21.15.105:0/3365406418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e54104eb0 0x7f2e541982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:42.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.324+0000 7f2e52d9d640 1 --2- 172.21.15.105:0/3365406418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e54102580 0x7f2e54197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:42.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.325+0000 7f2e4bfff640 1 -- 172.21.15.105:0/3365406418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e54102580 msgr2=0x7f2e54197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:42.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.325+0000 7f2e4bfff640 1 --2- 172.21.15.105:0/3365406418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e54102580 0x7f2e54197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.325+0000 7f2e4bfff640 1 -- 172.21.15.105:0/3365406418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e541077e0 msgr2=0x7f2e5419f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:42.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.325+0000 7f2e4bfff640 1 --2- 172.21.15.105:0/3365406418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e541077e0 0x7f2e5419f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.325+0000 7f2e4bfff640 1 -- 172.21.15.105:0/3365406418 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e5419f9b0 con 0x7f2e54104eb0 2024-09-16T02:02:42.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.325+0000 7f2e5359e640 1 --2- 172.21.15.105:0/3365406418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e541077e0 0x7f2e5419f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:02:42.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.325+0000 7f2e4bfff640 1 --2- 172.21.15.105:0/3365406418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e54104eb0 0x7f2e541982e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c00e9c0 tx=0x7f2e3c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:42.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.325+0000 7f2e50d99640 1 -- 172.21.15.105:0/3365406418 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c00ccc0 con 0x7f2e54104eb0 2024-09-16T02:02:42.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.325+0000 7f2e5923c640 1 -- 172.21.15.105:0/3365406418 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e5419fca0 con 0x7f2e54104eb0 2024-09-16T02:02:42.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.325+0000 7f2e50d99640 1 -- 172.21.15.105:0/3365406418 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e3c005020 con 0x7f2e54104eb0 2024-09-16T02:02:42.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.325+0000 7f2e50d99640 1 -- 172.21.15.105:0/3365406418 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c004a00 con 0x7f2e54104eb0 2024-09-16T02:02:42.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.325+0000 7f2e5923c640 1 -- 172.21.15.105:0/3365406418 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e541a01a0 con 0x7f2e54104eb0 2024-09-16T02:02:42.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.326+0000 7f2e5923c640 1 -- 172.21.15.105:0/3365406418 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e540fdc30 con 0x7f2e54104eb0 2024-09-16T02:02:42.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.328+0000 7f2e50d99640 1 -- 172.21.15.105:0/3365406418 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f2e3c0051d0 con 0x7f2e54104eb0 2024-09-16T02:02:42.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.329+0000 7f2e50d99640 1 --2- 172.21.15.105:0/3365406418 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f2e400a3250 0x7f2e400a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:42.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.329+0000 7f2e52d9d640 1 --2- 172.21.15.105:0/3365406418 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f2e400a3250 0x7f2e400a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:42.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.329+0000 7f2e50d99640 1 -- 172.21.15.105:0/3365406418 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f2e3c0cfa80 con 0x7f2e54104eb0 2024-09-16T02:02:42.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.329+0000 7f2e52d9d640 1 --2- 172.21.15.105:0/3365406418 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f2e400a3250 0x7f2e400a5710 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2e54199270 tx=0x7f2e44005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:42.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.333+0000 7f2e50d99640 1 -- 172.21.15.105:0/3365406418 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2e3c09c6e0 con 0x7f2e54104eb0 2024-09-16T02:02:42.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.510+0000 7f2e5923c640 1 -- 172.21.15.105:0/3365406418 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2e5402da10 con 0x7f2e54104eb0 2024-09-16T02:02:42.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.511+0000 7f2e50d99640 1 -- 172.21.15.105:0/3365406418 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) ==== 104+0+11 (secure 0 0 0) 0x7f2e3c09bfe0 con 0x7f2e54104eb0 2024-09-16T02:02:42.512 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-09-16T02:02:42.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.513+0000 7f2e5923c640 1 -- 172.21.15.105:0/3365406418 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f2e400a3250 msgr2=0x7f2e400a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:42.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.513+0000 7f2e5923c640 1 --2- 172.21.15.105:0/3365406418 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f2e400a3250 0x7f2e400a5710 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2e54199270 tx=0x7f2e44005f90 comp rx=0 tx=0).stop 2024-09-16T02:02:42.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.513+0000 7f2e5923c640 1 -- 172.21.15.105:0/3365406418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e54104eb0 msgr2=0x7f2e541982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:42.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.513+0000 7f2e5923c640 1 --2- 172.21.15.105:0/3365406418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e54104eb0 0x7f2e541982e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c00e9c0 tx=0x7f2e3c00ee80 comp rx=0 tx=0).stop 2024-09-16T02:02:42.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.513+0000 7f2e5923c640 1 -- 172.21.15.105:0/3365406418 shutdown_connections 2024-09-16T02:02:42.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.513+0000 7f2e5923c640 1 --2- 172.21.15.105:0/3365406418 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f2e400a3250 0x7f2e400a5710 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.513+0000 7f2e5923c640 1 --2- 172.21.15.105:0/3365406418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e54104eb0 0x7f2e541982e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.513+0000 7f2e5923c640 1 --2- 172.21.15.105:0/3365406418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e541077e0 0x7f2e5419f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.513+0000 7f2e5923c640 1 --2- 172.21.15.105:0/3365406418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e54102580 0x7f2e54197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.513+0000 7f2e5923c640 1 -- 172.21.15.105:0/3365406418 >> 172.21.15.105:0/3365406418 conn(0x7f2e540fc140 msgr2=0x7f2e54103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:42.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.513+0000 7f2e5923c640 1 -- 172.21.15.105:0/3365406418 shutdown_connections 2024-09-16T02:02:42.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.513+0000 7f2e5923c640 1 -- 172.21.15.105:0/3365406418 wait complete. 2024-09-16T02:02:42.525 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-16T02:02:42.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.635+0000 7efef648b640 1 -- 172.21.15.105:0/3272941406 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efef0101bf0 msgr2=0x7efef0102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:42.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.635+0000 7efef648b640 1 --2- 172.21.15.105:0/3272941406 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efef0101bf0 0x7efef0102070 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7efee4009900 tx=0x7efee402a410 comp rx=0 tx=0).stop 2024-09-16T02:02:42.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.635+0000 7efef648b640 1 -- 172.21.15.105:0/3272941406 shutdown_connections 2024-09-16T02:02:42.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.635+0000 7efef648b640 1 --2- 172.21.15.105:0/3272941406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efef01025b0 0x7efef0108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.635+0000 7efef648b640 1 --2- 172.21.15.105:0/3272941406 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efef0101bf0 0x7efef0102070 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.635+0000 7efef648b640 1 --2- 172.21.15.105:0/3272941406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efef0100a00 0x7efef0100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.635+0000 7efef648b640 1 -- 172.21.15.105:0/3272941406 >> 172.21.15.105:0/3272941406 conn(0x7efef00fc140 msgr2=0x7efef00fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:42.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.636+0000 7efef648b640 1 -- 172.21.15.105:0/3272941406 shutdown_connections 2024-09-16T02:02:42.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.636+0000 7efef648b640 1 -- 172.21.15.105:0/3272941406 wait complete. 2024-09-16T02:02:42.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.636+0000 7efef648b640 1 Processor -- start 2024-09-16T02:02:42.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.636+0000 7efef648b640 1 -- start start 2024-09-16T02:02:42.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.636+0000 7efef648b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efef0100a00 0x7efef0069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:42.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.636+0000 7efef648b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efef0101bf0 0x7efef019cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:42.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.637+0000 7efef648b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efef01025b0 0x7efef019f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:42.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.637+0000 7efef648b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efef010c510 con 0x7efef0101bf0 2024-09-16T02:02:42.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.637+0000 7efef648b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efef010c390 con 0x7efef0100a00 2024-09-16T02:02:42.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.637+0000 7efef648b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7efef010c690 con 0x7efef01025b0 2024-09-16T02:02:42.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.637+0000 7efeef7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efef0101bf0 0x7efef019cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:42.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.637+0000 7efeef7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efef0101bf0 0x7efef019cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60568/0 (socket says 172.21.15.105:60568) 2024-09-16T02:02:42.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.637+0000 7efeef7fe640 1 -- 172.21.15.105:0/2876079957 learned_addr learned my addr 172.21.15.105:0/2876079957 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:42.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.637+0000 7efef4a28640 1 --2- 172.21.15.105:0/2876079957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efef01025b0 0x7efef019f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:42.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.637+0000 7efeeffff640 1 --2- 172.21.15.105:0/2876079957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efef0100a00 0x7efef0069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:42.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.637+0000 7efeef7fe640 1 -- 172.21.15.105:0/2876079957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efef01025b0 msgr2=0x7efef019f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:42.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.637+0000 7efeef7fe640 1 --2- 172.21.15.105:0/2876079957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efef01025b0 0x7efef019f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.637+0000 7efeef7fe640 1 -- 172.21.15.105:0/2876079957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efef0100a00 msgr2=0x7efef0069f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:42.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.637+0000 7efeef7fe640 1 --2- 172.21.15.105:0/2876079957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efef0100a00 0x7efef0069f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.637+0000 7efeef7fe640 1 -- 172.21.15.105:0/2876079957 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef019f9d0 con 0x7efef0101bf0 2024-09-16T02:02:42.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.637+0000 7efeeffff640 1 --2- 172.21.15.105:0/2876079957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efef0100a00 0x7efef0069f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:02:42.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.637+0000 7efef4a28640 1 --2- 172.21.15.105:0/2876079957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efef01025b0 0x7efef019f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:02:42.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.637+0000 7efeef7fe640 1 --2- 172.21.15.105:0/2876079957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efef0101bf0 0x7efef019cac0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7efee4005940 tx=0x7efee40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:42.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.637+0000 7efeed7fa640 1 -- 172.21.15.105:0/2876079957 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee403f030 con 0x7efef0101bf0 2024-09-16T02:02:42.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.637+0000 7efeed7fa640 1 -- 172.21.15.105:0/2876079957 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efee40318f0 con 0x7efef0101bf0 2024-09-16T02:02:42.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.637+0000 7efeed7fa640 1 -- 172.21.15.105:0/2876079957 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee4035070 con 0x7efef0101bf0 2024-09-16T02:02:42.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.637+0000 7efef648b640 1 -- 172.21.15.105:0/2876079957 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efef019fc60 con 0x7efef0101bf0 2024-09-16T02:02:42.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.638+0000 7efef648b640 1 -- 172.21.15.105:0/2876079957 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efef01a0150 con 0x7efef0101bf0 2024-09-16T02:02:42.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.639+0000 7efef648b640 1 -- 172.21.15.105:0/2876079957 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efef0107f30 con 0x7efef0101bf0 2024-09-16T02:02:42.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.640+0000 7efeed7fa640 1 -- 172.21.15.105:0/2876079957 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7efee4042030 con 0x7efef0101bf0 2024-09-16T02:02:42.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.641+0000 7efeed7fa640 1 --2- 172.21.15.105:0/2876079957 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7efed80a3250 0x7efed80a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:42.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.641+0000 7efeed7fa640 1 -- 172.21.15.105:0/2876079957 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7efee40efdc0 con 0x7efef0101bf0 2024-09-16T02:02:42.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.641+0000 7efeeffff640 1 --2- 172.21.15.105:0/2876079957 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7efed80a3250 0x7efed80a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:42.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.642+0000 7efeeffff640 1 --2- 172.21.15.105:0/2876079957 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7efed80a3250 0x7efed80a5710 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efef0101a50 tx=0x7efee0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:42.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.645+0000 7efeed7fa640 1 -- 172.21.15.105:0/2876079957 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efee40bc430 con 0x7efef0101bf0 2024-09-16T02:02:42.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:42 smithi105 ceph-mon[31197]: pgmap v343: 97 pgs: 97 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 716 B/s wr, 10 op/s 2024-09-16T02:02:42.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:42 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2380259697' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-16T02:02:42.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:42 smithi105 ceph-mon[31197]: osdmap e83: 12 total, 12 up, 12 in 2024-09-16T02:02:42.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:42 smithi105 ceph-mon[31197]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T02:02:42.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:42 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/3978658784' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:02:42.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:42 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2141458111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T02:02:42.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:42 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:02:42.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:42 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/3365406418' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T02:02:42.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.825+0000 7efef648b640 1 -- 172.21.15.105:0/2876079957 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efef0062e20 con 0x7efef0101bf0 2024-09-16T02:02:42.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.826+0000 7efeed7fa640 1 -- 172.21.15.105:0/2876079957 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) ==== 100+0+11 (secure 0 0 0) 0x7efee40c0730 con 0x7efef0101bf0 2024-09-16T02:02:42.827 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-09-16T02:02:42.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.828+0000 7efef648b640 1 -- 172.21.15.105:0/2876079957 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7efed80a3250 msgr2=0x7efed80a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:42.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.828+0000 7efef648b640 1 --2- 172.21.15.105:0/2876079957 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7efed80a3250 0x7efed80a5710 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efef0101a50 tx=0x7efee0005f90 comp rx=0 tx=0).stop 2024-09-16T02:02:42.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.828+0000 7efef648b640 1 -- 172.21.15.105:0/2876079957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efef0101bf0 msgr2=0x7efef019cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:42.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.828+0000 7efef648b640 1 --2- 172.21.15.105:0/2876079957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efef0101bf0 0x7efef019cac0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7efee4005940 tx=0x7efee40042f0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.828+0000 7efef648b640 1 -- 172.21.15.105:0/2876079957 shutdown_connections 2024-09-16T02:02:42.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.828+0000 7efef648b640 1 --2- 172.21.15.105:0/2876079957 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7efed80a3250 0x7efed80a5710 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.828+0000 7efef648b640 1 --2- 172.21.15.105:0/2876079957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efef01025b0 0x7efef019f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.828+0000 7efef648b640 1 --2- 172.21.15.105:0/2876079957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efef0101bf0 0x7efef019cac0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.828+0000 7efef648b640 1 --2- 172.21.15.105:0/2876079957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efef0100a00 0x7efef0069f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.828+0000 7efef648b640 1 -- 172.21.15.105:0/2876079957 >> 172.21.15.105:0/2876079957 conn(0x7efef00fc140 msgr2=0x7efef0105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:42.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.828+0000 7efef648b640 1 -- 172.21.15.105:0/2876079957 shutdown_connections 2024-09-16T02:02:42.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:42.828+0000 7efef648b640 1 -- 172.21.15.105:0/2876079957 wait complete. 2024-09-16T02:02:42.840 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-16T02:02:42.840 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:02:42.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:42 smithi190 ceph-mon[34762]: pgmap v343: 97 pgs: 97 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 716 B/s wr, 10 op/s 2024-09-16T02:02:42.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:42 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/2380259697' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-16T02:02:42.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:42 smithi190 ceph-mon[34762]: osdmap e83: 12 total, 12 up, 12 in 2024-09-16T02:02:42.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:42 smithi190 ceph-mon[34762]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T02:02:42.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:42 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/3978658784' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:02:42.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:42 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/2141458111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T02:02:42.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:42 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:02:42.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:42 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/3365406418' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T02:02:42.900 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:42 smithi137 ceph-mon[34099]: pgmap v343: 97 pgs: 97 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 716 B/s wr, 10 op/s 2024-09-16T02:02:42.900 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:42 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/2380259697' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-16T02:02:42.900 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:42 smithi137 ceph-mon[34099]: osdmap e83: 12 total, 12 up, 12 in 2024-09-16T02:02:42.900 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:42 smithi137 ceph-mon[34099]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T02:02:42.900 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:42 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/3978658784' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:02:42.900 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:42 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/2141458111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T02:02:42.900 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:42 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:02:42.900 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:42 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/3365406418' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T02:02:43.446 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:02:43.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:43 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/2876079957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T02:02:43.887 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:43 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2876079957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T02:02:43.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:43 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/2876079957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T02:02:43.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.965+0000 7f5de97a8640 1 -- 172.21.15.105:0/1377694229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5de4074590 msgr2=0x7f5de4074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:43.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.965+0000 7f5de97a8640 1 --2- 172.21.15.105:0/1377694229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5de4074590 0x7f5de4074970 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4009900 tx=0x7f5dd402a480 comp rx=0 tx=0).stop 2024-09-16T02:02:43.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.965+0000 7f5de97a8640 1 -- 172.21.15.105:0/1377694229 shutdown_connections 2024-09-16T02:02:43.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.965+0000 7f5de97a8640 1 --2- 172.21.15.105:0/1377694229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5de4112b90 0x7f5de4114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:43.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.965+0000 7f5de97a8640 1 --2- 172.21.15.105:0/1377694229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5de4074eb0 0x7f5de4112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:43.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.965+0000 7f5de97a8640 1 --2- 172.21.15.105:0/1377694229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5de4074590 0x7f5de4074970 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:43.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.965+0000 7f5de97a8640 1 -- 172.21.15.105:0/1377694229 >> 172.21.15.105:0/1377694229 conn(0x7f5de40fd8c0 msgr2=0x7f5de40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:43.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.965+0000 7f5de97a8640 1 -- 172.21.15.105:0/1377694229 shutdown_connections 2024-09-16T02:02:43.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.965+0000 7f5de97a8640 1 -- 172.21.15.105:0/1377694229 wait complete. 2024-09-16T02:02:43.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.966+0000 7f5de97a8640 1 Processor -- start 2024-09-16T02:02:43.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.966+0000 7f5de97a8640 1 -- start start 2024-09-16T02:02:43.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.966+0000 7f5de97a8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5de4074590 0x7f5de41a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:43.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.966+0000 7f5de97a8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5de4074eb0 0x7f5de41a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:43.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.966+0000 7f5de97a8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5de4112b90 0x7f5de419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:43.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.966+0000 7f5de97a8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5de4118490 con 0x7f5de4074eb0 2024-09-16T02:02:43.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.966+0000 7f5de97a8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5de4118310 con 0x7f5de4074590 2024-09-16T02:02:43.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.966+0000 7f5de97a8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5de4118610 con 0x7f5de4112b90 2024-09-16T02:02:43.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.967+0000 7f5dda7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5de4074eb0 0x7f5de41a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:43.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.967+0000 7f5dda7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5de4074eb0 0x7f5de41a4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60574/0 (socket says 172.21.15.105:60574) 2024-09-16T02:02:43.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.967+0000 7f5dda7fc640 1 -- 172.21.15.105:0/1074913569 learned_addr learned my addr 172.21.15.105:0/1074913569 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:43.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.967+0000 7f5de2ffd640 1 --2- 172.21.15.105:0/1074913569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5de4074590 0x7f5de41a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:43.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.967+0000 7f5de37fe640 1 --2- 172.21.15.105:0/1074913569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5de4112b90 0x7f5de419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:43.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.967+0000 7f5dda7fc640 1 -- 172.21.15.105:0/1074913569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5de4112b90 msgr2=0x7f5de419e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:43.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.967+0000 7f5dda7fc640 1 --2- 172.21.15.105:0/1074913569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5de4112b90 0x7f5de419e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:43.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.967+0000 7f5dda7fc640 1 -- 172.21.15.105:0/1074913569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5de4074590 msgr2=0x7f5de41a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:43.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.967+0000 7f5dda7fc640 1 --2- 172.21.15.105:0/1074913569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5de4074590 0x7f5de41a46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:43.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.967+0000 7f5dda7fc640 1 -- 172.21.15.105:0/1074913569 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de419ef90 con 0x7f5de4074eb0 2024-09-16T02:02:43.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.967+0000 7f5dda7fc640 1 --2- 172.21.15.105:0/1074913569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5de4074eb0 0x7f5de41a4be0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc00b4a0 tx=0x7f5dcc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:43.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.967+0000 7f5de0ff9640 1 -- 172.21.15.105:0/1074913569 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc014070 con 0x7f5de4074eb0 2024-09-16T02:02:43.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.967+0000 7f5de0ff9640 1 -- 172.21.15.105:0/1074913569 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dcc010ac0 con 0x7f5de4074eb0 2024-09-16T02:02:43.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.967+0000 7f5de97a8640 1 -- 172.21.15.105:0/1074913569 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5de419f280 con 0x7f5de4074eb0 2024-09-16T02:02:43.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.967+0000 7f5de0ff9640 1 -- 172.21.15.105:0/1074913569 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc017c30 con 0x7f5de4074eb0 2024-09-16T02:02:43.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.967+0000 7f5de97a8640 1 -- 172.21.15.105:0/1074913569 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5de41ab570 con 0x7f5de4074eb0 2024-09-16T02:02:43.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.968+0000 7f5de97a8640 1 -- 172.21.15.105:0/1074913569 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5de4074970 con 0x7f5de4074eb0 2024-09-16T02:02:43.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.970+0000 7f5de0ff9640 1 -- 172.21.15.105:0/1074913569 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f5dcc017070 con 0x7f5de4074eb0 2024-09-16T02:02:43.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.970+0000 7f5de0ff9640 1 --2- 172.21.15.105:0/1074913569 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5dd00a3240 0x7f5dd00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:43.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.970+0000 7f5de2ffd640 1 --2- 172.21.15.105:0/1074913569 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5dd00a3240 0x7f5dd00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:43.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.970+0000 7f5de0ff9640 1 -- 172.21.15.105:0/1074913569 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f5dcc0cf010 con 0x7f5de4074eb0 2024-09-16T02:02:43.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.971+0000 7f5de2ffd640 1 --2- 172.21.15.105:0/1074913569 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5dd00a3240 0x7f5dd00a5700 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4000c00 tx=0x7f5dd4031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:43.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:43.973+0000 7f5de0ff9640 1 -- 172.21.15.105:0/1074913569 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5dcc09b680 con 0x7f5de4074eb0 2024-09-16T02:02:44.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:44.253+0000 7f5de97a8640 1 -- 172.21.15.105:0/1074913569 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5de419fd90 con 0x7f5de4074eb0 2024-09-16T02:02:44.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:44.254+0000 7f5de0ff9640 1 -- 172.21.15.105:0/1074913569 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11991 (secure 0 0 0) 0x7f5de419fd90 con 0x7f5de4074eb0 2024-09-16T02:02:44.255 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:02:44.255 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":28,"btime":"2024-09-16T02:02:41:554178+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2057699920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2057699920},{"type":"v1","addr":"172.21.15.105:6835","nonce":2057699920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1764886948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1764886948},{"type":"v1","addr":"172.21.15.105:6837","nonce":1764886948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T02:00:35.112266+0000","modified":"2024-09-16T02:02:40.885614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24469,"mds_2":24461,"mds_3":24457,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/4112062361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":4112062361},{"type":"v1","addr":"172.21.15.105:6839","nonce":4112062361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6841/163156514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":163156514},{"type":"v1","addr":"172.21.15.105:6841","nonce":163156514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/683519097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":683519097},{"type":"v1","addr":"172.21.15.190:6833","nonce":683519097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3025442862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3025442862},{"type":"v1","addr":"172.21.15.137:6833","nonce":3025442862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/763783602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":763783602},{"type":"v1","addr":"172.21.15.137:6835","nonce":763783602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.190:6835/1851042281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1851042281},{"type":"v1","addr":"172.21.15.190:6835","nonce":1851042281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/3181257549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3181257549},{"type":"v1","addr":"172.21.15.190:6837","nonce":3181257549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6837/2788541030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2788541030},{"type":"v1","addr":"172.21.15.137:6837","nonce":2788541030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.190:6839/1222573537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":1222573537},{"type":"v1","addr":"172.21.15.190:6839","nonce":1222573537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/3454924009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3454924009},{"type":"v1","addr":"172.21.15.137:6839","nonce":3454924009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24445,24457,24461,24469,24482]},"id":1}]} 2024-09-16T02:02:44.255 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 28 2024-09-16T02:02:44.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:44.257+0000 7f5de97a8640 1 -- 172.21.15.105:0/1074913569 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5dd00a3240 msgr2=0x7f5dd00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:44.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:44.257+0000 7f5de97a8640 1 --2- 172.21.15.105:0/1074913569 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5dd00a3240 0x7f5dd00a5700 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4000c00 tx=0x7f5dd4031400 comp rx=0 tx=0).stop 2024-09-16T02:02:44.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:44.257+0000 7f5de97a8640 1 -- 172.21.15.105:0/1074913569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5de4074eb0 msgr2=0x7f5de41a4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:44.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:44.257+0000 7f5de97a8640 1 --2- 172.21.15.105:0/1074913569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5de4074eb0 0x7f5de41a4be0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc00b4a0 tx=0x7f5dcc00b960 comp rx=0 tx=0).stop 2024-09-16T02:02:44.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:44.257+0000 7f5de97a8640 1 -- 172.21.15.105:0/1074913569 shutdown_connections 2024-09-16T02:02:44.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:44.257+0000 7f5de97a8640 1 --2- 172.21.15.105:0/1074913569 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5dd00a3240 0x7f5dd00a5700 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:44.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:44.257+0000 7f5de97a8640 1 --2- 172.21.15.105:0/1074913569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5de4112b90 0x7f5de419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:44.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:44.257+0000 7f5de97a8640 1 --2- 172.21.15.105:0/1074913569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5de4074eb0 0x7f5de41a4be0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:44.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:44.257+0000 7f5de97a8640 1 --2- 172.21.15.105:0/1074913569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5de4074590 0x7f5de41a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:44.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:44.257+0000 7f5de97a8640 1 -- 172.21.15.105:0/1074913569 >> 172.21.15.105:0/1074913569 conn(0x7f5de40fd8c0 msgr2=0x7f5de40ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:44.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:44.257+0000 7f5de97a8640 1 -- 172.21.15.105:0/1074913569 shutdown_connections 2024-09-16T02:02:44.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:44.257+0000 7f5de97a8640 1 -- 172.21.15.105:0/1074913569 wait complete. 2024-09-16T02:02:44.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:44 smithi190 ceph-mon[34762]: pgmap v345: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.4 KiB/s wr, 11 op/s; 22 B/s, 0 objects/s recovering 2024-09-16T02:02:44.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:44 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:02:44.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:44 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:02:44.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:44 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:02:44.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:44 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:02:44.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:44 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/1074913569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:02:44.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:44 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:02:44.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:44 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:02:44.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:44 smithi137 ceph-mon[34099]: pgmap v345: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.4 KiB/s wr, 11 op/s; 22 B/s, 0 objects/s recovering 2024-09-16T02:02:44.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:44 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:02:44.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:44 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:02:44.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:44 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:02:44.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:44 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:02:44.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:44 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/1074913569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:02:44.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:44 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:02:44.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:44 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:02:44.950 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:44 smithi105 ceph-mon[31197]: pgmap v345: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.4 KiB/s wr, 11 op/s; 22 B/s, 0 objects/s recovering 2024-09-16T02:02:44.950 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:44 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:02:44.950 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:44 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:02:44.950 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:44 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:02:44.951 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:44 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:02:44.951 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:44 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/1074913569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:02:44.951 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:44 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:02:44.951 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:44 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:02:44.980 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-16T02:02:44.980 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-16T02:02:44.981 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T02:02:45.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.090+0000 7f68173e4640 1 -- 172.21.15.105:0/1730660509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6810104ec0 msgr2=0x7f68101072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:45.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.090+0000 7f68173e4640 1 --2- 172.21.15.105:0/1730660509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6810104ec0 0x7f68101072b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6800009900 tx=0x7f680002a410 comp rx=0 tx=0).stop 2024-09-16T02:02:45.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.090+0000 7f68173e4640 1 -- 172.21.15.105:0/1730660509 shutdown_connections 2024-09-16T02:02:45.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.090+0000 7f68173e4640 1 --2- 172.21.15.105:0/1730660509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68101077f0 0x7f6810109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.090+0000 7f68173e4640 1 --2- 172.21.15.105:0/1730660509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6810104ec0 0x7f68101072b0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.090+0000 7f68173e4640 1 --2- 172.21.15.105:0/1730660509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6810102590 0x7f6810104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.090+0000 7f68173e4640 1 -- 172.21.15.105:0/1730660509 >> 172.21.15.105:0/1730660509 conn(0x7f68100fc170 msgr2=0x7f68100fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:45.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.091+0000 7f68173e4640 1 -- 172.21.15.105:0/1730660509 shutdown_connections 2024-09-16T02:02:45.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.091+0000 7f68173e4640 1 -- 172.21.15.105:0/1730660509 wait complete. 2024-09-16T02:02:45.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.091+0000 7f68173e4640 1 Processor -- start 2024-09-16T02:02:45.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.091+0000 7f68173e4640 1 -- start start 2024-09-16T02:02:45.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.091+0000 7f68173e4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6810102590 0x7f6810197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:45.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.091+0000 7f68173e4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6810104ec0 0x7f68101983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:45.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.091+0000 7f68173e4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68101077f0 0x7f681019f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:45.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.091+0000 7f68173e4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f681010d700 con 0x7f68101077f0 2024-09-16T02:02:45.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.091+0000 7f68173e4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f681010d580 con 0x7f6810102590 2024-09-16T02:02:45.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.091+0000 7f68173e4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f681010d880 con 0x7f6810104ec0 2024-09-16T02:02:45.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.091+0000 7f6815981640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68101077f0 0x7f681019f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:45.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.091+0000 7f6815981640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68101077f0 0x7f681019f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60598/0 (socket says 172.21.15.105:60598) 2024-09-16T02:02:45.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.091+0000 7f6815981640 1 -- 172.21.15.105:0/2151681043 learned_addr learned my addr 172.21.15.105:0/2151681043 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:45.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.092+0000 7f6815180640 1 --2- 172.21.15.105:0/2151681043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6810102590 0x7f6810197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:45.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.092+0000 7f681497f640 1 --2- 172.21.15.105:0/2151681043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6810104ec0 0x7f68101983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:45.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.092+0000 7f6815981640 1 -- 172.21.15.105:0/2151681043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6810104ec0 msgr2=0x7f68101983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:45.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.092+0000 7f6815981640 1 --2- 172.21.15.105:0/2151681043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6810104ec0 0x7f68101983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.092+0000 7f6815981640 1 -- 172.21.15.105:0/2151681043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6810102590 msgr2=0x7f6810197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:45.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.092+0000 7f6815981640 1 --2- 172.21.15.105:0/2151681043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6810102590 0x7f6810197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.092+0000 7f6815981640 1 -- 172.21.15.105:0/2151681043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f681019fad0 con 0x7f68101077f0 2024-09-16T02:02:45.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.092+0000 7f681497f640 1 --2- 172.21.15.105:0/2151681043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6810104ec0 0x7f68101983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:02:45.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.092+0000 7f6815180640 1 --2- 172.21.15.105:0/2151681043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6810102590 0x7f6810197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:02:45.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.092+0000 7f6815981640 1 --2- 172.21.15.105:0/2151681043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68101077f0 0x7f681019f460 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f680800eb40 tx=0x7f680800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:45.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.092+0000 7f68067fc640 1 -- 172.21.15.105:0/2151681043 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6808004b20 con 0x7f68101077f0 2024-09-16T02:02:45.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.092+0000 7f68067fc640 1 -- 172.21.15.105:0/2151681043 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6808004cc0 con 0x7f68101077f0 2024-09-16T02:02:45.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.092+0000 7f68173e4640 1 -- 172.21.15.105:0/2151681043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f681019fdc0 con 0x7f68101077f0 2024-09-16T02:02:45.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.092+0000 7f68067fc640 1 -- 172.21.15.105:0/2151681043 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68080107b0 con 0x7f68101077f0 2024-09-16T02:02:45.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.092+0000 7f68173e4640 1 -- 172.21.15.105:0/2151681043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68101a0360 con 0x7f68101077f0 2024-09-16T02:02:45.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.093+0000 7f68173e4640 1 -- 172.21.15.105:0/2151681043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68100fdc60 con 0x7f68101077f0 2024-09-16T02:02:45.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.094+0000 7f68067fc640 1 -- 172.21.15.105:0/2151681043 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f6808005020 con 0x7f68101077f0 2024-09-16T02:02:45.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.095+0000 7f68067fc640 1 --2- 172.21.15.105:0/2151681043 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f67fc0a32d0 0x7f67fc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:45.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.095+0000 7f6815180640 1 --2- 172.21.15.105:0/2151681043 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f67fc0a32d0 0x7f67fc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:45.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.095+0000 7f68067fc640 1 -- 172.21.15.105:0/2151681043 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f68080d01a0 con 0x7f68101077f0 2024-09-16T02:02:45.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.095+0000 7f6815180640 1 --2- 172.21.15.105:0/2151681043 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f67fc0a32d0 0x7f67fc0a5790 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f680c006fd0 tx=0x7f680c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:45.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.099+0000 7f68067fc640 1 -- 172.21.15.105:0/2151681043 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f680809ce60 con 0x7f68101077f0 2024-09-16T02:02:45.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.301+0000 7f68173e4640 1 -- 172.21.15.105:0/2151681043 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6810062630 con 0x7f67fc0a32d0 2024-09-16T02:02:45.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.307+0000 7f68067fc640 1 -- 172.21.15.105:0/2151681043 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259151 (secure 0 0 0) 0x7f6810062630 con 0x7f67fc0a32d0 2024-09-16T02:02:45.311 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:02:45.313 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-09-16T02:02:45.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.310+0000 7f68173e4640 1 -- 172.21.15.105:0/2151681043 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f67fc0a32d0 msgr2=0x7f67fc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:45.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.310+0000 7f68173e4640 1 --2- 172.21.15.105:0/2151681043 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f67fc0a32d0 0x7f67fc0a5790 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f680c006fd0 tx=0x7f680c008040 comp rx=0 tx=0).stop 2024-09-16T02:02:45.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.310+0000 7f68173e4640 1 -- 172.21.15.105:0/2151681043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68101077f0 msgr2=0x7f681019f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:45.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.310+0000 7f68173e4640 1 --2- 172.21.15.105:0/2151681043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68101077f0 0x7f681019f460 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f680800eb40 tx=0x7f680800c2d0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.310+0000 7f68173e4640 1 -- 172.21.15.105:0/2151681043 shutdown_connections 2024-09-16T02:02:45.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.310+0000 7f68173e4640 1 --2- 172.21.15.105:0/2151681043 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f67fc0a32d0 0x7f67fc0a5790 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.310+0000 7f68173e4640 1 --2- 172.21.15.105:0/2151681043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6810104ec0 0x7f68101983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.310+0000 7f68173e4640 1 --2- 172.21.15.105:0/2151681043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68101077f0 0x7f681019f460 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.310+0000 7f68173e4640 1 --2- 172.21.15.105:0/2151681043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6810102590 0x7f6810197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.310+0000 7f68173e4640 1 -- 172.21.15.105:0/2151681043 >> 172.21.15.105:0/2151681043 conn(0x7f68100fc170 msgr2=0x7f6810102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:45.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.310+0000 7f68173e4640 1 -- 172.21.15.105:0/2151681043 shutdown_connections 2024-09-16T02:02:45.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.310+0000 7f68173e4640 1 -- 172.21.15.105:0/2151681043 wait complete. 2024-09-16T02:02:45.322 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-09-16T02:02:43.475455+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658998,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1303,"num_read_kb":775,"num_write":441,"num_write_kb":3086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1181615,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":861,"ondisk_log_size":861,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372800,"kb_used_data":11052,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569024,"statfs":{"total":1151940427776,"available":1151558680576,"internally_reserved":0,"allocated":11317248,"data_stored":7191219,"data_compressed":191111,"data_compressed_allocated":253952,"data_compressed_original":2007040,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1980,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":47,"num_write":13,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":180,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002130"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.629463+0000","last_change":"2024-09-16T02:00:34.454121+0000","last_active":"2024-09-16T02:02:41.629463+0000","last_peered":"2024-09-16T02:02:41.629463+0000","last_clean":"2024-09-16T02:02:41.629463+0000","last_became_active":"2024-09-16T02:00:34.453810+0000","last_became_peered":"2024-09-16T02:00:34.453810+0000","last_unstale":"2024-09-16T02:02:41.629463+0000","last_undegraded":"2024-09-16T02:02:41.629463+0000","last_fullsized":"2024-09-16T02:02:41.629463+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:42:38.781928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.449067+0000","last_change":"2024-09-16T02:00:34.462020+0000","last_active":"2024-09-16T02:02:09.449067+0000","last_peered":"2024-09-16T02:02:09.449067+0000","last_clean":"2024-09-16T02:02:09.449067+0000","last_became_active":"2024-09-16T02:00:34.461680+0000","last_became_peered":"2024-09-16T02:00:34.461680+0000","last_unstale":"2024-09-16T02:02:09.449067+0000","last_undegraded":"2024-09-16T02:02:09.449067+0000","last_fullsized":"2024-09-16T02:02:09.449067+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:00:14.493643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.448082+0000","last_change":"2024-09-16T02:00:34.453034+0000","last_active":"2024-09-16T02:02:09.448082+0000","last_peered":"2024-09-16T02:02:09.448082+0000","last_clean":"2024-09-16T02:02:09.448082+0000","last_became_active":"2024-09-16T02:00:34.452719+0000","last_became_peered":"2024-09-16T02:00:34.452719+0000","last_unstale":"2024-09-16T02:02:09.448082+0000","last_undegraded":"2024-09-16T02:02:09.448082+0000","last_fullsized":"2024-09-16T02:02:09.448082+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:48:52.478501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.451501+0000","last_change":"2024-09-16T02:00:34.458156+0000","last_active":"2024-09-16T02:02:09.451501+0000","last_peered":"2024-09-16T02:02:09.451501+0000","last_clean":"2024-09-16T02:02:09.451501+0000","last_became_active":"2024-09-16T02:00:34.457831+0000","last_became_peered":"2024-09-16T02:00:34.457831+0000","last_unstale":"2024-09-16T02:02:09.451501+0000","last_undegraded":"2024-09-16T02:02:09.451501+0000","last_fullsized":"2024-09-16T02:02:09.451501+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:35:36.609046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.426325+0000","last_change":"2024-09-16T02:00:34.448638+0000","last_active":"2024-09-16T02:02:09.426325+0000","last_peered":"2024-09-16T02:02:09.426325+0000","last_clean":"2024-09-16T02:02:09.426325+0000","last_became_active":"2024-09-16T02:00:34.447993+0000","last_became_peered":"2024-09-16T02:00:34.447993+0000","last_unstale":"2024-09-16T02:02:09.426325+0000","last_undegraded":"2024-09-16T02:02:09.426325+0000","last_fullsized":"2024-09-16T02:02:09.426325+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:33:04.832053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.566521+0000","last_change":"2024-09-16T02:00:34.462651+0000","last_active":"2024-09-16T02:02:41.566521+0000","last_peered":"2024-09-16T02:02:41.566521+0000","last_clean":"2024-09-16T02:02:41.566521+0000","last_became_active":"2024-09-16T02:00:34.461837+0000","last_became_peered":"2024-09-16T02:00:34.461837+0000","last_unstale":"2024-09-16T02:02:41.566521+0000","last_undegraded":"2024-09-16T02:02:41.566521+0000","last_fullsized":"2024-09-16T02:02:41.566521+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:52:13.733591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.439301+0000","last_change":"2024-09-16T02:00:34.455362+0000","last_active":"2024-09-16T02:02:09.439301+0000","last_peered":"2024-09-16T02:02:09.439301+0000","last_clean":"2024-09-16T02:02:09.439301+0000","last_became_active":"2024-09-16T02:00:34.455029+0000","last_became_peered":"2024-09-16T02:00:34.455029+0000","last_unstale":"2024-09-16T02:02:09.439301+0000","last_undegraded":"2024-09-16T02:02:09.439301+0000","last_fullsized":"2024-09-16T02:02:09.439301+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:14:08.316845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.443005+0000","last_change":"2024-09-16T02:00:34.459578+0000","last_active":"2024-09-16T02:02:09.443005+0000","last_peered":"2024-09-16T02:02:09.443005+0000","last_clean":"2024-09-16T02:02:09.443005+0000","last_became_active":"2024-09-16T02:00:34.459288+0000","last_became_peered":"2024-09-16T02:00:34.459288+0000","last_unstale":"2024-09-16T02:02:09.443005+0000","last_undegraded":"2024-09-16T02:02:09.443005+0000","last_fullsized":"2024-09-16T02:02:09.443005+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:52:28.974276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.458685+0000","last_change":"2024-09-16T02:00:34.463011+0000","last_active":"2024-09-16T02:02:09.458685+0000","last_peered":"2024-09-16T02:02:09.458685+0000","last_clean":"2024-09-16T02:02:09.458685+0000","last_became_active":"2024-09-16T02:00:34.462668+0000","last_became_peered":"2024-09-16T02:00:34.462668+0000","last_unstale":"2024-09-16T02:02:09.458685+0000","last_undegraded":"2024-09-16T02:02:09.458685+0000","last_fullsized":"2024-09-16T02:02:09.458685+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:06:31.655869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.567949+0000","last_change":"2024-09-16T02:00:34.462241+0000","last_active":"2024-09-16T02:02:41.567949+0000","last_peered":"2024-09-16T02:02:41.567949+0000","last_clean":"2024-09-16T02:02:41.567949+0000","last_became_active":"2024-09-16T02:00:34.461925+0000","last_became_peered":"2024-09-16T02:00:34.461925+0000","last_unstale":"2024-09-16T02:02:41.567949+0000","last_undegraded":"2024-09-16T02:02:41.567949+0000","last_fullsized":"2024-09-16T02:02:41.567949+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:48:02.414471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.457960+0000","last_change":"2024-09-16T02:00:34.466132+0000","last_active":"2024-09-16T02:02:09.457960+0000","last_peered":"2024-09-16T02:02:09.457960+0000","last_clean":"2024-09-16T02:02:09.457960+0000","last_became_active":"2024-09-16T02:00:34.465750+0000","last_became_peered":"2024-09-16T02:00:34.465750+0000","last_unstale":"2024-09-16T02:02:09.457960+0000","last_undegraded":"2024-09-16T02:02:09.457960+0000","last_fullsized":"2024-09-16T02:02:09.457960+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:07:33.072432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.446261+0000","last_change":"2024-09-16T02:00:34.461759+0000","last_active":"2024-09-16T02:02:09.446261+0000","last_peered":"2024-09-16T02:02:09.446261+0000","last_clean":"2024-09-16T02:02:09.446261+0000","last_became_active":"2024-09-16T02:00:34.461385+0000","last_became_peered":"2024-09-16T02:00:34.461385+0000","last_unstale":"2024-09-16T02:02:09.446261+0000","last_undegraded":"2024-09-16T02:02:09.446261+0000","last_fullsized":"2024-09-16T02:02:09.446261+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:00:24.179172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.440847+0000","last_change":"2024-09-16T02:01:40.608361+0000","last_active":"2024-09-16T02:02:09.440847+0000","last_peered":"2024-09-16T02:02:09.440847+0000","last_clean":"2024-09-16T02:02:09.440847+0000","last_became_active":"2024-09-16T02:01:40.607954+0000","last_became_peered":"2024-09-16T02:01:40.607954+0000","last_unstale":"2024-09-16T02:02:09.440847+0000","last_undegraded":"2024-09-16T02:02:09.440847+0000","last_fullsized":"2024-09-16T02:02:09.440847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:18:31.244969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.443096+0000","last_change":"2024-09-16T02:00:34.460680+0000","last_active":"2024-09-16T02:02:09.443096+0000","last_peered":"2024-09-16T02:02:09.443096+0000","last_clean":"2024-09-16T02:02:09.443096+0000","last_became_active":"2024-09-16T02:00:34.460161+0000","last_became_peered":"2024-09-16T02:00:34.460161+0000","last_unstale":"2024-09-16T02:02:09.443096+0000","last_undegraded":"2024-09-16T02:02:09.443096+0000","last_fullsized":"2024-09-16T02:02:09.443096+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:38:19.848420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.444910+0000","last_change":"2024-09-16T02:00:34.454606+0000","last_active":"2024-09-16T02:02:09.444910+0000","last_peered":"2024-09-16T02:02:09.444910+0000","last_clean":"2024-09-16T02:02:09.444910+0000","last_became_active":"2024-09-16T02:00:34.454293+0000","last_became_peered":"2024-09-16T02:00:34.454293+0000","last_unstale":"2024-09-16T02:02:09.444910+0000","last_undegraded":"2024-09-16T02:02:09.444910+0000","last_fullsized":"2024-09-16T02:02:09.444910+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:55:54.466176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.455703+0000","last_change":"2024-09-16T02:00:34.452940+0000","last_active":"2024-09-16T02:02:09.455703+0000","last_peered":"2024-09-16T02:02:09.455703+0000","last_clean":"2024-09-16T02:02:09.455703+0000","last_became_active":"2024-09-16T02:00:34.452560+0000","last_became_peered":"2024-09-16T02:00:34.452560+0000","last_unstale":"2024-09-16T02:02:09.455703+0000","last_undegraded":"2024-09-16T02:02:09.455703+0000","last_fullsized":"2024-09-16T02:02:09.455703+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:42:11.564264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.629422+0000","last_change":"2024-09-16T02:00:34.453457+0000","last_active":"2024-09-16T02:02:41.629422+0000","last_peered":"2024-09-16T02:02:41.629422+0000","last_clean":"2024-09-16T02:02:41.629422+0000","last_became_active":"2024-09-16T02:00:34.453148+0000","last_became_peered":"2024-09-16T02:00:34.453148+0000","last_unstale":"2024-09-16T02:02:41.629422+0000","last_undegraded":"2024-09-16T02:02:41.629422+0000","last_fullsized":"2024-09-16T02:02:41.629422+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:28.456892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-16T02:02:40.559102+0000","last_change":"2024-09-16T02:00:34.457057+0000","last_active":"2024-09-16T02:02:40.559102+0000","last_peered":"2024-09-16T02:02:40.559102+0000","last_clean":"2024-09-16T02:02:40.559102+0000","last_became_active":"2024-09-16T02:00:34.456751+0000","last_became_peered":"2024-09-16T02:00:34.456751+0000","last_unstale":"2024-09-16T02:02:40.559102+0000","last_undegraded":"2024-09-16T02:02:40.559102+0000","last_fullsized":"2024-09-16T02:02:40.559102+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:32:32.207510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.443848+0000","last_change":"2024-09-16T02:00:34.448434+0000","last_active":"2024-09-16T02:02:09.443848+0000","last_peered":"2024-09-16T02:02:09.443848+0000","last_clean":"2024-09-16T02:02:09.443848+0000","last_became_active":"2024-09-16T02:00:34.448056+0000","last_became_peered":"2024-09-16T02:00:34.448056+0000","last_unstale":"2024-09-16T02:02:09.443848+0000","last_undegraded":"2024-09-16T02:02:09.443848+0000","last_fullsized":"2024-09-16T02:02:09.443848+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:15:28.162159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.629246+0000","last_change":"2024-09-16T02:00:34.459499+0000","last_active":"2024-09-16T02:02:41.629246+0000","last_peered":"2024-09-16T02:02:41.629246+0000","last_clean":"2024-09-16T02:02:41.629246+0000","last_became_active":"2024-09-16T02:00:34.459017+0000","last_became_peered":"2024-09-16T02:00:34.459017+0000","last_unstale":"2024-09-16T02:02:41.629246+0000","last_undegraded":"2024-09-16T02:02:41.629246+0000","last_fullsized":"2024-09-16T02:02:41.629246+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:41:36.866100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.447108+0000","last_change":"2024-09-16T02:00:34.448640+0000","last_active":"2024-09-16T02:02:09.447108+0000","last_peered":"2024-09-16T02:02:09.447108+0000","last_clean":"2024-09-16T02:02:09.447108+0000","last_became_active":"2024-09-16T02:00:34.447737+0000","last_became_peered":"2024-09-16T02:00:34.447737+0000","last_unstale":"2024-09-16T02:02:09.447108+0000","last_undegraded":"2024-09-16T02:02:09.447108+0000","last_fullsized":"2024-09-16T02:02:09.447108+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:20.192650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.458066+0000","last_change":"2024-09-16T02:00:34.451718+0000","last_active":"2024-09-16T02:02:09.458066+0000","last_peered":"2024-09-16T02:02:09.458066+0000","last_clean":"2024-09-16T02:02:09.458066+0000","last_became_active":"2024-09-16T02:00:34.451389+0000","last_became_peered":"2024-09-16T02:00:34.451389+0000","last_unstale":"2024-09-16T02:02:09.458066+0000","last_undegraded":"2024-09-16T02:02:09.458066+0000","last_fullsized":"2024-09-16T02:02:09.458066+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:13:19.296178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.438337+0000","last_change":"2024-09-16T02:00:34.458379+0000","last_active":"2024-09-16T02:02:09.438337+0000","last_peered":"2024-09-16T02:02:09.438337+0000","last_clean":"2024-09-16T02:02:09.438337+0000","last_became_active":"2024-09-16T02:00:34.458038+0000","last_became_peered":"2024-09-16T02:00:34.458038+0000","last_unstale":"2024-09-16T02:02:09.438337+0000","last_undegraded":"2024-09-16T02:02:09.438337+0000","last_fullsized":"2024-09-16T02:02:09.438337+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:34.669470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.442709+0000","last_change":"2024-09-16T02:00:34.450927+0000","last_active":"2024-09-16T02:02:09.442709+0000","last_peered":"2024-09-16T02:02:09.442709+0000","last_clean":"2024-09-16T02:02:09.442709+0000","last_became_active":"2024-09-16T02:00:34.447669+0000","last_became_peered":"2024-09-16T02:00:34.447669+0000","last_unstale":"2024-09-16T02:02:09.442709+0000","last_undegraded":"2024-09-16T02:02:09.442709+0000","last_fullsized":"2024-09-16T02:02:09.442709+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:57.861633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.437315+0000","last_change":"2024-09-16T02:00:34.454827+0000","last_active":"2024-09-16T02:02:09.437315+0000","last_peered":"2024-09-16T02:02:09.437315+0000","last_clean":"2024-09-16T02:02:09.437315+0000","last_became_active":"2024-09-16T02:00:34.454549+0000","last_became_peered":"2024-09-16T02:00:34.454549+0000","last_unstale":"2024-09-16T02:02:09.437315+0000","last_undegraded":"2024-09-16T02:02:09.437315+0000","last_fullsized":"2024-09-16T02:02:09.437315+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:53:45.440860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.454569+0000","last_change":"2024-09-16T02:00:34.455121+0000","last_active":"2024-09-16T02:02:09.454569+0000","last_peered":"2024-09-16T02:02:09.454569+0000","last_clean":"2024-09-16T02:02:09.454569+0000","last_became_active":"2024-09-16T02:00:34.454665+0000","last_became_peered":"2024-09-16T02:00:34.454665+0000","last_unstale":"2024-09-16T02:02:09.454569+0000","last_undegraded":"2024-09-16T02:02:09.454569+0000","last_fullsized":"2024-09-16T02:02:09.454569+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:31:42.771230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.629195+0000","last_change":"2024-09-16T02:00:34.458777+0000","last_active":"2024-09-16T02:02:41.629195+0000","last_peered":"2024-09-16T02:02:41.629195+0000","last_clean":"2024-09-16T02:02:41.629195+0000","last_became_active":"2024-09-16T02:00:34.457954+0000","last_became_peered":"2024-09-16T02:00:34.457954+0000","last_unstale":"2024-09-16T02:02:41.629195+0000","last_undegraded":"2024-09-16T02:02:41.629195+0000","last_fullsized":"2024-09-16T02:02:41.629195+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:58:28.032293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.439473+0000","last_change":"2024-09-16T02:00:34.444712+0000","last_active":"2024-09-16T02:02:09.439473+0000","last_peered":"2024-09-16T02:02:09.439473+0000","last_clean":"2024-09-16T02:02:09.439473+0000","last_became_active":"2024-09-16T02:00:34.444421+0000","last_became_peered":"2024-09-16T02:00:34.444421+0000","last_unstale":"2024-09-16T02:02:09.439473+0000","last_undegraded":"2024-09-16T02:02:09.439473+0000","last_fullsized":"2024-09-16T02:02:09.439473+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:36.557558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.446560+0000","last_change":"2024-09-16T02:00:34.454570+0000","last_active":"2024-09-16T02:02:09.446560+0000","last_peered":"2024-09-16T02:02:09.446560+0000","last_clean":"2024-09-16T02:02:09.446560+0000","last_became_active":"2024-09-16T02:00:34.454246+0000","last_became_peered":"2024-09-16T02:00:34.454246+0000","last_unstale":"2024-09-16T02:02:09.446560+0000","last_undegraded":"2024-09-16T02:02:09.446560+0000","last_fullsized":"2024-09-16T02:02:09.446560+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:36:18.634907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.443635+0000","last_change":"2024-09-16T02:00:34.449680+0000","last_active":"2024-09-16T02:02:09.443635+0000","last_peered":"2024-09-16T02:02:09.443635+0000","last_clean":"2024-09-16T02:02:09.443635+0000","last_became_active":"2024-09-16T02:00:34.449184+0000","last_became_peered":"2024-09-16T02:00:34.449184+0000","last_unstale":"2024-09-16T02:02:09.443635+0000","last_undegraded":"2024-09-16T02:02:09.443635+0000","last_fullsized":"2024-09-16T02:02:09.443635+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:28:34.867820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.453407+0000","last_change":"2024-09-16T02:00:34.455362+0000","last_active":"2024-09-16T02:02:09.453407+0000","last_peered":"2024-09-16T02:02:09.453407+0000","last_clean":"2024-09-16T02:02:09.453407+0000","last_became_active":"2024-09-16T02:00:34.455018+0000","last_became_peered":"2024-09-16T02:00:34.455018+0000","last_unstale":"2024-09-16T02:02:09.453407+0000","last_undegraded":"2024-09-16T02:02:09.453407+0000","last_fullsized":"2024-09-16T02:02:09.453407+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:38.278418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.629086+0000","last_change":"2024-09-16T02:00:34.459426+0000","last_active":"2024-09-16T02:02:41.629086+0000","last_peered":"2024-09-16T02:02:41.629086+0000","last_clean":"2024-09-16T02:02:41.629086+0000","last_became_active":"2024-09-16T02:00:34.459082+0000","last_became_peered":"2024-09-16T02:00:34.459082+0000","last_unstale":"2024-09-16T02:02:41.629086+0000","last_undegraded":"2024-09-16T02:02:41.629086+0000","last_fullsized":"2024-09-16T02:02:41.629086+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:59:42.226065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.449554+0000","last_change":"2024-09-16T02:00:34.450936+0000","last_active":"2024-09-16T02:02:09.449554+0000","last_peered":"2024-09-16T02:02:09.449554+0000","last_clean":"2024-09-16T02:02:09.449554+0000","last_became_active":"2024-09-16T02:00:34.450503+0000","last_became_peered":"2024-09-16T02:00:34.450503+0000","last_unstale":"2024-09-16T02:02:09.449554+0000","last_undegraded":"2024-09-16T02:02:09.449554+0000","last_fullsized":"2024-09-16T02:02:09.449554+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:35:55.806193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'5","reported_seq":141,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:37.688618+0000","last_change":"2024-09-16T02:00:32.443193+0000","last_active":"2024-09-16T02:02:37.688618+0000","last_peered":"2024-09-16T02:02:37.688618+0000","last_clean":"2024-09-16T02:02:37.688618+0000","last_became_active":"2024-09-16T02:00:32.442863+0000","last_became_peered":"2024-09-16T02:00:32.442863+0000","last_unstale":"2024-09-16T02:02:37.688618+0000","last_undegraded":"2024-09-16T02:02:37.688618+0000","last_fullsized":"2024-09-16T02:02:37.688618+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:19.734946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.444085+0000","last_change":"2024-09-16T02:01:40.570518+0000","last_active":"2024-09-16T02:02:09.444085+0000","last_peered":"2024-09-16T02:02:09.444085+0000","last_clean":"2024-09-16T02:02:09.444085+0000","last_became_active":"2024-09-16T02:01:40.570124+0000","last_became_peered":"2024-09-16T02:01:40.570124+0000","last_unstale":"2024-09-16T02:02:09.444085+0000","last_undegraded":"2024-09-16T02:02:09.444085+0000","last_fullsized":"2024-09-16T02:02:09.444085+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:44:21.062742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.566661+0000","last_change":"2024-09-16T02:01:40.572282+0000","last_active":"2024-09-16T02:02:41.566661+0000","last_peered":"2024-09-16T02:02:41.566661+0000","last_clean":"2024-09-16T02:02:41.566661+0000","last_became_active":"2024-09-16T02:01:40.564577+0000","last_became_peered":"2024-09-16T02:01:40.564577+0000","last_unstale":"2024-09-16T02:02:41.566661+0000","last_undegraded":"2024-09-16T02:02:41.566661+0000","last_fullsized":"2024-09-16T02:02:41.566661+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:19:40.834173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.559077+0000","last_change":"2024-09-16T02:00:32.438530+0000","last_active":"2024-09-16T02:01:40.559077+0000","last_peered":"2024-09-16T02:01:40.559077+0000","last_clean":"2024-09-16T02:01:40.559077+0000","last_became_active":"2024-09-16T02:00:32.438238+0000","last_became_peered":"2024-09-16T02:00:32.438238+0000","last_unstale":"2024-09-16T02:01:40.559077+0000","last_undegraded":"2024-09-16T02:01:40.559077+0000","last_fullsized":"2024-09-16T02:01:40.559077+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:52:32.799701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.441599+0000","last_change":"2024-09-16T02:00:34.456299+0000","last_active":"2024-09-16T02:02:09.441599+0000","last_peered":"2024-09-16T02:02:09.441599+0000","last_clean":"2024-09-16T02:02:09.441599+0000","last_became_active":"2024-09-16T02:00:34.455925+0000","last_became_peered":"2024-09-16T02:00:34.455925+0000","last_unstale":"2024-09-16T02:02:09.441599+0000","last_undegraded":"2024-09-16T02:02:09.441599+0000","last_fullsized":"2024-09-16T02:02:09.441599+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:33:27.162664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.565294+0000","last_change":"2024-09-16T02:00:32.442029+0000","last_active":"2024-09-16T02:01:40.565294+0000","last_peered":"2024-09-16T02:01:40.565294+0000","last_clean":"2024-09-16T02:01:40.565294+0000","last_became_active":"2024-09-16T02:00:32.441705+0000","last_became_peered":"2024-09-16T02:00:32.441705+0000","last_unstale":"2024-09-16T02:01:40.565294+0000","last_undegraded":"2024-09-16T02:01:40.565294+0000","last_fullsized":"2024-09-16T02:01:40.565294+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:50:54.725768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-16T02:02:40.558706+0000","last_change":"2024-09-16T02:00:34.457833+0000","last_active":"2024-09-16T02:02:40.558706+0000","last_peered":"2024-09-16T02:02:40.558706+0000","last_clean":"2024-09-16T02:02:40.558706+0000","last_became_active":"2024-09-16T02:00:34.457453+0000","last_became_peered":"2024-09-16T02:00:34.457453+0000","last_unstale":"2024-09-16T02:02:40.558706+0000","last_undegraded":"2024-09-16T02:02:40.558706+0000","last_fullsized":"2024-09-16T02:02:40.558706+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:21.777574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'6","reported_seq":276,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:42.572664+0000","last_change":"2024-09-16T02:02:41.576860+0000","last_active":"2024-09-16T02:02:42.572664+0000","last_peered":"2024-09-16T02:02:42.572664+0000","last_clean":"2024-09-16T02:02:42.572664+0000","last_became_active":"2024-09-16T02:02:41.570526+0000","last_became_peered":"2024-09-16T02:02:41.570526+0000","last_unstale":"2024-09-16T02:02:42.572664+0000","last_undegraded":"2024-09-16T02:02:42.572664+0000","last_fullsized":"2024-09-16T02:02:42.572664+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:19.875152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":230,"num_read_kb":230,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":180,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,9],"acting":[10,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.455614+0000","last_change":"2024-09-16T02:00:34.461746+0000","last_active":"2024-09-16T02:02:09.455614+0000","last_peered":"2024-09-16T02:02:09.455614+0000","last_clean":"2024-09-16T02:02:09.455614+0000","last_became_active":"2024-09-16T02:00:34.461441+0000","last_became_peered":"2024-09-16T02:00:34.461441+0000","last_unstale":"2024-09-16T02:02:09.455614+0000","last_undegraded":"2024-09-16T02:02:09.455614+0000","last_fullsized":"2024-09-16T02:02:09.455614+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:28:14.519159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564386+0000","last_change":"2024-09-16T02:00:32.443129+0000","last_active":"2024-09-16T02:01:40.564386+0000","last_peered":"2024-09-16T02:01:40.564386+0000","last_clean":"2024-09-16T02:01:40.564386+0000","last_became_active":"2024-09-16T02:00:32.442772+0000","last_became_peered":"2024-09-16T02:00:32.442772+0000","last_unstale":"2024-09-16T02:01:40.564386+0000","last_undegraded":"2024-09-16T02:01:40.564386+0000","last_fullsized":"2024-09-16T02:01:40.564386+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:10:03.964583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.445596+0000","last_change":"2024-09-16T02:00:34.457449+0000","last_active":"2024-09-16T02:02:09.445596+0000","last_peered":"2024-09-16T02:02:09.445596+0000","last_clean":"2024-09-16T02:02:09.445596+0000","last_became_active":"2024-09-16T02:00:34.457110+0000","last_became_peered":"2024-09-16T02:00:34.457110+0000","last_unstale":"2024-09-16T02:02:09.445596+0000","last_undegraded":"2024-09-16T02:02:09.445596+0000","last_fullsized":"2024-09-16T02:02:09.445596+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:35.662503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.565563+0000","last_change":"2024-09-16T02:00:32.587419+0000","last_active":"2024-09-16T02:01:40.565563+0000","last_peered":"2024-09-16T02:01:40.565563+0000","last_clean":"2024-09-16T02:01:40.565563+0000","last_became_active":"2024-09-16T02:00:32.587037+0000","last_became_peered":"2024-09-16T02:00:32.587037+0000","last_unstale":"2024-09-16T02:01:40.565563+0000","last_undegraded":"2024-09-16T02:01:40.565563+0000","last_fullsized":"2024-09-16T02:01:40.565563+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:11.835505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.456127+0000","last_change":"2024-09-16T02:00:34.456371+0000","last_active":"2024-09-16T02:02:09.456127+0000","last_peered":"2024-09-16T02:02:09.456127+0000","last_clean":"2024-09-16T02:02:09.456127+0000","last_became_active":"2024-09-16T02:00:34.456013+0000","last_became_peered":"2024-09-16T02:00:34.456013+0000","last_unstale":"2024-09-16T02:02:09.456127+0000","last_undegraded":"2024-09-16T02:02:09.456127+0000","last_fullsized":"2024-09-16T02:02:09.456127+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:30:15.847189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'33","reported_seq":299,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.694029+0000","last_change":"2024-09-16T02:00:32.444922+0000","last_active":"2024-09-16T02:02:41.694029+0000","last_peered":"2024-09-16T02:02:41.694029+0000","last_clean":"2024-09-16T02:02:41.694029+0000","last_became_active":"2024-09-16T02:00:32.444540+0000","last_became_peered":"2024-09-16T02:00:32.444540+0000","last_unstale":"2024-09-16T02:02:41.694029+0000","last_undegraded":"2024-09-16T02:02:41.694029+0000","last_fullsized":"2024-09-16T02:02:41.694029+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:59.183952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59955,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":240,"num_read_kb":178,"num_write":35,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.443938+0000","last_change":"2024-09-16T02:00:34.453366+0000","last_active":"2024-09-16T02:02:09.443938+0000","last_peered":"2024-09-16T02:02:09.443938+0000","last_clean":"2024-09-16T02:02:09.443938+0000","last_became_active":"2024-09-16T02:00:34.453048+0000","last_became_peered":"2024-09-16T02:00:34.453048+0000","last_unstale":"2024-09-16T02:02:09.443938+0000","last_undegraded":"2024-09-16T02:02:09.443938+0000","last_fullsized":"2024-09-16T02:02:09.443938+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:15:52.721893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.562558+0000","last_change":"2024-09-16T02:00:32.445516+0000","last_active":"2024-09-16T02:01:40.562558+0000","last_peered":"2024-09-16T02:01:40.562558+0000","last_clean":"2024-09-16T02:01:40.562558+0000","last_became_active":"2024-09-16T02:00:32.445092+0000","last_became_peered":"2024-09-16T02:00:32.445092+0000","last_unstale":"2024-09-16T02:01:40.562558+0000","last_undegraded":"2024-09-16T02:01:40.562558+0000","last_fullsized":"2024-09-16T02:01:40.562558+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:54.274692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.446974+0000","last_change":"2024-09-16T02:00:34.448450+0000","last_active":"2024-09-16T02:02:09.446974+0000","last_peered":"2024-09-16T02:02:09.446974+0000","last_clean":"2024-09-16T02:02:09.446974+0000","last_became_active":"2024-09-16T02:00:34.448116+0000","last_became_peered":"2024-09-16T02:00:34.448116+0000","last_unstale":"2024-09-16T02:02:09.446974+0000","last_undegraded":"2024-09-16T02:02:09.446974+0000","last_fullsized":"2024-09-16T02:02:09.446974+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:12:34.834798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.565309+0000","last_change":"2024-09-16T02:00:32.445427+0000","last_active":"2024-09-16T02:01:40.565309+0000","last_peered":"2024-09-16T02:01:40.565309+0000","last_clean":"2024-09-16T02:01:40.565309+0000","last_became_active":"2024-09-16T02:00:32.445058+0000","last_became_peered":"2024-09-16T02:00:32.445058+0000","last_unstale":"2024-09-16T02:01:40.565309+0000","last_undegraded":"2024-09-16T02:01:40.565309+0000","last_fullsized":"2024-09-16T02:01:40.565309+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:35:34.701331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.452607+0000","last_change":"2024-09-16T02:00:34.458479+0000","last_active":"2024-09-16T02:02:09.452607+0000","last_peered":"2024-09-16T02:02:09.452607+0000","last_clean":"2024-09-16T02:02:09.452607+0000","last_became_active":"2024-09-16T02:00:34.458135+0000","last_became_peered":"2024-09-16T02:00:34.458135+0000","last_unstale":"2024-09-16T02:02:09.452607+0000","last_undegraded":"2024-09-16T02:02:09.452607+0000","last_fullsized":"2024-09-16T02:02:09.452607+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:37:40.188861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'8","reported_seq":146,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:38.698284+0000","last_change":"2024-09-16T02:00:32.440319+0000","last_active":"2024-09-16T02:02:38.698284+0000","last_peered":"2024-09-16T02:02:38.698284+0000","last_clean":"2024-09-16T02:02:38.698284+0000","last_became_active":"2024-09-16T02:00:32.440005+0000","last_became_peered":"2024-09-16T02:00:32.440005+0000","last_unstale":"2024-09-16T02:02:38.698284+0000","last_undegraded":"2024-09-16T02:02:38.698284+0000","last_fullsized":"2024-09-16T02:02:38.698284+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:07:33.876577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1617,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.439440+0000","last_change":"2024-09-16T02:00:34.460468+0000","last_active":"2024-09-16T02:02:09.439440+0000","last_peered":"2024-09-16T02:02:09.439440+0000","last_clean":"2024-09-16T02:02:09.439440+0000","last_became_active":"2024-09-16T02:00:34.459190+0000","last_became_peered":"2024-09-16T02:00:34.459190+0000","last_unstale":"2024-09-16T02:02:09.439440+0000","last_undegraded":"2024-09-16T02:02:09.439440+0000","last_fullsized":"2024-09-16T02:02:09.439440+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:46:43.654991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.562649+0000","last_change":"2024-09-16T02:00:32.587700+0000","last_active":"2024-09-16T02:01:40.562649+0000","last_peered":"2024-09-16T02:01:40.562649+0000","last_clean":"2024-09-16T02:01:40.562649+0000","last_became_active":"2024-09-16T02:00:32.586964+0000","last_became_peered":"2024-09-16T02:00:32.586964+0000","last_unstale":"2024-09-16T02:01:40.562649+0000","last_undegraded":"2024-09-16T02:01:40.562649+0000","last_fullsized":"2024-09-16T02:01:40.562649+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:41.309918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.447532+0000","last_change":"2024-09-16T02:00:34.455652+0000","last_active":"2024-09-16T02:02:09.447532+0000","last_peered":"2024-09-16T02:02:09.447532+0000","last_clean":"2024-09-16T02:02:09.447532+0000","last_became_active":"2024-09-16T02:00:34.455306+0000","last_became_peered":"2024-09-16T02:00:34.455306+0000","last_unstale":"2024-09-16T02:02:09.447532+0000","last_undegraded":"2024-09-16T02:02:09.447532+0000","last_fullsized":"2024-09-16T02:02:09.447532+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:44:09.505096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564164+0000","last_change":"2024-09-16T02:00:32.441443+0000","last_active":"2024-09-16T02:01:40.564164+0000","last_peered":"2024-09-16T02:01:40.564164+0000","last_clean":"2024-09-16T02:01:40.564164+0000","last_became_active":"2024-09-16T02:00:32.441073+0000","last_became_peered":"2024-09-16T02:00:32.441073+0000","last_unstale":"2024-09-16T02:01:40.564164+0000","last_undegraded":"2024-09-16T02:01:40.564164+0000","last_fullsized":"2024-09-16T02:01:40.564164+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:16.865970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.629121+0000","last_change":"2024-09-16T02:01:40.611317+0000","last_active":"2024-09-16T02:02:41.629121+0000","last_peered":"2024-09-16T02:02:41.629121+0000","last_clean":"2024-09-16T02:02:41.629121+0000","last_became_active":"2024-09-16T02:01:40.610837+0000","last_became_peered":"2024-09-16T02:01:40.610837+0000","last_unstale":"2024-09-16T02:02:41.629121+0000","last_undegraded":"2024-09-16T02:02:41.629121+0000","last_fullsized":"2024-09-16T02:02:41.629121+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:19:55.376738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.617877+0000","last_change":"2024-09-16T02:01:40.617877+0000","last_active":"2024-09-16T02:01:40.617877+0000","last_peered":"2024-09-16T02:01:40.617877+0000","last_clean":"2024-09-16T02:01:40.617877+0000","last_became_active":"2024-09-16T02:01:40.611597+0000","last_became_peered":"2024-09-16T02:01:40.611597+0000","last_unstale":"2024-09-16T02:01:40.617877+0000","last_undegraded":"2024-09-16T02:01:40.617877+0000","last_fullsized":"2024-09-16T02:01:40.617877+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:53:35.796243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.449557+0000","last_change":"2024-09-16T02:00:34.461889+0000","last_active":"2024-09-16T02:02:09.449557+0000","last_peered":"2024-09-16T02:02:09.449557+0000","last_clean":"2024-09-16T02:02:09.449557+0000","last_became_active":"2024-09-16T02:00:34.461583+0000","last_became_peered":"2024-09-16T02:00:34.461583+0000","last_unstale":"2024-09-16T02:02:09.449557+0000","last_undegraded":"2024-09-16T02:02:09.449557+0000","last_fullsized":"2024-09-16T02:02:09.449557+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:32.598422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.565617+0000","last_change":"2024-09-16T02:00:32.445201+0000","last_active":"2024-09-16T02:01:40.565617+0000","last_peered":"2024-09-16T02:01:40.565617+0000","last_clean":"2024-09-16T02:01:40.565617+0000","last_became_active":"2024-09-16T02:00:32.444722+0000","last_became_peered":"2024-09-16T02:00:32.444722+0000","last_unstale":"2024-09-16T02:01:40.565617+0000","last_undegraded":"2024-09-16T02:01:40.565617+0000","last_fullsized":"2024-09-16T02:01:40.565617+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:41:11.657028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.445512+0000","last_change":"2024-09-16T02:01:40.613676+0000","last_active":"2024-09-16T02:02:09.445512+0000","last_peered":"2024-09-16T02:02:09.445512+0000","last_clean":"2024-09-16T02:02:09.445512+0000","last_became_active":"2024-09-16T02:01:40.608260+0000","last_became_peered":"2024-09-16T02:01:40.608260+0000","last_unstale":"2024-09-16T02:02:09.445512+0000","last_undegraded":"2024-09-16T02:02:09.445512+0000","last_fullsized":"2024-09-16T02:02:09.445512+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:30:59.004103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'6","reported_seq":142,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:38.698210+0000","last_change":"2024-09-16T02:00:32.860059+0000","last_active":"2024-09-16T02:02:38.698210+0000","last_peered":"2024-09-16T02:02:38.698210+0000","last_clean":"2024-09-16T02:02:38.698210+0000","last_became_active":"2024-09-16T02:00:32.859617+0000","last_became_peered":"2024-09-16T02:00:32.859617+0000","last_unstale":"2024-09-16T02:02:38.698210+0000","last_undegraded":"2024-09-16T02:02:38.698210+0000","last_fullsized":"2024-09-16T02:02:38.698210+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:41:50.248194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1665,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.457956+0000","last_change":"2024-09-16T02:00:34.454344+0000","last_active":"2024-09-16T02:02:09.457956+0000","last_peered":"2024-09-16T02:02:09.457956+0000","last_clean":"2024-09-16T02:02:09.457956+0000","last_became_active":"2024-09-16T02:00:34.454027+0000","last_became_peered":"2024-09-16T02:00:34.454027+0000","last_unstale":"2024-09-16T02:02:09.457956+0000","last_undegraded":"2024-09-16T02:02:09.457956+0000","last_fullsized":"2024-09-16T02:02:09.457956+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:33.020365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":335,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564402+0000","last_change":"2024-09-16T01:59:19.798159+0000","last_active":"2024-09-16T02:01:40.564402+0000","last_peered":"2024-09-16T02:01:40.564402+0000","last_clean":"2024-09-16T02:01:40.564402+0000","last_became_active":"2024-09-16T01:59:19.797688+0000","last_became_peered":"2024-09-16T01:59:19.797688+0000","last_unstale":"2024-09-16T02:01:40.564402+0000","last_undegraded":"2024-09-16T02:01:40.564402+0000","last_fullsized":"2024-09-16T02:01:40.564402+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:55:54.312750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:55:54.312750+0000","last_clean_scrub_stamp":"2024-09-16T01:55:54.312750+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:00:36.860158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.610325+0000","last_change":"2024-09-16T02:00:32.441925+0000","last_active":"2024-09-16T02:01:40.610325+0000","last_peered":"2024-09-16T02:01:40.610325+0000","last_clean":"2024-09-16T02:01:40.610325+0000","last_became_active":"2024-09-16T02:00:32.441565+0000","last_became_peered":"2024-09-16T02:00:32.441565+0000","last_unstale":"2024-09-16T02:01:40.610325+0000","last_undegraded":"2024-09-16T02:01:40.610325+0000","last_fullsized":"2024-09-16T02:01:40.610325+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:23:09.755034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.447256+0000","last_change":"2024-09-16T02:00:34.452986+0000","last_active":"2024-09-16T02:02:09.447256+0000","last_peered":"2024-09-16T02:02:09.447256+0000","last_clean":"2024-09-16T02:02:09.447256+0000","last_became_active":"2024-09-16T02:00:34.452688+0000","last_became_peered":"2024-09-16T02:00:34.452688+0000","last_unstale":"2024-09-16T02:02:09.447256+0000","last_undegraded":"2024-09-16T02:02:09.447256+0000","last_fullsized":"2024-09-16T02:02:09.447256+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:12.958209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564282+0000","last_change":"2024-09-16T02:00:32.443964+0000","last_active":"2024-09-16T02:01:40.564282+0000","last_peered":"2024-09-16T02:01:40.564282+0000","last_clean":"2024-09-16T02:01:40.564282+0000","last_became_active":"2024-09-16T02:00:32.443623+0000","last_became_peered":"2024-09-16T02:00:32.443623+0000","last_unstale":"2024-09-16T02:01:40.564282+0000","last_undegraded":"2024-09-16T02:01:40.564282+0000","last_fullsized":"2024-09-16T02:01:40.564282+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:30:34.406024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.444800+0000","last_change":"2024-09-16T02:00:34.449615+0000","last_active":"2024-09-16T02:02:09.444800+0000","last_peered":"2024-09-16T02:02:09.444800+0000","last_clean":"2024-09-16T02:02:09.444800+0000","last_became_active":"2024-09-16T02:00:34.449109+0000","last_became_peered":"2024-09-16T02:00:34.449109+0000","last_unstale":"2024-09-16T02:02:09.444800+0000","last_undegraded":"2024-09-16T02:02:09.444800+0000","last_fullsized":"2024-09-16T02:02:09.444800+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:45.036177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'7","reported_seq":143,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:38.910349+0000","last_change":"2024-09-16T02:00:32.587107+0000","last_active":"2024-09-16T02:02:38.910349+0000","last_peered":"2024-09-16T02:02:38.910349+0000","last_clean":"2024-09-16T02:02:38.910349+0000","last_became_active":"2024-09-16T02:00:32.586777+0000","last_became_peered":"2024-09-16T02:00:32.586777+0000","last_unstale":"2024-09-16T02:02:38.910349+0000","last_undegraded":"2024-09-16T02:02:38.910349+0000","last_fullsized":"2024-09-16T02:02:38.910349+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:03.968987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.445154+0000","last_change":"2024-09-16T02:01:40.570572+0000","last_active":"2024-09-16T02:02:09.445154+0000","last_peered":"2024-09-16T02:02:09.445154+0000","last_clean":"2024-09-16T02:02:09.445154+0000","last_became_active":"2024-09-16T02:01:40.570176+0000","last_became_peered":"2024-09-16T02:01:40.570176+0000","last_unstale":"2024-09-16T02:02:09.445154+0000","last_undegraded":"2024-09-16T02:02:09.445154+0000","last_fullsized":"2024-09-16T02:02:09.445154+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:24:13.138925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'6","reported_seq":144,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:39.691569+0000","last_change":"2024-09-16T02:00:32.443201+0000","last_active":"2024-09-16T02:02:39.691569+0000","last_peered":"2024-09-16T02:02:39.691569+0000","last_clean":"2024-09-16T02:02:39.691569+0000","last_became_active":"2024-09-16T02:00:32.442778+0000","last_became_peered":"2024-09-16T02:00:32.442778+0000","last_unstale":"2024-09-16T02:02:39.691569+0000","last_undegraded":"2024-09-16T02:02:39.691569+0000","last_fullsized":"2024-09-16T02:02:39.691569+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:43.933817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1083,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.566693+0000","last_change":"2024-09-16T02:00:34.463000+0000","last_active":"2024-09-16T02:02:41.566693+0000","last_peered":"2024-09-16T02:02:41.566693+0000","last_clean":"2024-09-16T02:02:41.566693+0000","last_became_active":"2024-09-16T02:00:34.461890+0000","last_became_peered":"2024-09-16T02:00:34.461890+0000","last_unstale":"2024-09-16T02:02:41.566693+0000","last_undegraded":"2024-09-16T02:02:41.566693+0000","last_fullsized":"2024-09-16T02:02:41.566693+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:21:15.161295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-16T02:02:40.558984+0000","last_change":"2024-09-16T02:00:32.586933+0000","last_active":"2024-09-16T02:02:40.558984+0000","last_peered":"2024-09-16T02:02:40.558984+0000","last_clean":"2024-09-16T02:02:40.558984+0000","last_became_active":"2024-09-16T02:00:32.586565+0000","last_became_peered":"2024-09-16T02:00:32.586565+0000","last_unstale":"2024-09-16T02:02:40.558984+0000","last_undegraded":"2024-09-16T02:02:40.558984+0000","last_fullsized":"2024-09-16T02:02:40.558984+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:45:07.130649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.567361+0000","last_change":"2024-09-16T02:00:34.460805+0000","last_active":"2024-09-16T02:02:41.567361+0000","last_peered":"2024-09-16T02:02:41.567361+0000","last_clean":"2024-09-16T02:02:41.567361+0000","last_became_active":"2024-09-16T02:00:34.460482+0000","last_became_peered":"2024-09-16T02:00:34.460482+0000","last_unstale":"2024-09-16T02:02:41.567361+0000","last_undegraded":"2024-09-16T02:02:41.567361+0000","last_fullsized":"2024-09-16T02:02:41.567361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:07:51.565053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564628+0000","last_change":"2024-09-16T02:00:32.587516+0000","last_active":"2024-09-16T02:01:40.564628+0000","last_peered":"2024-09-16T02:01:40.564628+0000","last_clean":"2024-09-16T02:01:40.564628+0000","last_became_active":"2024-09-16T02:00:32.587172+0000","last_became_peered":"2024-09-16T02:00:32.587172+0000","last_unstale":"2024-09-16T02:01:40.564628+0000","last_undegraded":"2024-09-16T02:01:40.564628+0000","last_fullsized":"2024-09-16T02:01:40.564628+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:52:29.018290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.444728+0000","last_change":"2024-09-16T02:00:34.446021+0000","last_active":"2024-09-16T02:02:09.444728+0000","last_peered":"2024-09-16T02:02:09.444728+0000","last_clean":"2024-09-16T02:02:09.444728+0000","last_became_active":"2024-09-16T02:00:34.445732+0000","last_became_peered":"2024-09-16T02:00:34.445732+0000","last_unstale":"2024-09-16T02:02:09.444728+0000","last_undegraded":"2024-09-16T02:02:09.444728+0000","last_fullsized":"2024-09-16T02:02:09.444728+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:51:17.960792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'16","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-16T02:02:40.559157+0000","last_change":"2024-09-16T02:00:32.439750+0000","last_active":"2024-09-16T02:02:40.559157+0000","last_peered":"2024-09-16T02:02:40.559157+0000","last_clean":"2024-09-16T02:02:40.559157+0000","last_became_active":"2024-09-16T02:00:32.439438+0000","last_became_peered":"2024-09-16T02:00:32.439438+0000","last_unstale":"2024-09-16T02:02:40.559157+0000","last_undegraded":"2024-09-16T02:02:40.559157+0000","last_fullsized":"2024-09-16T02:02:40.559157+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:50.764733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.447307+0000","last_change":"2024-09-16T02:00:34.453810+0000","last_active":"2024-09-16T02:02:09.447307+0000","last_peered":"2024-09-16T02:02:09.447307+0000","last_clean":"2024-09-16T02:02:09.447307+0000","last_became_active":"2024-09-16T02:00:34.453421+0000","last_became_peered":"2024-09-16T02:00:34.453421+0000","last_unstale":"2024-09-16T02:02:09.447307+0000","last_undegraded":"2024-09-16T02:02:09.447307+0000","last_fullsized":"2024-09-16T02:02:09.447307+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:23:50.405749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.610378+0000","last_change":"2024-09-16T02:00:32.860565+0000","last_active":"2024-09-16T02:01:40.610378+0000","last_peered":"2024-09-16T02:01:40.610378+0000","last_clean":"2024-09-16T02:01:40.610378+0000","last_became_active":"2024-09-16T02:00:32.860130+0000","last_became_peered":"2024-09-16T02:00:32.860130+0000","last_unstale":"2024-09-16T02:01:40.610378+0000","last_undegraded":"2024-09-16T02:01:40.610378+0000","last_fullsized":"2024-09-16T02:01:40.610378+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:09:55.759883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.442038+0000","last_change":"2024-09-16T02:01:40.564454+0000","last_active":"2024-09-16T02:02:09.442038+0000","last_peered":"2024-09-16T02:02:09.442038+0000","last_clean":"2024-09-16T02:02:09.442038+0000","last_became_active":"2024-09-16T02:01:40.563854+0000","last_became_peered":"2024-09-16T02:01:40.563854+0000","last_unstale":"2024-09-16T02:02:09.442038+0000","last_undegraded":"2024-09-16T02:02:09.442038+0000","last_fullsized":"2024-09-16T02:02:09.442038+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:56:30.610947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.563115+0000","last_change":"2024-09-16T02:00:32.587115+0000","last_active":"2024-09-16T02:01:40.563115+0000","last_peered":"2024-09-16T02:01:40.563115+0000","last_clean":"2024-09-16T02:01:40.563115+0000","last_became_active":"2024-09-16T02:00:32.586835+0000","last_became_peered":"2024-09-16T02:00:32.586835+0000","last_unstale":"2024-09-16T02:01:40.563115+0000","last_undegraded":"2024-09-16T02:01:40.563115+0000","last_fullsized":"2024-09-16T02:01:40.563115+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:12:58.820005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.410645+0000","last_change":"2024-09-16T02:00:34.457673+0000","last_active":"2024-09-16T02:02:09.410645+0000","last_peered":"2024-09-16T02:02:09.410645+0000","last_clean":"2024-09-16T02:02:09.410645+0000","last_became_active":"2024-09-16T02:00:34.457331+0000","last_became_peered":"2024-09-16T02:00:34.457331+0000","last_unstale":"2024-09-16T02:02:09.410645+0000","last_undegraded":"2024-09-16T02:02:09.410645+0000","last_fullsized":"2024-09-16T02:02:09.410645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:12:16.100036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.561350+0000","last_change":"2024-09-16T02:00:32.437528+0000","last_active":"2024-09-16T02:01:40.561350+0000","last_peered":"2024-09-16T02:01:40.561350+0000","last_clean":"2024-09-16T02:01:40.561350+0000","last_became_active":"2024-09-16T02:00:32.437080+0000","last_became_peered":"2024-09-16T02:00:32.437080+0000","last_unstale":"2024-09-16T02:01:40.561350+0000","last_undegraded":"2024-09-16T02:01:40.561350+0000","last_fullsized":"2024-09-16T02:01:40.561350+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:59:33.908026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.629149+0000","last_change":"2024-09-16T02:00:34.453504+0000","last_active":"2024-09-16T02:02:41.629149+0000","last_peered":"2024-09-16T02:02:41.629149+0000","last_clean":"2024-09-16T02:02:41.629149+0000","last_became_active":"2024-09-16T02:00:34.453212+0000","last_became_peered":"2024-09-16T02:00:34.453212+0000","last_unstale":"2024-09-16T02:02:41.629149+0000","last_undegraded":"2024-09-16T02:02:41.629149+0000","last_fullsized":"2024-09-16T02:02:41.629149+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:00:56.414548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.562417+0000","last_change":"2024-09-16T02:00:32.442282+0000","last_active":"2024-09-16T02:01:40.562417+0000","last_peered":"2024-09-16T02:01:40.562417+0000","last_clean":"2024-09-16T02:01:40.562417+0000","last_became_active":"2024-09-16T02:00:32.441910+0000","last_became_peered":"2024-09-16T02:00:32.441910+0000","last_unstale":"2024-09-16T02:01:40.562417+0000","last_undegraded":"2024-09-16T02:01:40.562417+0000","last_fullsized":"2024-09-16T02:01:40.562417+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:01:46.045021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.440363+0000","last_change":"2024-09-16T02:00:34.460177+0000","last_active":"2024-09-16T02:02:09.440363+0000","last_peered":"2024-09-16T02:02:09.440363+0000","last_clean":"2024-09-16T02:02:09.440363+0000","last_became_active":"2024-09-16T02:00:34.459874+0000","last_became_peered":"2024-09-16T02:00:34.459874+0000","last_unstale":"2024-09-16T02:02:09.440363+0000","last_undegraded":"2024-09-16T02:02:09.440363+0000","last_fullsized":"2024-09-16T02:02:09.440363+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:35.855591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.614997+0000","last_change":"2024-09-16T02:01:40.614997+0000","last_active":"2024-09-16T02:01:40.614997+0000","last_peered":"2024-09-16T02:01:40.614997+0000","last_clean":"2024-09-16T02:01:40.614997+0000","last_became_active":"2024-09-16T02:01:40.610889+0000","last_became_peered":"2024-09-16T02:01:40.610889+0000","last_unstale":"2024-09-16T02:01:40.614997+0000","last_undegraded":"2024-09-16T02:01:40.614997+0000","last_fullsized":"2024-09-16T02:01:40.614997+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:39:00.989594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.457243+0000","last_change":"2024-09-16T02:00:34.452964+0000","last_active":"2024-09-16T02:02:09.457243+0000","last_peered":"2024-09-16T02:02:09.457243+0000","last_clean":"2024-09-16T02:02:09.457243+0000","last_became_active":"2024-09-16T02:00:34.452617+0000","last_became_peered":"2024-09-16T02:00:34.452617+0000","last_unstale":"2024-09-16T02:02:09.457243+0000","last_undegraded":"2024-09-16T02:02:09.457243+0000","last_fullsized":"2024-09-16T02:02:09.457243+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:04.617970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.448809+0000","last_change":"2024-09-16T02:00:34.456585+0000","last_active":"2024-09-16T02:02:09.448809+0000","last_peered":"2024-09-16T02:02:09.448809+0000","last_clean":"2024-09-16T02:02:09.448809+0000","last_became_active":"2024-09-16T02:00:34.456199+0000","last_became_peered":"2024-09-16T02:00:34.456199+0000","last_unstale":"2024-09-16T02:02:09.448809+0000","last_undegraded":"2024-09-16T02:02:09.448809+0000","last_fullsized":"2024-09-16T02:02:09.448809+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:37:54.376785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.628991+0000","last_change":"2024-09-16T02:01:40.578904+0000","last_active":"2024-09-16T02:02:41.628991+0000","last_peered":"2024-09-16T02:02:41.628991+0000","last_clean":"2024-09-16T02:02:41.628991+0000","last_became_active":"2024-09-16T02:01:40.567221+0000","last_became_peered":"2024-09-16T02:01:40.567221+0000","last_unstale":"2024-09-16T02:02:41.628991+0000","last_undegraded":"2024-09-16T02:02:41.628991+0000","last_fullsized":"2024-09-16T02:02:41.628991+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:01.407145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.456236+0000","last_change":"2024-09-16T02:00:34.460925+0000","last_active":"2024-09-16T02:02:09.456236+0000","last_peered":"2024-09-16T02:02:09.456236+0000","last_clean":"2024-09-16T02:02:09.456236+0000","last_became_active":"2024-09-16T02:00:34.460581+0000","last_became_peered":"2024-09-16T02:00:34.460581+0000","last_unstale":"2024-09-16T02:02:09.456236+0000","last_undegraded":"2024-09-16T02:02:09.456236+0000","last_fullsized":"2024-09-16T02:02:09.456236+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:19:44.460695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564476+0000","last_change":"2024-09-16T02:00:32.443877+0000","last_active":"2024-09-16T02:01:40.564476+0000","last_peered":"2024-09-16T02:01:40.564476+0000","last_clean":"2024-09-16T02:01:40.564476+0000","last_became_active":"2024-09-16T02:00:32.443416+0000","last_became_peered":"2024-09-16T02:00:32.443416+0000","last_unstale":"2024-09-16T02:01:40.564476+0000","last_undegraded":"2024-09-16T02:01:40.564476+0000","last_fullsized":"2024-09-16T02:01:40.564476+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:36:57.367232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.447996+0000","last_change":"2024-09-16T02:00:34.453048+0000","last_active":"2024-09-16T02:02:09.447996+0000","last_peered":"2024-09-16T02:02:09.447996+0000","last_clean":"2024-09-16T02:02:09.447996+0000","last_became_active":"2024-09-16T02:00:34.452752+0000","last_became_peered":"2024-09-16T02:00:34.452752+0000","last_unstale":"2024-09-16T02:02:09.447996+0000","last_undegraded":"2024-09-16T02:02:09.447996+0000","last_fullsized":"2024-09-16T02:02:09.447996+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:55.970195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.561997+0000","last_change":"2024-09-16T02:00:32.586974+0000","last_active":"2024-09-16T02:01:40.561997+0000","last_peered":"2024-09-16T02:01:40.561997+0000","last_clean":"2024-09-16T02:01:40.561997+0000","last_became_active":"2024-09-16T02:00:32.586644+0000","last_became_peered":"2024-09-16T02:00:32.586644+0000","last_unstale":"2024-09-16T02:01:40.561997+0000","last_undegraded":"2024-09-16T02:01:40.561997+0000","last_fullsized":"2024-09-16T02:01:40.561997+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:52.913287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.442896+0000","last_change":"2024-09-16T02:00:34.447548+0000","last_active":"2024-09-16T02:02:09.442896+0000","last_peered":"2024-09-16T02:02:09.442896+0000","last_clean":"2024-09-16T02:02:09.442896+0000","last_became_active":"2024-09-16T02:00:34.447207+0000","last_became_peered":"2024-09-16T02:00:34.447207+0000","last_unstale":"2024-09-16T02:02:09.442896+0000","last_undegraded":"2024-09-16T02:02:09.442896+0000","last_fullsized":"2024-09-16T02:02:09.442896+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:22:02.494343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'14","reported_seq":154,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-16T02:02:40.692237+0000","last_change":"2024-09-16T02:00:32.441129+0000","last_active":"2024-09-16T02:02:40.692237+0000","last_peered":"2024-09-16T02:02:40.692237+0000","last_clean":"2024-09-16T02:02:40.692237+0000","last_became_active":"2024-09-16T02:00:32.440843+0000","last_became_peered":"2024-09-16T02:00:32.440843+0000","last_unstale":"2024-09-16T02:02:40.692237+0000","last_undegraded":"2024-09-16T02:02:40.692237+0000","last_fullsized":"2024-09-16T02:02:40.692237+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:17.721740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68396,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1177,"num_read_kb":666,"num_write":258,"num_write_kb":146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":762,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":206178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":222,"ondisk_log_size":222,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281576,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612648,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":917504,"data_stored":450434,"data_compressed":11572,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841581,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1069056,"data_stored":1101370,"data_compressed":47384,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005105,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":929792,"data_stored":440845,"data_compressed":8664,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234038,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":431894,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462971,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":1026622,"data_compressed":41575,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626497,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":897024,"data_stored":432310,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855429,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":1024281,"data_compressed":41575,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018954,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":432705,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247888,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":431523,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444117,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":978944,"data_stored":492408,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":958464,"data_stored":493178,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":433649,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T02:02:45.327 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T02:02:45.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.462+0000 7faa9ec2c640 1 -- 172.21.15.105:0/1199421934 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa981036e0 msgr2=0x7faa98109f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:45.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.462+0000 7faa9ec2c640 1 --2- 172.21.15.105:0/1199421934 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa981036e0 0x7faa98109f70 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7faa90009ca0 tx=0x7faa9002a8c0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.462+0000 7faa9ec2c640 1 -- 172.21.15.105:0/1199421934 shutdown_connections 2024-09-16T02:02:45.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.462+0000 7faa9ec2c640 1 --2- 172.21.15.105:0/1199421934 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa981036e0 0x7faa98109f70 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.462+0000 7faa9ec2c640 1 --2- 172.21.15.105:0/1199421934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa98102d20 0x7faa981031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.462+0000 7faa9ec2c640 1 --2- 172.21.15.105:0/1199421934 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa98101b30 0x7faa98101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.462+0000 7faa9ec2c640 1 -- 172.21.15.105:0/1199421934 >> 172.21.15.105:0/1199421934 conn(0x7faa980fd2b0 msgr2=0x7faa980ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:45.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.462+0000 7faa9ec2c640 1 -- 172.21.15.105:0/1199421934 shutdown_connections 2024-09-16T02:02:45.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.462+0000 7faa9ec2c640 1 -- 172.21.15.105:0/1199421934 wait complete. 2024-09-16T02:02:45.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.463+0000 7faa9ec2c640 1 Processor -- start 2024-09-16T02:02:45.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.463+0000 7faa9ec2c640 1 -- start start 2024-09-16T02:02:45.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.463+0000 7faa9ec2c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa98101b30 0x7faa98075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:45.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.463+0000 7faa9ec2c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa98102d20 0x7faa980745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:45.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.463+0000 7faa9ec2c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa981036e0 0x7faa98074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:45.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.463+0000 7faa9ec2c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7faa9810d640 con 0x7faa981036e0 2024-09-16T02:02:45.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.463+0000 7faa9ec2c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faa9810d4c0 con 0x7faa98101b30 2024-09-16T02:02:45.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.463+0000 7faa9ec2c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7faa9810d7c0 con 0x7faa98102d20 2024-09-16T02:02:45.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.463+0000 7faa9d1c9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa981036e0 0x7faa98074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:45.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.463+0000 7faa9d1c9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa981036e0 0x7faa98074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60614/0 (socket says 172.21.15.105:60614) 2024-09-16T02:02:45.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.463+0000 7faa9d1c9640 1 -- 172.21.15.105:0/2811883673 learned_addr learned my addr 172.21.15.105:0/2811883673 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:45.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.463+0000 7faa97fff640 1 --2- 172.21.15.105:0/2811883673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa98102d20 0x7faa980745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:45.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.463+0000 7faa9c9c8640 1 --2- 172.21.15.105:0/2811883673 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa98101b30 0x7faa98075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:45.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.463+0000 7faa9d1c9640 1 -- 172.21.15.105:0/2811883673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa98102d20 msgr2=0x7faa980745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:45.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.463+0000 7faa9d1c9640 1 --2- 172.21.15.105:0/2811883673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa98102d20 0x7faa980745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.463+0000 7faa9d1c9640 1 -- 172.21.15.105:0/2811883673 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa98101b30 msgr2=0x7faa98075fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:45.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.463+0000 7faa9d1c9640 1 --2- 172.21.15.105:0/2811883673 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa98101b30 0x7faa98075fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.463+0000 7faa9d1c9640 1 -- 172.21.15.105:0/2811883673 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa980753b0 con 0x7faa981036e0 2024-09-16T02:02:45.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.463+0000 7faa9c9c8640 1 --2- 172.21.15.105:0/2811883673 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa98101b30 0x7faa98075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:02:45.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.464+0000 7faa9d1c9640 1 --2- 172.21.15.105:0/2811883673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa981036e0 0x7faa98074b20 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7faa90009dc0 tx=0x7faa90003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:45.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.464+0000 7faa95ffb640 1 -- 172.21.15.105:0/2811883673 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa90032ec0 con 0x7faa981036e0 2024-09-16T02:02:45.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.464+0000 7faa95ffb640 1 -- 172.21.15.105:0/2811883673 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa900044f0 con 0x7faa981036e0 2024-09-16T02:02:45.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.464+0000 7faa9ec2c640 1 -- 172.21.15.105:0/2811883673 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa981a1e20 con 0x7faa981036e0 2024-09-16T02:02:45.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.464+0000 7faa95ffb640 1 -- 172.21.15.105:0/2811883673 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa90030d40 con 0x7faa981036e0 2024-09-16T02:02:45.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.464+0000 7faa9ec2c640 1 -- 172.21.15.105:0/2811883673 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa981a22e0 con 0x7faa981036e0 2024-09-16T02:02:45.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.465+0000 7faa9ec2c640 1 -- 172.21.15.105:0/2811883673 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa98108f10 con 0x7faa981036e0 2024-09-16T02:02:45.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.466+0000 7faa95ffb640 1 -- 172.21.15.105:0/2811883673 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7faa90004040 con 0x7faa981036e0 2024-09-16T02:02:45.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.467+0000 7faa95ffb640 1 --2- 172.21.15.105:0/2811883673 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7faa880a3470 0x7faa880a5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:45.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.467+0000 7faa9c9c8640 1 --2- 172.21.15.105:0/2811883673 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7faa880a3470 0x7faa880a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:45.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.467+0000 7faa95ffb640 1 -- 172.21.15.105:0/2811883673 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7faa900f0e90 con 0x7faa981036e0 2024-09-16T02:02:45.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.467+0000 7faa9c9c8640 1 --2- 172.21.15.105:0/2811883673 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7faa880a3470 0x7faa880a5930 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7faa8c005d00 tx=0x7faa8c005ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:45.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.471+0000 7faa95ffb640 1 -- 172.21.15.105:0/2811883673 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faa900bdb50 con 0x7faa981036e0 2024-09-16T02:02:45.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.637+0000 7faa9ec2c640 1 -- 172.21.15.105:0/2811883673 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faa98062630 con 0x7faa880a3470 2024-09-16T02:02:45.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.644+0000 7faa95ffb640 1 -- 172.21.15.105:0/2811883673 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259165 (secure 0 0 0) 0x7faa98062630 con 0x7faa880a3470 2024-09-16T02:02:45.648 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:02:45.650 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-09-16T02:02:45.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.647+0000 7faa9ec2c640 1 -- 172.21.15.105:0/2811883673 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7faa880a3470 msgr2=0x7faa880a5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:45.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.647+0000 7faa9ec2c640 1 --2- 172.21.15.105:0/2811883673 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7faa880a3470 0x7faa880a5930 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7faa8c005d00 tx=0x7faa8c005ef0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.647+0000 7faa9ec2c640 1 -- 172.21.15.105:0/2811883673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa981036e0 msgr2=0x7faa98074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:45.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.647+0000 7faa9ec2c640 1 --2- 172.21.15.105:0/2811883673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa981036e0 0x7faa98074b20 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7faa90009dc0 tx=0x7faa90003ba0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.647+0000 7faa9ec2c640 1 -- 172.21.15.105:0/2811883673 shutdown_connections 2024-09-16T02:02:45.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.647+0000 7faa9ec2c640 1 --2- 172.21.15.105:0/2811883673 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7faa880a3470 0x7faa880a5930 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.647+0000 7faa9ec2c640 1 --2- 172.21.15.105:0/2811883673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa98102d20 0x7faa980745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.647+0000 7faa9ec2c640 1 --2- 172.21.15.105:0/2811883673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa981036e0 0x7faa98074b20 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.647+0000 7faa9ec2c640 1 --2- 172.21.15.105:0/2811883673 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa98101b30 0x7faa98075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.647+0000 7faa9ec2c640 1 -- 172.21.15.105:0/2811883673 >> 172.21.15.105:0/2811883673 conn(0x7faa980fd2b0 msgr2=0x7faa981060e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:45.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.647+0000 7faa9ec2c640 1 -- 172.21.15.105:0/2811883673 shutdown_connections 2024-09-16T02:02:45.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.647+0000 7faa9ec2c640 1 -- 172.21.15.105:0/2811883673 wait complete. 2024-09-16T02:02:45.658 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:45 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:02:45.658 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:45 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:02:45.658 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:45 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:02:45.659 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-09-16T02:02:45.475852+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659493,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1324,"num_read_kb":783,"num_write":444,"num_write_kb":3088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1181615,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":863,"ondisk_log_size":863,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373016,"kb_used_data":11220,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568808,"statfs":{"total":1151940427776,"available":1151558459392,"internally_reserved":0,"allocated":11489280,"data_stored":7313522,"data_compressed":231774,"data_compressed_allocated":311296,"data_compressed_original":2121728,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2475,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":93,"num_read_kb":50,"num_write":16,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":180,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001842"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.629463+0000","last_change":"2024-09-16T02:00:34.454121+0000","last_active":"2024-09-16T02:02:41.629463+0000","last_peered":"2024-09-16T02:02:41.629463+0000","last_clean":"2024-09-16T02:02:41.629463+0000","last_became_active":"2024-09-16T02:00:34.453810+0000","last_became_peered":"2024-09-16T02:00:34.453810+0000","last_unstale":"2024-09-16T02:02:41.629463+0000","last_undegraded":"2024-09-16T02:02:41.629463+0000","last_fullsized":"2024-09-16T02:02:41.629463+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:42:38.781928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.449067+0000","last_change":"2024-09-16T02:00:34.462020+0000","last_active":"2024-09-16T02:02:09.449067+0000","last_peered":"2024-09-16T02:02:09.449067+0000","last_clean":"2024-09-16T02:02:09.449067+0000","last_became_active":"2024-09-16T02:00:34.461680+0000","last_became_peered":"2024-09-16T02:00:34.461680+0000","last_unstale":"2024-09-16T02:02:09.449067+0000","last_undegraded":"2024-09-16T02:02:09.449067+0000","last_fullsized":"2024-09-16T02:02:09.449067+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:00:14.493643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.448082+0000","last_change":"2024-09-16T02:00:34.453034+0000","last_active":"2024-09-16T02:02:09.448082+0000","last_peered":"2024-09-16T02:02:09.448082+0000","last_clean":"2024-09-16T02:02:09.448082+0000","last_became_active":"2024-09-16T02:00:34.452719+0000","last_became_peered":"2024-09-16T02:00:34.452719+0000","last_unstale":"2024-09-16T02:02:09.448082+0000","last_undegraded":"2024-09-16T02:02:09.448082+0000","last_fullsized":"2024-09-16T02:02:09.448082+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:48:52.478501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.564538+0000","last_change":"2024-09-16T02:00:34.458156+0000","last_active":"2024-09-16T02:02:41.564538+0000","last_peered":"2024-09-16T02:02:41.564538+0000","last_clean":"2024-09-16T02:02:41.564538+0000","last_became_active":"2024-09-16T02:00:34.457831+0000","last_became_peered":"2024-09-16T02:00:34.457831+0000","last_unstale":"2024-09-16T02:02:41.564538+0000","last_undegraded":"2024-09-16T02:02:41.564538+0000","last_fullsized":"2024-09-16T02:02:41.564538+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:35:36.609046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.566422+0000","last_change":"2024-09-16T02:00:34.448638+0000","last_active":"2024-09-16T02:02:41.566422+0000","last_peered":"2024-09-16T02:02:41.566422+0000","last_clean":"2024-09-16T02:02:41.566422+0000","last_became_active":"2024-09-16T02:00:34.447993+0000","last_became_peered":"2024-09-16T02:00:34.447993+0000","last_unstale":"2024-09-16T02:02:41.566422+0000","last_undegraded":"2024-09-16T02:02:41.566422+0000","last_fullsized":"2024-09-16T02:02:41.566422+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:33:04.832053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.566521+0000","last_change":"2024-09-16T02:00:34.462651+0000","last_active":"2024-09-16T02:02:41.566521+0000","last_peered":"2024-09-16T02:02:41.566521+0000","last_clean":"2024-09-16T02:02:41.566521+0000","last_became_active":"2024-09-16T02:00:34.461837+0000","last_became_peered":"2024-09-16T02:00:34.461837+0000","last_unstale":"2024-09-16T02:02:41.566521+0000","last_undegraded":"2024-09-16T02:02:41.566521+0000","last_fullsized":"2024-09-16T02:02:41.566521+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:52:13.733591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.439301+0000","last_change":"2024-09-16T02:00:34.455362+0000","last_active":"2024-09-16T02:02:09.439301+0000","last_peered":"2024-09-16T02:02:09.439301+0000","last_clean":"2024-09-16T02:02:09.439301+0000","last_became_active":"2024-09-16T02:00:34.455029+0000","last_became_peered":"2024-09-16T02:00:34.455029+0000","last_unstale":"2024-09-16T02:02:09.439301+0000","last_undegraded":"2024-09-16T02:02:09.439301+0000","last_fullsized":"2024-09-16T02:02:09.439301+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:14:08.316845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.606253+0000","last_change":"2024-09-16T02:00:34.459578+0000","last_active":"2024-09-16T02:02:41.606253+0000","last_peered":"2024-09-16T02:02:41.606253+0000","last_clean":"2024-09-16T02:02:41.606253+0000","last_became_active":"2024-09-16T02:00:34.459288+0000","last_became_peered":"2024-09-16T02:00:34.459288+0000","last_unstale":"2024-09-16T02:02:41.606253+0000","last_undegraded":"2024-09-16T02:02:41.606253+0000","last_fullsized":"2024-09-16T02:02:41.606253+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:52:28.974276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.569781+0000","last_change":"2024-09-16T02:00:34.463011+0000","last_active":"2024-09-16T02:02:41.569781+0000","last_peered":"2024-09-16T02:02:41.569781+0000","last_clean":"2024-09-16T02:02:41.569781+0000","last_became_active":"2024-09-16T02:00:34.462668+0000","last_became_peered":"2024-09-16T02:00:34.462668+0000","last_unstale":"2024-09-16T02:02:41.569781+0000","last_undegraded":"2024-09-16T02:02:41.569781+0000","last_fullsized":"2024-09-16T02:02:41.569781+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:06:31.655869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.567949+0000","last_change":"2024-09-16T02:00:34.462241+0000","last_active":"2024-09-16T02:02:41.567949+0000","last_peered":"2024-09-16T02:02:41.567949+0000","last_clean":"2024-09-16T02:02:41.567949+0000","last_became_active":"2024-09-16T02:00:34.461925+0000","last_became_peered":"2024-09-16T02:00:34.461925+0000","last_unstale":"2024-09-16T02:02:41.567949+0000","last_undegraded":"2024-09-16T02:02:41.567949+0000","last_fullsized":"2024-09-16T02:02:41.567949+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:48:02.414471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.457960+0000","last_change":"2024-09-16T02:00:34.466132+0000","last_active":"2024-09-16T02:02:09.457960+0000","last_peered":"2024-09-16T02:02:09.457960+0000","last_clean":"2024-09-16T02:02:09.457960+0000","last_became_active":"2024-09-16T02:00:34.465750+0000","last_became_peered":"2024-09-16T02:00:34.465750+0000","last_unstale":"2024-09-16T02:02:09.457960+0000","last_undegraded":"2024-09-16T02:02:09.457960+0000","last_fullsized":"2024-09-16T02:02:09.457960+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:07:33.072432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.564542+0000","last_change":"2024-09-16T02:00:34.461759+0000","last_active":"2024-09-16T02:02:41.564542+0000","last_peered":"2024-09-16T02:02:41.564542+0000","last_clean":"2024-09-16T02:02:41.564542+0000","last_became_active":"2024-09-16T02:00:34.461385+0000","last_became_peered":"2024-09-16T02:00:34.461385+0000","last_unstale":"2024-09-16T02:02:41.564542+0000","last_undegraded":"2024-09-16T02:02:41.564542+0000","last_fullsized":"2024-09-16T02:02:41.564542+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:00:24.179172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.607349+0000","last_change":"2024-09-16T02:01:40.608361+0000","last_active":"2024-09-16T02:02:41.607349+0000","last_peered":"2024-09-16T02:02:41.607349+0000","last_clean":"2024-09-16T02:02:41.607349+0000","last_became_active":"2024-09-16T02:01:40.607954+0000","last_became_peered":"2024-09-16T02:01:40.607954+0000","last_unstale":"2024-09-16T02:02:41.607349+0000","last_undegraded":"2024-09-16T02:02:41.607349+0000","last_fullsized":"2024-09-16T02:02:41.607349+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:18:31.244969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.443096+0000","last_change":"2024-09-16T02:00:34.460680+0000","last_active":"2024-09-16T02:02:09.443096+0000","last_peered":"2024-09-16T02:02:09.443096+0000","last_clean":"2024-09-16T02:02:09.443096+0000","last_became_active":"2024-09-16T02:00:34.460161+0000","last_became_peered":"2024-09-16T02:00:34.460161+0000","last_unstale":"2024-09-16T02:02:09.443096+0000","last_undegraded":"2024-09-16T02:02:09.443096+0000","last_fullsized":"2024-09-16T02:02:09.443096+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:38:19.848420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.444910+0000","last_change":"2024-09-16T02:00:34.454606+0000","last_active":"2024-09-16T02:02:09.444910+0000","last_peered":"2024-09-16T02:02:09.444910+0000","last_clean":"2024-09-16T02:02:09.444910+0000","last_became_active":"2024-09-16T02:00:34.454293+0000","last_became_peered":"2024-09-16T02:00:34.454293+0000","last_unstale":"2024-09-16T02:02:09.444910+0000","last_undegraded":"2024-09-16T02:02:09.444910+0000","last_fullsized":"2024-09-16T02:02:09.444910+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:55:54.466176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.568781+0000","last_change":"2024-09-16T02:00:34.452940+0000","last_active":"2024-09-16T02:02:41.568781+0000","last_peered":"2024-09-16T02:02:41.568781+0000","last_clean":"2024-09-16T02:02:41.568781+0000","last_became_active":"2024-09-16T02:00:34.452560+0000","last_became_peered":"2024-09-16T02:00:34.452560+0000","last_unstale":"2024-09-16T02:02:41.568781+0000","last_undegraded":"2024-09-16T02:02:41.568781+0000","last_fullsized":"2024-09-16T02:02:41.568781+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:42:11.564264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.629422+0000","last_change":"2024-09-16T02:00:34.453457+0000","last_active":"2024-09-16T02:02:41.629422+0000","last_peered":"2024-09-16T02:02:41.629422+0000","last_clean":"2024-09-16T02:02:41.629422+0000","last_became_active":"2024-09-16T02:00:34.453148+0000","last_became_peered":"2024-09-16T02:00:34.453148+0000","last_unstale":"2024-09-16T02:02:41.629422+0000","last_undegraded":"2024-09-16T02:02:41.629422+0000","last_fullsized":"2024-09-16T02:02:41.629422+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:28.456892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-16T02:02:40.559102+0000","last_change":"2024-09-16T02:00:34.457057+0000","last_active":"2024-09-16T02:02:40.559102+0000","last_peered":"2024-09-16T02:02:40.559102+0000","last_clean":"2024-09-16T02:02:40.559102+0000","last_became_active":"2024-09-16T02:00:34.456751+0000","last_became_peered":"2024-09-16T02:00:34.456751+0000","last_unstale":"2024-09-16T02:02:40.559102+0000","last_undegraded":"2024-09-16T02:02:40.559102+0000","last_fullsized":"2024-09-16T02:02:40.559102+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:32:32.207510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.571309+0000","last_change":"2024-09-16T02:00:34.448434+0000","last_active":"2024-09-16T02:02:41.571309+0000","last_peered":"2024-09-16T02:02:41.571309+0000","last_clean":"2024-09-16T02:02:41.571309+0000","last_became_active":"2024-09-16T02:00:34.448056+0000","last_became_peered":"2024-09-16T02:00:34.448056+0000","last_unstale":"2024-09-16T02:02:41.571309+0000","last_undegraded":"2024-09-16T02:02:41.571309+0000","last_fullsized":"2024-09-16T02:02:41.571309+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:15:28.162159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.629246+0000","last_change":"2024-09-16T02:00:34.459499+0000","last_active":"2024-09-16T02:02:41.629246+0000","last_peered":"2024-09-16T02:02:41.629246+0000","last_clean":"2024-09-16T02:02:41.629246+0000","last_became_active":"2024-09-16T02:00:34.459017+0000","last_became_peered":"2024-09-16T02:00:34.459017+0000","last_unstale":"2024-09-16T02:02:41.629246+0000","last_undegraded":"2024-09-16T02:02:41.629246+0000","last_fullsized":"2024-09-16T02:02:41.629246+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:41:36.866100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.607427+0000","last_change":"2024-09-16T02:00:34.448640+0000","last_active":"2024-09-16T02:02:41.607427+0000","last_peered":"2024-09-16T02:02:41.607427+0000","last_clean":"2024-09-16T02:02:41.607427+0000","last_became_active":"2024-09-16T02:00:34.447737+0000","last_became_peered":"2024-09-16T02:00:34.447737+0000","last_unstale":"2024-09-16T02:02:41.607427+0000","last_undegraded":"2024-09-16T02:02:41.607427+0000","last_fullsized":"2024-09-16T02:02:41.607427+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:20.192650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.569353+0000","last_change":"2024-09-16T02:00:34.451718+0000","last_active":"2024-09-16T02:02:41.569353+0000","last_peered":"2024-09-16T02:02:41.569353+0000","last_clean":"2024-09-16T02:02:41.569353+0000","last_became_active":"2024-09-16T02:00:34.451389+0000","last_became_peered":"2024-09-16T02:00:34.451389+0000","last_unstale":"2024-09-16T02:02:41.569353+0000","last_undegraded":"2024-09-16T02:02:41.569353+0000","last_fullsized":"2024-09-16T02:02:41.569353+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:13:19.296178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.606335+0000","last_change":"2024-09-16T02:00:34.458379+0000","last_active":"2024-09-16T02:02:41.606335+0000","last_peered":"2024-09-16T02:02:41.606335+0000","last_clean":"2024-09-16T02:02:41.606335+0000","last_became_active":"2024-09-16T02:00:34.458038+0000","last_became_peered":"2024-09-16T02:00:34.458038+0000","last_unstale":"2024-09-16T02:02:41.606335+0000","last_undegraded":"2024-09-16T02:02:41.606335+0000","last_fullsized":"2024-09-16T02:02:41.606335+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:34.669470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.565299+0000","last_change":"2024-09-16T02:00:34.450927+0000","last_active":"2024-09-16T02:02:41.565299+0000","last_peered":"2024-09-16T02:02:41.565299+0000","last_clean":"2024-09-16T02:02:41.565299+0000","last_became_active":"2024-09-16T02:00:34.447669+0000","last_became_peered":"2024-09-16T02:00:34.447669+0000","last_unstale":"2024-09-16T02:02:41.565299+0000","last_undegraded":"2024-09-16T02:02:41.565299+0000","last_fullsized":"2024-09-16T02:02:41.565299+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:57.861633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.606779+0000","last_change":"2024-09-16T02:00:34.454827+0000","last_active":"2024-09-16T02:02:41.606779+0000","last_peered":"2024-09-16T02:02:41.606779+0000","last_clean":"2024-09-16T02:02:41.606779+0000","last_became_active":"2024-09-16T02:00:34.454549+0000","last_became_peered":"2024-09-16T02:00:34.454549+0000","last_unstale":"2024-09-16T02:02:41.606779+0000","last_undegraded":"2024-09-16T02:02:41.606779+0000","last_fullsized":"2024-09-16T02:02:41.606779+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:53:45.440860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.571467+0000","last_change":"2024-09-16T02:00:34.455121+0000","last_active":"2024-09-16T02:02:41.571467+0000","last_peered":"2024-09-16T02:02:41.571467+0000","last_clean":"2024-09-16T02:02:41.571467+0000","last_became_active":"2024-09-16T02:00:34.454665+0000","last_became_peered":"2024-09-16T02:00:34.454665+0000","last_unstale":"2024-09-16T02:02:41.571467+0000","last_undegraded":"2024-09-16T02:02:41.571467+0000","last_fullsized":"2024-09-16T02:02:41.571467+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:31:42.771230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.629195+0000","last_change":"2024-09-16T02:00:34.458777+0000","last_active":"2024-09-16T02:02:41.629195+0000","last_peered":"2024-09-16T02:02:41.629195+0000","last_clean":"2024-09-16T02:02:41.629195+0000","last_became_active":"2024-09-16T02:00:34.457954+0000","last_became_peered":"2024-09-16T02:00:34.457954+0000","last_unstale":"2024-09-16T02:02:41.629195+0000","last_undegraded":"2024-09-16T02:02:41.629195+0000","last_fullsized":"2024-09-16T02:02:41.629195+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:58:28.032293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.607136+0000","last_change":"2024-09-16T02:00:34.444712+0000","last_active":"2024-09-16T02:02:41.607136+0000","last_peered":"2024-09-16T02:02:41.607136+0000","last_clean":"2024-09-16T02:02:41.607136+0000","last_became_active":"2024-09-16T02:00:34.444421+0000","last_became_peered":"2024-09-16T02:00:34.444421+0000","last_unstale":"2024-09-16T02:02:41.607136+0000","last_undegraded":"2024-09-16T02:02:41.607136+0000","last_fullsized":"2024-09-16T02:02:41.607136+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:36.557558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.446560+0000","last_change":"2024-09-16T02:00:34.454570+0000","last_active":"2024-09-16T02:02:09.446560+0000","last_peered":"2024-09-16T02:02:09.446560+0000","last_clean":"2024-09-16T02:02:09.446560+0000","last_became_active":"2024-09-16T02:00:34.454246+0000","last_became_peered":"2024-09-16T02:00:34.454246+0000","last_unstale":"2024-09-16T02:02:09.446560+0000","last_undegraded":"2024-09-16T02:02:09.446560+0000","last_fullsized":"2024-09-16T02:02:09.446560+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:36:18.634907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.571164+0000","last_change":"2024-09-16T02:00:34.449680+0000","last_active":"2024-09-16T02:02:41.571164+0000","last_peered":"2024-09-16T02:02:41.571164+0000","last_clean":"2024-09-16T02:02:41.571164+0000","last_became_active":"2024-09-16T02:00:34.449184+0000","last_became_peered":"2024-09-16T02:00:34.449184+0000","last_unstale":"2024-09-16T02:02:41.571164+0000","last_undegraded":"2024-09-16T02:02:41.571164+0000","last_fullsized":"2024-09-16T02:02:41.571164+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:28:34.867820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.571034+0000","last_change":"2024-09-16T02:00:34.455362+0000","last_active":"2024-09-16T02:02:41.571034+0000","last_peered":"2024-09-16T02:02:41.571034+0000","last_clean":"2024-09-16T02:02:41.571034+0000","last_became_active":"2024-09-16T02:00:34.455018+0000","last_became_peered":"2024-09-16T02:00:34.455018+0000","last_unstale":"2024-09-16T02:02:41.571034+0000","last_undegraded":"2024-09-16T02:02:41.571034+0000","last_fullsized":"2024-09-16T02:02:41.571034+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:38.278418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.629086+0000","last_change":"2024-09-16T02:00:34.459426+0000","last_active":"2024-09-16T02:02:41.629086+0000","last_peered":"2024-09-16T02:02:41.629086+0000","last_clean":"2024-09-16T02:02:41.629086+0000","last_became_active":"2024-09-16T02:00:34.459082+0000","last_became_peered":"2024-09-16T02:00:34.459082+0000","last_unstale":"2024-09-16T02:02:41.629086+0000","last_undegraded":"2024-09-16T02:02:41.629086+0000","last_fullsized":"2024-09-16T02:02:41.629086+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:59:42.226065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.565886+0000","last_change":"2024-09-16T02:00:34.450936+0000","last_active":"2024-09-16T02:02:41.565886+0000","last_peered":"2024-09-16T02:02:41.565886+0000","last_clean":"2024-09-16T02:02:41.565886+0000","last_became_active":"2024-09-16T02:00:34.450503+0000","last_became_peered":"2024-09-16T02:00:34.450503+0000","last_unstale":"2024-09-16T02:02:41.565886+0000","last_undegraded":"2024-09-16T02:02:41.565886+0000","last_fullsized":"2024-09-16T02:02:41.565886+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:35:55.806193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":151,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:42.573075+0000","last_change":"2024-09-16T02:00:32.443193+0000","last_active":"2024-09-16T02:02:42.573075+0000","last_peered":"2024-09-16T02:02:42.573075+0000","last_clean":"2024-09-16T02:02:42.573075+0000","last_became_active":"2024-09-16T02:00:32.442863+0000","last_became_peered":"2024-09-16T02:00:32.442863+0000","last_unstale":"2024-09-16T02:02:42.573075+0000","last_undegraded":"2024-09-16T02:02:42.573075+0000","last_fullsized":"2024-09-16T02:02:42.573075+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:19.734946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1117,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.565455+0000","last_change":"2024-09-16T02:01:40.570518+0000","last_active":"2024-09-16T02:02:41.565455+0000","last_peered":"2024-09-16T02:02:41.565455+0000","last_clean":"2024-09-16T02:02:41.565455+0000","last_became_active":"2024-09-16T02:01:40.570124+0000","last_became_peered":"2024-09-16T02:01:40.570124+0000","last_unstale":"2024-09-16T02:02:41.565455+0000","last_undegraded":"2024-09-16T02:02:41.565455+0000","last_fullsized":"2024-09-16T02:02:41.565455+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:44:21.062742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.566661+0000","last_change":"2024-09-16T02:01:40.572282+0000","last_active":"2024-09-16T02:02:41.566661+0000","last_peered":"2024-09-16T02:02:41.566661+0000","last_clean":"2024-09-16T02:02:41.566661+0000","last_became_active":"2024-09-16T02:01:40.564577+0000","last_became_peered":"2024-09-16T02:01:40.564577+0000","last_unstale":"2024-09-16T02:02:41.566661+0000","last_undegraded":"2024-09-16T02:02:41.566661+0000","last_fullsized":"2024-09-16T02:02:41.566661+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:19:40.834173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.606793+0000","last_change":"2024-09-16T02:00:32.438530+0000","last_active":"2024-09-16T02:02:41.606793+0000","last_peered":"2024-09-16T02:02:41.606793+0000","last_clean":"2024-09-16T02:02:41.606793+0000","last_became_active":"2024-09-16T02:00:32.438238+0000","last_became_peered":"2024-09-16T02:00:32.438238+0000","last_unstale":"2024-09-16T02:02:41.606793+0000","last_undegraded":"2024-09-16T02:02:41.606793+0000","last_fullsized":"2024-09-16T02:02:41.606793+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:52:32.799701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.606323+0000","last_change":"2024-09-16T02:00:34.456299+0000","last_active":"2024-09-16T02:02:41.606323+0000","last_peered":"2024-09-16T02:02:41.606323+0000","last_clean":"2024-09-16T02:02:41.606323+0000","last_became_active":"2024-09-16T02:00:34.455925+0000","last_became_peered":"2024-09-16T02:00:34.455925+0000","last_unstale":"2024-09-16T02:02:41.606323+0000","last_undegraded":"2024-09-16T02:02:41.606323+0000","last_fullsized":"2024-09-16T02:02:41.606323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:33:27.162664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.566251+0000","last_change":"2024-09-16T02:00:32.442029+0000","last_active":"2024-09-16T02:02:41.566251+0000","last_peered":"2024-09-16T02:02:41.566251+0000","last_clean":"2024-09-16T02:02:41.566251+0000","last_became_active":"2024-09-16T02:00:32.441705+0000","last_became_peered":"2024-09-16T02:00:32.441705+0000","last_unstale":"2024-09-16T02:02:41.566251+0000","last_undegraded":"2024-09-16T02:02:41.566251+0000","last_fullsized":"2024-09-16T02:02:41.566251+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:50:54.725768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-16T02:02:40.558706+0000","last_change":"2024-09-16T02:00:34.457833+0000","last_active":"2024-09-16T02:02:40.558706+0000","last_peered":"2024-09-16T02:02:40.558706+0000","last_clean":"2024-09-16T02:02:40.558706+0000","last_became_active":"2024-09-16T02:00:34.457453+0000","last_became_peered":"2024-09-16T02:00:34.457453+0000","last_unstale":"2024-09-16T02:02:40.558706+0000","last_undegraded":"2024-09-16T02:02:40.558706+0000","last_fullsized":"2024-09-16T02:02:40.558706+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:21.777574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'6","reported_seq":276,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:42.572664+0000","last_change":"2024-09-16T02:02:41.576860+0000","last_active":"2024-09-16T02:02:42.572664+0000","last_peered":"2024-09-16T02:02:42.572664+0000","last_clean":"2024-09-16T02:02:42.572664+0000","last_became_active":"2024-09-16T02:02:41.570526+0000","last_became_peered":"2024-09-16T02:02:41.570526+0000","last_unstale":"2024-09-16T02:02:42.572664+0000","last_undegraded":"2024-09-16T02:02:42.572664+0000","last_fullsized":"2024-09-16T02:02:42.572664+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:19.875152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":230,"num_read_kb":230,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":180,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,9],"acting":[10,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.569687+0000","last_change":"2024-09-16T02:00:34.461746+0000","last_active":"2024-09-16T02:02:41.569687+0000","last_peered":"2024-09-16T02:02:41.569687+0000","last_clean":"2024-09-16T02:02:41.569687+0000","last_became_active":"2024-09-16T02:00:34.461441+0000","last_became_peered":"2024-09-16T02:00:34.461441+0000","last_unstale":"2024-09-16T02:02:41.569687+0000","last_undegraded":"2024-09-16T02:02:41.569687+0000","last_fullsized":"2024-09-16T02:02:41.569687+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:28:14.519159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564386+0000","last_change":"2024-09-16T02:00:32.443129+0000","last_active":"2024-09-16T02:01:40.564386+0000","last_peered":"2024-09-16T02:01:40.564386+0000","last_clean":"2024-09-16T02:01:40.564386+0000","last_became_active":"2024-09-16T02:00:32.442772+0000","last_became_peered":"2024-09-16T02:00:32.442772+0000","last_unstale":"2024-09-16T02:01:40.564386+0000","last_undegraded":"2024-09-16T02:01:40.564386+0000","last_fullsized":"2024-09-16T02:01:40.564386+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:10:03.964583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.445596+0000","last_change":"2024-09-16T02:00:34.457449+0000","last_active":"2024-09-16T02:02:09.445596+0000","last_peered":"2024-09-16T02:02:09.445596+0000","last_clean":"2024-09-16T02:02:09.445596+0000","last_became_active":"2024-09-16T02:00:34.457110+0000","last_became_peered":"2024-09-16T02:00:34.457110+0000","last_unstale":"2024-09-16T02:02:09.445596+0000","last_undegraded":"2024-09-16T02:02:09.445596+0000","last_fullsized":"2024-09-16T02:02:09.445596+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:35.662503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.565645+0000","last_change":"2024-09-16T02:00:32.587419+0000","last_active":"2024-09-16T02:02:41.565645+0000","last_peered":"2024-09-16T02:02:41.565645+0000","last_clean":"2024-09-16T02:02:41.565645+0000","last_became_active":"2024-09-16T02:00:32.587037+0000","last_became_peered":"2024-09-16T02:00:32.587037+0000","last_unstale":"2024-09-16T02:02:41.565645+0000","last_undegraded":"2024-09-16T02:02:41.565645+0000","last_fullsized":"2024-09-16T02:02:41.565645+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:11.835505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.569632+0000","last_change":"2024-09-16T02:00:34.456371+0000","last_active":"2024-09-16T02:02:41.569632+0000","last_peered":"2024-09-16T02:02:41.569632+0000","last_clean":"2024-09-16T02:02:41.569632+0000","last_became_active":"2024-09-16T02:00:34.456013+0000","last_became_peered":"2024-09-16T02:00:34.456013+0000","last_unstale":"2024-09-16T02:02:41.569632+0000","last_undegraded":"2024-09-16T02:02:41.569632+0000","last_fullsized":"2024-09-16T02:02:41.569632+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:30:15.847189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'33","reported_seq":299,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.694029+0000","last_change":"2024-09-16T02:00:32.444922+0000","last_active":"2024-09-16T02:02:41.694029+0000","last_peered":"2024-09-16T02:02:41.694029+0000","last_clean":"2024-09-16T02:02:41.694029+0000","last_became_active":"2024-09-16T02:00:32.444540+0000","last_became_peered":"2024-09-16T02:00:32.444540+0000","last_unstale":"2024-09-16T02:02:41.694029+0000","last_undegraded":"2024-09-16T02:02:41.694029+0000","last_fullsized":"2024-09-16T02:02:41.694029+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:59.183952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59955,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":240,"num_read_kb":178,"num_write":35,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.566234+0000","last_change":"2024-09-16T02:00:34.453366+0000","last_active":"2024-09-16T02:02:41.566234+0000","last_peered":"2024-09-16T02:02:41.566234+0000","last_clean":"2024-09-16T02:02:41.566234+0000","last_became_active":"2024-09-16T02:00:34.453048+0000","last_became_peered":"2024-09-16T02:00:34.453048+0000","last_unstale":"2024-09-16T02:02:41.566234+0000","last_undegraded":"2024-09-16T02:02:41.566234+0000","last_fullsized":"2024-09-16T02:02:41.566234+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:15:52.721893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.571068+0000","last_change":"2024-09-16T02:00:32.445516+0000","last_active":"2024-09-16T02:02:41.571068+0000","last_peered":"2024-09-16T02:02:41.571068+0000","last_clean":"2024-09-16T02:02:41.571068+0000","last_became_active":"2024-09-16T02:00:32.445092+0000","last_became_peered":"2024-09-16T02:00:32.445092+0000","last_unstale":"2024-09-16T02:02:41.571068+0000","last_undegraded":"2024-09-16T02:02:41.571068+0000","last_fullsized":"2024-09-16T02:02:41.571068+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:54.274692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.606896+0000","last_change":"2024-09-16T02:00:34.448450+0000","last_active":"2024-09-16T02:02:41.606896+0000","last_peered":"2024-09-16T02:02:41.606896+0000","last_clean":"2024-09-16T02:02:41.606896+0000","last_became_active":"2024-09-16T02:00:34.448116+0000","last_became_peered":"2024-09-16T02:00:34.448116+0000","last_unstale":"2024-09-16T02:02:41.606896+0000","last_undegraded":"2024-09-16T02:02:41.606896+0000","last_fullsized":"2024-09-16T02:02:41.606896+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:12:34.834798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.565670+0000","last_change":"2024-09-16T02:00:32.445427+0000","last_active":"2024-09-16T02:02:41.565670+0000","last_peered":"2024-09-16T02:02:41.565670+0000","last_clean":"2024-09-16T02:02:41.565670+0000","last_became_active":"2024-09-16T02:00:32.445058+0000","last_became_peered":"2024-09-16T02:00:32.445058+0000","last_unstale":"2024-09-16T02:02:41.565670+0000","last_undegraded":"2024-09-16T02:02:41.565670+0000","last_fullsized":"2024-09-16T02:02:41.565670+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:35:34.701331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.452607+0000","last_change":"2024-09-16T02:00:34.458479+0000","last_active":"2024-09-16T02:02:09.452607+0000","last_peered":"2024-09-16T02:02:09.452607+0000","last_clean":"2024-09-16T02:02:09.452607+0000","last_became_active":"2024-09-16T02:00:34.458135+0000","last_became_peered":"2024-09-16T02:00:34.458135+0000","last_unstale":"2024-09-16T02:02:09.452607+0000","last_undegraded":"2024-09-16T02:02:09.452607+0000","last_fullsized":"2024-09-16T02:02:09.452607+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:37:40.188861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'8","reported_seq":156,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:42.706504+0000","last_change":"2024-09-16T02:00:32.440319+0000","last_active":"2024-09-16T02:02:42.706504+0000","last_peered":"2024-09-16T02:02:42.706504+0000","last_clean":"2024-09-16T02:02:42.706504+0000","last_became_active":"2024-09-16T02:00:32.440005+0000","last_became_peered":"2024-09-16T02:00:32.440005+0000","last_unstale":"2024-09-16T02:02:42.706504+0000","last_undegraded":"2024-09-16T02:02:42.706504+0000","last_fullsized":"2024-09-16T02:02:42.706504+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:07:33.876577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1617,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.439440+0000","last_change":"2024-09-16T02:00:34.460468+0000","last_active":"2024-09-16T02:02:09.439440+0000","last_peered":"2024-09-16T02:02:09.439440+0000","last_clean":"2024-09-16T02:02:09.439440+0000","last_became_active":"2024-09-16T02:00:34.459190+0000","last_became_peered":"2024-09-16T02:00:34.459190+0000","last_unstale":"2024-09-16T02:02:09.439440+0000","last_undegraded":"2024-09-16T02:02:09.439440+0000","last_fullsized":"2024-09-16T02:02:09.439440+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:46:43.654991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.565709+0000","last_change":"2024-09-16T02:00:32.587700+0000","last_active":"2024-09-16T02:02:41.565709+0000","last_peered":"2024-09-16T02:02:41.565709+0000","last_clean":"2024-09-16T02:02:41.565709+0000","last_became_active":"2024-09-16T02:00:32.586964+0000","last_became_peered":"2024-09-16T02:00:32.586964+0000","last_unstale":"2024-09-16T02:02:41.565709+0000","last_undegraded":"2024-09-16T02:02:41.565709+0000","last_fullsized":"2024-09-16T02:02:41.565709+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:41.309918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.565323+0000","last_change":"2024-09-16T02:00:34.455652+0000","last_active":"2024-09-16T02:02:41.565323+0000","last_peered":"2024-09-16T02:02:41.565323+0000","last_clean":"2024-09-16T02:02:41.565323+0000","last_became_active":"2024-09-16T02:00:34.455306+0000","last_became_peered":"2024-09-16T02:00:34.455306+0000","last_unstale":"2024-09-16T02:02:41.565323+0000","last_undegraded":"2024-09-16T02:02:41.565323+0000","last_fullsized":"2024-09-16T02:02:41.565323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:44:09.505096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'3","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.566304+0000","last_change":"2024-09-16T02:00:32.441443+0000","last_active":"2024-09-16T02:02:41.566304+0000","last_peered":"2024-09-16T02:02:41.566304+0000","last_clean":"2024-09-16T02:02:41.566304+0000","last_became_active":"2024-09-16T02:00:32.441073+0000","last_became_peered":"2024-09-16T02:00:32.441073+0000","last_unstale":"2024-09-16T02:02:41.566304+0000","last_undegraded":"2024-09-16T02:02:41.566304+0000","last_fullsized":"2024-09-16T02:02:41.566304+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:16.865970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.629121+0000","last_change":"2024-09-16T02:01:40.611317+0000","last_active":"2024-09-16T02:02:41.629121+0000","last_peered":"2024-09-16T02:02:41.629121+0000","last_clean":"2024-09-16T02:02:41.629121+0000","last_became_active":"2024-09-16T02:01:40.610837+0000","last_became_peered":"2024-09-16T02:01:40.610837+0000","last_unstale":"2024-09-16T02:02:41.629121+0000","last_undegraded":"2024-09-16T02:02:41.629121+0000","last_fullsized":"2024-09-16T02:02:41.629121+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:19:55.376738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.607108+0000","last_change":"2024-09-16T02:01:40.617877+0000","last_active":"2024-09-16T02:02:41.607108+0000","last_peered":"2024-09-16T02:02:41.607108+0000","last_clean":"2024-09-16T02:02:41.607108+0000","last_became_active":"2024-09-16T02:01:40.611597+0000","last_became_peered":"2024-09-16T02:01:40.611597+0000","last_unstale":"2024-09-16T02:02:41.607108+0000","last_undegraded":"2024-09-16T02:02:41.607108+0000","last_fullsized":"2024-09-16T02:02:41.607108+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:53:35.796243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.449557+0000","last_change":"2024-09-16T02:00:34.461889+0000","last_active":"2024-09-16T02:02:09.449557+0000","last_peered":"2024-09-16T02:02:09.449557+0000","last_clean":"2024-09-16T02:02:09.449557+0000","last_became_active":"2024-09-16T02:00:34.461583+0000","last_became_peered":"2024-09-16T02:00:34.461583+0000","last_unstale":"2024-09-16T02:02:09.449557+0000","last_undegraded":"2024-09-16T02:02:09.449557+0000","last_fullsized":"2024-09-16T02:02:09.449557+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:32.598422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.565762+0000","last_change":"2024-09-16T02:00:32.445201+0000","last_active":"2024-09-16T02:02:41.565762+0000","last_peered":"2024-09-16T02:02:41.565762+0000","last_clean":"2024-09-16T02:02:41.565762+0000","last_became_active":"2024-09-16T02:00:32.444722+0000","last_became_peered":"2024-09-16T02:00:32.444722+0000","last_unstale":"2024-09-16T02:02:41.565762+0000","last_undegraded":"2024-09-16T02:02:41.565762+0000","last_fullsized":"2024-09-16T02:02:41.565762+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:41:11.657028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.445512+0000","last_change":"2024-09-16T02:01:40.613676+0000","last_active":"2024-09-16T02:02:09.445512+0000","last_peered":"2024-09-16T02:02:09.445512+0000","last_clean":"2024-09-16T02:02:09.445512+0000","last_became_active":"2024-09-16T02:01:40.608260+0000","last_became_peered":"2024-09-16T02:01:40.608260+0000","last_unstale":"2024-09-16T02:02:09.445512+0000","last_undegraded":"2024-09-16T02:02:09.445512+0000","last_fullsized":"2024-09-16T02:02:09.445512+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:30:59.004103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'6","reported_seq":151,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:42.573509+0000","last_change":"2024-09-16T02:00:32.860059+0000","last_active":"2024-09-16T02:02:42.573509+0000","last_peered":"2024-09-16T02:02:42.573509+0000","last_clean":"2024-09-16T02:02:42.573509+0000","last_became_active":"2024-09-16T02:00:32.859617+0000","last_became_peered":"2024-09-16T02:00:32.859617+0000","last_unstale":"2024-09-16T02:02:42.573509+0000","last_undegraded":"2024-09-16T02:02:42.573509+0000","last_fullsized":"2024-09-16T02:02:42.573509+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:41:50.248194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1665,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.457956+0000","last_change":"2024-09-16T02:00:34.454344+0000","last_active":"2024-09-16T02:02:09.457956+0000","last_peered":"2024-09-16T02:02:09.457956+0000","last_clean":"2024-09-16T02:02:09.457956+0000","last_became_active":"2024-09-16T02:00:34.454027+0000","last_became_peered":"2024-09-16T02:00:34.454027+0000","last_unstale":"2024-09-16T02:02:09.457956+0000","last_undegraded":"2024-09-16T02:02:09.457956+0000","last_fullsized":"2024-09-16T02:02:09.457956+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:33.020365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":335,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564402+0000","last_change":"2024-09-16T01:59:19.798159+0000","last_active":"2024-09-16T02:01:40.564402+0000","last_peered":"2024-09-16T02:01:40.564402+0000","last_clean":"2024-09-16T02:01:40.564402+0000","last_became_active":"2024-09-16T01:59:19.797688+0000","last_became_peered":"2024-09-16T01:59:19.797688+0000","last_unstale":"2024-09-16T02:01:40.564402+0000","last_undegraded":"2024-09-16T02:01:40.564402+0000","last_fullsized":"2024-09-16T02:01:40.564402+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:55:54.312750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:55:54.312750+0000","last_clean_scrub_stamp":"2024-09-16T01:55:54.312750+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:00:36.860158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.564285+0000","last_change":"2024-09-16T02:00:32.441925+0000","last_active":"2024-09-16T02:02:41.564285+0000","last_peered":"2024-09-16T02:02:41.564285+0000","last_clean":"2024-09-16T02:02:41.564285+0000","last_became_active":"2024-09-16T02:00:32.441565+0000","last_became_peered":"2024-09-16T02:00:32.441565+0000","last_unstale":"2024-09-16T02:02:41.564285+0000","last_undegraded":"2024-09-16T02:02:41.564285+0000","last_fullsized":"2024-09-16T02:02:41.564285+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:23:09.755034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.606600+0000","last_change":"2024-09-16T02:00:34.452986+0000","last_active":"2024-09-16T02:02:41.606600+0000","last_peered":"2024-09-16T02:02:41.606600+0000","last_clean":"2024-09-16T02:02:41.606600+0000","last_became_active":"2024-09-16T02:00:34.452688+0000","last_became_peered":"2024-09-16T02:00:34.452688+0000","last_unstale":"2024-09-16T02:02:41.606600+0000","last_undegraded":"2024-09-16T02:02:41.606600+0000","last_fullsized":"2024-09-16T02:02:41.606600+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:12.958209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564282+0000","last_change":"2024-09-16T02:00:32.443964+0000","last_active":"2024-09-16T02:01:40.564282+0000","last_peered":"2024-09-16T02:01:40.564282+0000","last_clean":"2024-09-16T02:01:40.564282+0000","last_became_active":"2024-09-16T02:00:32.443623+0000","last_became_peered":"2024-09-16T02:00:32.443623+0000","last_unstale":"2024-09-16T02:01:40.564282+0000","last_undegraded":"2024-09-16T02:01:40.564282+0000","last_fullsized":"2024-09-16T02:01:40.564282+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:30:34.406024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.606679+0000","last_change":"2024-09-16T02:00:34.449615+0000","last_active":"2024-09-16T02:02:41.606679+0000","last_peered":"2024-09-16T02:02:41.606679+0000","last_clean":"2024-09-16T02:02:41.606679+0000","last_became_active":"2024-09-16T02:00:34.449109+0000","last_became_peered":"2024-09-16T02:00:34.449109+0000","last_unstale":"2024-09-16T02:02:41.606679+0000","last_undegraded":"2024-09-16T02:02:41.606679+0000","last_fullsized":"2024-09-16T02:02:41.606679+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:45.036177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'7","reported_seq":152,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:42.705716+0000","last_change":"2024-09-16T02:00:32.587107+0000","last_active":"2024-09-16T02:02:42.705716+0000","last_peered":"2024-09-16T02:02:42.705716+0000","last_clean":"2024-09-16T02:02:42.705716+0000","last_became_active":"2024-09-16T02:00:32.586777+0000","last_became_peered":"2024-09-16T02:00:32.586777+0000","last_unstale":"2024-09-16T02:02:42.705716+0000","last_undegraded":"2024-09-16T02:02:42.705716+0000","last_fullsized":"2024-09-16T02:02:42.705716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:03.968987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.565315+0000","last_change":"2024-09-16T02:01:40.570572+0000","last_active":"2024-09-16T02:02:41.565315+0000","last_peered":"2024-09-16T02:02:41.565315+0000","last_clean":"2024-09-16T02:02:41.565315+0000","last_became_active":"2024-09-16T02:01:40.570176+0000","last_became_peered":"2024-09-16T02:01:40.570176+0000","last_unstale":"2024-09-16T02:02:41.565315+0000","last_undegraded":"2024-09-16T02:02:41.565315+0000","last_fullsized":"2024-09-16T02:02:41.565315+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:24:13.138925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'6","reported_seq":144,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:39.691569+0000","last_change":"2024-09-16T02:00:32.443201+0000","last_active":"2024-09-16T02:02:39.691569+0000","last_peered":"2024-09-16T02:02:39.691569+0000","last_clean":"2024-09-16T02:02:39.691569+0000","last_became_active":"2024-09-16T02:00:32.442778+0000","last_became_peered":"2024-09-16T02:00:32.442778+0000","last_unstale":"2024-09-16T02:02:39.691569+0000","last_undegraded":"2024-09-16T02:02:39.691569+0000","last_fullsized":"2024-09-16T02:02:39.691569+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:43.933817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1083,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.566693+0000","last_change":"2024-09-16T02:00:34.463000+0000","last_active":"2024-09-16T02:02:41.566693+0000","last_peered":"2024-09-16T02:02:41.566693+0000","last_clean":"2024-09-16T02:02:41.566693+0000","last_became_active":"2024-09-16T02:00:34.461890+0000","last_became_peered":"2024-09-16T02:00:34.461890+0000","last_unstale":"2024-09-16T02:02:41.566693+0000","last_undegraded":"2024-09-16T02:02:41.566693+0000","last_fullsized":"2024-09-16T02:02:41.566693+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:21:15.161295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-16T02:02:40.558984+0000","last_change":"2024-09-16T02:00:32.586933+0000","last_active":"2024-09-16T02:02:40.558984+0000","last_peered":"2024-09-16T02:02:40.558984+0000","last_clean":"2024-09-16T02:02:40.558984+0000","last_became_active":"2024-09-16T02:00:32.586565+0000","last_became_peered":"2024-09-16T02:00:32.586565+0000","last_unstale":"2024-09-16T02:02:40.558984+0000","last_undegraded":"2024-09-16T02:02:40.558984+0000","last_fullsized":"2024-09-16T02:02:40.558984+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:45:07.130649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.567361+0000","last_change":"2024-09-16T02:00:34.460805+0000","last_active":"2024-09-16T02:02:41.567361+0000","last_peered":"2024-09-16T02:02:41.567361+0000","last_clean":"2024-09-16T02:02:41.567361+0000","last_became_active":"2024-09-16T02:00:34.460482+0000","last_became_peered":"2024-09-16T02:00:34.460482+0000","last_unstale":"2024-09-16T02:02:41.567361+0000","last_undegraded":"2024-09-16T02:02:41.567361+0000","last_fullsized":"2024-09-16T02:02:41.567361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:07:51.565053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.566451+0000","last_change":"2024-09-16T02:00:32.587516+0000","last_active":"2024-09-16T02:02:41.566451+0000","last_peered":"2024-09-16T02:02:41.566451+0000","last_clean":"2024-09-16T02:02:41.566451+0000","last_became_active":"2024-09-16T02:00:32.587172+0000","last_became_peered":"2024-09-16T02:00:32.587172+0000","last_unstale":"2024-09-16T02:02:41.566451+0000","last_undegraded":"2024-09-16T02:02:41.566451+0000","last_fullsized":"2024-09-16T02:02:41.566451+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:52:29.018290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.607223+0000","last_change":"2024-09-16T02:00:34.446021+0000","last_active":"2024-09-16T02:02:41.607223+0000","last_peered":"2024-09-16T02:02:41.607223+0000","last_clean":"2024-09-16T02:02:41.607223+0000","last_became_active":"2024-09-16T02:00:34.445732+0000","last_became_peered":"2024-09-16T02:00:34.445732+0000","last_unstale":"2024-09-16T02:02:41.607223+0000","last_undegraded":"2024-09-16T02:02:41.607223+0000","last_fullsized":"2024-09-16T02:02:41.607223+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:51:17.960792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'16","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-16T02:02:40.559157+0000","last_change":"2024-09-16T02:00:32.439750+0000","last_active":"2024-09-16T02:02:40.559157+0000","last_peered":"2024-09-16T02:02:40.559157+0000","last_clean":"2024-09-16T02:02:40.559157+0000","last_became_active":"2024-09-16T02:00:32.439438+0000","last_became_peered":"2024-09-16T02:00:32.439438+0000","last_unstale":"2024-09-16T02:02:40.559157+0000","last_undegraded":"2024-09-16T02:02:40.559157+0000","last_fullsized":"2024-09-16T02:02:40.559157+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:50.764733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.606872+0000","last_change":"2024-09-16T02:00:34.453810+0000","last_active":"2024-09-16T02:02:41.606872+0000","last_peered":"2024-09-16T02:02:41.606872+0000","last_clean":"2024-09-16T02:02:41.606872+0000","last_became_active":"2024-09-16T02:00:34.453421+0000","last_became_peered":"2024-09-16T02:00:34.453421+0000","last_unstale":"2024-09-16T02:02:41.606872+0000","last_undegraded":"2024-09-16T02:02:41.606872+0000","last_fullsized":"2024-09-16T02:02:41.606872+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:23:50.405749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.564776+0000","last_change":"2024-09-16T02:00:32.860565+0000","last_active":"2024-09-16T02:02:41.564776+0000","last_peered":"2024-09-16T02:02:41.564776+0000","last_clean":"2024-09-16T02:02:41.564776+0000","last_became_active":"2024-09-16T02:00:32.860130+0000","last_became_peered":"2024-09-16T02:00:32.860130+0000","last_unstale":"2024-09-16T02:02:41.564776+0000","last_undegraded":"2024-09-16T02:02:41.564776+0000","last_fullsized":"2024-09-16T02:02:41.564776+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:09:55.759883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.442038+0000","last_change":"2024-09-16T02:01:40.564454+0000","last_active":"2024-09-16T02:02:09.442038+0000","last_peered":"2024-09-16T02:02:09.442038+0000","last_clean":"2024-09-16T02:02:09.442038+0000","last_became_active":"2024-09-16T02:01:40.563854+0000","last_became_peered":"2024-09-16T02:01:40.563854+0000","last_unstale":"2024-09-16T02:02:09.442038+0000","last_undegraded":"2024-09-16T02:02:09.442038+0000","last_fullsized":"2024-09-16T02:02:09.442038+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:56:30.610947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.632971+0000","last_change":"2024-09-16T02:02:41.632561+0000","last_active":"2024-09-16T02:02:41.632971+0000","last_peered":"2024-09-16T02:02:41.632971+0000","last_clean":"2024-09-16T02:02:41.632971+0000","last_became_active":"2024-09-16T02:02:41.630051+0000","last_became_peered":"2024-09-16T02:02:41.630051+0000","last_unstale":"2024-09-16T02:02:41.632971+0000","last_undegraded":"2024-09-16T02:02:41.632971+0000","last_fullsized":"2024-09-16T02:02:41.632971+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:49:57.569311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.565707+0000","last_change":"2024-09-16T02:00:34.457673+0000","last_active":"2024-09-16T02:02:41.565707+0000","last_peered":"2024-09-16T02:02:41.565707+0000","last_clean":"2024-09-16T02:02:41.565707+0000","last_became_active":"2024-09-16T02:00:34.457331+0000","last_became_peered":"2024-09-16T02:00:34.457331+0000","last_unstale":"2024-09-16T02:02:41.565707+0000","last_undegraded":"2024-09-16T02:02:41.565707+0000","last_fullsized":"2024-09-16T02:02:41.565707+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:12:16.100036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.606509+0000","last_change":"2024-09-16T02:00:32.437528+0000","last_active":"2024-09-16T02:02:41.606509+0000","last_peered":"2024-09-16T02:02:41.606509+0000","last_clean":"2024-09-16T02:02:41.606509+0000","last_became_active":"2024-09-16T02:00:32.437080+0000","last_became_peered":"2024-09-16T02:00:32.437080+0000","last_unstale":"2024-09-16T02:02:41.606509+0000","last_undegraded":"2024-09-16T02:02:41.606509+0000","last_fullsized":"2024-09-16T02:02:41.606509+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:59:33.908026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.629149+0000","last_change":"2024-09-16T02:00:34.453504+0000","last_active":"2024-09-16T02:02:41.629149+0000","last_peered":"2024-09-16T02:02:41.629149+0000","last_clean":"2024-09-16T02:02:41.629149+0000","last_became_active":"2024-09-16T02:00:34.453212+0000","last_became_peered":"2024-09-16T02:00:34.453212+0000","last_unstale":"2024-09-16T02:02:41.629149+0000","last_undegraded":"2024-09-16T02:02:41.629149+0000","last_fullsized":"2024-09-16T02:02:41.629149+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:00:56.414548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'4","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.571093+0000","last_change":"2024-09-16T02:00:32.442282+0000","last_active":"2024-09-16T02:02:41.571093+0000","last_peered":"2024-09-16T02:02:41.571093+0000","last_clean":"2024-09-16T02:02:41.571093+0000","last_became_active":"2024-09-16T02:00:32.441910+0000","last_became_peered":"2024-09-16T02:00:32.441910+0000","last_unstale":"2024-09-16T02:02:41.571093+0000","last_undegraded":"2024-09-16T02:02:41.571093+0000","last_fullsized":"2024-09-16T02:02:41.571093+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:01:46.045021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.606559+0000","last_change":"2024-09-16T02:00:34.460177+0000","last_active":"2024-09-16T02:02:41.606559+0000","last_peered":"2024-09-16T02:02:41.606559+0000","last_clean":"2024-09-16T02:02:41.606559+0000","last_became_active":"2024-09-16T02:00:34.459874+0000","last_became_peered":"2024-09-16T02:00:34.459874+0000","last_unstale":"2024-09-16T02:02:41.606559+0000","last_undegraded":"2024-09-16T02:02:41.606559+0000","last_fullsized":"2024-09-16T02:02:41.606559+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:35.855591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":55,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.611163+0000","last_change":"2024-09-16T02:02:41.611163+0000","last_active":"2024-09-16T02:02:41.611163+0000","last_peered":"2024-09-16T02:02:41.611163+0000","last_clean":"2024-09-16T02:02:41.611163+0000","last_became_active":"2024-09-16T02:02:41.608020+0000","last_became_peered":"2024-09-16T02:02:41.608020+0000","last_unstale":"2024-09-16T02:02:41.611163+0000","last_undegraded":"2024-09-16T02:02:41.611163+0000","last_fullsized":"2024-09-16T02:02:41.611163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:43:57.184447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,6],"acting":[8,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.457243+0000","last_change":"2024-09-16T02:00:34.452964+0000","last_active":"2024-09-16T02:02:09.457243+0000","last_peered":"2024-09-16T02:02:09.457243+0000","last_clean":"2024-09-16T02:02:09.457243+0000","last_became_active":"2024-09-16T02:00:34.452617+0000","last_became_peered":"2024-09-16T02:00:34.452617+0000","last_unstale":"2024-09-16T02:02:09.457243+0000","last_undegraded":"2024-09-16T02:02:09.457243+0000","last_fullsized":"2024-09-16T02:02:09.457243+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:04.617970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.565533+0000","last_change":"2024-09-16T02:00:34.456585+0000","last_active":"2024-09-16T02:02:41.565533+0000","last_peered":"2024-09-16T02:02:41.565533+0000","last_clean":"2024-09-16T02:02:41.565533+0000","last_became_active":"2024-09-16T02:00:34.456199+0000","last_became_peered":"2024-09-16T02:00:34.456199+0000","last_unstale":"2024-09-16T02:02:41.565533+0000","last_undegraded":"2024-09-16T02:02:41.565533+0000","last_fullsized":"2024-09-16T02:02:41.565533+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:37:54.376785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.628991+0000","last_change":"2024-09-16T02:01:40.578904+0000","last_active":"2024-09-16T02:02:41.628991+0000","last_peered":"2024-09-16T02:02:41.628991+0000","last_clean":"2024-09-16T02:02:41.628991+0000","last_became_active":"2024-09-16T02:01:40.567221+0000","last_became_peered":"2024-09-16T02:01:40.567221+0000","last_unstale":"2024-09-16T02:02:41.628991+0000","last_undegraded":"2024-09-16T02:02:41.628991+0000","last_fullsized":"2024-09-16T02:02:41.628991+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:01.407145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.569386+0000","last_change":"2024-09-16T02:00:34.460925+0000","last_active":"2024-09-16T02:02:41.569386+0000","last_peered":"2024-09-16T02:02:41.569386+0000","last_clean":"2024-09-16T02:02:41.569386+0000","last_became_active":"2024-09-16T02:00:34.460581+0000","last_became_peered":"2024-09-16T02:00:34.460581+0000","last_unstale":"2024-09-16T02:02:41.569386+0000","last_undegraded":"2024-09-16T02:02:41.569386+0000","last_fullsized":"2024-09-16T02:02:41.569386+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:19:44.460695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.566351+0000","last_change":"2024-09-16T02:00:32.443877+0000","last_active":"2024-09-16T02:02:41.566351+0000","last_peered":"2024-09-16T02:02:41.566351+0000","last_clean":"2024-09-16T02:02:41.566351+0000","last_became_active":"2024-09-16T02:00:32.443416+0000","last_became_peered":"2024-09-16T02:00:32.443416+0000","last_unstale":"2024-09-16T02:02:41.566351+0000","last_undegraded":"2024-09-16T02:02:41.566351+0000","last_fullsized":"2024-09-16T02:02:41.566351+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:36:57.367232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.447996+0000","last_change":"2024-09-16T02:00:34.453048+0000","last_active":"2024-09-16T02:02:09.447996+0000","last_peered":"2024-09-16T02:02:09.447996+0000","last_clean":"2024-09-16T02:02:09.447996+0000","last_became_active":"2024-09-16T02:00:34.452752+0000","last_became_peered":"2024-09-16T02:00:34.452752+0000","last_unstale":"2024-09-16T02:02:09.447996+0000","last_undegraded":"2024-09-16T02:02:09.447996+0000","last_fullsized":"2024-09-16T02:02:09.447996+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:55.970195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.606728+0000","last_change":"2024-09-16T02:00:32.586974+0000","last_active":"2024-09-16T02:02:41.606728+0000","last_peered":"2024-09-16T02:02:41.606728+0000","last_clean":"2024-09-16T02:02:41.606728+0000","last_became_active":"2024-09-16T02:00:32.586644+0000","last_became_peered":"2024-09-16T02:00:32.586644+0000","last_unstale":"2024-09-16T02:02:41.606728+0000","last_undegraded":"2024-09-16T02:02:41.606728+0000","last_fullsized":"2024-09-16T02:02:41.606728+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:52.913287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.606607+0000","last_change":"2024-09-16T02:00:34.447548+0000","last_active":"2024-09-16T02:02:41.606607+0000","last_peered":"2024-09-16T02:02:41.606607+0000","last_clean":"2024-09-16T02:02:41.606607+0000","last_became_active":"2024-09-16T02:00:34.447207+0000","last_became_peered":"2024-09-16T02:00:34.447207+0000","last_unstale":"2024-09-16T02:02:41.606607+0000","last_undegraded":"2024-09-16T02:02:41.606607+0000","last_fullsized":"2024-09-16T02:02:41.606607+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:22:02.494343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'14","reported_seq":154,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-16T02:02:40.692237+0000","last_change":"2024-09-16T02:00:32.441129+0000","last_active":"2024-09-16T02:02:40.692237+0000","last_peered":"2024-09-16T02:02:40.692237+0000","last_clean":"2024-09-16T02:02:40.692237+0000","last_became_active":"2024-09-16T02:00:32.440843+0000","last_became_peered":"2024-09-16T02:00:32.440843+0000","last_unstale":"2024-09-16T02:02:40.692237+0000","last_undegraded":"2024-09-16T02:02:40.692237+0000","last_fullsized":"2024-09-16T02:02:40.692237+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:17.721740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68891,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1198,"num_read_kb":674,"num_write":261,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":762,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":206673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":224,"ondisk_log_size":224,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281576,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612648,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":917504,"data_stored":450434,"data_compressed":11572,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841581,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1069056,"data_stored":1101370,"data_compressed":47384,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005105,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":929792,"data_stored":440845,"data_compressed":8664,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234039,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":929792,"data_stored":449083,"data_compressed":11572,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462971,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":1026622,"data_compressed":41575,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626498,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":921600,"data_stored":449499,"data_compressed":11572,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855430,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":991232,"data_stored":1041785,"data_compressed":47384,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018955,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":921600,"data_stored":449894,"data_compressed":11572,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247889,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":933888,"data_stored":448712,"data_compressed":11572,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444118,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1003520,"data_stored":511082,"data_compressed":11572,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":991232,"data_stored":510547,"data_compressed":11572,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":433649,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":62382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":3135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T02:02:45.663 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-16T02:02:45.664 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T02:02:45.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.799+0000 7fe353a7b640 1 -- 172.21.15.105:0/1035610562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe34c107850 msgr2=0x7fe34c109c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:45.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.799+0000 7fe353a7b640 1 --2- 172.21.15.105:0/1035610562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe34c107850 0x7fe34c109c60 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe344009ca0 tx=0x7fe34402a8c0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.799+0000 7fe353a7b640 1 -- 172.21.15.105:0/1035610562 shutdown_connections 2024-09-16T02:02:45.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.799+0000 7fe353a7b640 1 --2- 172.21.15.105:0/1035610562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe34c107850 0x7fe34c109c60 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.799+0000 7fe353a7b640 1 --2- 172.21.15.105:0/1035610562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe34c104f20 0x7fe34c107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.799+0000 7fe353a7b640 1 --2- 172.21.15.105:0/1035610562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe34c1025f0 0x7fe34c1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.799+0000 7fe353a7b640 1 -- 172.21.15.105:0/1035610562 >> 172.21.15.105:0/1035610562 conn(0x7fe34c0fc170 msgr2=0x7fe34c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:45.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.799+0000 7fe353a7b640 1 -- 172.21.15.105:0/1035610562 shutdown_connections 2024-09-16T02:02:45.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.799+0000 7fe353a7b640 1 -- 172.21.15.105:0/1035610562 wait complete. 2024-09-16T02:02:45.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.799+0000 7fe353a7b640 1 Processor -- start 2024-09-16T02:02:45.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.799+0000 7fe353a7b640 1 -- start start 2024-09-16T02:02:45.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.799+0000 7fe353a7b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe34c1025f0 0x7fe34c101f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:45.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.799+0000 7fe353a7b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe34c104f20 0x7fe34c1005a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:45.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.799+0000 7fe353a7b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe34c107850 0x7fe34c100ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:45.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.799+0000 7fe353a7b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe34c10d680 con 0x7fe34c1025f0 2024-09-16T02:02:45.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.799+0000 7fe353a7b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe34c10d500 con 0x7fe34c104f20 2024-09-16T02:02:45.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.799+0000 7fe353a7b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe34c10d800 con 0x7fe34c107850 2024-09-16T02:02:45.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.800+0000 7fe351817640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe34c1025f0 0x7fe34c101f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:45.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.800+0000 7fe351817640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe34c1025f0 0x7fe34c101f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60624/0 (socket says 172.21.15.105:60624) 2024-09-16T02:02:45.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.800+0000 7fe351817640 1 -- 172.21.15.105:0/2269741817 learned_addr learned my addr 172.21.15.105:0/2269741817 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:45.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.800+0000 7fe351817640 1 -- 172.21.15.105:0/2269741817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe34c107850 msgr2=0x7fe34c100ae0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T02:02:45.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.800+0000 7fe351817640 1 --2- 172.21.15.105:0/2269741817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe34c107850 0x7fe34c100ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.800+0000 7fe351016640 1 --2- 172.21.15.105:0/2269741817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe34c104f20 0x7fe34c1005a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:45.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.800+0000 7fe351817640 1 -- 172.21.15.105:0/2269741817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe34c104f20 msgr2=0x7fe34c1005a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:45.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.800+0000 7fe351817640 1 --2- 172.21.15.105:0/2269741817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe34c104f20 0x7fe34c1005a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.800+0000 7fe351817640 1 -- 172.21.15.105:0/2269741817 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe34c101370 con 0x7fe34c1025f0 2024-09-16T02:02:45.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.800+0000 7fe351016640 1 --2- 172.21.15.105:0/2269741817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe34c104f20 0x7fe34c1005a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:02:45.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.800+0000 7fe351817640 1 --2- 172.21.15.105:0/2269741817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe34c1025f0 0x7fe34c101f60 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe34800d770 tx=0x7fe34800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:45.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.800+0000 7fe342ffd640 1 -- 172.21.15.105:0/2269741817 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe348017070 con 0x7fe34c1025f0 2024-09-16T02:02:45.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.800+0000 7fe342ffd640 1 -- 172.21.15.105:0/2269741817 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3480054d0 con 0x7fe34c1025f0 2024-09-16T02:02:45.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.800+0000 7fe353a7b640 1 -- 172.21.15.105:0/2269741817 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe34c1a1ef0 con 0x7fe34c1025f0 2024-09-16T02:02:45.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.800+0000 7fe342ffd640 1 -- 172.21.15.105:0/2269741817 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe348004cf0 con 0x7fe34c1025f0 2024-09-16T02:02:45.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.800+0000 7fe353a7b640 1 -- 172.21.15.105:0/2269741817 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe34c1a22f0 con 0x7fe34c1025f0 2024-09-16T02:02:45.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.801+0000 7fe353a7b640 1 -- 172.21.15.105:0/2269741817 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe34c0fdc60 con 0x7fe34c1025f0 2024-09-16T02:02:45.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.802+0000 7fe342ffd640 1 -- 172.21.15.105:0/2269741817 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fe348004e90 con 0x7fe34c1025f0 2024-09-16T02:02:45.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.803+0000 7fe342ffd640 1 --2- 172.21.15.105:0/2269741817 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe3380a3250 0x7fe3380a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:45.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.803+0000 7fe351016640 1 --2- 172.21.15.105:0/2269741817 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe3380a3250 0x7fe3380a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:45.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.803+0000 7fe342ffd640 1 -- 172.21.15.105:0/2269741817 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fe3480cf890 con 0x7fe34c1025f0 2024-09-16T02:02:45.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.804+0000 7fe351016640 1 --2- 172.21.15.105:0/2269741817 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe3380a3250 0x7fe3380a5710 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe34c101cd0 tx=0x7fe33c005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:45.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.806+0000 7fe342ffd640 1 -- 172.21.15.105:0/2269741817 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe34809c550 con 0x7fe34c1025f0 2024-09-16T02:02:45.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:45 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:02:45.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:45 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:02:45.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:45 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:02:45.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:45 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:02:45.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:45 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:02:45.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:45 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:02:45.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.971+0000 7fe353a7b640 1 -- 172.21.15.105:0/2269741817 --> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe34c062630 con 0x7fe3380a3250 2024-09-16T02:02:45.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.977+0000 7fe342ffd640 1 -- 172.21.15.105:0/2269741817 <== mgr.14150 v2:172.21.15.105:6800/275626334 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259165 (secure 0 0 0) 0x7fe34c062630 con 0x7fe3380a3250 2024-09-16T02:02:45.980 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:02:45.981 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-09-16T02:02:45.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.980+0000 7fe353a7b640 1 -- 172.21.15.105:0/2269741817 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe3380a3250 msgr2=0x7fe3380a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:45.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.980+0000 7fe353a7b640 1 --2- 172.21.15.105:0/2269741817 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe3380a3250 0x7fe3380a5710 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe34c101cd0 tx=0x7fe33c005e50 comp rx=0 tx=0).stop 2024-09-16T02:02:45.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.980+0000 7fe353a7b640 1 -- 172.21.15.105:0/2269741817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe34c1025f0 msgr2=0x7fe34c101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:45.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.980+0000 7fe353a7b640 1 --2- 172.21.15.105:0/2269741817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe34c1025f0 0x7fe34c101f60 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe34800d770 tx=0x7fe34800dc30 comp rx=0 tx=0).stop 2024-09-16T02:02:45.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.980+0000 7fe353a7b640 1 -- 172.21.15.105:0/2269741817 shutdown_connections 2024-09-16T02:02:45.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.980+0000 7fe353a7b640 1 --2- 172.21.15.105:0/2269741817 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe3380a3250 0x7fe3380a5710 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.980+0000 7fe353a7b640 1 --2- 172.21.15.105:0/2269741817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe34c107850 0x7fe34c100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.980+0000 7fe353a7b640 1 --2- 172.21.15.105:0/2269741817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe34c104f20 0x7fe34c1005a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.980+0000 7fe353a7b640 1 --2- 172.21.15.105:0/2269741817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe34c1025f0 0x7fe34c101f60 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.980+0000 7fe353a7b640 1 -- 172.21.15.105:0/2269741817 >> 172.21.15.105:0/2269741817 conn(0x7fe34c0fc170 msgr2=0x7fe34c102f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:45.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.980+0000 7fe353a7b640 1 -- 172.21.15.105:0/2269741817 shutdown_connections 2024-09-16T02:02:45.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:45.980+0000 7fe353a7b640 1 -- 172.21.15.105:0/2269741817 wait complete. 2024-09-16T02:02:45.991 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-09-16T02:02:45.475852+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659493,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1324,"num_read_kb":783,"num_write":444,"num_write_kb":3088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1181615,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":863,"ondisk_log_size":863,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373016,"kb_used_data":11220,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568808,"statfs":{"total":1151940427776,"available":1151558459392,"internally_reserved":0,"allocated":11489280,"data_stored":7313522,"data_compressed":231774,"data_compressed_allocated":311296,"data_compressed_original":2121728,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2475,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":93,"num_read_kb":50,"num_write":16,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":180,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001842"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.629463+0000","last_change":"2024-09-16T02:00:34.454121+0000","last_active":"2024-09-16T02:02:41.629463+0000","last_peered":"2024-09-16T02:02:41.629463+0000","last_clean":"2024-09-16T02:02:41.629463+0000","last_became_active":"2024-09-16T02:00:34.453810+0000","last_became_peered":"2024-09-16T02:00:34.453810+0000","last_unstale":"2024-09-16T02:02:41.629463+0000","last_undegraded":"2024-09-16T02:02:41.629463+0000","last_fullsized":"2024-09-16T02:02:41.629463+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:42:38.781928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.449067+0000","last_change":"2024-09-16T02:00:34.462020+0000","last_active":"2024-09-16T02:02:09.449067+0000","last_peered":"2024-09-16T02:02:09.449067+0000","last_clean":"2024-09-16T02:02:09.449067+0000","last_became_active":"2024-09-16T02:00:34.461680+0000","last_became_peered":"2024-09-16T02:00:34.461680+0000","last_unstale":"2024-09-16T02:02:09.449067+0000","last_undegraded":"2024-09-16T02:02:09.449067+0000","last_fullsized":"2024-09-16T02:02:09.449067+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:00:14.493643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.448082+0000","last_change":"2024-09-16T02:00:34.453034+0000","last_active":"2024-09-16T02:02:09.448082+0000","last_peered":"2024-09-16T02:02:09.448082+0000","last_clean":"2024-09-16T02:02:09.448082+0000","last_became_active":"2024-09-16T02:00:34.452719+0000","last_became_peered":"2024-09-16T02:00:34.452719+0000","last_unstale":"2024-09-16T02:02:09.448082+0000","last_undegraded":"2024-09-16T02:02:09.448082+0000","last_fullsized":"2024-09-16T02:02:09.448082+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:48:52.478501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.564538+0000","last_change":"2024-09-16T02:00:34.458156+0000","last_active":"2024-09-16T02:02:41.564538+0000","last_peered":"2024-09-16T02:02:41.564538+0000","last_clean":"2024-09-16T02:02:41.564538+0000","last_became_active":"2024-09-16T02:00:34.457831+0000","last_became_peered":"2024-09-16T02:00:34.457831+0000","last_unstale":"2024-09-16T02:02:41.564538+0000","last_undegraded":"2024-09-16T02:02:41.564538+0000","last_fullsized":"2024-09-16T02:02:41.564538+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:35:36.609046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.566422+0000","last_change":"2024-09-16T02:00:34.448638+0000","last_active":"2024-09-16T02:02:41.566422+0000","last_peered":"2024-09-16T02:02:41.566422+0000","last_clean":"2024-09-16T02:02:41.566422+0000","last_became_active":"2024-09-16T02:00:34.447993+0000","last_became_peered":"2024-09-16T02:00:34.447993+0000","last_unstale":"2024-09-16T02:02:41.566422+0000","last_undegraded":"2024-09-16T02:02:41.566422+0000","last_fullsized":"2024-09-16T02:02:41.566422+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:33:04.832053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.566521+0000","last_change":"2024-09-16T02:00:34.462651+0000","last_active":"2024-09-16T02:02:41.566521+0000","last_peered":"2024-09-16T02:02:41.566521+0000","last_clean":"2024-09-16T02:02:41.566521+0000","last_became_active":"2024-09-16T02:00:34.461837+0000","last_became_peered":"2024-09-16T02:00:34.461837+0000","last_unstale":"2024-09-16T02:02:41.566521+0000","last_undegraded":"2024-09-16T02:02:41.566521+0000","last_fullsized":"2024-09-16T02:02:41.566521+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:52:13.733591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.439301+0000","last_change":"2024-09-16T02:00:34.455362+0000","last_active":"2024-09-16T02:02:09.439301+0000","last_peered":"2024-09-16T02:02:09.439301+0000","last_clean":"2024-09-16T02:02:09.439301+0000","last_became_active":"2024-09-16T02:00:34.455029+0000","last_became_peered":"2024-09-16T02:00:34.455029+0000","last_unstale":"2024-09-16T02:02:09.439301+0000","last_undegraded":"2024-09-16T02:02:09.439301+0000","last_fullsized":"2024-09-16T02:02:09.439301+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:14:08.316845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.606253+0000","last_change":"2024-09-16T02:00:34.459578+0000","last_active":"2024-09-16T02:02:41.606253+0000","last_peered":"2024-09-16T02:02:41.606253+0000","last_clean":"2024-09-16T02:02:41.606253+0000","last_became_active":"2024-09-16T02:00:34.459288+0000","last_became_peered":"2024-09-16T02:00:34.459288+0000","last_unstale":"2024-09-16T02:02:41.606253+0000","last_undegraded":"2024-09-16T02:02:41.606253+0000","last_fullsized":"2024-09-16T02:02:41.606253+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:52:28.974276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.569781+0000","last_change":"2024-09-16T02:00:34.463011+0000","last_active":"2024-09-16T02:02:41.569781+0000","last_peered":"2024-09-16T02:02:41.569781+0000","last_clean":"2024-09-16T02:02:41.569781+0000","last_became_active":"2024-09-16T02:00:34.462668+0000","last_became_peered":"2024-09-16T02:00:34.462668+0000","last_unstale":"2024-09-16T02:02:41.569781+0000","last_undegraded":"2024-09-16T02:02:41.569781+0000","last_fullsized":"2024-09-16T02:02:41.569781+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:06:31.655869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.567949+0000","last_change":"2024-09-16T02:00:34.462241+0000","last_active":"2024-09-16T02:02:41.567949+0000","last_peered":"2024-09-16T02:02:41.567949+0000","last_clean":"2024-09-16T02:02:41.567949+0000","last_became_active":"2024-09-16T02:00:34.461925+0000","last_became_peered":"2024-09-16T02:00:34.461925+0000","last_unstale":"2024-09-16T02:02:41.567949+0000","last_undegraded":"2024-09-16T02:02:41.567949+0000","last_fullsized":"2024-09-16T02:02:41.567949+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:48:02.414471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.457960+0000","last_change":"2024-09-16T02:00:34.466132+0000","last_active":"2024-09-16T02:02:09.457960+0000","last_peered":"2024-09-16T02:02:09.457960+0000","last_clean":"2024-09-16T02:02:09.457960+0000","last_became_active":"2024-09-16T02:00:34.465750+0000","last_became_peered":"2024-09-16T02:00:34.465750+0000","last_unstale":"2024-09-16T02:02:09.457960+0000","last_undegraded":"2024-09-16T02:02:09.457960+0000","last_fullsized":"2024-09-16T02:02:09.457960+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:07:33.072432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.564542+0000","last_change":"2024-09-16T02:00:34.461759+0000","last_active":"2024-09-16T02:02:41.564542+0000","last_peered":"2024-09-16T02:02:41.564542+0000","last_clean":"2024-09-16T02:02:41.564542+0000","last_became_active":"2024-09-16T02:00:34.461385+0000","last_became_peered":"2024-09-16T02:00:34.461385+0000","last_unstale":"2024-09-16T02:02:41.564542+0000","last_undegraded":"2024-09-16T02:02:41.564542+0000","last_fullsized":"2024-09-16T02:02:41.564542+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:00:24.179172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.607349+0000","last_change":"2024-09-16T02:01:40.608361+0000","last_active":"2024-09-16T02:02:41.607349+0000","last_peered":"2024-09-16T02:02:41.607349+0000","last_clean":"2024-09-16T02:02:41.607349+0000","last_became_active":"2024-09-16T02:01:40.607954+0000","last_became_peered":"2024-09-16T02:01:40.607954+0000","last_unstale":"2024-09-16T02:02:41.607349+0000","last_undegraded":"2024-09-16T02:02:41.607349+0000","last_fullsized":"2024-09-16T02:02:41.607349+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:18:31.244969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.443096+0000","last_change":"2024-09-16T02:00:34.460680+0000","last_active":"2024-09-16T02:02:09.443096+0000","last_peered":"2024-09-16T02:02:09.443096+0000","last_clean":"2024-09-16T02:02:09.443096+0000","last_became_active":"2024-09-16T02:00:34.460161+0000","last_became_peered":"2024-09-16T02:00:34.460161+0000","last_unstale":"2024-09-16T02:02:09.443096+0000","last_undegraded":"2024-09-16T02:02:09.443096+0000","last_fullsized":"2024-09-16T02:02:09.443096+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:38:19.848420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.444910+0000","last_change":"2024-09-16T02:00:34.454606+0000","last_active":"2024-09-16T02:02:09.444910+0000","last_peered":"2024-09-16T02:02:09.444910+0000","last_clean":"2024-09-16T02:02:09.444910+0000","last_became_active":"2024-09-16T02:00:34.454293+0000","last_became_peered":"2024-09-16T02:00:34.454293+0000","last_unstale":"2024-09-16T02:02:09.444910+0000","last_undegraded":"2024-09-16T02:02:09.444910+0000","last_fullsized":"2024-09-16T02:02:09.444910+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:55:54.466176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.568781+0000","last_change":"2024-09-16T02:00:34.452940+0000","last_active":"2024-09-16T02:02:41.568781+0000","last_peered":"2024-09-16T02:02:41.568781+0000","last_clean":"2024-09-16T02:02:41.568781+0000","last_became_active":"2024-09-16T02:00:34.452560+0000","last_became_peered":"2024-09-16T02:00:34.452560+0000","last_unstale":"2024-09-16T02:02:41.568781+0000","last_undegraded":"2024-09-16T02:02:41.568781+0000","last_fullsized":"2024-09-16T02:02:41.568781+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:42:11.564264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.629422+0000","last_change":"2024-09-16T02:00:34.453457+0000","last_active":"2024-09-16T02:02:41.629422+0000","last_peered":"2024-09-16T02:02:41.629422+0000","last_clean":"2024-09-16T02:02:41.629422+0000","last_became_active":"2024-09-16T02:00:34.453148+0000","last_became_peered":"2024-09-16T02:00:34.453148+0000","last_unstale":"2024-09-16T02:02:41.629422+0000","last_undegraded":"2024-09-16T02:02:41.629422+0000","last_fullsized":"2024-09-16T02:02:41.629422+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:28.456892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-16T02:02:40.559102+0000","last_change":"2024-09-16T02:00:34.457057+0000","last_active":"2024-09-16T02:02:40.559102+0000","last_peered":"2024-09-16T02:02:40.559102+0000","last_clean":"2024-09-16T02:02:40.559102+0000","last_became_active":"2024-09-16T02:00:34.456751+0000","last_became_peered":"2024-09-16T02:00:34.456751+0000","last_unstale":"2024-09-16T02:02:40.559102+0000","last_undegraded":"2024-09-16T02:02:40.559102+0000","last_fullsized":"2024-09-16T02:02:40.559102+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:32:32.207510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.571309+0000","last_change":"2024-09-16T02:00:34.448434+0000","last_active":"2024-09-16T02:02:41.571309+0000","last_peered":"2024-09-16T02:02:41.571309+0000","last_clean":"2024-09-16T02:02:41.571309+0000","last_became_active":"2024-09-16T02:00:34.448056+0000","last_became_peered":"2024-09-16T02:00:34.448056+0000","last_unstale":"2024-09-16T02:02:41.571309+0000","last_undegraded":"2024-09-16T02:02:41.571309+0000","last_fullsized":"2024-09-16T02:02:41.571309+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:15:28.162159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.629246+0000","last_change":"2024-09-16T02:00:34.459499+0000","last_active":"2024-09-16T02:02:41.629246+0000","last_peered":"2024-09-16T02:02:41.629246+0000","last_clean":"2024-09-16T02:02:41.629246+0000","last_became_active":"2024-09-16T02:00:34.459017+0000","last_became_peered":"2024-09-16T02:00:34.459017+0000","last_unstale":"2024-09-16T02:02:41.629246+0000","last_undegraded":"2024-09-16T02:02:41.629246+0000","last_fullsized":"2024-09-16T02:02:41.629246+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:41:36.866100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.607427+0000","last_change":"2024-09-16T02:00:34.448640+0000","last_active":"2024-09-16T02:02:41.607427+0000","last_peered":"2024-09-16T02:02:41.607427+0000","last_clean":"2024-09-16T02:02:41.607427+0000","last_became_active":"2024-09-16T02:00:34.447737+0000","last_became_peered":"2024-09-16T02:00:34.447737+0000","last_unstale":"2024-09-16T02:02:41.607427+0000","last_undegraded":"2024-09-16T02:02:41.607427+0000","last_fullsized":"2024-09-16T02:02:41.607427+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:20.192650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.569353+0000","last_change":"2024-09-16T02:00:34.451718+0000","last_active":"2024-09-16T02:02:41.569353+0000","last_peered":"2024-09-16T02:02:41.569353+0000","last_clean":"2024-09-16T02:02:41.569353+0000","last_became_active":"2024-09-16T02:00:34.451389+0000","last_became_peered":"2024-09-16T02:00:34.451389+0000","last_unstale":"2024-09-16T02:02:41.569353+0000","last_undegraded":"2024-09-16T02:02:41.569353+0000","last_fullsized":"2024-09-16T02:02:41.569353+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:13:19.296178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.606335+0000","last_change":"2024-09-16T02:00:34.458379+0000","last_active":"2024-09-16T02:02:41.606335+0000","last_peered":"2024-09-16T02:02:41.606335+0000","last_clean":"2024-09-16T02:02:41.606335+0000","last_became_active":"2024-09-16T02:00:34.458038+0000","last_became_peered":"2024-09-16T02:00:34.458038+0000","last_unstale":"2024-09-16T02:02:41.606335+0000","last_undegraded":"2024-09-16T02:02:41.606335+0000","last_fullsized":"2024-09-16T02:02:41.606335+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:34.669470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.565299+0000","last_change":"2024-09-16T02:00:34.450927+0000","last_active":"2024-09-16T02:02:41.565299+0000","last_peered":"2024-09-16T02:02:41.565299+0000","last_clean":"2024-09-16T02:02:41.565299+0000","last_became_active":"2024-09-16T02:00:34.447669+0000","last_became_peered":"2024-09-16T02:00:34.447669+0000","last_unstale":"2024-09-16T02:02:41.565299+0000","last_undegraded":"2024-09-16T02:02:41.565299+0000","last_fullsized":"2024-09-16T02:02:41.565299+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:57.861633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.606779+0000","last_change":"2024-09-16T02:00:34.454827+0000","last_active":"2024-09-16T02:02:41.606779+0000","last_peered":"2024-09-16T02:02:41.606779+0000","last_clean":"2024-09-16T02:02:41.606779+0000","last_became_active":"2024-09-16T02:00:34.454549+0000","last_became_peered":"2024-09-16T02:00:34.454549+0000","last_unstale":"2024-09-16T02:02:41.606779+0000","last_undegraded":"2024-09-16T02:02:41.606779+0000","last_fullsized":"2024-09-16T02:02:41.606779+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:53:45.440860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.571467+0000","last_change":"2024-09-16T02:00:34.455121+0000","last_active":"2024-09-16T02:02:41.571467+0000","last_peered":"2024-09-16T02:02:41.571467+0000","last_clean":"2024-09-16T02:02:41.571467+0000","last_became_active":"2024-09-16T02:00:34.454665+0000","last_became_peered":"2024-09-16T02:00:34.454665+0000","last_unstale":"2024-09-16T02:02:41.571467+0000","last_undegraded":"2024-09-16T02:02:41.571467+0000","last_fullsized":"2024-09-16T02:02:41.571467+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:31:42.771230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.629195+0000","last_change":"2024-09-16T02:00:34.458777+0000","last_active":"2024-09-16T02:02:41.629195+0000","last_peered":"2024-09-16T02:02:41.629195+0000","last_clean":"2024-09-16T02:02:41.629195+0000","last_became_active":"2024-09-16T02:00:34.457954+0000","last_became_peered":"2024-09-16T02:00:34.457954+0000","last_unstale":"2024-09-16T02:02:41.629195+0000","last_undegraded":"2024-09-16T02:02:41.629195+0000","last_fullsized":"2024-09-16T02:02:41.629195+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:58:28.032293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.607136+0000","last_change":"2024-09-16T02:00:34.444712+0000","last_active":"2024-09-16T02:02:41.607136+0000","last_peered":"2024-09-16T02:02:41.607136+0000","last_clean":"2024-09-16T02:02:41.607136+0000","last_became_active":"2024-09-16T02:00:34.444421+0000","last_became_peered":"2024-09-16T02:00:34.444421+0000","last_unstale":"2024-09-16T02:02:41.607136+0000","last_undegraded":"2024-09-16T02:02:41.607136+0000","last_fullsized":"2024-09-16T02:02:41.607136+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:36.557558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.446560+0000","last_change":"2024-09-16T02:00:34.454570+0000","last_active":"2024-09-16T02:02:09.446560+0000","last_peered":"2024-09-16T02:02:09.446560+0000","last_clean":"2024-09-16T02:02:09.446560+0000","last_became_active":"2024-09-16T02:00:34.454246+0000","last_became_peered":"2024-09-16T02:00:34.454246+0000","last_unstale":"2024-09-16T02:02:09.446560+0000","last_undegraded":"2024-09-16T02:02:09.446560+0000","last_fullsized":"2024-09-16T02:02:09.446560+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:36:18.634907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.571164+0000","last_change":"2024-09-16T02:00:34.449680+0000","last_active":"2024-09-16T02:02:41.571164+0000","last_peered":"2024-09-16T02:02:41.571164+0000","last_clean":"2024-09-16T02:02:41.571164+0000","last_became_active":"2024-09-16T02:00:34.449184+0000","last_became_peered":"2024-09-16T02:00:34.449184+0000","last_unstale":"2024-09-16T02:02:41.571164+0000","last_undegraded":"2024-09-16T02:02:41.571164+0000","last_fullsized":"2024-09-16T02:02:41.571164+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:28:34.867820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.571034+0000","last_change":"2024-09-16T02:00:34.455362+0000","last_active":"2024-09-16T02:02:41.571034+0000","last_peered":"2024-09-16T02:02:41.571034+0000","last_clean":"2024-09-16T02:02:41.571034+0000","last_became_active":"2024-09-16T02:00:34.455018+0000","last_became_peered":"2024-09-16T02:00:34.455018+0000","last_unstale":"2024-09-16T02:02:41.571034+0000","last_undegraded":"2024-09-16T02:02:41.571034+0000","last_fullsized":"2024-09-16T02:02:41.571034+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:38.278418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.629086+0000","last_change":"2024-09-16T02:00:34.459426+0000","last_active":"2024-09-16T02:02:41.629086+0000","last_peered":"2024-09-16T02:02:41.629086+0000","last_clean":"2024-09-16T02:02:41.629086+0000","last_became_active":"2024-09-16T02:00:34.459082+0000","last_became_peered":"2024-09-16T02:00:34.459082+0000","last_unstale":"2024-09-16T02:02:41.629086+0000","last_undegraded":"2024-09-16T02:02:41.629086+0000","last_fullsized":"2024-09-16T02:02:41.629086+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:59:42.226065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.565886+0000","last_change":"2024-09-16T02:00:34.450936+0000","last_active":"2024-09-16T02:02:41.565886+0000","last_peered":"2024-09-16T02:02:41.565886+0000","last_clean":"2024-09-16T02:02:41.565886+0000","last_became_active":"2024-09-16T02:00:34.450503+0000","last_became_peered":"2024-09-16T02:00:34.450503+0000","last_unstale":"2024-09-16T02:02:41.565886+0000","last_undegraded":"2024-09-16T02:02:41.565886+0000","last_fullsized":"2024-09-16T02:02:41.565886+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:35:55.806193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":151,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:42.573075+0000","last_change":"2024-09-16T02:00:32.443193+0000","last_active":"2024-09-16T02:02:42.573075+0000","last_peered":"2024-09-16T02:02:42.573075+0000","last_clean":"2024-09-16T02:02:42.573075+0000","last_became_active":"2024-09-16T02:00:32.442863+0000","last_became_peered":"2024-09-16T02:00:32.442863+0000","last_unstale":"2024-09-16T02:02:42.573075+0000","last_undegraded":"2024-09-16T02:02:42.573075+0000","last_fullsized":"2024-09-16T02:02:42.573075+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:19.734946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1117,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.565455+0000","last_change":"2024-09-16T02:01:40.570518+0000","last_active":"2024-09-16T02:02:41.565455+0000","last_peered":"2024-09-16T02:02:41.565455+0000","last_clean":"2024-09-16T02:02:41.565455+0000","last_became_active":"2024-09-16T02:01:40.570124+0000","last_became_peered":"2024-09-16T02:01:40.570124+0000","last_unstale":"2024-09-16T02:02:41.565455+0000","last_undegraded":"2024-09-16T02:02:41.565455+0000","last_fullsized":"2024-09-16T02:02:41.565455+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:44:21.062742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.566661+0000","last_change":"2024-09-16T02:01:40.572282+0000","last_active":"2024-09-16T02:02:41.566661+0000","last_peered":"2024-09-16T02:02:41.566661+0000","last_clean":"2024-09-16T02:02:41.566661+0000","last_became_active":"2024-09-16T02:01:40.564577+0000","last_became_peered":"2024-09-16T02:01:40.564577+0000","last_unstale":"2024-09-16T02:02:41.566661+0000","last_undegraded":"2024-09-16T02:02:41.566661+0000","last_fullsized":"2024-09-16T02:02:41.566661+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:19:40.834173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.606793+0000","last_change":"2024-09-16T02:00:32.438530+0000","last_active":"2024-09-16T02:02:41.606793+0000","last_peered":"2024-09-16T02:02:41.606793+0000","last_clean":"2024-09-16T02:02:41.606793+0000","last_became_active":"2024-09-16T02:00:32.438238+0000","last_became_peered":"2024-09-16T02:00:32.438238+0000","last_unstale":"2024-09-16T02:02:41.606793+0000","last_undegraded":"2024-09-16T02:02:41.606793+0000","last_fullsized":"2024-09-16T02:02:41.606793+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:52:32.799701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.606323+0000","last_change":"2024-09-16T02:00:34.456299+0000","last_active":"2024-09-16T02:02:41.606323+0000","last_peered":"2024-09-16T02:02:41.606323+0000","last_clean":"2024-09-16T02:02:41.606323+0000","last_became_active":"2024-09-16T02:00:34.455925+0000","last_became_peered":"2024-09-16T02:00:34.455925+0000","last_unstale":"2024-09-16T02:02:41.606323+0000","last_undegraded":"2024-09-16T02:02:41.606323+0000","last_fullsized":"2024-09-16T02:02:41.606323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:33:27.162664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.566251+0000","last_change":"2024-09-16T02:00:32.442029+0000","last_active":"2024-09-16T02:02:41.566251+0000","last_peered":"2024-09-16T02:02:41.566251+0000","last_clean":"2024-09-16T02:02:41.566251+0000","last_became_active":"2024-09-16T02:00:32.441705+0000","last_became_peered":"2024-09-16T02:00:32.441705+0000","last_unstale":"2024-09-16T02:02:41.566251+0000","last_undegraded":"2024-09-16T02:02:41.566251+0000","last_fullsized":"2024-09-16T02:02:41.566251+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:50:54.725768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-16T02:02:40.558706+0000","last_change":"2024-09-16T02:00:34.457833+0000","last_active":"2024-09-16T02:02:40.558706+0000","last_peered":"2024-09-16T02:02:40.558706+0000","last_clean":"2024-09-16T02:02:40.558706+0000","last_became_active":"2024-09-16T02:00:34.457453+0000","last_became_peered":"2024-09-16T02:00:34.457453+0000","last_unstale":"2024-09-16T02:02:40.558706+0000","last_undegraded":"2024-09-16T02:02:40.558706+0000","last_fullsized":"2024-09-16T02:02:40.558706+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:21.777574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'6","reported_seq":276,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:42.572664+0000","last_change":"2024-09-16T02:02:41.576860+0000","last_active":"2024-09-16T02:02:42.572664+0000","last_peered":"2024-09-16T02:02:42.572664+0000","last_clean":"2024-09-16T02:02:42.572664+0000","last_became_active":"2024-09-16T02:02:41.570526+0000","last_became_peered":"2024-09-16T02:02:41.570526+0000","last_unstale":"2024-09-16T02:02:42.572664+0000","last_undegraded":"2024-09-16T02:02:42.572664+0000","last_fullsized":"2024-09-16T02:02:42.572664+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:19.875152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":230,"num_read_kb":230,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":180,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,9],"acting":[10,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.569687+0000","last_change":"2024-09-16T02:00:34.461746+0000","last_active":"2024-09-16T02:02:41.569687+0000","last_peered":"2024-09-16T02:02:41.569687+0000","last_clean":"2024-09-16T02:02:41.569687+0000","last_became_active":"2024-09-16T02:00:34.461441+0000","last_became_peered":"2024-09-16T02:00:34.461441+0000","last_unstale":"2024-09-16T02:02:41.569687+0000","last_undegraded":"2024-09-16T02:02:41.569687+0000","last_fullsized":"2024-09-16T02:02:41.569687+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:28:14.519159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564386+0000","last_change":"2024-09-16T02:00:32.443129+0000","last_active":"2024-09-16T02:01:40.564386+0000","last_peered":"2024-09-16T02:01:40.564386+0000","last_clean":"2024-09-16T02:01:40.564386+0000","last_became_active":"2024-09-16T02:00:32.442772+0000","last_became_peered":"2024-09-16T02:00:32.442772+0000","last_unstale":"2024-09-16T02:01:40.564386+0000","last_undegraded":"2024-09-16T02:01:40.564386+0000","last_fullsized":"2024-09-16T02:01:40.564386+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:10:03.964583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.445596+0000","last_change":"2024-09-16T02:00:34.457449+0000","last_active":"2024-09-16T02:02:09.445596+0000","last_peered":"2024-09-16T02:02:09.445596+0000","last_clean":"2024-09-16T02:02:09.445596+0000","last_became_active":"2024-09-16T02:00:34.457110+0000","last_became_peered":"2024-09-16T02:00:34.457110+0000","last_unstale":"2024-09-16T02:02:09.445596+0000","last_undegraded":"2024-09-16T02:02:09.445596+0000","last_fullsized":"2024-09-16T02:02:09.445596+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:35.662503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.565645+0000","last_change":"2024-09-16T02:00:32.587419+0000","last_active":"2024-09-16T02:02:41.565645+0000","last_peered":"2024-09-16T02:02:41.565645+0000","last_clean":"2024-09-16T02:02:41.565645+0000","last_became_active":"2024-09-16T02:00:32.587037+0000","last_became_peered":"2024-09-16T02:00:32.587037+0000","last_unstale":"2024-09-16T02:02:41.565645+0000","last_undegraded":"2024-09-16T02:02:41.565645+0000","last_fullsized":"2024-09-16T02:02:41.565645+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:11.835505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.569632+0000","last_change":"2024-09-16T02:00:34.456371+0000","last_active":"2024-09-16T02:02:41.569632+0000","last_peered":"2024-09-16T02:02:41.569632+0000","last_clean":"2024-09-16T02:02:41.569632+0000","last_became_active":"2024-09-16T02:00:34.456013+0000","last_became_peered":"2024-09-16T02:00:34.456013+0000","last_unstale":"2024-09-16T02:02:41.569632+0000","last_undegraded":"2024-09-16T02:02:41.569632+0000","last_fullsized":"2024-09-16T02:02:41.569632+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:30:15.847189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'33","reported_seq":299,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.694029+0000","last_change":"2024-09-16T02:00:32.444922+0000","last_active":"2024-09-16T02:02:41.694029+0000","last_peered":"2024-09-16T02:02:41.694029+0000","last_clean":"2024-09-16T02:02:41.694029+0000","last_became_active":"2024-09-16T02:00:32.444540+0000","last_became_peered":"2024-09-16T02:00:32.444540+0000","last_unstale":"2024-09-16T02:02:41.694029+0000","last_undegraded":"2024-09-16T02:02:41.694029+0000","last_fullsized":"2024-09-16T02:02:41.694029+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:59.183952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59955,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":240,"num_read_kb":178,"num_write":35,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.566234+0000","last_change":"2024-09-16T02:00:34.453366+0000","last_active":"2024-09-16T02:02:41.566234+0000","last_peered":"2024-09-16T02:02:41.566234+0000","last_clean":"2024-09-16T02:02:41.566234+0000","last_became_active":"2024-09-16T02:00:34.453048+0000","last_became_peered":"2024-09-16T02:00:34.453048+0000","last_unstale":"2024-09-16T02:02:41.566234+0000","last_undegraded":"2024-09-16T02:02:41.566234+0000","last_fullsized":"2024-09-16T02:02:41.566234+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:15:52.721893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.571068+0000","last_change":"2024-09-16T02:00:32.445516+0000","last_active":"2024-09-16T02:02:41.571068+0000","last_peered":"2024-09-16T02:02:41.571068+0000","last_clean":"2024-09-16T02:02:41.571068+0000","last_became_active":"2024-09-16T02:00:32.445092+0000","last_became_peered":"2024-09-16T02:00:32.445092+0000","last_unstale":"2024-09-16T02:02:41.571068+0000","last_undegraded":"2024-09-16T02:02:41.571068+0000","last_fullsized":"2024-09-16T02:02:41.571068+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:54.274692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.606896+0000","last_change":"2024-09-16T02:00:34.448450+0000","last_active":"2024-09-16T02:02:41.606896+0000","last_peered":"2024-09-16T02:02:41.606896+0000","last_clean":"2024-09-16T02:02:41.606896+0000","last_became_active":"2024-09-16T02:00:34.448116+0000","last_became_peered":"2024-09-16T02:00:34.448116+0000","last_unstale":"2024-09-16T02:02:41.606896+0000","last_undegraded":"2024-09-16T02:02:41.606896+0000","last_fullsized":"2024-09-16T02:02:41.606896+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:12:34.834798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.565670+0000","last_change":"2024-09-16T02:00:32.445427+0000","last_active":"2024-09-16T02:02:41.565670+0000","last_peered":"2024-09-16T02:02:41.565670+0000","last_clean":"2024-09-16T02:02:41.565670+0000","last_became_active":"2024-09-16T02:00:32.445058+0000","last_became_peered":"2024-09-16T02:00:32.445058+0000","last_unstale":"2024-09-16T02:02:41.565670+0000","last_undegraded":"2024-09-16T02:02:41.565670+0000","last_fullsized":"2024-09-16T02:02:41.565670+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:35:34.701331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.452607+0000","last_change":"2024-09-16T02:00:34.458479+0000","last_active":"2024-09-16T02:02:09.452607+0000","last_peered":"2024-09-16T02:02:09.452607+0000","last_clean":"2024-09-16T02:02:09.452607+0000","last_became_active":"2024-09-16T02:00:34.458135+0000","last_became_peered":"2024-09-16T02:00:34.458135+0000","last_unstale":"2024-09-16T02:02:09.452607+0000","last_undegraded":"2024-09-16T02:02:09.452607+0000","last_fullsized":"2024-09-16T02:02:09.452607+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:37:40.188861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'8","reported_seq":156,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:42.706504+0000","last_change":"2024-09-16T02:00:32.440319+0000","last_active":"2024-09-16T02:02:42.706504+0000","last_peered":"2024-09-16T02:02:42.706504+0000","last_clean":"2024-09-16T02:02:42.706504+0000","last_became_active":"2024-09-16T02:00:32.440005+0000","last_became_peered":"2024-09-16T02:00:32.440005+0000","last_unstale":"2024-09-16T02:02:42.706504+0000","last_undegraded":"2024-09-16T02:02:42.706504+0000","last_fullsized":"2024-09-16T02:02:42.706504+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:07:33.876577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1617,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.439440+0000","last_change":"2024-09-16T02:00:34.460468+0000","last_active":"2024-09-16T02:02:09.439440+0000","last_peered":"2024-09-16T02:02:09.439440+0000","last_clean":"2024-09-16T02:02:09.439440+0000","last_became_active":"2024-09-16T02:00:34.459190+0000","last_became_peered":"2024-09-16T02:00:34.459190+0000","last_unstale":"2024-09-16T02:02:09.439440+0000","last_undegraded":"2024-09-16T02:02:09.439440+0000","last_fullsized":"2024-09-16T02:02:09.439440+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:46:43.654991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.565709+0000","last_change":"2024-09-16T02:00:32.587700+0000","last_active":"2024-09-16T02:02:41.565709+0000","last_peered":"2024-09-16T02:02:41.565709+0000","last_clean":"2024-09-16T02:02:41.565709+0000","last_became_active":"2024-09-16T02:00:32.586964+0000","last_became_peered":"2024-09-16T02:00:32.586964+0000","last_unstale":"2024-09-16T02:02:41.565709+0000","last_undegraded":"2024-09-16T02:02:41.565709+0000","last_fullsized":"2024-09-16T02:02:41.565709+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:41.309918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.565323+0000","last_change":"2024-09-16T02:00:34.455652+0000","last_active":"2024-09-16T02:02:41.565323+0000","last_peered":"2024-09-16T02:02:41.565323+0000","last_clean":"2024-09-16T02:02:41.565323+0000","last_became_active":"2024-09-16T02:00:34.455306+0000","last_became_peered":"2024-09-16T02:00:34.455306+0000","last_unstale":"2024-09-16T02:02:41.565323+0000","last_undegraded":"2024-09-16T02:02:41.565323+0000","last_fullsized":"2024-09-16T02:02:41.565323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:44:09.505096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'3","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.566304+0000","last_change":"2024-09-16T02:00:32.441443+0000","last_active":"2024-09-16T02:02:41.566304+0000","last_peered":"2024-09-16T02:02:41.566304+0000","last_clean":"2024-09-16T02:02:41.566304+0000","last_became_active":"2024-09-16T02:00:32.441073+0000","last_became_peered":"2024-09-16T02:00:32.441073+0000","last_unstale":"2024-09-16T02:02:41.566304+0000","last_undegraded":"2024-09-16T02:02:41.566304+0000","last_fullsized":"2024-09-16T02:02:41.566304+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:16.865970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.629121+0000","last_change":"2024-09-16T02:01:40.611317+0000","last_active":"2024-09-16T02:02:41.629121+0000","last_peered":"2024-09-16T02:02:41.629121+0000","last_clean":"2024-09-16T02:02:41.629121+0000","last_became_active":"2024-09-16T02:01:40.610837+0000","last_became_peered":"2024-09-16T02:01:40.610837+0000","last_unstale":"2024-09-16T02:02:41.629121+0000","last_undegraded":"2024-09-16T02:02:41.629121+0000","last_fullsized":"2024-09-16T02:02:41.629121+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:19:55.376738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.607108+0000","last_change":"2024-09-16T02:01:40.617877+0000","last_active":"2024-09-16T02:02:41.607108+0000","last_peered":"2024-09-16T02:02:41.607108+0000","last_clean":"2024-09-16T02:02:41.607108+0000","last_became_active":"2024-09-16T02:01:40.611597+0000","last_became_peered":"2024-09-16T02:01:40.611597+0000","last_unstale":"2024-09-16T02:02:41.607108+0000","last_undegraded":"2024-09-16T02:02:41.607108+0000","last_fullsized":"2024-09-16T02:02:41.607108+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:53:35.796243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.449557+0000","last_change":"2024-09-16T02:00:34.461889+0000","last_active":"2024-09-16T02:02:09.449557+0000","last_peered":"2024-09-16T02:02:09.449557+0000","last_clean":"2024-09-16T02:02:09.449557+0000","last_became_active":"2024-09-16T02:00:34.461583+0000","last_became_peered":"2024-09-16T02:00:34.461583+0000","last_unstale":"2024-09-16T02:02:09.449557+0000","last_undegraded":"2024-09-16T02:02:09.449557+0000","last_fullsized":"2024-09-16T02:02:09.449557+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:32.598422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.565762+0000","last_change":"2024-09-16T02:00:32.445201+0000","last_active":"2024-09-16T02:02:41.565762+0000","last_peered":"2024-09-16T02:02:41.565762+0000","last_clean":"2024-09-16T02:02:41.565762+0000","last_became_active":"2024-09-16T02:00:32.444722+0000","last_became_peered":"2024-09-16T02:00:32.444722+0000","last_unstale":"2024-09-16T02:02:41.565762+0000","last_undegraded":"2024-09-16T02:02:41.565762+0000","last_fullsized":"2024-09-16T02:02:41.565762+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:41:11.657028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.445512+0000","last_change":"2024-09-16T02:01:40.613676+0000","last_active":"2024-09-16T02:02:09.445512+0000","last_peered":"2024-09-16T02:02:09.445512+0000","last_clean":"2024-09-16T02:02:09.445512+0000","last_became_active":"2024-09-16T02:01:40.608260+0000","last_became_peered":"2024-09-16T02:01:40.608260+0000","last_unstale":"2024-09-16T02:02:09.445512+0000","last_undegraded":"2024-09-16T02:02:09.445512+0000","last_fullsized":"2024-09-16T02:02:09.445512+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:30:59.004103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'6","reported_seq":151,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:42.573509+0000","last_change":"2024-09-16T02:00:32.860059+0000","last_active":"2024-09-16T02:02:42.573509+0000","last_peered":"2024-09-16T02:02:42.573509+0000","last_clean":"2024-09-16T02:02:42.573509+0000","last_became_active":"2024-09-16T02:00:32.859617+0000","last_became_peered":"2024-09-16T02:00:32.859617+0000","last_unstale":"2024-09-16T02:02:42.573509+0000","last_undegraded":"2024-09-16T02:02:42.573509+0000","last_fullsized":"2024-09-16T02:02:42.573509+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:41:50.248194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1665,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.457956+0000","last_change":"2024-09-16T02:00:34.454344+0000","last_active":"2024-09-16T02:02:09.457956+0000","last_peered":"2024-09-16T02:02:09.457956+0000","last_clean":"2024-09-16T02:02:09.457956+0000","last_became_active":"2024-09-16T02:00:34.454027+0000","last_became_peered":"2024-09-16T02:00:34.454027+0000","last_unstale":"2024-09-16T02:02:09.457956+0000","last_undegraded":"2024-09-16T02:02:09.457956+0000","last_fullsized":"2024-09-16T02:02:09.457956+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:33.020365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":335,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564402+0000","last_change":"2024-09-16T01:59:19.798159+0000","last_active":"2024-09-16T02:01:40.564402+0000","last_peered":"2024-09-16T02:01:40.564402+0000","last_clean":"2024-09-16T02:01:40.564402+0000","last_became_active":"2024-09-16T01:59:19.797688+0000","last_became_peered":"2024-09-16T01:59:19.797688+0000","last_unstale":"2024-09-16T02:01:40.564402+0000","last_undegraded":"2024-09-16T02:01:40.564402+0000","last_fullsized":"2024-09-16T02:01:40.564402+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:55:54.312750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:55:54.312750+0000","last_clean_scrub_stamp":"2024-09-16T01:55:54.312750+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:00:36.860158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.564285+0000","last_change":"2024-09-16T02:00:32.441925+0000","last_active":"2024-09-16T02:02:41.564285+0000","last_peered":"2024-09-16T02:02:41.564285+0000","last_clean":"2024-09-16T02:02:41.564285+0000","last_became_active":"2024-09-16T02:00:32.441565+0000","last_became_peered":"2024-09-16T02:00:32.441565+0000","last_unstale":"2024-09-16T02:02:41.564285+0000","last_undegraded":"2024-09-16T02:02:41.564285+0000","last_fullsized":"2024-09-16T02:02:41.564285+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:23:09.755034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.606600+0000","last_change":"2024-09-16T02:00:34.452986+0000","last_active":"2024-09-16T02:02:41.606600+0000","last_peered":"2024-09-16T02:02:41.606600+0000","last_clean":"2024-09-16T02:02:41.606600+0000","last_became_active":"2024-09-16T02:00:34.452688+0000","last_became_peered":"2024-09-16T02:00:34.452688+0000","last_unstale":"2024-09-16T02:02:41.606600+0000","last_undegraded":"2024-09-16T02:02:41.606600+0000","last_fullsized":"2024-09-16T02:02:41.606600+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:12.958209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:01:40.564282+0000","last_change":"2024-09-16T02:00:32.443964+0000","last_active":"2024-09-16T02:01:40.564282+0000","last_peered":"2024-09-16T02:01:40.564282+0000","last_clean":"2024-09-16T02:01:40.564282+0000","last_became_active":"2024-09-16T02:00:32.443623+0000","last_became_peered":"2024-09-16T02:00:32.443623+0000","last_unstale":"2024-09-16T02:01:40.564282+0000","last_undegraded":"2024-09-16T02:01:40.564282+0000","last_fullsized":"2024-09-16T02:01:40.564282+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:30:34.406024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.606679+0000","last_change":"2024-09-16T02:00:34.449615+0000","last_active":"2024-09-16T02:02:41.606679+0000","last_peered":"2024-09-16T02:02:41.606679+0000","last_clean":"2024-09-16T02:02:41.606679+0000","last_became_active":"2024-09-16T02:00:34.449109+0000","last_became_peered":"2024-09-16T02:00:34.449109+0000","last_unstale":"2024-09-16T02:02:41.606679+0000","last_undegraded":"2024-09-16T02:02:41.606679+0000","last_fullsized":"2024-09-16T02:02:41.606679+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:45.036177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'7","reported_seq":152,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:42.705716+0000","last_change":"2024-09-16T02:00:32.587107+0000","last_active":"2024-09-16T02:02:42.705716+0000","last_peered":"2024-09-16T02:02:42.705716+0000","last_clean":"2024-09-16T02:02:42.705716+0000","last_became_active":"2024-09-16T02:00:32.586777+0000","last_became_peered":"2024-09-16T02:00:32.586777+0000","last_unstale":"2024-09-16T02:02:42.705716+0000","last_undegraded":"2024-09-16T02:02:42.705716+0000","last_fullsized":"2024-09-16T02:02:42.705716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:03.968987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.565315+0000","last_change":"2024-09-16T02:01:40.570572+0000","last_active":"2024-09-16T02:02:41.565315+0000","last_peered":"2024-09-16T02:02:41.565315+0000","last_clean":"2024-09-16T02:02:41.565315+0000","last_became_active":"2024-09-16T02:01:40.570176+0000","last_became_peered":"2024-09-16T02:01:40.570176+0000","last_unstale":"2024-09-16T02:02:41.565315+0000","last_undegraded":"2024-09-16T02:02:41.565315+0000","last_fullsized":"2024-09-16T02:02:41.565315+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:24:13.138925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'6","reported_seq":144,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:39.691569+0000","last_change":"2024-09-16T02:00:32.443201+0000","last_active":"2024-09-16T02:02:39.691569+0000","last_peered":"2024-09-16T02:02:39.691569+0000","last_clean":"2024-09-16T02:02:39.691569+0000","last_became_active":"2024-09-16T02:00:32.442778+0000","last_became_peered":"2024-09-16T02:00:32.442778+0000","last_unstale":"2024-09-16T02:02:39.691569+0000","last_undegraded":"2024-09-16T02:02:39.691569+0000","last_fullsized":"2024-09-16T02:02:39.691569+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:43.933817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1083,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.566693+0000","last_change":"2024-09-16T02:00:34.463000+0000","last_active":"2024-09-16T02:02:41.566693+0000","last_peered":"2024-09-16T02:02:41.566693+0000","last_clean":"2024-09-16T02:02:41.566693+0000","last_became_active":"2024-09-16T02:00:34.461890+0000","last_became_peered":"2024-09-16T02:00:34.461890+0000","last_unstale":"2024-09-16T02:02:41.566693+0000","last_undegraded":"2024-09-16T02:02:41.566693+0000","last_fullsized":"2024-09-16T02:02:41.566693+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:21:15.161295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-16T02:02:40.558984+0000","last_change":"2024-09-16T02:00:32.586933+0000","last_active":"2024-09-16T02:02:40.558984+0000","last_peered":"2024-09-16T02:02:40.558984+0000","last_clean":"2024-09-16T02:02:40.558984+0000","last_became_active":"2024-09-16T02:00:32.586565+0000","last_became_peered":"2024-09-16T02:00:32.586565+0000","last_unstale":"2024-09-16T02:02:40.558984+0000","last_undegraded":"2024-09-16T02:02:40.558984+0000","last_fullsized":"2024-09-16T02:02:40.558984+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:45:07.130649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.567361+0000","last_change":"2024-09-16T02:00:34.460805+0000","last_active":"2024-09-16T02:02:41.567361+0000","last_peered":"2024-09-16T02:02:41.567361+0000","last_clean":"2024-09-16T02:02:41.567361+0000","last_became_active":"2024-09-16T02:00:34.460482+0000","last_became_peered":"2024-09-16T02:00:34.460482+0000","last_unstale":"2024-09-16T02:02:41.567361+0000","last_undegraded":"2024-09-16T02:02:41.567361+0000","last_fullsized":"2024-09-16T02:02:41.567361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:07:51.565053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.566451+0000","last_change":"2024-09-16T02:00:32.587516+0000","last_active":"2024-09-16T02:02:41.566451+0000","last_peered":"2024-09-16T02:02:41.566451+0000","last_clean":"2024-09-16T02:02:41.566451+0000","last_became_active":"2024-09-16T02:00:32.587172+0000","last_became_peered":"2024-09-16T02:00:32.587172+0000","last_unstale":"2024-09-16T02:02:41.566451+0000","last_undegraded":"2024-09-16T02:02:41.566451+0000","last_fullsized":"2024-09-16T02:02:41.566451+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:52:29.018290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.607223+0000","last_change":"2024-09-16T02:00:34.446021+0000","last_active":"2024-09-16T02:02:41.607223+0000","last_peered":"2024-09-16T02:02:41.607223+0000","last_clean":"2024-09-16T02:02:41.607223+0000","last_became_active":"2024-09-16T02:00:34.445732+0000","last_became_peered":"2024-09-16T02:00:34.445732+0000","last_unstale":"2024-09-16T02:02:41.607223+0000","last_undegraded":"2024-09-16T02:02:41.607223+0000","last_fullsized":"2024-09-16T02:02:41.607223+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:51:17.960792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'16","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-16T02:02:40.559157+0000","last_change":"2024-09-16T02:00:32.439750+0000","last_active":"2024-09-16T02:02:40.559157+0000","last_peered":"2024-09-16T02:02:40.559157+0000","last_clean":"2024-09-16T02:02:40.559157+0000","last_became_active":"2024-09-16T02:00:32.439438+0000","last_became_peered":"2024-09-16T02:00:32.439438+0000","last_unstale":"2024-09-16T02:02:40.559157+0000","last_undegraded":"2024-09-16T02:02:40.559157+0000","last_fullsized":"2024-09-16T02:02:40.559157+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:50.764733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.606872+0000","last_change":"2024-09-16T02:00:34.453810+0000","last_active":"2024-09-16T02:02:41.606872+0000","last_peered":"2024-09-16T02:02:41.606872+0000","last_clean":"2024-09-16T02:02:41.606872+0000","last_became_active":"2024-09-16T02:00:34.453421+0000","last_became_peered":"2024-09-16T02:00:34.453421+0000","last_unstale":"2024-09-16T02:02:41.606872+0000","last_undegraded":"2024-09-16T02:02:41.606872+0000","last_fullsized":"2024-09-16T02:02:41.606872+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:23:50.405749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.564776+0000","last_change":"2024-09-16T02:00:32.860565+0000","last_active":"2024-09-16T02:02:41.564776+0000","last_peered":"2024-09-16T02:02:41.564776+0000","last_clean":"2024-09-16T02:02:41.564776+0000","last_became_active":"2024-09-16T02:00:32.860130+0000","last_became_peered":"2024-09-16T02:00:32.860130+0000","last_unstale":"2024-09-16T02:02:41.564776+0000","last_undegraded":"2024-09-16T02:02:41.564776+0000","last_fullsized":"2024-09-16T02:02:41.564776+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:09:55.759883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.442038+0000","last_change":"2024-09-16T02:01:40.564454+0000","last_active":"2024-09-16T02:02:09.442038+0000","last_peered":"2024-09-16T02:02:09.442038+0000","last_clean":"2024-09-16T02:02:09.442038+0000","last_became_active":"2024-09-16T02:01:40.563854+0000","last_became_peered":"2024-09-16T02:01:40.563854+0000","last_unstale":"2024-09-16T02:02:09.442038+0000","last_undegraded":"2024-09-16T02:02:09.442038+0000","last_fullsized":"2024-09-16T02:02:09.442038+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:56:30.610947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.632971+0000","last_change":"2024-09-16T02:02:41.632561+0000","last_active":"2024-09-16T02:02:41.632971+0000","last_peered":"2024-09-16T02:02:41.632971+0000","last_clean":"2024-09-16T02:02:41.632971+0000","last_became_active":"2024-09-16T02:02:41.630051+0000","last_became_peered":"2024-09-16T02:02:41.630051+0000","last_unstale":"2024-09-16T02:02:41.632971+0000","last_undegraded":"2024-09-16T02:02:41.632971+0000","last_fullsized":"2024-09-16T02:02:41.632971+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:49:57.569311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.565707+0000","last_change":"2024-09-16T02:00:34.457673+0000","last_active":"2024-09-16T02:02:41.565707+0000","last_peered":"2024-09-16T02:02:41.565707+0000","last_clean":"2024-09-16T02:02:41.565707+0000","last_became_active":"2024-09-16T02:00:34.457331+0000","last_became_peered":"2024-09-16T02:00:34.457331+0000","last_unstale":"2024-09-16T02:02:41.565707+0000","last_undegraded":"2024-09-16T02:02:41.565707+0000","last_fullsized":"2024-09-16T02:02:41.565707+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:12:16.100036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.606509+0000","last_change":"2024-09-16T02:00:32.437528+0000","last_active":"2024-09-16T02:02:41.606509+0000","last_peered":"2024-09-16T02:02:41.606509+0000","last_clean":"2024-09-16T02:02:41.606509+0000","last_became_active":"2024-09-16T02:00:32.437080+0000","last_became_peered":"2024-09-16T02:00:32.437080+0000","last_unstale":"2024-09-16T02:02:41.606509+0000","last_undegraded":"2024-09-16T02:02:41.606509+0000","last_fullsized":"2024-09-16T02:02:41.606509+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:59:33.908026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.629149+0000","last_change":"2024-09-16T02:00:34.453504+0000","last_active":"2024-09-16T02:02:41.629149+0000","last_peered":"2024-09-16T02:02:41.629149+0000","last_clean":"2024-09-16T02:02:41.629149+0000","last_became_active":"2024-09-16T02:00:34.453212+0000","last_became_peered":"2024-09-16T02:00:34.453212+0000","last_unstale":"2024-09-16T02:02:41.629149+0000","last_undegraded":"2024-09-16T02:02:41.629149+0000","last_fullsized":"2024-09-16T02:02:41.629149+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:00:56.414548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'4","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.571093+0000","last_change":"2024-09-16T02:00:32.442282+0000","last_active":"2024-09-16T02:02:41.571093+0000","last_peered":"2024-09-16T02:02:41.571093+0000","last_clean":"2024-09-16T02:02:41.571093+0000","last_became_active":"2024-09-16T02:00:32.441910+0000","last_became_peered":"2024-09-16T02:00:32.441910+0000","last_unstale":"2024-09-16T02:02:41.571093+0000","last_undegraded":"2024-09-16T02:02:41.571093+0000","last_fullsized":"2024-09-16T02:02:41.571093+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:01:46.045021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.606559+0000","last_change":"2024-09-16T02:00:34.460177+0000","last_active":"2024-09-16T02:02:41.606559+0000","last_peered":"2024-09-16T02:02:41.606559+0000","last_clean":"2024-09-16T02:02:41.606559+0000","last_became_active":"2024-09-16T02:00:34.459874+0000","last_became_peered":"2024-09-16T02:00:34.459874+0000","last_unstale":"2024-09-16T02:02:41.606559+0000","last_undegraded":"2024-09-16T02:02:41.606559+0000","last_fullsized":"2024-09-16T02:02:41.606559+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:35.855591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":55,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.611163+0000","last_change":"2024-09-16T02:02:41.611163+0000","last_active":"2024-09-16T02:02:41.611163+0000","last_peered":"2024-09-16T02:02:41.611163+0000","last_clean":"2024-09-16T02:02:41.611163+0000","last_became_active":"2024-09-16T02:02:41.608020+0000","last_became_peered":"2024-09-16T02:02:41.608020+0000","last_unstale":"2024-09-16T02:02:41.611163+0000","last_undegraded":"2024-09-16T02:02:41.611163+0000","last_fullsized":"2024-09-16T02:02:41.611163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:43:57.184447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,6],"acting":[8,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.457243+0000","last_change":"2024-09-16T02:00:34.452964+0000","last_active":"2024-09-16T02:02:09.457243+0000","last_peered":"2024-09-16T02:02:09.457243+0000","last_clean":"2024-09-16T02:02:09.457243+0000","last_became_active":"2024-09-16T02:00:34.452617+0000","last_became_peered":"2024-09-16T02:00:34.452617+0000","last_unstale":"2024-09-16T02:02:09.457243+0000","last_undegraded":"2024-09-16T02:02:09.457243+0000","last_fullsized":"2024-09-16T02:02:09.457243+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:04.617970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.565533+0000","last_change":"2024-09-16T02:00:34.456585+0000","last_active":"2024-09-16T02:02:41.565533+0000","last_peered":"2024-09-16T02:02:41.565533+0000","last_clean":"2024-09-16T02:02:41.565533+0000","last_became_active":"2024-09-16T02:00:34.456199+0000","last_became_peered":"2024-09-16T02:00:34.456199+0000","last_unstale":"2024-09-16T02:02:41.565533+0000","last_undegraded":"2024-09-16T02:02:41.565533+0000","last_fullsized":"2024-09-16T02:02:41.565533+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:37:54.376785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.628991+0000","last_change":"2024-09-16T02:01:40.578904+0000","last_active":"2024-09-16T02:02:41.628991+0000","last_peered":"2024-09-16T02:02:41.628991+0000","last_clean":"2024-09-16T02:02:41.628991+0000","last_became_active":"2024-09-16T02:01:40.567221+0000","last_became_peered":"2024-09-16T02:01:40.567221+0000","last_unstale":"2024-09-16T02:02:41.628991+0000","last_undegraded":"2024-09-16T02:02:41.628991+0000","last_fullsized":"2024-09-16T02:02:41.628991+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:01.407145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.569386+0000","last_change":"2024-09-16T02:00:34.460925+0000","last_active":"2024-09-16T02:02:41.569386+0000","last_peered":"2024-09-16T02:02:41.569386+0000","last_clean":"2024-09-16T02:02:41.569386+0000","last_became_active":"2024-09-16T02:00:34.460581+0000","last_became_peered":"2024-09-16T02:00:34.460581+0000","last_unstale":"2024-09-16T02:02:41.569386+0000","last_undegraded":"2024-09-16T02:02:41.569386+0000","last_fullsized":"2024-09-16T02:02:41.569386+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:19:44.460695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.566351+0000","last_change":"2024-09-16T02:00:32.443877+0000","last_active":"2024-09-16T02:02:41.566351+0000","last_peered":"2024-09-16T02:02:41.566351+0000","last_clean":"2024-09-16T02:02:41.566351+0000","last_became_active":"2024-09-16T02:00:32.443416+0000","last_became_peered":"2024-09-16T02:00:32.443416+0000","last_unstale":"2024-09-16T02:02:41.566351+0000","last_undegraded":"2024-09-16T02:02:41.566351+0000","last_fullsized":"2024-09-16T02:02:41.566351+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:36:57.367232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:02:09.447996+0000","last_change":"2024-09-16T02:00:34.453048+0000","last_active":"2024-09-16T02:02:09.447996+0000","last_peered":"2024-09-16T02:02:09.447996+0000","last_clean":"2024-09-16T02:02:09.447996+0000","last_became_active":"2024-09-16T02:00:34.452752+0000","last_became_peered":"2024-09-16T02:00:34.452752+0000","last_unstale":"2024-09-16T02:02:09.447996+0000","last_undegraded":"2024-09-16T02:02:09.447996+0000","last_fullsized":"2024-09-16T02:02:09.447996+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:55.970195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.606728+0000","last_change":"2024-09-16T02:00:32.586974+0000","last_active":"2024-09-16T02:02:41.606728+0000","last_peered":"2024-09-16T02:02:41.606728+0000","last_clean":"2024-09-16T02:02:41.606728+0000","last_became_active":"2024-09-16T02:00:32.586644+0000","last_became_peered":"2024-09-16T02:00:32.586644+0000","last_unstale":"2024-09-16T02:02:41.606728+0000","last_undegraded":"2024-09-16T02:02:41.606728+0000","last_fullsized":"2024-09-16T02:02:41.606728+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:52.913287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T02:02:41.606607+0000","last_change":"2024-09-16T02:00:34.447548+0000","last_active":"2024-09-16T02:02:41.606607+0000","last_peered":"2024-09-16T02:02:41.606607+0000","last_clean":"2024-09-16T02:02:41.606607+0000","last_became_active":"2024-09-16T02:00:34.447207+0000","last_became_peered":"2024-09-16T02:00:34.447207+0000","last_unstale":"2024-09-16T02:02:41.606607+0000","last_undegraded":"2024-09-16T02:02:41.606607+0000","last_fullsized":"2024-09-16T02:02:41.606607+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:33.428129+0000","last_clean_scrub_stamp":"2024-09-16T02:00:33.428129+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:22:02.494343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'14","reported_seq":154,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-16T02:02:40.692237+0000","last_change":"2024-09-16T02:00:32.441129+0000","last_active":"2024-09-16T02:02:40.692237+0000","last_peered":"2024-09-16T02:02:40.692237+0000","last_clean":"2024-09-16T02:02:40.692237+0000","last_became_active":"2024-09-16T02:00:32.440843+0000","last_became_peered":"2024-09-16T02:00:32.440843+0000","last_unstale":"2024-09-16T02:02:40.692237+0000","last_undegraded":"2024-09-16T02:02:40.692237+0000","last_fullsized":"2024-09-16T02:02:40.692237+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:00:31.414503+0000","last_clean_scrub_stamp":"2024-09-16T02:00:31.414503+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:17.721740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68891,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1198,"num_read_kb":674,"num_write":261,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":762,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":206673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":224,"ondisk_log_size":224,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281576,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612648,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":917504,"data_stored":450434,"data_compressed":11572,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841581,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1069056,"data_stored":1101370,"data_compressed":47384,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005105,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":929792,"data_stored":440845,"data_compressed":8664,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234039,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":929792,"data_stored":449083,"data_compressed":11572,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462971,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":1026622,"data_compressed":41575,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626498,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":921600,"data_stored":449499,"data_compressed":11572,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855430,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":991232,"data_stored":1041785,"data_compressed":47384,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018955,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":921600,"data_stored":449894,"data_compressed":11572,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247889,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":933888,"data_stored":448712,"data_compressed":11572,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444118,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1003520,"data_stored":511082,"data_compressed":11572,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":991232,"data_stored":510547,"data_compressed":11572,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":433649,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35812,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":62382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":3135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T02:02:45.997 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:02:46.275 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:02:46.671 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:46 smithi105 ceph-mon[31197]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:02:46.672 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:46 smithi105 ceph-mon[31197]: pgmap v346: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.6 KiB/s wr, 13 op/s; 22 B/s, 0 objects/s recovering 2024-09-16T02:02:46.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.751+0000 7fcf946b2640 1 -- 172.21.15.105:0/1493565840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf8c102ef0 msgr2=0x7fcf8c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:46.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.751+0000 7fcf946b2640 1 --2- 172.21.15.105:0/1493565840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf8c102ef0 0x7fcf8c10da40 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fcf84009ca0 tx=0x7fcf8402a950 comp rx=0 tx=0).stop 2024-09-16T02:02:46.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.751+0000 7fcf946b2640 1 -- 172.21.15.105:0/1493565840 shutdown_connections 2024-09-16T02:02:46.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.751+0000 7fcf946b2640 1 --2- 172.21.15.105:0/1493565840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf8c102570 0x7fcf8c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:46.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.751+0000 7fcf946b2640 1 --2- 172.21.15.105:0/1493565840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf8c102ef0 0x7fcf8c10da40 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:46.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.751+0000 7fcf946b2640 1 --2- 172.21.15.105:0/1493565840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf8c100e20 0x7fcf8c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:46.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.751+0000 7fcf946b2640 1 -- 172.21.15.105:0/1493565840 >> 172.21.15.105:0/1493565840 conn(0x7fcf8c0fc7d0 msgr2=0x7fcf8c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:46.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.751+0000 7fcf946b2640 1 -- 172.21.15.105:0/1493565840 shutdown_connections 2024-09-16T02:02:46.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.751+0000 7fcf946b2640 1 -- 172.21.15.105:0/1493565840 wait complete. 2024-09-16T02:02:46.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.751+0000 7fcf946b2640 1 Processor -- start 2024-09-16T02:02:46.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.751+0000 7fcf946b2640 1 -- start start 2024-09-16T02:02:46.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.752+0000 7fcf946b2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf8c100e20 0x7fcf8c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:46.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.752+0000 7fcf946b2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf8c102570 0x7fcf8c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:46.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.752+0000 7fcf946b2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf8c102ef0 0x7fcf8c19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:46.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.752+0000 7fcf946b2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8c111fa0 con 0x7fcf8c102570 2024-09-16T02:02:46.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.752+0000 7fcf946b2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8c111e20 con 0x7fcf8c102ef0 2024-09-16T02:02:46.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.752+0000 7fcf946b2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8c112120 con 0x7fcf8c100e20 2024-09-16T02:02:46.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.752+0000 7fcf91c4d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf8c102570 0x7fcf8c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:46.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.752+0000 7fcf91c4d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf8c102570 0x7fcf8c198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60646/0 (socket says 172.21.15.105:60646) 2024-09-16T02:02:46.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.752+0000 7fcf91c4d640 1 -- 172.21.15.105:0/2521620266 learned_addr learned my addr 172.21.15.105:0/2521620266 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:46.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.752+0000 7fcf9244e640 1 --2- 172.21.15.105:0/2521620266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf8c100e20 0x7fcf8c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:46.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.752+0000 7fcf92c4f640 1 --2- 172.21.15.105:0/2521620266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf8c102ef0 0x7fcf8c19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:46.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.752+0000 7fcf91c4d640 1 -- 172.21.15.105:0/2521620266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf8c100e20 msgr2=0x7fcf8c198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:46.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.752+0000 7fcf91c4d640 1 --2- 172.21.15.105:0/2521620266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf8c100e20 0x7fcf8c198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:46.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.752+0000 7fcf91c4d640 1 -- 172.21.15.105:0/2521620266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf8c102ef0 msgr2=0x7fcf8c19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:46.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.752+0000 7fcf91c4d640 1 --2- 172.21.15.105:0/2521620266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf8c102ef0 0x7fcf8c19cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:46.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.752+0000 7fcf91c4d640 1 -- 172.21.15.105:0/2521620266 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf8c19d3f0 con 0x7fcf8c102570 2024-09-16T02:02:46.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.752+0000 7fcf91c4d640 1 --2- 172.21.15.105:0/2521620266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf8c102570 0x7fcf8c198980 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fcf8800b4a0 tx=0x7fcf8800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:46.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.752+0000 7fcf7f7fe640 1 -- 172.21.15.105:0/2521620266 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf88014070 con 0x7fcf8c102570 2024-09-16T02:02:46.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.752+0000 7fcf946b2640 1 -- 172.21.15.105:0/2521620266 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf8c19d6e0 con 0x7fcf8c102570 2024-09-16T02:02:46.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.752+0000 7fcf7f7fe640 1 -- 172.21.15.105:0/2521620266 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf88010ac0 con 0x7fcf8c102570 2024-09-16T02:02:46.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.752+0000 7fcf7f7fe640 1 -- 172.21.15.105:0/2521620266 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf88017c30 con 0x7fcf8c102570 2024-09-16T02:02:46.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.752+0000 7fcf946b2640 1 -- 172.21.15.105:0/2521620266 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf8c1a4fc0 con 0x7fcf8c102570 2024-09-16T02:02:46.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.753+0000 7fcf946b2640 1 -- 172.21.15.105:0/2521620266 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf8c101200 con 0x7fcf8c102570 2024-09-16T02:02:46.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.754+0000 7fcf7f7fe640 1 -- 172.21.15.105:0/2521620266 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fcf88017070 con 0x7fcf8c102570 2024-09-16T02:02:46.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.755+0000 7fcf7f7fe640 1 --2- 172.21.15.105:0/2521620266 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fcf780a3240 0x7fcf780a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:46.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.755+0000 7fcf9244e640 1 --2- 172.21.15.105:0/2521620266 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fcf780a3240 0x7fcf780a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:46.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.755+0000 7fcf7f7fe640 1 -- 172.21.15.105:0/2521620266 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fcf880cee10 con 0x7fcf8c102570 2024-09-16T02:02:46.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.756+0000 7fcf9244e640 1 --2- 172.21.15.105:0/2521620266 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fcf780a3240 0x7fcf780a5700 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcf800058c0 tx=0x7fcf8000a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:46.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.759+0000 7fcf7f7fe640 1 -- 172.21.15.105:0/2521620266 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcf8809bad0 con 0x7fcf8c102570 2024-09-16T02:02:46.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:46 smithi190 ceph-mon[34762]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:02:46.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:46 smithi190 ceph-mon[34762]: pgmap v346: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.6 KiB/s wr, 13 op/s; 22 B/s, 0 objects/s recovering 2024-09-16T02:02:46.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:46 smithi137 ceph-mon[34099]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:02:46.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:46 smithi137 ceph-mon[34099]: pgmap v346: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.6 KiB/s wr, 13 op/s; 22 B/s, 0 objects/s recovering 2024-09-16T02:02:46.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.996+0000 7fcf946b2640 1 -- 172.21.15.105:0/2521620266 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf8c109090 con 0x7fcf8c102570 2024-09-16T02:02:46.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.997+0000 7fcf7f7fe640 1 -- 172.21.15.105:0/2521620266 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11991 (secure 0 0 0) 0x7fcf8809b3d0 con 0x7fcf8c102570 2024-09-16T02:02:46.998 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:02:46.998 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":28,"btime":"2024-09-16T02:02:41:554178+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2057699920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2057699920},{"type":"v1","addr":"172.21.15.105:6835","nonce":2057699920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1764886948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1764886948},{"type":"v1","addr":"172.21.15.105:6837","nonce":1764886948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T02:00:35.112266+0000","modified":"2024-09-16T02:02:40.885614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24469,"mds_2":24461,"mds_3":24457,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/4112062361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":4112062361},{"type":"v1","addr":"172.21.15.105:6839","nonce":4112062361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6841/163156514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":163156514},{"type":"v1","addr":"172.21.15.105:6841","nonce":163156514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/683519097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":683519097},{"type":"v1","addr":"172.21.15.190:6833","nonce":683519097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3025442862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3025442862},{"type":"v1","addr":"172.21.15.137:6833","nonce":3025442862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/763783602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":763783602},{"type":"v1","addr":"172.21.15.137:6835","nonce":763783602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.190:6835/1851042281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1851042281},{"type":"v1","addr":"172.21.15.190:6835","nonce":1851042281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/3181257549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3181257549},{"type":"v1","addr":"172.21.15.190:6837","nonce":3181257549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6837/2788541030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2788541030},{"type":"v1","addr":"172.21.15.137:6837","nonce":2788541030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.190:6839/1222573537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":1222573537},{"type":"v1","addr":"172.21.15.190:6839","nonce":1222573537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/3454924009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3454924009},{"type":"v1","addr":"172.21.15.137:6839","nonce":3454924009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24445,24457,24461,24469,24482]},"id":1}]} 2024-09-16T02:02:46.999 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 28 2024-09-16T02:02:47.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.999+0000 7fcf946b2640 1 -- 172.21.15.105:0/2521620266 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fcf780a3240 msgr2=0x7fcf780a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:47.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.999+0000 7fcf946b2640 1 --2- 172.21.15.105:0/2521620266 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fcf780a3240 0x7fcf780a5700 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcf800058c0 tx=0x7fcf8000a4c0 comp rx=0 tx=0).stop 2024-09-16T02:02:47.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.999+0000 7fcf946b2640 1 -- 172.21.15.105:0/2521620266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf8c102570 msgr2=0x7fcf8c198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:47.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:46.999+0000 7fcf946b2640 1 --2- 172.21.15.105:0/2521620266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf8c102570 0x7fcf8c198980 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fcf8800b4a0 tx=0x7fcf8800b960 comp rx=0 tx=0).stop 2024-09-16T02:02:47.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:47.000+0000 7fcf946b2640 1 -- 172.21.15.105:0/2521620266 shutdown_connections 2024-09-16T02:02:47.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:47.000+0000 7fcf946b2640 1 --2- 172.21.15.105:0/2521620266 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fcf780a3240 0x7fcf780a5700 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:47.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:47.000+0000 7fcf946b2640 1 --2- 172.21.15.105:0/2521620266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf8c102ef0 0x7fcf8c19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:47.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:47.000+0000 7fcf946b2640 1 --2- 172.21.15.105:0/2521620266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf8c102570 0x7fcf8c198980 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:47.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:47.000+0000 7fcf946b2640 1 --2- 172.21.15.105:0/2521620266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf8c100e20 0x7fcf8c198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:47.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:47.000+0000 7fcf946b2640 1 -- 172.21.15.105:0/2521620266 >> 172.21.15.105:0/2521620266 conn(0x7fcf8c0fc7d0 msgr2=0x7fcf8c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:47.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:47.000+0000 7fcf946b2640 1 -- 172.21.15.105:0/2521620266 shutdown_connections 2024-09-16T02:02:47.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:47.000+0000 7fcf946b2640 1 -- 172.21.15.105:0/2521620266 wait complete. 2024-09-16T02:02:47.823 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T02:02:47.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:47 smithi190 ceph-mon[34762]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:02:47.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:47 smithi190 ceph-mon[34762]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:02:47.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:47 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/2521620266' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:02:47.846 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:47 smithi105 ceph-mon[31197]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:02:47.846 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:47 smithi105 ceph-mon[31197]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:02:47.846 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:47 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2521620266' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:02:47.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:47 smithi137 ceph-mon[34099]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:02:47.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:47 smithi137 ceph-mon[34099]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:02:47.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:47 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/2521620266' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:02:48.073 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:02:48.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.521+0000 7f4103028640 1 -- 172.21.15.105:0/3708099799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40fc102570 msgr2=0x7f40fc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:48.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.521+0000 7f4103028640 1 --2- 172.21.15.105:0/3708099799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40fc102570 0x7f40fc1029b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f40ec009900 tx=0x7f40ec02a480 comp rx=0 tx=0).stop 2024-09-16T02:02:48.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.521+0000 7f4103028640 1 -- 172.21.15.105:0/3708099799 shutdown_connections 2024-09-16T02:02:48.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.521+0000 7f4103028640 1 --2- 172.21.15.105:0/3708099799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40fc102ef0 0x7f40fc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:48.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.521+0000 7f4103028640 1 --2- 172.21.15.105:0/3708099799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40fc102570 0x7f40fc1029b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:48.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.521+0000 7f4103028640 1 --2- 172.21.15.105:0/3708099799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40fc100e20 0x7f40fc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:48.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.521+0000 7f4103028640 1 -- 172.21.15.105:0/3708099799 >> 172.21.15.105:0/3708099799 conn(0x7f40fc0fc7d0 msgr2=0x7f40fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:48.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.521+0000 7f4103028640 1 -- 172.21.15.105:0/3708099799 shutdown_connections 2024-09-16T02:02:48.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.521+0000 7f4103028640 1 -- 172.21.15.105:0/3708099799 wait complete. 2024-09-16T02:02:48.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.521+0000 7f4103028640 1 Processor -- start 2024-09-16T02:02:48.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.522+0000 7f4103028640 1 -- start start 2024-09-16T02:02:48.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.522+0000 7f4103028640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40fc100e20 0x7f40fc199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:48.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.522+0000 7f4103028640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40fc102570 0x7f40fc1998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:48.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.522+0000 7f4103028640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40fc102ef0 0x7f40fc1a4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:48.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.522+0000 7f4103028640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f40fc110ed0 con 0x7f40fc102570 2024-09-16T02:02:48.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.522+0000 7f4103028640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f40fc110d50 con 0x7f40fc100e20 2024-09-16T02:02:48.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.522+0000 7f4103028640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f40fc111050 con 0x7f40fc102ef0 2024-09-16T02:02:48.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.522+0000 7f40fbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40fc102570 0x7f40fc1998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:48.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.522+0000 7f40fbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40fc102570 0x7f40fc1998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51008/0 (socket says 172.21.15.105:51008) 2024-09-16T02:02:48.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.522+0000 7f40fbfff640 1 -- 172.21.15.105:0/665484510 learned_addr learned my addr 172.21.15.105:0/665484510 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:48.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.522+0000 7f4100dc4640 1 --2- 172.21.15.105:0/665484510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40fc100e20 0x7f40fc199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:48.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.522+0000 7f40fbfff640 1 -- 172.21.15.105:0/665484510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40fc102ef0 msgr2=0x7f40fc1a4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:48.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.522+0000 7f40fbfff640 1 --2- 172.21.15.105:0/665484510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40fc102ef0 0x7f40fc1a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:48.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.522+0000 7f40fbfff640 1 -- 172.21.15.105:0/665484510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40fc100e20 msgr2=0x7f40fc199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:48.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.522+0000 7f40fbfff640 1 --2- 172.21.15.105:0/665484510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40fc100e20 0x7f40fc199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:48.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.522+0000 7f40fbfff640 1 -- 172.21.15.105:0/665484510 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40fc1a4940 con 0x7f40fc102570 2024-09-16T02:02:48.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.522+0000 7f4100dc4640 1 --2- 172.21.15.105:0/665484510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40fc100e20 0x7f40fc199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:02:48.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.522+0000 7f40fbfff640 1 --2- 172.21.15.105:0/665484510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40fc102570 0x7f40fc1998c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f40ec006010 tx=0x7f40ec0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:48.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.522+0000 7f40f9ffb640 1 -- 172.21.15.105:0/665484510 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec03f070 con 0x7f40fc102570 2024-09-16T02:02:48.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.523+0000 7f40f9ffb640 1 -- 172.21.15.105:0/665484510 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40ec003aa0 con 0x7f40fc102570 2024-09-16T02:02:48.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.523+0000 7f4103028640 1 -- 172.21.15.105:0/665484510 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40fc1a4b10 con 0x7f40fc102570 2024-09-16T02:02:48.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.523+0000 7f40f9ffb640 1 -- 172.21.15.105:0/665484510 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec035030 con 0x7f40fc102570 2024-09-16T02:02:48.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.523+0000 7f4103028640 1 -- 172.21.15.105:0/665484510 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40fc1a4fc0 con 0x7f40fc102570 2024-09-16T02:02:48.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.524+0000 7f4103028640 1 -- 172.21.15.105:0/665484510 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40fc101200 con 0x7f40fc102570 2024-09-16T02:02:48.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.525+0000 7f40f9ffb640 1 -- 172.21.15.105:0/665484510 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f40ec042070 con 0x7f40fc102570 2024-09-16T02:02:48.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.525+0000 7f40f9ffb640 1 --2- 172.21.15.105:0/665484510 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f40e80a32c0 0x7f40e80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:48.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.526+0000 7f4100dc4640 1 --2- 172.21.15.105:0/665484510 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f40e80a32c0 0x7f40e80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:48.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.526+0000 7f40f9ffb640 1 -- 172.21.15.105:0/665484510 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f40ec0f0390 con 0x7f40fc102570 2024-09-16T02:02:48.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.526+0000 7f4100dc4640 1 --2- 172.21.15.105:0/665484510 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f40e80a32c0 0x7f40e80a5780 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f40f4007eb0 tx=0x7f40f40073c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:48.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.530+0000 7f40f9ffb640 1 -- 172.21.15.105:0/665484510 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f40ec0bd050 con 0x7f40fc102570 2024-09-16T02:02:48.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.700+0000 7f4103028640 1 -- 172.21.15.105:0/665484510 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f40fc109090 con 0x7f40fc102570 2024-09-16T02:02:48.701 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:48 smithi105 ceph-mon[31197]: pgmap v347: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 10 op/s; 22 B/s, 0 objects/s recovering 2024-09-16T02:02:48.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.701+0000 7f40f9ffb640 1 -- 172.21.15.105:0/665484510 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22962 (secure 0 0 0) 0x7f40ec0bc950 con 0x7f40fc102570 2024-09-16T02:02:48.703 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:02:48.703 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":83,"fsid":"423b7a2a-73ce-11ef-bceb-c7b262605968","created":"2024-09-16T01:52:44.388033+0000","modified":"2024-09-16T02:02:41.553985+0000","last_up_change":"2024-09-16T01:59:42.218034+0000","last_in_change":"2024-09-16T01:59:18.785252+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T01:55:53.680808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T02:00:31.286117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T02:00:33.310045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a60d6c72-da8b-4b02-bb33-672be795e7f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6803","nonce":1226660290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6805","nonce":1226660290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6809","nonce":1226660290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6807","nonce":1226660290}]},"public_addr":"172.21.15.105:6803/1226660290","cluster_addr":"172.21.15.105:6805/1226660290","heartbeat_back_addr":"172.21.15.105:6809/1226660290","heartbeat_front_addr":"172.21.15.105:6807/1226660290","state":["exists","up"]},{"osd":1,"uuid":"b1d9ae84-b9a0-4a7e-b0fe-c402d46809fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6801","nonce":34753956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6803","nonce":34753956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6807","nonce":34753956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6805","nonce":34753956}]},"public_addr":"172.21.15.137:6801/34753956","cluster_addr":"172.21.15.137:6803/34753956","heartbeat_back_addr":"172.21.15.137:6807/34753956","heartbeat_front_addr":"172.21.15.137:6805/34753956","state":["exists","up"]},{"osd":2,"uuid":"137faae6-2e8b-44b5-8976-3d95a6eef1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6801","nonce":1159531918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6803","nonce":1159531918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6807","nonce":1159531918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6805","nonce":1159531918}]},"public_addr":"172.21.15.190:6801/1159531918","cluster_addr":"172.21.15.190:6803/1159531918","heartbeat_back_addr":"172.21.15.190:6807/1159531918","heartbeat_front_addr":"172.21.15.190:6805/1159531918","state":["exists","up"]},{"osd":3,"uuid":"5743c37b-5cd8-42b6-81c8-5cf5afc71e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6811","nonce":604905679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6813","nonce":604905679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6817","nonce":604905679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6815","nonce":604905679}]},"public_addr":"172.21.15.105:6811/604905679","cluster_addr":"172.21.15.105:6813/604905679","heartbeat_back_addr":"172.21.15.105:6817/604905679","heartbeat_front_addr":"172.21.15.105:6815/604905679","state":["exists","up"]},{"osd":4,"uuid":"e439d4a2-b54c-4de9-97f6-3510afe57f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6809","nonce":3758109169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6811","nonce":3758109169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6815","nonce":3758109169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6813","nonce":3758109169}]},"public_addr":"172.21.15.137:6809/3758109169","cluster_addr":"172.21.15.137:6811/3758109169","heartbeat_back_addr":"172.21.15.137:6815/3758109169","heartbeat_front_addr":"172.21.15.137:6813/3758109169","state":["exists","up"]},{"osd":5,"uuid":"d1e2a7d5-99f8-40fa-af6e-0305cc0af677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6809","nonce":2086773754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6811","nonce":2086773754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6815","nonce":2086773754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6813","nonce":2086773754}]},"public_addr":"172.21.15.190:6809/2086773754","cluster_addr":"172.21.15.190:6811/2086773754","heartbeat_back_addr":"172.21.15.190:6815/2086773754","heartbeat_front_addr":"172.21.15.190:6813/2086773754","state":["exists","up"]},{"osd":6,"uuid":"cd418159-a0f6-44c7-9213-c010b132fb56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6819","nonce":4012018725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6821","nonce":4012018725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6825","nonce":4012018725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6823","nonce":4012018725}]},"public_addr":"172.21.15.105:6819/4012018725","cluster_addr":"172.21.15.105:6821/4012018725","heartbeat_back_addr":"172.21.15.105:6825/4012018725","heartbeat_front_addr":"172.21.15.105:6823/4012018725","state":["exists","up"]},{"osd":7,"uuid":"76c15863-801e-4a72-be73-ba9df46c15b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6817","nonce":888678288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6819","nonce":888678288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6823","nonce":888678288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6821","nonce":888678288}]},"public_addr":"172.21.15.137:6817/888678288","cluster_addr":"172.21.15.137:6819/888678288","heartbeat_back_addr":"172.21.15.137:6823/888678288","heartbeat_front_addr":"172.21.15.137:6821/888678288","state":["exists","up"]},{"osd":8,"uuid":"6718597c-6f42-48ee-9493-edb2396fe707","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6817","nonce":2132638341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6819","nonce":2132638341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6823","nonce":2132638341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6821","nonce":2132638341}]},"public_addr":"172.21.15.190:6817/2132638341","cluster_addr":"172.21.15.190:6819/2132638341","heartbeat_back_addr":"172.21.15.190:6823/2132638341","heartbeat_front_addr":"172.21.15.190:6821/2132638341","state":["exists","up"]},{"osd":9,"uuid":"5178edc0-a2e8-491c-821c-6f41ec5aeb2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6827","nonce":3375034979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6829","nonce":3375034979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6833","nonce":3375034979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6831","nonce":3375034979}]},"public_addr":"172.21.15.105:6827/3375034979","cluster_addr":"172.21.15.105:6829/3375034979","heartbeat_back_addr":"172.21.15.105:6833/3375034979","heartbeat_front_addr":"172.21.15.105:6831/3375034979","state":["exists","up"]},{"osd":10,"uuid":"ad145d3e-a6e2-4dd7-bea3-bcd01eb3413e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6825","nonce":318562234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6827","nonce":318562234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6831","nonce":318562234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6829","nonce":318562234}]},"public_addr":"172.21.15.137:6825/318562234","cluster_addr":"172.21.15.137:6827/318562234","heartbeat_back_addr":"172.21.15.137:6831/318562234","heartbeat_front_addr":"172.21.15.137:6829/318562234","state":["exists","up"]},{"osd":11,"uuid":"6d44bced-3a8b-4763-a93e-0ea34acb4cb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6825","nonce":3820460689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6827","nonce":3820460689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6831","nonce":3820460689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6829","nonce":3820460689}]},"public_addr":"172.21.15.190:6825/3820460689","cluster_addr":"172.21.15.190:6827/3820460689","heartbeat_back_addr":"172.21.15.190:6831/3820460689","heartbeat_front_addr":"172.21.15.190:6829/3820460689","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:55:02.759033+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:55:26.829142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:55:51.120200+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:56:17.277870+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:56:42.313701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:07.013416+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:32.477916+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:58.643764+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:23.454908+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:48.869082+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:14.921573+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:40.908993+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":7,"to":6}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.17","mappings":[{"from":5,"to":1}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":7}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:0/2453739657":"2024-09-17T01:53:39.351314+0000","172.21.15.105:6801/1469189484":"2024-09-17T01:53:39.351314+0000","172.21.15.105:0/3466166879":"2024-09-17T01:53:39.351314+0000","172.21.15.105:6801/1268348742":"2024-09-17T01:53:15.522227+0000","172.21.15.105:0/2897248478":"2024-09-17T01:53:39.351314+0000","172.21.15.105:0/3709363371":"2024-09-17T01:53:15.522227+0000","172.21.15.105:6800/1268348742":"2024-09-17T01:53:15.522227+0000","172.21.15.105:6800/1469189484":"2024-09-17T01:53:39.351314+0000","172.21.15.105:0/1198176299":"2024-09-17T01:53:15.522227+0000","172.21.15.105:0/3031917848":"2024-09-17T01:53:15.522227+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T02:02:48.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.704+0000 7f4103028640 1 -- 172.21.15.105:0/665484510 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f40e80a32c0 msgr2=0x7f40e80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:48.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.704+0000 7f4103028640 1 --2- 172.21.15.105:0/665484510 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f40e80a32c0 0x7f40e80a5780 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f40f4007eb0 tx=0x7f40f40073c0 comp rx=0 tx=0).stop 2024-09-16T02:02:48.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.704+0000 7f4103028640 1 -- 172.21.15.105:0/665484510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40fc102570 msgr2=0x7f40fc1998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:48.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.704+0000 7f4103028640 1 --2- 172.21.15.105:0/665484510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40fc102570 0x7f40fc1998c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f40ec006010 tx=0x7f40ec0042f0 comp rx=0 tx=0).stop 2024-09-16T02:02:48.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.704+0000 7f4103028640 1 -- 172.21.15.105:0/665484510 shutdown_connections 2024-09-16T02:02:48.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.704+0000 7f4103028640 1 --2- 172.21.15.105:0/665484510 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f40e80a32c0 0x7f40e80a5780 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:48.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.704+0000 7f4103028640 1 --2- 172.21.15.105:0/665484510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40fc102ef0 0x7f40fc1a4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:48.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.704+0000 7f4103028640 1 --2- 172.21.15.105:0/665484510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40fc102570 0x7f40fc1998c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:48.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.704+0000 7f4103028640 1 --2- 172.21.15.105:0/665484510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40fc100e20 0x7f40fc199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:48.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.704+0000 7f4103028640 1 -- 172.21.15.105:0/665484510 >> 172.21.15.105:0/665484510 conn(0x7f40fc0fc7d0 msgr2=0x7f40fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:48.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.704+0000 7f4103028640 1 -- 172.21.15.105:0/665484510 shutdown_connections 2024-09-16T02:02:48.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:48.704+0000 7f4103028640 1 -- 172.21.15.105:0/665484510 wait complete. 2024-09-16T02:02:48.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:48 smithi137 ceph-mon[34099]: pgmap v347: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 10 op/s; 22 B/s, 0 objects/s recovering 2024-09-16T02:02:49.016 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:48 smithi190 ceph-mon[34762]: pgmap v347: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 10 op/s; 22 B/s, 0 objects/s recovering 2024-09-16T02:02:49.397 DEBUG:tasks.fwd_scrub:Yielding 2024-09-16T02:02:49.397 INFO:teuthology.run_tasks:Running task workunit... 2024-09-16T02:02:49.411 INFO:tasks.workunit:Pulling workunits from ref ad35db06e8924912f01e32b35576297c0f1352b4 2024-09-16T02:02:49.411 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-16T02:02:49.411 INFO:tasks.workunit:timeout=3h 2024-09-16T02:02:49.411 INFO:tasks.workunit:cleanup=True 2024-09-16T02:02:49.412 DEBUG:teuthology.orchestra.run.smithi105:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-16T02:02:49.412 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-16T02:02:49.412 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T02:02:49.412 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T02:02:49.453 INFO:teuthology.orchestra.run.smithi105.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-16T02:02:49.453 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-09-16T02:02:49.453 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 78h/120d Inode: 1 Links: 2 2024-09-16T02:02:49.453 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T02:02:49.453 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:fusefs_t:s0 2024-09-16T02:02:49.454 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-09-16 02:00:50.927550307 +0000 2024-09-16T02:02:49.454 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-09-16 02:00:50.927550307 +0000 2024-09-16T02:02:49.454 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-09-16 02:02:40.526161341 +0000 2024-09-16T02:02:49.454 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: - 2024-09-16T02:02:49.454 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-16T02:02:49.454 DEBUG:teuthology.orchestra.run.smithi105:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-16T02:02:49.523 DEBUG:teuthology.orchestra.run.smithi105:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout ad35db06e8924912f01e32b35576297c0f1352b4 2024-09-16T02:02:49.580 INFO:tasks.workunit.client.0.smithi105.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-16T02:02:49.691 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:02:49.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:49 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/665484510' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:02:49.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:49 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/665484510' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:02:49.977 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:49 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/665484510' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:02:50.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.184+0000 7f56534f1640 1 -- 172.21.15.105:0/2464493384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f564c101dd0 msgr2=0x7f564c1021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:50.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.184+0000 7f56534f1640 1 --2- 172.21.15.105:0/2464493384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f564c101dd0 0x7f564c1021b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5644009ca0 tx=0x7f564402a8c0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.184+0000 7f56534f1640 1 -- 172.21.15.105:0/2464493384 shutdown_connections 2024-09-16T02:02:50.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.184+0000 7f56534f1640 1 --2- 172.21.15.105:0/2464493384 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f564c102780 0x7f564c1067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.184+0000 7f56534f1640 1 --2- 172.21.15.105:0/2464493384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f564c106f00 0x7f564c10ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.184+0000 7f56534f1640 1 --2- 172.21.15.105:0/2464493384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f564c101dd0 0x7f564c1021b0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.184+0000 7f56534f1640 1 -- 172.21.15.105:0/2464493384 >> 172.21.15.105:0/2464493384 conn(0x7f564c0fd8b0 msgr2=0x7f564c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:50.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.184+0000 7f56534f1640 1 -- 172.21.15.105:0/2464493384 shutdown_connections 2024-09-16T02:02:50.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.184+0000 7f56534f1640 1 -- 172.21.15.105:0/2464493384 wait complete. 2024-09-16T02:02:50.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.185+0000 7f56534f1640 1 Processor -- start 2024-09-16T02:02:50.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.185+0000 7f56534f1640 1 -- start start 2024-09-16T02:02:50.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.185+0000 7f56534f1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f564c101dd0 0x7f564c198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:50.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.185+0000 7f56534f1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f564c102780 0x7f564c198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:50.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.185+0000 7f56534f1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f564c106f00 0x7f564c19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:50.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.185+0000 7f56534f1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f564c111e10 con 0x7f564c106f00 2024-09-16T02:02:50.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.185+0000 7f56534f1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f564c111c90 con 0x7f564c102780 2024-09-16T02:02:50.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.185+0000 7f56534f1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f564c111f90 con 0x7f564c101dd0 2024-09-16T02:02:50.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.185+0000 7f5651a8e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f564c106f00 0x7f564c19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:50.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.185+0000 7f5651a8e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f564c106f00 0x7f564c19cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51036/0 (socket says 172.21.15.105:51036) 2024-09-16T02:02:50.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.185+0000 7f5651a8e640 1 -- 172.21.15.105:0/4147254392 learned_addr learned my addr 172.21.15.105:0/4147254392 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:50.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.185+0000 7f5650a8c640 1 --2- 172.21.15.105:0/4147254392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f564c102780 0x7f564c198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:50.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.185+0000 7f565128d640 1 --2- 172.21.15.105:0/4147254392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f564c101dd0 0x7f564c198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:50.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.186+0000 7f5651a8e640 1 -- 172.21.15.105:0/4147254392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f564c101dd0 msgr2=0x7f564c198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:50.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.186+0000 7f5651a8e640 1 --2- 172.21.15.105:0/4147254392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f564c101dd0 0x7f564c198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.186+0000 7f5651a8e640 1 -- 172.21.15.105:0/4147254392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f564c102780 msgr2=0x7f564c198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:50.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.186+0000 7f5651a8e640 1 --2- 172.21.15.105:0/4147254392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f564c102780 0x7f564c198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.186+0000 7f5651a8e640 1 -- 172.21.15.105:0/4147254392 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f564c19d300 con 0x7f564c106f00 2024-09-16T02:02:50.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.186+0000 7f5651a8e640 1 --2- 172.21.15.105:0/4147254392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f564c106f00 0x7f564c19cc20 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f564800e970 tx=0x7f564800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:50.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.186+0000 7f56427fc640 1 -- 172.21.15.105:0/4147254392 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564800ccc0 con 0x7f564c106f00 2024-09-16T02:02:50.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.186+0000 7f56534f1640 1 -- 172.21.15.105:0/4147254392 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f564c19d5f0 con 0x7f564c106f00 2024-09-16T02:02:50.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.186+0000 7f56427fc640 1 -- 172.21.15.105:0/4147254392 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f564800ce60 con 0x7f564c106f00 2024-09-16T02:02:50.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.186+0000 7f56427fc640 1 -- 172.21.15.105:0/4147254392 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5648010430 con 0x7f564c106f00 2024-09-16T02:02:50.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.186+0000 7f56534f1640 1 -- 172.21.15.105:0/4147254392 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f564c1a4ed0 con 0x7f564c106f00 2024-09-16T02:02:50.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.189+0000 7f56558ca740 1 Processor -- start 2024-09-16T02:02:50.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.189+0000 7f56427fc640 1 -- 172.21.15.105:0/4147254392 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f5648004f90 con 0x7f564c106f00 2024-09-16T02:02:50.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.189+0000 7f56558ca740 1 -- start start 2024-09-16T02:02:50.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.189+0000 7f56558ca740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c7e083b60 0x561c7e083f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:50.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.189+0000 7f56558ca740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c7e084480 0x561c7e08e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:50.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.189+0000 7f56558ca740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c7e08ed40 0x561c7e091130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:50.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.189+0000 7f56558ca740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561c7dfe9280 con 0x561c7e083b60 2024-09-16T02:02:50.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.189+0000 7f56558ca740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561c7df90b10 con 0x561c7e08ed40 2024-09-16T02:02:50.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.189+0000 7f56558ca740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561c7dfe5240 con 0x561c7e084480 2024-09-16T02:02:50.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.189+0000 7f5650a8c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c7e083b60 0x561c7e083f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:50.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.189+0000 7f5650a8c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c7e083b60 0x561c7e083f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51046/0 (socket says 172.21.15.105:51046) 2024-09-16T02:02:50.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.189+0000 7f5650a8c640 1 -- 172.21.15.105:0/2562725663 learned_addr learned my addr 172.21.15.105:0/2562725663 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:50.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.189+0000 7f56427fc640 1 --2- 172.21.15.105:0/4147254392 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f56380a3240 0x7f56380a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:50.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.190+0000 7f5650a8c640 1 -- 172.21.15.105:0/2562725663 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c7e084480 msgr2=0x561c7e08e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:50.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.190+0000 7f5650a8c640 1 --2- 172.21.15.105:0/2562725663 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c7e084480 0x561c7e08e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.190+0000 7f5650a8c640 1 -- 172.21.15.105:0/2562725663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c7e08ed40 msgr2=0x561c7e091130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T02:02:50.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.190+0000 7f5650a8c640 1 --2- 172.21.15.105:0/2562725663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c7e08ed40 0x561c7e091130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.190+0000 7f5650a8c640 1 -- 172.21.15.105:0/2562725663 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c7df9d1f0 con 0x561c7e083b60 2024-09-16T02:02:50.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.190+0000 7f56427fc640 1 -- 172.21.15.105:0/4147254392 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f56480cfcb0 con 0x7f564c106f00 2024-09-16T02:02:50.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.190+0000 7f5651a8e640 1 --2- 172.21.15.105:0/4147254392 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f56380a3240 0x7f56380a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:50.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.190+0000 7f5650a8c640 1 --2- 172.21.15.105:0/2562725663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c7e083b60 0x561c7e083f40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f563c00b690 tx=0x7f563c00bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c91dc5745ea90ff6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:50.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.190+0000 7f56377fe640 1 -- 172.21.15.105:0/2562725663 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563c003bd0 con 0x561c7e083b60 2024-09-16T02:02:50.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.190+0000 7f56377fe640 1 -- 172.21.15.105:0/2562725663 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f563c015070 con 0x561c7e083b60 2024-09-16T02:02:50.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.190+0000 7f56377fe640 1 -- 172.21.15.105:0/2562725663 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563c004d70 con 0x561c7e083b60 2024-09-16T02:02:50.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.191+0000 7f56558ca740 1 -- 172.21.15.105:0/2562725663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c7e083b60 msgr2=0x561c7e083f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:50.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.191+0000 7f56558ca740 1 --2- 172.21.15.105:0/2562725663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c7e083b60 0x561c7e083f40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f563c00b690 tx=0x7f563c00bb50 comp rx=0 tx=0).stop 2024-09-16T02:02:50.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.191+0000 7f5651a8e640 1 --2- 172.21.15.105:0/4147254392 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f56380a3240 0x7f56380a5700 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f564800c6a0 tx=0x7f5648009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:50.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.191+0000 7f56558ca740 1 -- 172.21.15.105:0/2562725663 shutdown_connections 2024-09-16T02:02:50.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.191+0000 7f56558ca740 1 --2- 172.21.15.105:0/2562725663 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c7e084480 0x561c7e08e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.191+0000 7f56558ca740 1 --2- 172.21.15.105:0/2562725663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c7e08ed40 0x561c7e091130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.191+0000 7f56558ca740 1 --2- 172.21.15.105:0/2562725663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c7e083b60 0x561c7e083f40 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.191+0000 7f56558ca740 1 -- 172.21.15.105:0/2562725663 >> 172.21.15.105:0/2562725663 conn(0x561c7e0800f0 msgr2=0x561c7e0804d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:50.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.191+0000 7f56558ca740 1 -- 172.21.15.105:0/2562725663 shutdown_connections 2024-09-16T02:02:50.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.191+0000 7f56558ca740 1 -- 172.21.15.105:0/2562725663 wait complete. 2024-09-16T02:02:50.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.191+0000 7f56558ca740 1 -- start start 2024-09-16T02:02:50.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.191+0000 7f56558ca740 1 Processor -- start 2024-09-16T02:02:50.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.192+0000 7f56558ca740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c7e083b60 0x561c7e08bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:50.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.192+0000 7f56558ca740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c7e084480 0x561c7e08c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:50.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.192+0000 7f56558ca740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c7e08ed40 0x561c7e08c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:50.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.192+0000 7f56558ca740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561c7e091910 con 0x561c7e08ed40 2024-09-16T02:02:50.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.192+0000 7f56558ca740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561c7dfa3f10 con 0x561c7e084480 2024-09-16T02:02:50.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.192+0000 7f56558ca740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561c7e091a90 con 0x561c7e083b60 2024-09-16T02:02:50.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.192+0000 7f56558ca740 10 client.0 _ll_register_callbacks cb 0x561c7df6ca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:02:50.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.192+0000 7f5650a8c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c7e08ed40 0x561c7e08c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:50.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.192+0000 7f5650a8c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c7e08ed40 0x561c7e08c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51056/0 (socket says 172.21.15.105:51056) 2024-09-16T02:02:50.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.192+0000 7f5650a8c640 1 -- 172.21.15.105:0/4245439031 learned_addr learned my addr 172.21.15.105:0/4245439031 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:50.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.192+0000 7f5650a8c640 1 --2- 172.21.15.105:0/4245439031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c7e083b60 0x561c7e08bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:50.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.192+0000 7f565128d640 1 --2- 172.21.15.105:0/4245439031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c7e084480 0x561c7e08c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:50.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.192+0000 7f565128d640 1 -- 172.21.15.105:0/4245439031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c7e083b60 msgr2=0x561c7e08bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:50.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.192+0000 7f565128d640 1 --2- 172.21.15.105:0/4245439031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c7e083b60 0x561c7e08bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.192+0000 7f565128d640 1 -- 172.21.15.105:0/4245439031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c7e08ed40 msgr2=0x561c7e08c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:50.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.192+0000 7f565128d640 1 --2- 172.21.15.105:0/4245439031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c7e08ed40 0x561c7e08c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.192+0000 7f565128d640 1 -- 172.21.15.105:0/4245439031 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c7dd5a6a0 con 0x561c7e084480 2024-09-16T02:02:50.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.192+0000 7f5650a8c640 1 --2- 172.21.15.105:0/4245439031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c7e08ed40 0x561c7e08c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:02:50.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.192+0000 7f565128d640 1 --2- 172.21.15.105:0/4245439031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c7e084480 0x561c7e08c1e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5644000f80 tx=0x7f5644009a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4744dc2bd69336c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:50.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.192+0000 7f5634ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-16T02:02:50.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.193+0000 7f5634ff9640 1 -- 172.21.15.105:0/4245439031 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5644031860 con 0x561c7e084480 2024-09-16T02:02:50.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.193+0000 7f5634ff9640 1 -- 172.21.15.105:0/4245439031 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f5644003970 con 0x561c7e084480 2024-09-16T02:02:50.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.193+0000 7f56558ca740 1 -- 172.21.15.105:0/4245439031 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f563c015070 con 0x561c7e084480 2024-09-16T02:02:50.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.193+0000 7f5634ff9640 1 -- 172.21.15.105:0/4245439031 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5644003b70 con 0x561c7e084480 2024-09-16T02:02:50.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.193+0000 7f56558ca740 10 client.24791 fetch_fsmap learned FSMap version 28 2024-09-16T02:02:50.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.193+0000 7f56558ca740 1 -- 172.21.15.105:0/4245439031 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561c7e09d590 con 0x561c7e084480 2024-09-16T02:02:50.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.193+0000 7f5634ff9640 1 -- 172.21.15.105:0/4245439031 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f563c015070 con 0x561c7e084480 2024-09-16T02:02:50.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.194+0000 7f5634ff9640 1 -- 172.21.15.105:0/4245439031 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f56440306e0 con 0x561c7e084480 2024-09-16T02:02:50.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.194+0000 7f56558ca740 10 client.24791 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:02:50.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.194+0000 7f56558ca740 10 client.24791 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:02:50.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.194+0000 7f56558ca740 20 client.24791 populate_metadata read hostname 'smithi105' 2024-09-16T02:02:50.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.194+0000 7f56558ca740 1 --2- 172.21.15.105:0/4245439031 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x561c7e09e9a0 0x561c7e0a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:50.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.194+0000 7f56558ca740 4 client.24791 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:02:50.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.194+0000 7f56558ca740 1 -- 172.21.15.105:0/4245439031 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c7dff67f0 con 0x561c7e09e9a0 2024-09-16T02:02:50.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.194+0000 7f5650a8c640 1 --2- 172.21.15.105:0/4245439031 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x561c7e09e9a0 0x561c7e0a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:50.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.194+0000 7f5650a8c640 1 --2- 172.21.15.105:0/4245439031 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x561c7e09e9a0 0x561c7e0a0dc0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65fe96bf42bd0113 server_cookie=33f8f6976c2b5fb in_seq=0 out_seq=0 2024-09-16T02:02:50.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.194+0000 7f5634ff9640 10 client.24791 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:02:50.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.195+0000 7f5634ff9640 1 -- 172.21.15.105:0/4245439031 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561c7dff67f0 con 0x561c7e09e9a0 2024-09-16T02:02:50.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.195+0000 7f5634ff9640 10 client.24791 handle_command_reply: tid=0 2024-09-16T02:02:50.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.195+0000 7f56558ca740 1 client.24791 shutdown 2024-09-16T02:02:50.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.195+0000 7f56558ca740 1 -- 172.21.15.105:0/4245439031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c7e084480 msgr2=0x561c7e08c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:50.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.195+0000 7f56558ca740 1 --2- 172.21.15.105:0/4245439031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c7e084480 0x561c7e08c1e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5644000f80 tx=0x7f5644009a10 comp rx=0 tx=0).stop 2024-09-16T02:02:50.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.195+0000 7f56558ca740 1 -- 172.21.15.105:0/4245439031 shutdown_connections 2024-09-16T02:02:50.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.195+0000 7f56558ca740 1 --2- 172.21.15.105:0/4245439031 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x561c7e09e9a0 0x561c7e0a0dc0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.195+0000 7f56558ca740 1 --2- 172.21.15.105:0/4245439031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c7e08ed40 0x561c7e08c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.195+0000 7f56558ca740 1 --2- 172.21.15.105:0/4245439031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c7e084480 0x561c7e08c1e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.195+0000 7f56558ca740 1 --2- 172.21.15.105:0/4245439031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c7e083b60 0x561c7e08bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.195+0000 7f5634ff9640 0 client.24791 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:02:50.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.195+0000 7f56558ca740 1 -- 172.21.15.105:0/4245439031 >> 172.21.15.105:0/4245439031 conn(0x561c7e0800f0 msgr2=0x561c7e085e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:50.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.196+0000 7f56558ca740 1 -- 172.21.15.105:0/4245439031 shutdown_connections 2024-09-16T02:02:50.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.196+0000 7f56558ca740 1 -- 172.21.15.105:0/4245439031 wait complete. 2024-09-16T02:02:50.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.196+0000 7f56558ca740 20 client.24791 trim_cache size 0 max 16384 2024-09-16T02:02:50.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.213+0000 7f56558ca740 1 Processor -- start 2024-09-16T02:02:50.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.213+0000 7f56558ca740 1 -- start start 2024-09-16T02:02:50.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.213+0000 7f56558ca740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c7e083b60 0x561c7e08c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:50.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.213+0000 7f56558ca740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c7e084480 0x561c7e0a6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:50.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.213+0000 7f56558ca740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c7e08ed40 0x561c7e0a6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:50.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.213+0000 7f56558ca740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f564c1ad6c0 con 0x561c7e083b60 2024-09-16T02:02:50.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.213+0000 7f56558ca740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f564c1af200 con 0x561c7e08ed40 2024-09-16T02:02:50.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.213+0000 7f56558ca740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f564c1add90 con 0x561c7e084480 2024-09-16T02:02:50.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.213+0000 7f565128d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c7e084480 0x561c7e0a6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:50.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.214+0000 7f565128d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c7e084480 0x561c7e0a6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.105:41660/0 (socket says 172.21.15.105:41660) 2024-09-16T02:02:50.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.214+0000 7f565128d640 1 -- 172.21.15.105:0/2482830586 learned_addr learned my addr 172.21.15.105:0/2482830586 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:50.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.214+0000 7f5650a8c640 1 --2- 172.21.15.105:0/2482830586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c7e083b60 0x561c7e08c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:50.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.214+0000 7f565128d640 1 -- 172.21.15.105:0/2482830586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c7e08ed40 msgr2=0x561c7e0a6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:50.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.214+0000 7f5650a8c640 1 --2- 172.21.15.105:0/2482830586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c7e08ed40 0x561c7e0a6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:50.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.214+0000 7f565128d640 1 --2- 172.21.15.105:0/2482830586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c7e08ed40 0x561c7e0a6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.214+0000 7f565128d640 1 -- 172.21.15.105:0/2482830586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c7e083b60 msgr2=0x561c7e08c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:50.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.214+0000 7f565128d640 1 --2- 172.21.15.105:0/2482830586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c7e083b60 0x561c7e08c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.214+0000 7f565128d640 1 -- 172.21.15.105:0/2482830586 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c7e08a1f0 con 0x561c7e084480 2024-09-16T02:02:50.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.214+0000 7f5650a8c640 1 --2- 172.21.15.105:0/2482830586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c7e08ed40 0x561c7e0a6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:02:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.214+0000 7f565128d640 1 --2- 172.21.15.105:0/2482830586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c7e084480 0x561c7e0a6690 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5644009ca0 tx=0x7f564402f310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f6a3c0db171cdc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.214+0000 7f5636ffd640 1 -- 172.21.15.105:0/2482830586 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5644030d10 con 0x561c7e084480 2024-09-16T02:02:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.214+0000 7f5636ffd640 1 -- 172.21.15.105:0/2482830586 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5644036030 con 0x561c7e084480 2024-09-16T02:02:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.214+0000 7f5636ffd640 1 -- 172.21.15.105:0/2482830586 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5644045a40 con 0x561c7e084480 2024-09-16T02:02:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.215+0000 7f56558ca740 1 -- 172.21.15.105:0/2482830586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c7e084480 msgr2=0x561c7e0a6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.215+0000 7f56558ca740 1 --2- 172.21.15.105:0/2482830586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c7e084480 0x561c7e0a6690 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5644009ca0 tx=0x7f564402f310 comp rx=0 tx=0).stop 2024-09-16T02:02:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.215+0000 7f56558ca740 1 -- 172.21.15.105:0/2482830586 shutdown_connections 2024-09-16T02:02:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.215+0000 7f56558ca740 1 --2- 172.21.15.105:0/2482830586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c7e084480 0x561c7e0a6690 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.215+0000 7f56558ca740 1 --2- 172.21.15.105:0/2482830586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c7e08ed40 0x561c7e0a6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.215+0000 7f56558ca740 1 --2- 172.21.15.105:0/2482830586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c7e083b60 0x561c7e08c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.215+0000 7f56558ca740 1 -- 172.21.15.105:0/2482830586 >> 172.21.15.105:0/2482830586 conn(0x561c7e0800f0 msgr2=0x561c7e08dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.215+0000 7f56558ca740 1 -- 172.21.15.105:0/2482830586 shutdown_connections 2024-09-16T02:02:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.215+0000 7f56558ca740 1 -- 172.21.15.105:0/2482830586 wait complete. 2024-09-16T02:02:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.215+0000 7f56558ca740 1 -- start start 2024-09-16T02:02:50.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.216+0000 7f56558ca740 1 Processor -- start 2024-09-16T02:02:50.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.216+0000 7f56558ca740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c7e083b60 0x561c7e0975c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:50.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.216+0000 7f56558ca740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c7e084480 0x561c7e0928f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:50.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.216+0000 7f56558ca740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c7e08ed40 0x561c7e093030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:50.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.216+0000 7f56558ca740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f564c1aeb30 con 0x561c7e084480 2024-09-16T02:02:50.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.216+0000 7f56558ca740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f564c1ae460 con 0x561c7e08ed40 2024-09-16T02:02:50.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.216+0000 7f56558ca740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f564c1ab6a0 con 0x561c7e083b60 2024-09-16T02:02:50.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.216+0000 7f56558ca740 10 client.0 _ll_register_callbacks cb 0x7f5644009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:02:50.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.216+0000 7f565128d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c7e084480 0x561c7e0928f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:50.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.216+0000 7f5650a8c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c7e083b60 0x561c7e0975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:50.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.216+0000 7f5650a8c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c7e083b60 0x561c7e0975c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.105:41662/0 (socket says 172.21.15.105:41662) 2024-09-16T02:02:50.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.216+0000 7f5650a8c640 1 -- 172.21.15.105:0/1242436577 learned_addr learned my addr 172.21.15.105:0/1242436577 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:50.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.216+0000 7f5650a8c640 1 --2- 172.21.15.105:0/1242436577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c7e08ed40 0x561c7e093030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:50.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.217+0000 7f565128d640 1 -- 172.21.15.105:0/1242436577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c7e083b60 msgr2=0x561c7e0975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:50.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.217+0000 7f565128d640 1 --2- 172.21.15.105:0/1242436577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c7e083b60 0x561c7e0975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.217+0000 7f565128d640 1 -- 172.21.15.105:0/1242436577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c7e08ed40 msgr2=0x561c7e093030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:50.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.217+0000 7f565128d640 1 --2- 172.21.15.105:0/1242436577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c7e08ed40 0x561c7e093030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.217+0000 7f565128d640 1 -- 172.21.15.105:0/1242436577 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c7e108660 con 0x561c7e084480 2024-09-16T02:02:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.217+0000 7f5650a8c640 1 --2- 172.21.15.105:0/1242436577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c7e08ed40 0x561c7e093030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:02:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.217+0000 7f565128d640 1 --2- 172.21.15.105:0/1242436577 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c7e084480 0x561c7e0928f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x561c7e0919f0 tx=0x7f56440305c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d38672e971ae1d75 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.217+0000 7f5634ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:02:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.217+0000 7f5634ff9640 1 -- 172.21.15.105:0/1242436577 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56440306c0 con 0x561c7e084480 2024-09-16T02:02:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.217+0000 7f5634ff9640 1 -- 172.21.15.105:0/1242436577 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f5644045480 con 0x561c7e084480 2024-09-16T02:02:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.217+0000 7f56558ca740 1 -- 172.21.15.105:0/1242436577 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5644036030 con 0x561c7e084480 2024-09-16T02:02:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.217+0000 7f5634ff9640 1 -- 172.21.15.105:0/1242436577 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5644030860 con 0x561c7e084480 2024-09-16T02:02:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.217+0000 7f56558ca740 10 client.15330 fetch_fsmap learned FSMap version 28 2024-09-16T02:02:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.217+0000 7f56558ca740 1 -- 172.21.15.105:0/1242436577 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561c7e1097b0 con 0x561c7e084480 2024-09-16T02:02:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.218+0000 7f5634ff9640 1 -- 172.21.15.105:0/1242436577 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5644036030 con 0x561c7e084480 2024-09-16T02:02:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.218+0000 7f5634ff9640 1 -- 172.21.15.105:0/1242436577 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f564402f6c0 con 0x561c7e084480 2024-09-16T02:02:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.218+0000 7f56558ca740 10 client.15330 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:02:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.218+0000 7f56558ca740 10 client.15330 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:02:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.218+0000 7f56558ca740 20 client.15330 populate_metadata read hostname 'smithi105' 2024-09-16T02:02:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.218+0000 7f56558ca740 1 --2- 172.21.15.105:0/1242436577 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x561c7e109ce0 0x561c7e10c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.218+0000 7f56558ca740 4 client.15330 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T02:02:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.218+0000 7f56558ca740 1 -- 172.21.15.105:0/1242436577 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561c7dff0d00 con 0x561c7e109ce0 2024-09-16T02:02:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.218+0000 7f5650a8c640 1 --2- 172.21.15.105:0/1242436577 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x561c7e109ce0 0x561c7e10c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.219+0000 7f5634ff9640 10 client.15330 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:02:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.219+0000 7f5650a8c640 1 --2- 172.21.15.105:0/1242436577 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x561c7e109ce0 0x561c7e10c0d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a691acabb17a641f server_cookie=f2d0a0f9f2b24178 in_seq=0 out_seq=0 2024-09-16T02:02:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.219+0000 7f5634ff9640 1 -- 172.21.15.105:0/1242436577 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561c7dff0d00 con 0x561c7e109ce0 2024-09-16T02:02:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.219+0000 7f5634ff9640 10 client.15330 handle_command_reply: tid=0 2024-09-16T02:02:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.219+0000 7f56558ca740 1 client.15330 shutdown 2024-09-16T02:02:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.220+0000 7f56558ca740 1 -- 172.21.15.105:0/1242436577 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c7e084480 msgr2=0x561c7e0928f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.220+0000 7f56558ca740 1 --2- 172.21.15.105:0/1242436577 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c7e084480 0x561c7e0928f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x561c7e0919f0 tx=0x7f56440305c0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.220+0000 7f56558ca740 1 -- 172.21.15.105:0/1242436577 shutdown_connections 2024-09-16T02:02:50.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.220+0000 7f56558ca740 1 --2- 172.21.15.105:0/1242436577 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x561c7e109ce0 0x561c7e10c0d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.220+0000 7f56558ca740 1 --2- 172.21.15.105:0/1242436577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c7e08ed40 0x561c7e093030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.220+0000 7f56558ca740 1 --2- 172.21.15.105:0/1242436577 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c7e084480 0x561c7e0928f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.220+0000 7f56558ca740 1 --2- 172.21.15.105:0/1242436577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c7e083b60 0x561c7e0975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.220+0000 7f56558ca740 1 -- 172.21.15.105:0/1242436577 >> 172.21.15.105:0/1242436577 conn(0x561c7e0800f0 msgr2=0x561c7e0855b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:50.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.220+0000 7f5634ff9640 0 client.15330 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:02:50.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.220+0000 7f56558ca740 1 -- 172.21.15.105:0/1242436577 shutdown_connections 2024-09-16T02:02:50.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.220+0000 7f56558ca740 1 -- 172.21.15.105:0/1242436577 wait complete. 2024-09-16T02:02:50.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.220+0000 7f56558ca740 20 client.15330 trim_cache size 0 max 16384 2024-09-16T02:02:50.225 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-16T02:02:50.225 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-09-16T02:02:50.226 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "7957fb27-356e-4393-bfb0-74fedd022abd", 2024-09-16T02:02:50.226 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-09-16T02:02:50.226 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-16T02:02:50.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.221+0000 7f56534f1640 1 -- 172.21.15.105:0/4147254392 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f56380a3240 msgr2=0x7f56380a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:50.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.221+0000 7f56534f1640 1 --2- 172.21.15.105:0/4147254392 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f56380a3240 0x7f56380a5700 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f564800c6a0 tx=0x7f5648009310 comp rx=0 tx=0).stop 2024-09-16T02:02:50.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.221+0000 7f56534f1640 1 -- 172.21.15.105:0/4147254392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f564c106f00 msgr2=0x7f564c19cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:50.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.221+0000 7f56534f1640 1 --2- 172.21.15.105:0/4147254392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f564c106f00 0x7f564c19cc20 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f564800e970 tx=0x7f564800ed70 comp rx=0 tx=0).stop 2024-09-16T02:02:50.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.221+0000 7f56534f1640 1 -- 172.21.15.105:0/4147254392 shutdown_connections 2024-09-16T02:02:50.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.221+0000 7f56534f1640 1 --2- 172.21.15.105:0/4147254392 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f56380a3240 0x7f56380a5700 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.221+0000 7f56534f1640 1 --2- 172.21.15.105:0/4147254392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f564c106f00 0x7f564c19cc20 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.221+0000 7f56534f1640 1 --2- 172.21.15.105:0/4147254392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f564c102780 0x7f564c198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.221+0000 7f56534f1640 1 --2- 172.21.15.105:0/4147254392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f564c101dd0 0x7f564c198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.221+0000 7f56534f1640 1 -- 172.21.15.105:0/4147254392 >> 172.21.15.105:0/4147254392 conn(0x7f564c0fd8b0 msgr2=0x7f564c0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:50.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.221+0000 7f56534f1640 1 -- 172.21.15.105:0/4147254392 shutdown_connections 2024-09-16T02:02:50.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:50.221+0000 7f56534f1640 1 -- 172.21.15.105:0/4147254392 wait complete. 2024-09-16T02:02:50.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:50 smithi137 ceph-mon[34099]: pgmap v348: 97 pgs: 97 active+clean; 648 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.5 KiB/s wr, 11 op/s; 20 B/s, 0 objects/s recovering 2024-09-16T02:02:50.980 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:50 smithi105 ceph-mon[31197]: pgmap v348: 97 pgs: 97 active+clean; 648 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.5 KiB/s wr, 11 op/s; 20 B/s, 0 objects/s recovering 2024-09-16T02:02:51.008 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T02:02:51.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:50 smithi190 ceph-mon[34762]: pgmap v348: 97 pgs: 97 active+clean; 648 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.5 KiB/s wr, 11 op/s; 20 B/s, 0 objects/s recovering 2024-09-16T02:02:51.293 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:02:51.684 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:51 smithi105 ceph-mon[31197]: scrub queued for path: / 2024-09-16T02:02:51.684 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:51 smithi105 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-16T02:02:51.684 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:51 smithi105 ceph-mon[31197]: scrub summary: active paths [/] 2024-09-16T02:02:51.684 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:51 smithi105 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-16T02:02:51.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.765+0000 7f243c7ca640 1 -- 172.21.15.105:0/3169946829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2434100a80 msgr2=0x7f2434100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:51.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.765+0000 7f243c7ca640 1 --2- 172.21.15.105:0/3169946829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2434100a80 0x7f2434100ee0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2428009930 tx=0x7f242802a420 comp rx=0 tx=0).stop 2024-09-16T02:02:51.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.765+0000 7f243c7ca640 1 -- 172.21.15.105:0/3169946829 shutdown_connections 2024-09-16T02:02:51.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.765+0000 7f243c7ca640 1 --2- 172.21.15.105:0/3169946829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2434101420 0x7f243410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:51.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.765+0000 7f243c7ca640 1 --2- 172.21.15.105:0/3169946829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2434100a80 0x7f2434100ee0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:51.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.765+0000 7f243c7ca640 1 --2- 172.21.15.105:0/3169946829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2434106a80 0x7f2434106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:51.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.765+0000 7f243c7ca640 1 -- 172.21.15.105:0/3169946829 >> 172.21.15.105:0/3169946829 conn(0x7f24340fc7a0 msgr2=0x7f24340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:51.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.765+0000 7f243c7ca640 1 -- 172.21.15.105:0/3169946829 shutdown_connections 2024-09-16T02:02:51.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.765+0000 7f243c7ca640 1 -- 172.21.15.105:0/3169946829 wait complete. 2024-09-16T02:02:51.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.766+0000 7f243c7ca640 1 Processor -- start 2024-09-16T02:02:51.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.766+0000 7f243c7ca640 1 -- start start 2024-09-16T02:02:51.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.766+0000 7f243c7ca640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2434100a80 0x7f24341992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:51.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.766+0000 7f243c7ca640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2434101420 0x7f2434199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:51.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.766+0000 7f243c7ca640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2434106a80 0x7f24341a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:51.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.766+0000 7f243c7ca640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2434110d50 con 0x7f2434100a80 2024-09-16T02:02:51.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.766+0000 7f243c7ca640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2434110bd0 con 0x7f2434101420 2024-09-16T02:02:51.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.766+0000 7f243c7ca640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2434110ed0 con 0x7f2434106a80 2024-09-16T02:02:51.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.766+0000 7f243a566640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2434100a80 0x7f24341992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:51.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.766+0000 7f243a566640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2434100a80 0x7f24341992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51100/0 (socket says 172.21.15.105:51100) 2024-09-16T02:02:51.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.766+0000 7f243a566640 1 -- 172.21.15.105:0/3819152876 learned_addr learned my addr 172.21.15.105:0/3819152876 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:51.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.766+0000 7f2439d65640 1 --2- 172.21.15.105:0/3819152876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2434101420 0x7f2434199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:51.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.766+0000 7f243ad67640 1 --2- 172.21.15.105:0/3819152876 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2434106a80 0x7f24341a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:51.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.766+0000 7f243a566640 1 -- 172.21.15.105:0/3819152876 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2434106a80 msgr2=0x7f24341a4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:51.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.766+0000 7f243a566640 1 --2- 172.21.15.105:0/3819152876 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2434106a80 0x7f24341a4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:51.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.766+0000 7f243a566640 1 -- 172.21.15.105:0/3819152876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2434101420 msgr2=0x7f2434199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:51.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.766+0000 7f243a566640 1 --2- 172.21.15.105:0/3819152876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2434101420 0x7f2434199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:51.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.766+0000 7f243a566640 1 -- 172.21.15.105:0/3819152876 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24341a48a0 con 0x7f2434100a80 2024-09-16T02:02:51.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.766+0000 7f2439d65640 1 --2- 172.21.15.105:0/3819152876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2434101420 0x7f2434199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:02:51.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.766+0000 7f243a566640 1 --2- 172.21.15.105:0/3819152876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2434100a80 0x7f24341992e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f243000b4a0 tx=0x7f243000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:51.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.767+0000 7f24277fe640 1 -- 172.21.15.105:0/3819152876 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2430014070 con 0x7f2434100a80 2024-09-16T02:02:51.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.767+0000 7f24277fe640 1 -- 172.21.15.105:0/3819152876 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2430010ac0 con 0x7f2434100a80 2024-09-16T02:02:51.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.767+0000 7f243c7ca640 1 -- 172.21.15.105:0/3819152876 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24341a4a70 con 0x7f2434100a80 2024-09-16T02:02:51.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.767+0000 7f24277fe640 1 -- 172.21.15.105:0/3819152876 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2430017c30 con 0x7f2434100a80 2024-09-16T02:02:51.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.767+0000 7f243c7ca640 1 -- 172.21.15.105:0/3819152876 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24341a4f30 con 0x7f2434100a80 2024-09-16T02:02:51.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.769+0000 7f24277fe640 1 -- 172.21.15.105:0/3819152876 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f2430017070 con 0x7f2434100a80 2024-09-16T02:02:51.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.769+0000 7f243eba3740 1 Processor -- start 2024-09-16T02:02:51.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.770+0000 7f243eba3740 1 -- start start 2024-09-16T02:02:51.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.770+0000 7f243eba3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d722464a0 0x560d72246880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:51.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.770+0000 7f243eba3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560d72246dc0 0x560d72251140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:51.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.770+0000 7f243eba3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d72251680 0x560d72253a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:51.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.770+0000 7f243eba3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560d721ac2e0 con 0x560d722464a0 2024-09-16T02:02:51.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.770+0000 7f243eba3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560d72153ce0 con 0x560d72251680 2024-09-16T02:02:51.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.770+0000 7f243eba3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560d721a82a0 con 0x560d72246dc0 2024-09-16T02:02:51.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.770+0000 7f243ad67640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d722464a0 0x560d72246880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:51.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.770+0000 7f243ad67640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d722464a0 0x560d72246880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51116/0 (socket says 172.21.15.105:51116) 2024-09-16T02:02:51.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.770+0000 7f24277fe640 1 --2- 172.21.15.105:0/3819152876 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f24200a3240 0x7f24200a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:51.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.770+0000 7f243ad67640 1 -- 172.21.15.105:0/2141147141 learned_addr learned my addr 172.21.15.105:0/2141147141 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:51.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.770+0000 7f243ad67640 1 --2- 172.21.15.105:0/3819152876 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f24200a3240 0x7f24200a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:51.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.770+0000 7f2439d65640 1 --2- 172.21.15.105:0/2141147141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d72251680 0x560d72253a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:51.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.770+0000 7f24277fe640 1 -- 172.21.15.105:0/3819152876 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f24300cee70 con 0x7f2434100a80 2024-09-16T02:02:51.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.770+0000 7f243a566640 1 --2- 172.21.15.105:0/2141147141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560d72246dc0 0x560d72251140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:51.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.770+0000 7f243ad67640 1 -- 172.21.15.105:0/2141147141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560d72246dc0 msgr2=0x560d72251140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:51.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.770+0000 7f243ad67640 1 --2- 172.21.15.105:0/2141147141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560d72246dc0 0x560d72251140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:51.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.770+0000 7f243ad67640 1 -- 172.21.15.105:0/2141147141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d72251680 msgr2=0x560d72253a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:51.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.770+0000 7f243ad67640 1 --2- 172.21.15.105:0/2141147141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d72251680 0x560d72253a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:51.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.770+0000 7f243ad67640 1 -- 172.21.15.105:0/2141147141 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d72138440 con 0x560d722464a0 2024-09-16T02:02:51.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.770+0000 7f243a566640 1 --2- 172.21.15.105:0/2141147141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560d72246dc0 0x560d72251140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:02:51.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.770+0000 7f243ad67640 1 --2- 172.21.15.105:0/2141147141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d722464a0 0x560d72246880 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f242c013a70 tx=0x7f242c013f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=444eb8a1873d4fd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:51.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.771+0000 7f2424ff9640 1 -- 172.21.15.105:0/2141147141 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c013540 con 0x560d722464a0 2024-09-16T02:02:51.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.771+0000 7f2424ff9640 1 -- 172.21.15.105:0/2141147141 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f242c022070 con 0x560d722464a0 2024-09-16T02:02:51.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.771+0000 7f2424ff9640 1 -- 172.21.15.105:0/2141147141 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c01cc20 con 0x560d722464a0 2024-09-16T02:02:51.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.771+0000 7f243ad67640 1 --2- 172.21.15.105:0/3819152876 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f24200a3240 0x7f24200a5700 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f242c021000 tx=0x7f242c01f000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:51.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.771+0000 7f243eba3740 1 -- 172.21.15.105:0/2141147141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d722464a0 msgr2=0x560d72246880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:51.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.771+0000 7f243eba3740 1 --2- 172.21.15.105:0/2141147141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d722464a0 0x560d72246880 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f242c013a70 tx=0x7f242c013f30 comp rx=0 tx=0).stop 2024-09-16T02:02:51.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.771+0000 7f243eba3740 1 -- 172.21.15.105:0/2141147141 shutdown_connections 2024-09-16T02:02:51.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.771+0000 7f243eba3740 1 --2- 172.21.15.105:0/2141147141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560d72246dc0 0x560d72251140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:51.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.771+0000 7f243eba3740 1 --2- 172.21.15.105:0/2141147141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d72251680 0x560d72253a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:51.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.771+0000 7f243eba3740 1 --2- 172.21.15.105:0/2141147141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d722464a0 0x560d72246880 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:51.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.771+0000 7f243eba3740 1 -- 172.21.15.105:0/2141147141 >> 172.21.15.105:0/2141147141 conn(0x560d72242a30 msgr2=0x560d72242e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:51.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.772+0000 7f243eba3740 1 -- 172.21.15.105:0/2141147141 shutdown_connections 2024-09-16T02:02:51.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.772+0000 7f243eba3740 1 -- 172.21.15.105:0/2141147141 wait complete. 2024-09-16T02:02:51.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.772+0000 7f243eba3740 1 -- start start 2024-09-16T02:02:51.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.772+0000 7f243eba3740 1 Processor -- start 2024-09-16T02:02:51.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.772+0000 7f243eba3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560d722464a0 0x560d72250f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:51.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.772+0000 7f243eba3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d72246dc0 0x560d7224e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:51.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.772+0000 7f243eba3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d72251680 0x560d7224e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:51.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.772+0000 7f243eba3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560d72254250 con 0x560d72246dc0 2024-09-16T02:02:51.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.772+0000 7f243eba3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560d72167050 con 0x560d72251680 2024-09-16T02:02:51.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.772+0000 7f243eba3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560d722543d0 con 0x560d722464a0 2024-09-16T02:02:51.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.772+0000 7f243eba3740 10 client.0 _ll_register_callbacks cb 0x560d721a7160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:02:51.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.772+0000 7f2439d65640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d72246dc0 0x560d7224e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:51.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.772+0000 7f2439d65640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d72246dc0 0x560d7224e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51118/0 (socket says 172.21.15.105:51118) 2024-09-16T02:02:51.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.772+0000 7f2439d65640 1 -- 172.21.15.105:0/1640258165 learned_addr learned my addr 172.21.15.105:0/1640258165 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:51.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.772+0000 7f243ad67640 1 --2- 172.21.15.105:0/1640258165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d72251680 0x560d7224e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:51.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.772+0000 7f243a566640 1 --2- 172.21.15.105:0/1640258165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560d722464a0 0x560d72250f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:51.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.773+0000 7f2439d65640 1 -- 172.21.15.105:0/1640258165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560d722464a0 msgr2=0x560d72250f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:51.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.773+0000 7f2439d65640 1 --2- 172.21.15.105:0/1640258165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560d722464a0 0x560d72250f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:51.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.773+0000 7f2439d65640 1 -- 172.21.15.105:0/1640258165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d72251680 msgr2=0x560d7224e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:51.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.773+0000 7f2439d65640 1 --2- 172.21.15.105:0/1640258165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d72251680 0x560d7224e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:51.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.773+0000 7f2439d65640 1 -- 172.21.15.105:0/1640258165 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d71f1d3c0 con 0x560d72246dc0 2024-09-16T02:02:51.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.773+0000 7f2439d65640 1 --2- 172.21.15.105:0/1640258165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d72246dc0 0x560d7224e230 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2428031440 tx=0x7f2428031890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c73a06ac4a6904 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:51.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.773+0000 7f241dffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:02:51.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.773+0000 7f241dffb640 1 -- 172.21.15.105:0/1640258165 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2428035070 con 0x560d72246dc0 2024-09-16T02:02:51.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.773+0000 7f241dffb640 1 -- 172.21.15.105:0/1640258165 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f2428004240 con 0x560d72246dc0 2024-09-16T02:02:51.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.773+0000 7f243eba3740 1 -- 172.21.15.105:0/1640258165 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f242c022070 con 0x560d72246dc0 2024-09-16T02:02:51.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.773+0000 7f241dffb640 1 -- 172.21.15.105:0/1640258165 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2428004440 con 0x560d72246dc0 2024-09-16T02:02:51.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.773+0000 7f243eba3740 10 client.15342 fetch_fsmap learned FSMap version 28 2024-09-16T02:02:51.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.773+0000 7f243eba3740 1 -- 172.21.15.105:0/1640258165 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560d7225fb30 con 0x560d72246dc0 2024-09-16T02:02:51.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.773+0000 7f241dffb640 1 -- 172.21.15.105:0/1640258165 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f242c022070 con 0x560d72246dc0 2024-09-16T02:02:51.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.773+0000 7f241dffb640 1 -- 172.21.15.105:0/1640258165 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f242804d050 con 0x560d72246dc0 2024-09-16T02:02:51.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.774+0000 7f243eba3740 10 client.15342 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:02:51.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.774+0000 7f243eba3740 10 client.15342 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:02:51.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.774+0000 7f243eba3740 20 client.15342 populate_metadata read hostname 'smithi105' 2024-09-16T02:02:51.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.774+0000 7f243eba3740 1 --2- 172.21.15.105:0/1640258165 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x560d72260ff0 0x560d72263450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:51.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.774+0000 7f243eba3740 4 client.15342 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:02:51.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.774+0000 7f243eba3740 1 -- 172.21.15.105:0/1640258165 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d721b9850 con 0x560d72260ff0 2024-09-16T02:02:51.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.774+0000 7f243a566640 1 --2- 172.21.15.105:0/1640258165 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x560d72260ff0 0x560d72263450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:51.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.774+0000 7f241dffb640 10 client.15342 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:02:51.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.774+0000 7f243a566640 1 --2- 172.21.15.105:0/1640258165 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x560d72260ff0 0x560d72263450 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f490a1583462ba15 server_cookie=1d21897d950d97f5 in_seq=0 out_seq=0 2024-09-16T02:02:51.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.775+0000 7f241dffb640 1 -- 172.21.15.105:0/1640258165 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560d721b9850 con 0x560d72260ff0 2024-09-16T02:02:51.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.775+0000 7f241dffb640 10 client.15342 handle_command_reply: tid=0 2024-09-16T02:02:51.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.775+0000 7f243eba3740 1 client.15342 shutdown 2024-09-16T02:02:51.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.775+0000 7f243eba3740 1 -- 172.21.15.105:0/1640258165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d72246dc0 msgr2=0x560d7224e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:51.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.775+0000 7f243eba3740 1 --2- 172.21.15.105:0/1640258165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d72246dc0 0x560d7224e230 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2428031440 tx=0x7f2428031890 comp rx=0 tx=0).stop 2024-09-16T02:02:51.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.775+0000 7f243eba3740 1 -- 172.21.15.105:0/1640258165 shutdown_connections 2024-09-16T02:02:51.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.775+0000 7f243eba3740 1 --2- 172.21.15.105:0/1640258165 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x560d72260ff0 0x560d72263450 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:51.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.775+0000 7f243eba3740 1 --2- 172.21.15.105:0/1640258165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d72251680 0x560d7224e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:51.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.775+0000 7f243eba3740 1 --2- 172.21.15.105:0/1640258165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d72246dc0 0x560d7224e230 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:51.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.775+0000 7f241dffb640 0 client.15342 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:02:51.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.775+0000 7f243eba3740 1 --2- 172.21.15.105:0/1640258165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560d722464a0 0x560d72250f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:51.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.775+0000 7f243eba3740 1 -- 172.21.15.105:0/1640258165 >> 172.21.15.105:0/1640258165 conn(0x560d72242a30 msgr2=0x560d72241b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:51.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.775+0000 7f243eba3740 1 -- 172.21.15.105:0/1640258165 shutdown_connections 2024-09-16T02:02:51.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.775+0000 7f243eba3740 1 -- 172.21.15.105:0/1640258165 wait complete. 2024-09-16T02:02:51.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.775+0000 7f243eba3740 20 client.15342 trim_cache size 0 max 16384 2024-09-16T02:02:51.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.792+0000 7f243eba3740 1 Processor -- start 2024-09-16T02:02:51.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.793+0000 7f243eba3740 1 -- start start 2024-09-16T02:02:51.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.793+0000 7f243eba3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d722464a0 0x560d7225d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:51.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.793+0000 7f243eba3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d72246dc0 0x560d72268d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:51.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.793+0000 7f243eba3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560d72251680 0x560d72269260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:51.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.793+0000 7f243eba3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f24341ad6e0 con 0x560d72246dc0 2024-09-16T02:02:51.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.793+0000 7f243eba3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f24341af220 con 0x560d722464a0 2024-09-16T02:02:51.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.793+0000 7f243eba3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f24341addb0 con 0x560d72251680 2024-09-16T02:02:51.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.793+0000 7f2439d65640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d72246dc0 0x560d72268d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:51.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.793+0000 7f2439d65640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d72246dc0 0x560d72268d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51124/0 (socket says 172.21.15.105:51124) 2024-09-16T02:02:51.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.793+0000 7f2439d65640 1 -- 172.21.15.105:0/3729423899 learned_addr learned my addr 172.21.15.105:0/3729423899 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:51.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.793+0000 7f2439d65640 1 -- 172.21.15.105:0/3729423899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560d72251680 msgr2=0x560d72269260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:51.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.793+0000 7f2439d65640 1 --2- 172.21.15.105:0/3729423899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560d72251680 0x560d72269260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:51.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.793+0000 7f2439d65640 1 -- 172.21.15.105:0/3729423899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d722464a0 msgr2=0x560d7225d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:51.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.793+0000 7f2439d65640 1 --2- 172.21.15.105:0/3729423899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d722464a0 0x560d7225d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:51.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.793+0000 7f2439d65640 1 -- 172.21.15.105:0/3729423899 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d72250ba0 con 0x560d72246dc0 2024-09-16T02:02:51.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.793+0000 7f2439d65640 1 --2- 172.21.15.105:0/3729423899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d72246dc0 0x560d72268d20 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2428004ca0 tx=0x7f2428009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538055f2cf158d7e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:51.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.793+0000 7f241ffff640 1 -- 172.21.15.105:0/3729423899 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2428035040 con 0x560d72246dc0 2024-09-16T02:02:51.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.794+0000 7f241ffff640 1 -- 172.21.15.105:0/3729423899 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2428031db0 con 0x560d72246dc0 2024-09-16T02:02:51.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.794+0000 7f241ffff640 1 -- 172.21.15.105:0/3729423899 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24280053d0 con 0x560d72246dc0 2024-09-16T02:02:51.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.794+0000 7f243eba3740 1 -- 172.21.15.105:0/3729423899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d72246dc0 msgr2=0x560d72268d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:51.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.794+0000 7f243eba3740 1 --2- 172.21.15.105:0/3729423899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d72246dc0 0x560d72268d20 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2428004ca0 tx=0x7f2428009250 comp rx=0 tx=0).stop 2024-09-16T02:02:51.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.794+0000 7f243eba3740 1 -- 172.21.15.105:0/3729423899 shutdown_connections 2024-09-16T02:02:51.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.794+0000 7f243eba3740 1 --2- 172.21.15.105:0/3729423899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560d72251680 0x560d72269260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:51.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.794+0000 7f243eba3740 1 --2- 172.21.15.105:0/3729423899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d72246dc0 0x560d72268d20 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:51.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.794+0000 7f243eba3740 1 --2- 172.21.15.105:0/3729423899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d722464a0 0x560d7225d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:51.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.794+0000 7f243eba3740 1 -- 172.21.15.105:0/3729423899 >> 172.21.15.105:0/3729423899 conn(0x560d72242a30 msgr2=0x560d7224d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:51.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.794+0000 7f243eba3740 1 -- 172.21.15.105:0/3729423899 shutdown_connections 2024-09-16T02:02:51.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.794+0000 7f243eba3740 1 -- 172.21.15.105:0/3729423899 wait complete. 2024-09-16T02:02:51.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.795+0000 7f243eba3740 1 -- start start 2024-09-16T02:02:51.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.795+0000 7f243eba3740 1 Processor -- start 2024-09-16T02:02:51.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.795+0000 7f243eba3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560d722464a0 0x560d72259b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:51.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.795+0000 7f243eba3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d72246dc0 0x560d7224adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:51.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.795+0000 7f243eba3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d72251680 0x560d7224b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:51.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.795+0000 7f243eba3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f24341aeb50 con 0x560d72251680 2024-09-16T02:02:51.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.795+0000 7f243eba3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f24341ae480 con 0x560d72246dc0 2024-09-16T02:02:51.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.795+0000 7f243eba3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f24341ab6c0 con 0x560d722464a0 2024-09-16T02:02:51.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.795+0000 7f243eba3740 10 client.0 _ll_register_callbacks cb 0x7f2428009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:02:51.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.795+0000 7f243ad67640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d72251680 0x560d7224b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:51.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.795+0000 7f243ad67640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d72251680 0x560d7224b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51134/0 (socket says 172.21.15.105:51134) 2024-09-16T02:02:51.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.795+0000 7f243ad67640 1 -- 172.21.15.105:0/998691729 learned_addr learned my addr 172.21.15.105:0/998691729 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:02:51.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.795+0000 7f243a566640 1 --2- 172.21.15.105:0/998691729 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560d722464a0 0x560d72259b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:51.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.795+0000 7f2439d65640 1 --2- 172.21.15.105:0/998691729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d72246dc0 0x560d7224adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:51.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.795+0000 7f243ad67640 1 -- 172.21.15.105:0/998691729 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560d722464a0 msgr2=0x560d72259b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:51.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.795+0000 7f243ad67640 1 --2- 172.21.15.105:0/998691729 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560d722464a0 0x560d72259b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:51.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.795+0000 7f243ad67640 1 -- 172.21.15.105:0/998691729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d72246dc0 msgr2=0x560d7224adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:02:51.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.795+0000 7f243ad67640 1 --2- 172.21.15.105:0/998691729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d72246dc0 0x560d7224adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:51.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.795+0000 7f243ad67640 1 -- 172.21.15.105:0/998691729 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d7224b920 con 0x560d72251680 2024-09-16T02:02:51.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.795+0000 7f2439d65640 1 --2- 172.21.15.105:0/998691729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d72246dc0 0x560d7224adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:02:51.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.795+0000 7f243a566640 1 --2- 172.21.15.105:0/998691729 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560d722464a0 0x560d72259b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:02:51.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.796+0000 7f243ad67640 1 --2- 172.21.15.105:0/998691729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d72251680 0x560d7224b330 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f242c005c40 tx=0x7f242c009a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=501de6b675b56d08 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:51.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.796+0000 7f241dffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:02:51.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.796+0000 7f241dffb640 1 -- 172.21.15.105:0/998691729 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c034060 con 0x560d72251680 2024-09-16T02:02:51.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.796+0000 7f243eba3740 1 -- 172.21.15.105:0/998691729 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2428031db0 con 0x560d72251680 2024-09-16T02:02:51.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.796+0000 7f241dffb640 1 -- 172.21.15.105:0/998691729 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f242c013640 con 0x560d72251680 2024-09-16T02:02:51.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.796+0000 7f241dffb640 1 -- 172.21.15.105:0/998691729 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f242c011e10 con 0x560d72251680 2024-09-16T02:02:51.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.796+0000 7f243eba3740 10 client.15348 fetch_fsmap learned FSMap version 28 2024-09-16T02:02:51.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.796+0000 7f243eba3740 1 -- 172.21.15.105:0/998691729 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560d722cbb00 con 0x560d72251680 2024-09-16T02:02:51.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.796+0000 7f241dffb640 1 -- 172.21.15.105:0/998691729 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2428031db0 con 0x560d72251680 2024-09-16T02:02:51.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.796+0000 7f241dffb640 1 -- 172.21.15.105:0/998691729 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f242c038030 con 0x560d72251680 2024-09-16T02:02:51.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.796+0000 7f243eba3740 10 client.15348 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:02:51.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.796+0000 7f243eba3740 10 client.15348 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:02:51.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.796+0000 7f243eba3740 20 client.15348 populate_metadata read hostname 'smithi105' 2024-09-16T02:02:51.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.796+0000 7f243eba3740 1 --2- 172.21.15.105:0/998691729 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x560d72260ff0 0x560d722cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:51.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.796+0000 7f243eba3740 4 client.15348 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T02:02:51.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.796+0000 7f243eba3740 1 -- 172.21.15.105:0/998691729 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560d721b3d60 con 0x560d72260ff0 2024-09-16T02:02:51.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.797+0000 7f243a566640 1 --2- 172.21.15.105:0/998691729 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x560d72260ff0 0x560d722cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:51.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.797+0000 7f241dffb640 10 client.15348 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:02:51.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.797+0000 7f243a566640 1 --2- 172.21.15.105:0/998691729 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x560d72260ff0 0x560d722cdfb0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e015ab93dd341ae server_cookie=893efe9de882cbff in_seq=0 out_seq=0 2024-09-16T02:02:51.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.797+0000 7f241dffb640 1 -- 172.21.15.105:0/998691729 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560d721b3d60 con 0x560d72260ff0 2024-09-16T02:02:51.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.797+0000 7f241dffb640 10 client.15348 handle_command_reply: tid=0 2024-09-16T02:02:51.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.797+0000 7f243eba3740 1 client.15348 shutdown 2024-09-16T02:02:51.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.797+0000 7f243eba3740 1 -- 172.21.15.105:0/998691729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d72251680 msgr2=0x560d7224b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:51.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.797+0000 7f243eba3740 1 --2- 172.21.15.105:0/998691729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d72251680 0x560d7224b330 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f242c005c40 tx=0x7f242c009a80 comp rx=0 tx=0).stop 2024-09-16T02:02:51.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.797+0000 7f243eba3740 1 -- 172.21.15.105:0/998691729 shutdown_connections 2024-09-16T02:02:51.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.797+0000 7f243eba3740 1 --2- 172.21.15.105:0/998691729 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x560d72260ff0 0x560d722cdfb0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:51.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.797+0000 7f243eba3740 1 --2- 172.21.15.105:0/998691729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d72251680 0x560d7224b330 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:51.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.797+0000 7f243eba3740 1 --2- 172.21.15.105:0/998691729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d72246dc0 0x560d7224adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:51.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.797+0000 7f241dffb640 0 client.15348 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:02:51.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.797+0000 7f243eba3740 1 --2- 172.21.15.105:0/998691729 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560d722464a0 0x560d72259b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:51.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.797+0000 7f243eba3740 1 -- 172.21.15.105:0/998691729 >> 172.21.15.105:0/998691729 conn(0x560d72242a30 msgr2=0x560d72248f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:51.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.798+0000 7f243eba3740 1 -- 172.21.15.105:0/998691729 shutdown_connections 2024-09-16T02:02:51.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.798+0000 7f243eba3740 1 -- 172.21.15.105:0/998691729 wait complete. 2024-09-16T02:02:51.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.798+0000 7f243eba3740 20 client.15348 trim_cache size 0 max 16384 2024-09-16T02:02:51.821 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-16T02:02:51.821 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T02:02:51.821 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-09-16T02:02:51.821 INFO:teuthology.orchestra.run.smithi105.stdout: "7957fb27-356e-4393-bfb0-74fedd022abd": { 2024-09-16T02:02:51.821 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-09-16T02:02:51.821 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "7957fb27-356e-4393-bfb0-74fedd022abd", 2024-09-16T02:02:51.821 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-09-16T02:02:51.822 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:02:51.822 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:02:51.822 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-16T02:02:51.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.798+0000 7f243c7ca640 1 -- 172.21.15.105:0/3819152876 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f24200a3240 msgr2=0x7f24200a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:51.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.798+0000 7f243c7ca640 1 --2- 172.21.15.105:0/3819152876 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f24200a3240 0x7f24200a5700 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f242c021000 tx=0x7f242c01f000 comp rx=0 tx=0).stop 2024-09-16T02:02:51.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.798+0000 7f243c7ca640 1 -- 172.21.15.105:0/3819152876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2434100a80 msgr2=0x7f24341992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:51.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.798+0000 7f243c7ca640 1 --2- 172.21.15.105:0/3819152876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2434100a80 0x7f24341992e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f243000b4a0 tx=0x7f243000b960 comp rx=0 tx=0).stop 2024-09-16T02:02:51.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.798+0000 7f243c7ca640 1 -- 172.21.15.105:0/3819152876 shutdown_connections 2024-09-16T02:02:51.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.798+0000 7f243c7ca640 1 --2- 172.21.15.105:0/3819152876 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f24200a3240 0x7f24200a5700 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:51.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.798+0000 7f243c7ca640 1 --2- 172.21.15.105:0/3819152876 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2434106a80 0x7f24341a4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:51.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.798+0000 7f243c7ca640 1 --2- 172.21.15.105:0/3819152876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2434101420 0x7f2434199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:51.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.798+0000 7f243c7ca640 1 --2- 172.21.15.105:0/3819152876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2434100a80 0x7f24341992e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:51.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.798+0000 7f243c7ca640 1 -- 172.21.15.105:0/3819152876 >> 172.21.15.105:0/3819152876 conn(0x7f24340fc7a0 msgr2=0x7f24340fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:51.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.798+0000 7f243c7ca640 1 -- 172.21.15.105:0/3819152876 shutdown_connections 2024-09-16T02:02:51.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:02:51.798+0000 7f243c7ca640 1 -- 172.21.15.105:0/3819152876 wait complete. 2024-09-16T02:02:51.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:51 smithi137 ceph-mon[34099]: scrub queued for path: / 2024-09-16T02:02:51.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:51 smithi137 ceph-mon[34099]: scrub summary: idle+waiting paths [/] 2024-09-16T02:02:51.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:51 smithi137 ceph-mon[34099]: scrub summary: active paths [/] 2024-09-16T02:02:51.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:51 smithi137 ceph-mon[34099]: scrub summary: idle+waiting paths [/] 2024-09-16T02:02:52.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:51 smithi190 ceph-mon[34762]: scrub queued for path: / 2024-09-16T02:02:52.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:51 smithi190 ceph-mon[34762]: scrub summary: idle+waiting paths [/] 2024-09-16T02:02:52.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:51 smithi190 ceph-mon[34762]: scrub summary: active paths [/] 2024-09-16T02:02:52.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:51 smithi190 ceph-mon[34762]: scrub summary: idle+waiting paths [/] 2024-09-16T02:02:52.493 INFO:tasks.cephfs.filesystem:scrub status for tag:7957fb27-356e-4393-bfb0-74fedd022abd - {'path': '/', 'tag': '7957fb27-356e-4393-bfb0-74fedd022abd', 'options': 'recursive,force'} 2024-09-16T02:02:52.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:52 smithi105 ceph-mon[31197]: pgmap v349: 97 pgs: 97 active+clean; 648 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 11 op/s; 17 B/s, 0 objects/s recovering 2024-09-16T02:02:52.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:52 smithi137 ceph-mon[34099]: pgmap v349: 97 pgs: 97 active+clean; 648 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 11 op/s; 17 B/s, 0 objects/s recovering 2024-09-16T02:02:53.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:52 smithi190 ceph-mon[34762]: pgmap v349: 97 pgs: 97 active+clean; 648 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 11 op/s; 17 B/s, 0 objects/s recovering 2024-09-16T02:02:53.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:53 smithi137 ceph-mon[34099]: pgmap v350: 97 pgs: 97 active+clean; 655 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s; 15 B/s, 0 objects/s recovering 2024-09-16T02:02:54.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:53 smithi105 ceph-mon[31197]: pgmap v350: 97 pgs: 97 active+clean; 655 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s; 15 B/s, 0 objects/s recovering 2024-09-16T02:02:54.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:53 smithi190 ceph-mon[34762]: pgmap v350: 97 pgs: 97 active+clean; 655 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s; 15 B/s, 0 objects/s recovering 2024-09-16T02:02:56.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:56 smithi105 ceph-mon[31197]: pgmap v351: 97 pgs: 97 active+clean; 655 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.3 KiB/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T02:02:56.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:56 smithi190 ceph-mon[34762]: pgmap v351: 97 pgs: 97 active+clean; 655 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.3 KiB/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T02:02:56.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:56 smithi137 ceph-mon[34099]: pgmap v351: 97 pgs: 97 active+clean; 655 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.3 KiB/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T02:02:58.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:02:58 smithi105 ceph-mon[31197]: pgmap v352: 97 pgs: 97 active+clean; 655 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-16T02:02:58.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:02:58 smithi190 ceph-mon[34762]: pgmap v352: 97 pgs: 97 active+clean; 655 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-16T02:02:58.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:02:58 smithi137 ceph-mon[34099]: pgmap v352: 97 pgs: 97 active+clean; 655 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-16T02:03:00.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:00 smithi105 ceph-mon[31197]: pgmap v353: 97 pgs: 97 active+clean; 655 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-16T02:03:00.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:00 smithi190 ceph-mon[34762]: pgmap v353: 97 pgs: 97 active+clean; 655 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-16T02:03:00.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:00 smithi137 ceph-mon[34099]: pgmap v353: 97 pgs: 97 active+clean; 655 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-16T02:03:02.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:02 smithi105 ceph-mon[31197]: pgmap v354: 97 pgs: 97 active+clean; 655 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-16T02:03:02.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:02 smithi190 ceph-mon[34762]: pgmap v354: 97 pgs: 97 active+clean; 655 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-16T02:03:02.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:02 smithi137 ceph-mon[34099]: pgmap v354: 97 pgs: 97 active+clean; 655 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-16T02:03:04.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:04 smithi190 ceph-mon[34762]: pgmap v355: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-09-16T02:03:04.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:04 smithi137 ceph-mon[34099]: pgmap v355: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-09-16T02:03:05.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:04 smithi105 ceph-mon[31197]: pgmap v355: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-09-16T02:03:05.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:05 smithi190 ceph-mon[34762]: scrub summary: idle 2024-09-16T02:03:05.868 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:05 smithi105 ceph-mon[31197]: scrub summary: idle 2024-09-16T02:03:05.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:05 smithi137 ceph-mon[34099]: scrub summary: idle 2024-09-16T02:03:06.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:06 smithi190 ceph-mon[34762]: pgmap v356: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-16T02:03:06.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:06 smithi137 ceph-mon[34099]: pgmap v356: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-16T02:03:06.921 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:06 smithi105 ceph-mon[31197]: pgmap v356: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-16T02:03:08.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:08 smithi190 ceph-mon[34762]: pgmap v357: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-16T02:03:08.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:08 smithi137 ceph-mon[34099]: pgmap v357: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-16T02:03:09.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:08 smithi105 ceph-mon[31197]: pgmap v357: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-16T02:03:10.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:10 smithi190 ceph-mon[34762]: pgmap v358: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-16T02:03:10.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:10 smithi137 ceph-mon[34099]: pgmap v358: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-16T02:03:11.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:10 smithi105 ceph-mon[31197]: pgmap v358: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-16T02:03:12.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:12 smithi190 ceph-mon[34762]: pgmap v359: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-16T02:03:12.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:12 smithi137 ceph-mon[34099]: pgmap v359: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-16T02:03:13.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:12 smithi105 ceph-mon[31197]: pgmap v359: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-16T02:03:14.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:14 smithi190 ceph-mon[34762]: pgmap v360: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-16T02:03:14.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:14 smithi137 ceph-mon[34099]: pgmap v360: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-16T02:03:15.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:14 smithi105 ceph-mon[31197]: pgmap v360: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-16T02:03:16.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:16 smithi105 ceph-mon[31197]: pgmap v361: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T02:03:16.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:16 smithi190 ceph-mon[34762]: pgmap v361: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T02:03:16.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:16 smithi137 ceph-mon[34099]: pgmap v361: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T02:03:18.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:18 smithi190 ceph-mon[34762]: pgmap v362: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T02:03:18.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:18 smithi137 ceph-mon[34099]: pgmap v362: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T02:03:19.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:18 smithi105 ceph-mon[31197]: pgmap v362: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T02:03:20.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:20 smithi190 ceph-mon[34762]: pgmap v363: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-16T02:03:20.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:20 smithi137 ceph-mon[34099]: pgmap v363: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-16T02:03:21.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:20 smithi105 ceph-mon[31197]: pgmap v363: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-16T02:03:22.494 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T02:03:22.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:22 smithi105 ceph-mon[31197]: pgmap v364: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T02:03:22.811 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:03:22.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:22 smithi190 ceph-mon[34762]: pgmap v364: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T02:03:22.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:22 smithi137 ceph-mon[34099]: pgmap v364: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T02:03:23.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.324+0000 7fda57253640 1 -- 172.21.15.105:0/3022574346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda50103780 msgr2=0x7fda50107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:23.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.324+0000 7fda57253640 1 --2- 172.21.15.105:0/3022574346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda50103780 0x7fda50107bc0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fda40009930 tx=0x7fda4002a420 comp rx=0 tx=0).stop 2024-09-16T02:03:23.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.324+0000 7fda57253640 1 -- 172.21.15.105:0/3022574346 shutdown_connections 2024-09-16T02:03:23.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.324+0000 7fda57253640 1 --2- 172.21.15.105:0/3022574346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda50108300 0x7fda5010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:23.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.325+0000 7fda57253640 1 --2- 172.21.15.105:0/3022574346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda50103780 0x7fda50107bc0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:23.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.325+0000 7fda57253640 1 --2- 172.21.15.105:0/3022574346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda50102dd0 0x7fda501031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:23.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.325+0000 7fda57253640 1 -- 172.21.15.105:0/3022574346 >> 172.21.15.105:0/3022574346 conn(0x7fda500fc7d0 msgr2=0x7fda500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:23.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.325+0000 7fda57253640 1 -- 172.21.15.105:0/3022574346 shutdown_connections 2024-09-16T02:03:23.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.325+0000 7fda57253640 1 -- 172.21.15.105:0/3022574346 wait complete. 2024-09-16T02:03:23.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.325+0000 7fda57253640 1 Processor -- start 2024-09-16T02:03:23.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.326+0000 7fda57253640 1 -- start start 2024-09-16T02:03:23.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.326+0000 7fda57253640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda50102dd0 0x7fda501a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:23.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.326+0000 7fda57253640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda50103780 0x7fda501a4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:23.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.326+0000 7fda57253640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda50108300 0x7fda5010ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:23.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.326+0000 7fda57253640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fda50111f00 con 0x7fda50103780 2024-09-16T02:03:23.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.326+0000 7fda54fef640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda50102dd0 0x7fda501a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:23.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.326+0000 7fda47fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda50103780 0x7fda501a4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:23.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.326+0000 7fda57253640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fda50111d80 con 0x7fda50102dd0 2024-09-16T02:03:23.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.326+0000 7fda57253640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fda50112080 con 0x7fda50108300 2024-09-16T02:03:23.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.326+0000 7fda54fef640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda50102dd0 0x7fda501a4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.105:49034/0 (socket says 172.21.15.105:49034) 2024-09-16T02:03:23.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.327+0000 7fda54fef640 1 -- 172.21.15.105:0/1026683343 learned_addr learned my addr 172.21.15.105:0/1026683343 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:23.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.327+0000 7fda557f0640 1 --2- 172.21.15.105:0/1026683343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda50108300 0x7fda5010ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:23.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.327+0000 7fda47fff640 1 -- 172.21.15.105:0/1026683343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda50108300 msgr2=0x7fda5010ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:23.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.327+0000 7fda47fff640 1 --2- 172.21.15.105:0/1026683343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda50108300 0x7fda5010ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:23.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.327+0000 7fda47fff640 1 -- 172.21.15.105:0/1026683343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda50102dd0 msgr2=0x7fda501a4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:23.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.327+0000 7fda47fff640 1 --2- 172.21.15.105:0/1026683343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda50102dd0 0x7fda501a4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:23.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.327+0000 7fda47fff640 1 -- 172.21.15.105:0/1026683343 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda5010e4e0 con 0x7fda50103780 2024-09-16T02:03:23.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.327+0000 7fda47fff640 1 --2- 172.21.15.105:0/1026683343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda50103780 0x7fda501a4cd0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fda4002a940 tx=0x7fda400045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:23.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.328+0000 7fda45ffb640 1 -- 172.21.15.105:0/1026683343 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4003f070 con 0x7fda50103780 2024-09-16T02:03:23.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.328+0000 7fda45ffb640 1 -- 172.21.15.105:0/1026683343 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda40003ca0 con 0x7fda50103780 2024-09-16T02:03:23.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.328+0000 7fda45ffb640 1 -- 172.21.15.105:0/1026683343 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda40035070 con 0x7fda50103780 2024-09-16T02:03:23.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.328+0000 7fda57253640 1 -- 172.21.15.105:0/1026683343 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda5010e770 con 0x7fda50103780 2024-09-16T02:03:23.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.328+0000 7fda57253640 1 -- 172.21.15.105:0/1026683343 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda5010eb10 con 0x7fda50103780 2024-09-16T02:03:23.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.331+0000 7fda5962c740 1 Processor -- start 2024-09-16T02:03:23.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.331+0000 7fda5962c740 1 -- start start 2024-09-16T02:03:23.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.332+0000 7fda45ffb640 1 -- 172.21.15.105:0/1026683343 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fda4003b640 con 0x7fda50103780 2024-09-16T02:03:23.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.332+0000 7fda5962c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558efc4c74a0 0x558efc4c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:23.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.332+0000 7fda5962c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efc4c7dc0 0x558efc4d2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:23.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.332+0000 7fda5962c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558efc4d2680 0x558efc4d4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:23.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.332+0000 7fda5962c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558efc42d2e0 con 0x558efc4d2680 2024-09-16T02:03:23.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.332+0000 7fda5962c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558efc3d4ce0 con 0x558efc4c7dc0 2024-09-16T02:03:23.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.332+0000 7fda5962c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558efc4292a0 con 0x558efc4c74a0 2024-09-16T02:03:23.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.332+0000 7fda47fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558efc4d2680 0x558efc4d4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:23.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.332+0000 7fda47fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558efc4d2680 0x558efc4d4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35976/0 (socket says 172.21.15.105:35976) 2024-09-16T02:03:23.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.332+0000 7fda47fff640 1 -- 172.21.15.105:0/3656612588 learned_addr learned my addr 172.21.15.105:0/3656612588 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:23.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.332+0000 7fda54fef640 1 --2- 172.21.15.105:0/3656612588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efc4c7dc0 0x558efc4d2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:23.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.332+0000 7fda47fff640 1 -- 172.21.15.105:0/3656612588 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558efc4c74a0 msgr2=0x558efc4c7880 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T02:03:23.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.332+0000 7fda47fff640 1 --2- 172.21.15.105:0/3656612588 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558efc4c74a0 0x558efc4c7880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:23.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.332+0000 7fda47fff640 1 -- 172.21.15.105:0/3656612588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efc4c7dc0 msgr2=0x558efc4d2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:23.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.332+0000 7fda47fff640 1 --2- 172.21.15.105:0/3656612588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efc4c7dc0 0x558efc4d2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:23.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.332+0000 7fda47fff640 1 -- 172.21.15.105:0/3656612588 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558efc3b9440 con 0x558efc4d2680 2024-09-16T02:03:23.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.333+0000 7fda47fff640 1 --2- 172.21.15.105:0/3656612588 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558efc4d2680 0x558efc4d4a70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fda400f3ce0 tx=0x7fda400f3dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46567eb117d680e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:23.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.333+0000 7fda36ffd640 1 -- 172.21.15.105:0/3656612588 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda400f1030 con 0x558efc4d2680 2024-09-16T02:03:23.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.333+0000 7fda36ffd640 1 -- 172.21.15.105:0/3656612588 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda400fd070 con 0x558efc4d2680 2024-09-16T02:03:23.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.333+0000 7fda36ffd640 1 -- 172.21.15.105:0/3656612588 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda400f66a0 con 0x558efc4d2680 2024-09-16T02:03:23.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.334+0000 7fda45ffb640 1 --2- 172.21.15.105:0/1026683343 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fda380a3340 0x7fda380a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:23.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.334+0000 7fda557f0640 1 --2- 172.21.15.105:0/1026683343 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fda380a3340 0x7fda380a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:23.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.334+0000 7fda45ffb640 1 -- 172.21.15.105:0/1026683343 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fda400efdc0 con 0x7fda50103780 2024-09-16T02:03:23.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.334+0000 7fda5962c740 1 -- 172.21.15.105:0/3656612588 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558efc4d2680 msgr2=0x558efc4d4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:23.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.334+0000 7fda5962c740 1 --2- 172.21.15.105:0/3656612588 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558efc4d2680 0x558efc4d4a70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fda400f3ce0 tx=0x7fda400f3dd0 comp rx=0 tx=0).stop 2024-09-16T02:03:23.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.335+0000 7fda5962c740 1 -- 172.21.15.105:0/3656612588 shutdown_connections 2024-09-16T02:03:23.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.335+0000 7fda5962c740 1 --2- 172.21.15.105:0/3656612588 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558efc4d2680 0x558efc4d4a70 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:23.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.335+0000 7fda5962c740 1 --2- 172.21.15.105:0/3656612588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efc4c7dc0 0x558efc4d2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:23.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.335+0000 7fda5962c740 1 --2- 172.21.15.105:0/3656612588 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558efc4c74a0 0x558efc4c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:23.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.335+0000 7fda557f0640 1 --2- 172.21.15.105:0/1026683343 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fda380a3340 0x7fda380a5800 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fda48009370 tx=0x7fda480092a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:23.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.335+0000 7fda5962c740 1 -- 172.21.15.105:0/3656612588 >> 172.21.15.105:0/3656612588 conn(0x558efc4c3a30 msgr2=0x558efc4c3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:23.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.335+0000 7fda5962c740 1 -- 172.21.15.105:0/3656612588 shutdown_connections 2024-09-16T02:03:23.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.335+0000 7fda5962c740 1 -- 172.21.15.105:0/3656612588 wait complete. 2024-09-16T02:03:23.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.335+0000 7fda5962c740 1 -- start start 2024-09-16T02:03:23.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.335+0000 7fda5962c740 1 Processor -- start 2024-09-16T02:03:23.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.336+0000 7fda5962c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558efc4c74a0 0x558efc4d2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:23.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.336+0000 7fda5962c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efc4c7dc0 0x558efc4cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:23.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.336+0000 7fda5962c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558efc4d2680 0x558efc4cf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:23.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.336+0000 7fda5962c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558efc4d5010 con 0x558efc4d2680 2024-09-16T02:03:23.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.336+0000 7fda5962c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558efc3e8050 con 0x558efc4c7dc0 2024-09-16T02:03:23.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.336+0000 7fda5962c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558efc4d5190 con 0x558efc4c74a0 2024-09-16T02:03:23.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.336+0000 7fda54fef640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558efc4c74a0 0x558efc4d2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:23.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.336+0000 7fda5962c740 10 client.0 _ll_register_callbacks cb 0x558efc428160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:03:23.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.336+0000 7fda54fef640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558efc4c74a0 0x558efc4d2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.105:47738/0 (socket says 172.21.15.105:47738) 2024-09-16T02:03:23.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.336+0000 7fda557f0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558efc4d2680 0x558efc4cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:23.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.336+0000 7fda54fef640 1 -- 172.21.15.105:0/3550459872 learned_addr learned my addr 172.21.15.105:0/3550459872 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:23.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.336+0000 7fda47fff640 1 --2- 172.21.15.105:0/3550459872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efc4c7dc0 0x558efc4cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:23.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.336+0000 7fda54fef640 1 -- 172.21.15.105:0/3550459872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efc4c7dc0 msgr2=0x558efc4cf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:23.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.336+0000 7fda54fef640 1 --2- 172.21.15.105:0/3550459872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efc4c7dc0 0x558efc4cf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:23.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.336+0000 7fda54fef640 1 -- 172.21.15.105:0/3550459872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558efc4d2680 msgr2=0x558efc4cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:23.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.337+0000 7fda54fef640 1 --2- 172.21.15.105:0/3550459872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558efc4d2680 0x558efc4cf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:23.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.337+0000 7fda54fef640 1 -- 172.21.15.105:0/3550459872 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558efc19e3c0 con 0x558efc4c74a0 2024-09-16T02:03:23.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.337+0000 7fda557f0640 1 --2- 172.21.15.105:0/3550459872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558efc4d2680 0x558efc4cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:03:23.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.337+0000 7fda54fef640 1 --2- 172.21.15.105:0/3550459872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558efc4c74a0 0x558efc4d2000 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fda4c00bc20 tx=0x7fda4c0039c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a292aa418b876f62 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:23.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.337+0000 7fda2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-16T02:03:23.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.337+0000 7fda2ffff640 1 -- 172.21.15.105:0/3550459872 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4c011650 con 0x558efc4c74a0 2024-09-16T02:03:23.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.337+0000 7fda2ffff640 1 -- 172.21.15.105:0/3550459872 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fda4c0117f0 con 0x558efc4c74a0 2024-09-16T02:03:23.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.337+0000 7fda5962c740 1 -- 172.21.15.105:0/3550459872 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda400fd070 con 0x558efc4c74a0 2024-09-16T02:03:23.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.337+0000 7fda2ffff640 1 -- 172.21.15.105:0/3550459872 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fda4c0119f0 con 0x558efc4c74a0 2024-09-16T02:03:23.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.337+0000 7fda5962c740 10 client.24799 fetch_fsmap learned FSMap version 28 2024-09-16T02:03:23.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.337+0000 7fda5962c740 1 -- 172.21.15.105:0/3550459872 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558efc4e09d0 con 0x558efc4c74a0 2024-09-16T02:03:23.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.338+0000 7fda2ffff640 1 -- 172.21.15.105:0/3550459872 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda400fd070 con 0x558efc4c74a0 2024-09-16T02:03:23.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.338+0000 7fda2ffff640 1 -- 172.21.15.105:0/3550459872 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fda4c01bbe0 con 0x558efc4c74a0 2024-09-16T02:03:23.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.338+0000 7fda5962c740 10 client.24799 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:03:23.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.338+0000 7fda5962c740 10 client.24799 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:03:23.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.338+0000 7fda5962c740 20 client.24799 populate_metadata read hostname 'smithi105' 2024-09-16T02:03:23.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.338+0000 7fda5962c740 1 --2- 172.21.15.105:0/3550459872 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x558efc4e1e90 0x558efc4e42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:23.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.338+0000 7fda5962c740 4 client.24799 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:03:23.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.338+0000 7fda5962c740 1 -- 172.21.15.105:0/3550459872 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558efc43a850 con 0x558efc4e1e90 2024-09-16T02:03:23.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.338+0000 7fda47fff640 1 --2- 172.21.15.105:0/3550459872 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x558efc4e1e90 0x558efc4e42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:23.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.339+0000 7fda2ffff640 10 client.24799 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:03:23.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.339+0000 7fda47fff640 1 --2- 172.21.15.105:0/3550459872 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x558efc4e1e90 0x558efc4e42a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c24f9f54e99a275 server_cookie=660b06a723991230 in_seq=0 out_seq=0 2024-09-16T02:03:23.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.340+0000 7fda2ffff640 1 -- 172.21.15.105:0/3550459872 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558efc43a850 con 0x558efc4e1e90 2024-09-16T02:03:23.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.340+0000 7fda2ffff640 10 client.24799 handle_command_reply: tid=0 2024-09-16T02:03:23.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.340+0000 7fda5962c740 1 client.24799 shutdown 2024-09-16T02:03:23.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.340+0000 7fda5962c740 1 -- 172.21.15.105:0/3550459872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558efc4c74a0 msgr2=0x558efc4d2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:23.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.340+0000 7fda5962c740 1 --2- 172.21.15.105:0/3550459872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558efc4c74a0 0x558efc4d2000 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fda4c00bc20 tx=0x7fda4c0039c0 comp rx=0 tx=0).stop 2024-09-16T02:03:23.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.340+0000 7fda5962c740 1 -- 172.21.15.105:0/3550459872 shutdown_connections 2024-09-16T02:03:23.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.340+0000 7fda5962c740 1 --2- 172.21.15.105:0/3550459872 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x558efc4e1e90 0x558efc4e42a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:23.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.340+0000 7fda2ffff640 0 client.24799 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:03:23.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.340+0000 7fda5962c740 1 --2- 172.21.15.105:0/3550459872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558efc4d2680 0x558efc4cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:23.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.340+0000 7fda5962c740 1 --2- 172.21.15.105:0/3550459872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efc4c7dc0 0x558efc4cf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:23.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.340+0000 7fda5962c740 1 --2- 172.21.15.105:0/3550459872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558efc4c74a0 0x558efc4d2000 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:23.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.340+0000 7fda5962c740 1 -- 172.21.15.105:0/3550459872 >> 172.21.15.105:0/3550459872 conn(0x558efc4c3a30 msgr2=0x558efc4c9710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:23.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.340+0000 7fda5962c740 1 -- 172.21.15.105:0/3550459872 shutdown_connections 2024-09-16T02:03:23.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.340+0000 7fda5962c740 1 -- 172.21.15.105:0/3550459872 wait complete. 2024-09-16T02:03:23.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.340+0000 7fda5962c740 20 client.24799 trim_cache size 0 max 16384 2024-09-16T02:03:23.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.373+0000 7fda5962c740 1 Processor -- start 2024-09-16T02:03:23.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.374+0000 7fda5962c740 1 -- start start 2024-09-16T02:03:23.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.374+0000 7fda5962c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558efc4c74a0 0x558efc4de1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:23.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.374+0000 7fda5962c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558efc4c7dc0 0x558efc4cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:23.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.374+0000 7fda5962c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efc4d2680 0x558efc4e9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:23.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.374+0000 7fda5962c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fda501b3c40 con 0x558efc4c7dc0 2024-09-16T02:03:23.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.374+0000 7fda5962c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fda501b5780 con 0x558efc4d2680 2024-09-16T02:03:23.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.374+0000 7fda5962c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fda501b4310 con 0x558efc4c74a0 2024-09-16T02:03:23.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.374+0000 7fda47fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558efc4c7dc0 0x558efc4cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:23.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.374+0000 7fda47fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558efc4c7dc0 0x558efc4cdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35992/0 (socket says 172.21.15.105:35992) 2024-09-16T02:03:23.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.374+0000 7fda47fff640 1 -- 172.21.15.105:0/2649692413 learned_addr learned my addr 172.21.15.105:0/2649692413 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:23.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.374+0000 7fda54fef640 1 --2- 172.21.15.105:0/2649692413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558efc4c74a0 0x558efc4de1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:23.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.374+0000 7fda47fff640 1 -- 172.21.15.105:0/2649692413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558efc4c74a0 msgr2=0x558efc4de1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:23.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.374+0000 7fda47fff640 1 --2- 172.21.15.105:0/2649692413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558efc4c74a0 0x558efc4de1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:23.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.374+0000 7fda47fff640 1 -- 172.21.15.105:0/2649692413 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efc4d2680 msgr2=0x558efc4e9b70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T02:03:23.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.374+0000 7fda47fff640 1 --2- 172.21.15.105:0/2649692413 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efc4d2680 0x558efc4e9b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:23.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.374+0000 7fda47fff640 1 -- 172.21.15.105:0/2649692413 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558efc4ce260 con 0x558efc4c7dc0 2024-09-16T02:03:23.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.375+0000 7fda47fff640 1 --2- 172.21.15.105:0/2649692413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558efc4c7dc0 0x558efc4cdb80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fda400f3fd0 tx=0x7fda400f3dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b180c3f836f36b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:23.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.375+0000 7fda367fc640 1 -- 172.21.15.105:0/2649692413 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda400f1070 con 0x558efc4c7dc0 2024-09-16T02:03:23.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.375+0000 7fda367fc640 1 -- 172.21.15.105:0/2649692413 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda400fd040 con 0x558efc4c7dc0 2024-09-16T02:03:23.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.375+0000 7fda367fc640 1 -- 172.21.15.105:0/2649692413 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda40048040 con 0x558efc4c7dc0 2024-09-16T02:03:23.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.376+0000 7fda5962c740 1 -- 172.21.15.105:0/2649692413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558efc4c7dc0 msgr2=0x558efc4cdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:23.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.376+0000 7fda5962c740 1 --2- 172.21.15.105:0/2649692413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558efc4c7dc0 0x558efc4cdb80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fda400f3fd0 tx=0x7fda400f3dd0 comp rx=0 tx=0).stop 2024-09-16T02:03:23.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.376+0000 7fda5962c740 1 -- 172.21.15.105:0/2649692413 shutdown_connections 2024-09-16T02:03:23.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.376+0000 7fda5962c740 1 --2- 172.21.15.105:0/2649692413 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efc4d2680 0x558efc4e9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:23.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.376+0000 7fda5962c740 1 --2- 172.21.15.105:0/2649692413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558efc4c7dc0 0x558efc4cdb80 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:23.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.376+0000 7fda5962c740 1 --2- 172.21.15.105:0/2649692413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558efc4c74a0 0x558efc4de1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:23.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.376+0000 7fda5962c740 1 -- 172.21.15.105:0/2649692413 >> 172.21.15.105:0/2649692413 conn(0x558efc4c3a30 msgr2=0x558efc4ca8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:23.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.376+0000 7fda5962c740 1 -- 172.21.15.105:0/2649692413 shutdown_connections 2024-09-16T02:03:23.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.376+0000 7fda5962c740 1 -- 172.21.15.105:0/2649692413 wait complete. 2024-09-16T02:03:23.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.377+0000 7fda5962c740 1 -- start start 2024-09-16T02:03:23.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.377+0000 7fda5962c740 1 Processor -- start 2024-09-16T02:03:23.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.377+0000 7fda5962c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efc4c74a0 0x558efc4d6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:23.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.377+0000 7fda5962c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558efc4c7dc0 0x558efc4d66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:23.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.377+0000 7fda5962c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558efc4d2680 0x558efc4d6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:23.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.377+0000 7fda5962c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fda501b50b0 con 0x558efc4d2680 2024-09-16T02:03:23.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.377+0000 7fda5962c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fda501b49e0 con 0x558efc4c74a0 2024-09-16T02:03:23.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.377+0000 7fda5962c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fda501b1c20 con 0x558efc4c7dc0 2024-09-16T02:03:23.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.377+0000 7fda5962c740 10 client.0 _ll_register_callbacks cb 0x7fda4c004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:03:23.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.377+0000 7fda557f0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558efc4d2680 0x558efc4d6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:23.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.377+0000 7fda557f0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558efc4d2680 0x558efc4d6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35998/0 (socket says 172.21.15.105:35998) 2024-09-16T02:03:23.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.377+0000 7fda557f0640 1 -- 172.21.15.105:0/281194752 learned_addr learned my addr 172.21.15.105:0/281194752 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:23.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.378+0000 7fda54fef640 1 --2- 172.21.15.105:0/281194752 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efc4c74a0 0x558efc4d6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:23.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.378+0000 7fda47fff640 1 --2- 172.21.15.105:0/281194752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558efc4c7dc0 0x558efc4d66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:23.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.378+0000 7fda557f0640 1 -- 172.21.15.105:0/281194752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558efc4c7dc0 msgr2=0x558efc4d66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:23.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.378+0000 7fda557f0640 1 --2- 172.21.15.105:0/281194752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558efc4c7dc0 0x558efc4d66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:23.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.378+0000 7fda557f0640 1 -- 172.21.15.105:0/281194752 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efc4c74a0 msgr2=0x558efc4d6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:23.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.378+0000 7fda557f0640 1 --2- 172.21.15.105:0/281194752 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efc4c74a0 0x558efc4d6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:23.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.378+0000 7fda557f0640 1 -- 172.21.15.105:0/281194752 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558efc4cfba0 con 0x558efc4d2680 2024-09-16T02:03:23.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.378+0000 7fda54fef640 1 --2- 172.21.15.105:0/281194752 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efc4c74a0 0x558efc4d6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:03:23.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.378+0000 7fda557f0640 1 --2- 172.21.15.105:0/281194752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558efc4d2680 0x558efc4d6be0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fda48011c00 tx=0x7fda48004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f57191c6135417 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:23.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.378+0000 7fda2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:03:23.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.378+0000 7fda2ffff640 1 -- 172.21.15.105:0/281194752 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda48010c60 con 0x558efc4d2680 2024-09-16T02:03:23.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.378+0000 7fda2ffff640 1 -- 172.21.15.105:0/281194752 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fda48010e00 con 0x558efc4d2680 2024-09-16T02:03:23.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.379+0000 7fda5962c740 1 -- 172.21.15.105:0/281194752 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda400fd040 con 0x558efc4d2680 2024-09-16T02:03:23.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.379+0000 7fda2ffff640 1 -- 172.21.15.105:0/281194752 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fda48028470 con 0x558efc4d2680 2024-09-16T02:03:23.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.379+0000 7fda5962c740 10 client.15366 fetch_fsmap learned FSMap version 28 2024-09-16T02:03:23.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.379+0000 7fda5962c740 1 -- 172.21.15.105:0/281194752 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558efc54cda0 con 0x558efc4d2680 2024-09-16T02:03:23.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.379+0000 7fda2ffff640 1 -- 172.21.15.105:0/281194752 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda400fd040 con 0x558efc4d2680 2024-09-16T02:03:23.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.379+0000 7fda2ffff640 1 -- 172.21.15.105:0/281194752 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fda4801f800 con 0x558efc4d2680 2024-09-16T02:03:23.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.379+0000 7fda5962c740 10 client.15366 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:03:23.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.379+0000 7fda5962c740 10 client.15366 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:03:23.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.379+0000 7fda5962c740 20 client.15366 populate_metadata read hostname 'smithi105' 2024-09-16T02:03:23.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.379+0000 7fda5962c740 1 --2- 172.21.15.105:0/281194752 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x558efc4e1e90 0x558efc54f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:23.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.379+0000 7fda5962c740 4 client.15366 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T02:03:23.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.379+0000 7fda5962c740 1 -- 172.21.15.105:0/281194752 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558efc434d60 con 0x558efc4e1e90 2024-09-16T02:03:23.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.380+0000 7fda54fef640 1 --2- 172.21.15.105:0/281194752 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x558efc4e1e90 0x558efc54f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:23.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.380+0000 7fda2ffff640 10 client.15366 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:03:23.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.380+0000 7fda54fef640 1 --2- 172.21.15.105:0/281194752 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x558efc4e1e90 0x558efc54f560 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22cdff49f92f5064 server_cookie=817ba3bfb28d34c in_seq=0 out_seq=0 2024-09-16T02:03:23.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.380+0000 7fda2ffff640 1 -- 172.21.15.105:0/281194752 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558efc434d60 con 0x558efc4e1e90 2024-09-16T02:03:23.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.380+0000 7fda2ffff640 10 client.15366 handle_command_reply: tid=0 2024-09-16T02:03:23.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.380+0000 7fda5962c740 1 client.15366 shutdown 2024-09-16T02:03:23.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.381+0000 7fda5962c740 1 -- 172.21.15.105:0/281194752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558efc4d2680 msgr2=0x558efc4d6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:23.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.381+0000 7fda5962c740 1 --2- 172.21.15.105:0/281194752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558efc4d2680 0x558efc4d6be0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fda48011c00 tx=0x7fda48004a40 comp rx=0 tx=0).stop 2024-09-16T02:03:23.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.381+0000 7fda5962c740 1 -- 172.21.15.105:0/281194752 shutdown_connections 2024-09-16T02:03:23.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.381+0000 7fda5962c740 1 --2- 172.21.15.105:0/281194752 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x558efc4e1e90 0x558efc54f560 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:23.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.381+0000 7fda5962c740 1 --2- 172.21.15.105:0/281194752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558efc4c7dc0 0x558efc4d66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:23.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.381+0000 7fda5962c740 1 --2- 172.21.15.105:0/281194752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558efc4d2680 0x558efc4d6be0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:23.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.381+0000 7fda5962c740 1 --2- 172.21.15.105:0/281194752 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efc4c74a0 0x558efc4d6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:23.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.381+0000 7fda5962c740 1 -- 172.21.15.105:0/281194752 >> 172.21.15.105:0/281194752 conn(0x558efc4c3a30 msgr2=0x558efc4ca8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:23.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.381+0000 7fda2ffff640 0 client.15366 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:03:23.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.381+0000 7fda5962c740 1 -- 172.21.15.105:0/281194752 shutdown_connections 2024-09-16T02:03:23.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.381+0000 7fda5962c740 1 -- 172.21.15.105:0/281194752 wait complete. 2024-09-16T02:03:23.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.381+0000 7fda5962c740 20 client.15366 trim_cache size 0 max 16384 2024-09-16T02:03:23.388 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-16T02:03:23.388 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-09-16T02:03:23.388 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-09-16T02:03:23.389 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-16T02:03:23.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.382+0000 7fda57253640 1 -- 172.21.15.105:0/1026683343 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fda380a3340 msgr2=0x7fda380a5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:23.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.382+0000 7fda57253640 1 --2- 172.21.15.105:0/1026683343 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fda380a3340 0x7fda380a5800 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fda48009370 tx=0x7fda480092a0 comp rx=0 tx=0).stop 2024-09-16T02:03:23.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.382+0000 7fda57253640 1 -- 172.21.15.105:0/1026683343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda50103780 msgr2=0x7fda501a4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:23.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.382+0000 7fda57253640 1 --2- 172.21.15.105:0/1026683343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda50103780 0x7fda501a4cd0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fda4002a940 tx=0x7fda400045e0 comp rx=0 tx=0).stop 2024-09-16T02:03:23.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.382+0000 7fda57253640 1 -- 172.21.15.105:0/1026683343 shutdown_connections 2024-09-16T02:03:23.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.382+0000 7fda57253640 1 --2- 172.21.15.105:0/1026683343 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fda380a3340 0x7fda380a5800 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:23.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.382+0000 7fda57253640 1 --2- 172.21.15.105:0/1026683343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda50108300 0x7fda5010ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:23.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.382+0000 7fda57253640 1 --2- 172.21.15.105:0/1026683343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda50103780 0x7fda501a4cd0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:23.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.382+0000 7fda57253640 1 --2- 172.21.15.105:0/1026683343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda50102dd0 0x7fda501a4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:23.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.382+0000 7fda57253640 1 -- 172.21.15.105:0/1026683343 >> 172.21.15.105:0/1026683343 conn(0x7fda500fc7d0 msgr2=0x7fda50103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:23.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.382+0000 7fda57253640 1 -- 172.21.15.105:0/1026683343 shutdown_connections 2024-09-16T02:03:23.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:23.382+0000 7fda57253640 1 -- 172.21.15.105:0/1026683343 wait complete. 2024-09-16T02:03:24.256 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T02:03:24.256 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:03:24.578 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:03:24.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:24 smithi190 ceph-mon[34762]: pgmap v365: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-16T02:03:24.886 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:24 smithi105 ceph-mon[31197]: pgmap v365: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-16T02:03:24.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:24 smithi137 ceph-mon[34099]: pgmap v365: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-16T02:03:25.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.175+0000 7ff7811eb640 1 -- 172.21.15.105:0/2212534733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff77c102ef0 msgr2=0x7ff77c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:25.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.175+0000 7ff7811eb640 1 --2- 172.21.15.105:0/2212534733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff77c102ef0 0x7ff77c10da40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff774009ca0 tx=0x7ff77402a950 comp rx=0 tx=0).stop 2024-09-16T02:03:25.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.175+0000 7ff7811eb640 1 -- 172.21.15.105:0/2212534733 shutdown_connections 2024-09-16T02:03:25.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.175+0000 7ff7811eb640 1 --2- 172.21.15.105:0/2212534733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff77c102570 0x7ff77c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:25.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.175+0000 7ff7811eb640 1 --2- 172.21.15.105:0/2212534733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff77c102ef0 0x7ff77c10da40 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:25.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.175+0000 7ff7811eb640 1 --2- 172.21.15.105:0/2212534733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff77c100e20 0x7ff77c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:25.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.175+0000 7ff7811eb640 1 -- 172.21.15.105:0/2212534733 >> 172.21.15.105:0/2212534733 conn(0x7ff77c0fc7d0 msgr2=0x7ff77c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:25.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.175+0000 7ff7811eb640 1 -- 172.21.15.105:0/2212534733 shutdown_connections 2024-09-16T02:03:25.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.175+0000 7ff7811eb640 1 -- 172.21.15.105:0/2212534733 wait complete. 2024-09-16T02:03:25.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.176+0000 7ff7811eb640 1 Processor -- start 2024-09-16T02:03:25.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.176+0000 7ff7811eb640 1 -- start start 2024-09-16T02:03:25.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.176+0000 7ff7811eb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff77c100e20 0x7ff77c19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:25.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.177+0000 7ff7811eb640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff77c102570 0x7ff77c19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:25.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.177+0000 7ff7811eb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff77c102ef0 0x7ff77c19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:25.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.177+0000 7ff7811eb640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff77c10fd30 con 0x7ff77c102ef0 2024-09-16T02:03:25.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.177+0000 7ff77b59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff77c102ef0 0x7ff77c19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:25.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.177+0000 7ff77b59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff77c102ef0 0x7ff77c19ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36014/0 (socket says 172.21.15.105:36014) 2024-09-16T02:03:25.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.177+0000 7ff77b59e640 1 -- 172.21.15.105:0/2759422367 learned_addr learned my addr 172.21.15.105:0/2759422367 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:25.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.177+0000 7ff77259c640 1 --2- 172.21.15.105:0/2759422367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff77c102570 0x7ff77c19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:25.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.177+0000 7ff7811eb640 1 -- 172.21.15.105:0/2759422367 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff77c10fbb0 con 0x7ff77c100e20 2024-09-16T02:03:25.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.177+0000 7ff7811eb640 1 -- 172.21.15.105:0/2759422367 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff77c10feb0 con 0x7ff77c102570 2024-09-16T02:03:25.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.177+0000 7ff77ad9d640 1 --2- 172.21.15.105:0/2759422367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff77c100e20 0x7ff77c19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:25.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.178+0000 7ff77259c640 1 -- 172.21.15.105:0/2759422367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff77c100e20 msgr2=0x7ff77c19a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:25.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.178+0000 7ff77259c640 1 --2- 172.21.15.105:0/2759422367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff77c100e20 0x7ff77c19a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:25.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.178+0000 7ff77259c640 1 -- 172.21.15.105:0/2759422367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff77c102ef0 msgr2=0x7ff77c19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:25.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.178+0000 7ff77259c640 1 --2- 172.21.15.105:0/2759422367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff77c102ef0 0x7ff77c19ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:25.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.178+0000 7ff77259c640 1 -- 172.21.15.105:0/2759422367 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff77c19f640 con 0x7ff77c102570 2024-09-16T02:03:25.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.178+0000 7ff77b59e640 1 --2- 172.21.15.105:0/2759422367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff77c102ef0 0x7ff77c19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:03:25.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.178+0000 7ff77259c640 1 --2- 172.21.15.105:0/2759422367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff77c102570 0x7ff77c19abd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff76400e9f0 tx=0x7ff76400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:25.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.178+0000 7ff778d99640 1 -- 172.21.15.105:0/2759422367 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76400ccc0 con 0x7ff77c102570 2024-09-16T02:03:25.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.178+0000 7ff7811eb640 1 -- 172.21.15.105:0/2759422367 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff77c19f930 con 0x7ff77c102570 2024-09-16T02:03:25.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.178+0000 7ff778d99640 1 -- 172.21.15.105:0/2759422367 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff764005020 con 0x7ff77c102570 2024-09-16T02:03:25.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.179+0000 7ff778d99640 1 -- 172.21.15.105:0/2759422367 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff764004a00 con 0x7ff77c102570 2024-09-16T02:03:25.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.179+0000 7ff7811eb640 1 -- 172.21.15.105:0/2759422367 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff77c1a7210 con 0x7ff77c102570 2024-09-16T02:03:25.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.180+0000 7ff7811eb640 1 -- 172.21.15.105:0/2759422367 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff77c101200 con 0x7ff77c102570 2024-09-16T02:03:25.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.183+0000 7ff778d99640 1 -- 172.21.15.105:0/2759422367 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7ff764004ba0 con 0x7ff77c102570 2024-09-16T02:03:25.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.184+0000 7ff778d99640 1 --2- 172.21.15.105:0/2759422367 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff7680a32b0 0x7ff7680a5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:25.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.184+0000 7ff778d99640 1 -- 172.21.15.105:0/2759422367 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7ff7640cf8e0 con 0x7ff77c102570 2024-09-16T02:03:25.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.184+0000 7ff77ad9d640 1 --2- 172.21.15.105:0/2759422367 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff7680a32b0 0x7ff7680a5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:25.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.185+0000 7ff77ad9d640 1 --2- 172.21.15.105:0/2759422367 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff7680a32b0 0x7ff7680a5770 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff76c009c20 tx=0x7ff76c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:25.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.186+0000 7ff778d99640 1 -- 172.21.15.105:0/2759422367 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff7640051d0 con 0x7ff77c102570 2024-09-16T02:03:25.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.603+0000 7ff7811eb640 1 -- 172.21.15.105:0/2759422367 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff77c19b9f0 con 0x7ff77c102570 2024-09-16T02:03:25.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.605+0000 7ff778d99640 1 -- 172.21.15.105:0/2759422367 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11991 (secure 0 0 0) 0x7ff77c19b9f0 con 0x7ff77c102570 2024-09-16T02:03:25.607 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:03:25.607 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":28,"btime":"2024-09-16T02:02:41:554178+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2057699920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2057699920},{"type":"v1","addr":"172.21.15.105:6835","nonce":2057699920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1764886948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1764886948},{"type":"v1","addr":"172.21.15.105:6837","nonce":1764886948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T02:00:35.112266+0000","modified":"2024-09-16T02:02:40.885614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24469,"mds_2":24461,"mds_3":24457,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/4112062361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":4112062361},{"type":"v1","addr":"172.21.15.105:6839","nonce":4112062361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6841/163156514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":163156514},{"type":"v1","addr":"172.21.15.105:6841","nonce":163156514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/683519097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":683519097},{"type":"v1","addr":"172.21.15.190:6833","nonce":683519097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3025442862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3025442862},{"type":"v1","addr":"172.21.15.137:6833","nonce":3025442862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/763783602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":763783602},{"type":"v1","addr":"172.21.15.137:6835","nonce":763783602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.190:6835/1851042281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1851042281},{"type":"v1","addr":"172.21.15.190:6835","nonce":1851042281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/3181257549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3181257549},{"type":"v1","addr":"172.21.15.190:6837","nonce":3181257549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6837/2788541030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2788541030},{"type":"v1","addr":"172.21.15.137:6837","nonce":2788541030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.190:6839/1222573537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":1222573537},{"type":"v1","addr":"172.21.15.190:6839","nonce":1222573537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/3454924009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3454924009},{"type":"v1","addr":"172.21.15.137:6839","nonce":3454924009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24445,24457,24461,24469,24482]},"id":1}]} 2024-09-16T02:03:25.607 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 28 2024-09-16T02:03:25.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.607+0000 7ff7811eb640 1 -- 172.21.15.105:0/2759422367 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff7680a32b0 msgr2=0x7ff7680a5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:25.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.607+0000 7ff7811eb640 1 --2- 172.21.15.105:0/2759422367 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff7680a32b0 0x7ff7680a5770 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff76c009c20 tx=0x7ff76c005f90 comp rx=0 tx=0).stop 2024-09-16T02:03:25.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.607+0000 7ff7811eb640 1 -- 172.21.15.105:0/2759422367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff77c102570 msgr2=0x7ff77c19abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:25.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.607+0000 7ff7811eb640 1 --2- 172.21.15.105:0/2759422367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff77c102570 0x7ff77c19abd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff76400e9f0 tx=0x7ff76400eeb0 comp rx=0 tx=0).stop 2024-09-16T02:03:25.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.608+0000 7ff7811eb640 1 -- 172.21.15.105:0/2759422367 shutdown_connections 2024-09-16T02:03:25.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.608+0000 7ff7811eb640 1 --2- 172.21.15.105:0/2759422367 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff7680a32b0 0x7ff7680a5770 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:25.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.608+0000 7ff7811eb640 1 --2- 172.21.15.105:0/2759422367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff77c102570 0x7ff77c19abd0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:25.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.608+0000 7ff7811eb640 1 --2- 172.21.15.105:0/2759422367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff77c102ef0 0x7ff77c19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:25.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.608+0000 7ff7811eb640 1 --2- 172.21.15.105:0/2759422367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff77c100e20 0x7ff77c19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:25.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.608+0000 7ff7811eb640 1 -- 172.21.15.105:0/2759422367 >> 172.21.15.105:0/2759422367 conn(0x7ff77c0fc7d0 msgr2=0x7ff77c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:25.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.608+0000 7ff7811eb640 1 -- 172.21.15.105:0/2759422367 shutdown_connections 2024-09-16T02:03:25.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:25.608+0000 7ff7811eb640 1 -- 172.21.15.105:0/2759422367 wait complete. 2024-09-16T02:03:26.346 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-16T02:03:26.625 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:03:26.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:26 smithi190 ceph-mon[34762]: pgmap v366: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T02:03:26.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:26 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/2759422367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:03:26.904 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:26 smithi105 ceph-mon[31197]: pgmap v366: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T02:03:26.904 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:26 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2759422367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:03:26.920 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:26 smithi137 ceph-mon[34099]: pgmap v366: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T02:03:26.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:26 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/2759422367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:03:27.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.162+0000 7fd0ae6b6640 1 -- 172.21.15.105:0/3842973343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0a81082a0 msgr2=0x7fd0a810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:27.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.162+0000 7fd0ae6b6640 1 --2- 172.21.15.105:0/3842973343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0a81082a0 0x7fd0a810ec40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009cd0 tx=0x7fd0a002a910 comp rx=0 tx=0).stop 2024-09-16T02:03:27.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.162+0000 7fd0ae6b6640 1 -- 172.21.15.105:0/3842973343 shutdown_connections 2024-09-16T02:03:27.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.162+0000 7fd0ae6b6640 1 --2- 172.21.15.105:0/3842973343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0a81082a0 0x7fd0a810ec40 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:27.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.162+0000 7fd0ae6b6640 1 --2- 172.21.15.105:0/3842973343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0a81036e0 0x7fd0a8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:27.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.162+0000 7fd0ae6b6640 1 --2- 172.21.15.105:0/3842973343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0a8102d30 0x7fd0a8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:27.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.162+0000 7fd0ae6b6640 1 -- 172.21.15.105:0/3842973343 >> 172.21.15.105:0/3842973343 conn(0x7fd0a80fc7d0 msgr2=0x7fd0a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:27.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.162+0000 7fd0ae6b6640 1 -- 172.21.15.105:0/3842973343 shutdown_connections 2024-09-16T02:03:27.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.162+0000 7fd0ae6b6640 1 -- 172.21.15.105:0/3842973343 wait complete. 2024-09-16T02:03:27.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.163+0000 7fd0ae6b6640 1 Processor -- start 2024-09-16T02:03:27.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.163+0000 7fd0ae6b6640 1 -- start start 2024-09-16T02:03:27.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.164+0000 7fd0ae6b6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0a8102d30 0x7fd0a81985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:27.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.164+0000 7fd0ae6b6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0a81036e0 0x7fd0a8198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:27.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.164+0000 7fd0ae6b6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0a81082a0 0x7fd0a819ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:27.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.164+0000 7fd0ae6b6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a8112060 con 0x7fd0a81036e0 2024-09-16T02:03:27.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.164+0000 7fd0ae6b6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a8111ee0 con 0x7fd0a81082a0 2024-09-16T02:03:27.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.164+0000 7fd0ae6b6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a81121e0 con 0x7fd0a8102d30 2024-09-16T02:03:27.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.164+0000 7fd0a77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0a81036e0 0x7fd0a8198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:27.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.164+0000 7fd0acc53640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0a81082a0 0x7fd0a819ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:27.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.164+0000 7fd0acc53640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0a81082a0 0x7fd0a819ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.105:49096/0 (socket says 172.21.15.105:49096) 2024-09-16T02:03:27.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.164+0000 7fd0a77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0a81036e0 0x7fd0a8198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36038/0 (socket says 172.21.15.105:36038) 2024-09-16T02:03:27.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.164+0000 7fd0acc53640 1 -- 172.21.15.105:0/3630018554 learned_addr learned my addr 172.21.15.105:0/3630018554 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:27.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.164+0000 7fd0a77fe640 1 -- 172.21.15.105:0/3630018554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0a8102d30 msgr2=0x7fd0a81985e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T02:03:27.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.164+0000 7fd0a77fe640 1 --2- 172.21.15.105:0/3630018554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0a8102d30 0x7fd0a81985e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:27.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.164+0000 7fd0a77fe640 1 -- 172.21.15.105:0/3630018554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0a81082a0 msgr2=0x7fd0a819ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:27.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.164+0000 7fd0a77fe640 1 --2- 172.21.15.105:0/3630018554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0a81082a0 0x7fd0a819ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:27.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.164+0000 7fd0a77fe640 1 -- 172.21.15.105:0/3630018554 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a819d590 con 0x7fd0a81036e0 2024-09-16T02:03:27.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.165+0000 7fd0a77fe640 1 --2- 172.21.15.105:0/3630018554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0a81036e0 0x7fd0a8198b20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd0a80697d0 tx=0x7fd09c00b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:27.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.165+0000 7fd0a57fa640 1 -- 172.21.15.105:0/3630018554 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c015020 con 0x7fd0a81036e0 2024-09-16T02:03:27.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.165+0000 7fd0ae6b6640 1 -- 172.21.15.105:0/3630018554 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a819d880 con 0x7fd0a81036e0 2024-09-16T02:03:27.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.165+0000 7fd0a57fa640 1 -- 172.21.15.105:0/3630018554 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd09c011ce0 con 0x7fd0a81036e0 2024-09-16T02:03:27.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.165+0000 7fd0a57fa640 1 -- 172.21.15.105:0/3630018554 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c010da0 con 0x7fd0a81036e0 2024-09-16T02:03:27.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.165+0000 7fd0ae6b6640 1 -- 172.21.15.105:0/3630018554 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a81a5160 con 0x7fd0a81036e0 2024-09-16T02:03:27.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.168+0000 7fd0a57fa640 1 -- 172.21.15.105:0/3630018554 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fd09c018020 con 0x7fd0a81036e0 2024-09-16T02:03:27.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.168+0000 7fd0a57fa640 1 --2- 172.21.15.105:0/3630018554 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fd0900a32c0 0x7fd0900a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:27.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.168+0000 7fd0b0a8f740 1 Processor -- start 2024-09-16T02:03:27.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.169+0000 7fd0a57fa640 1 -- 172.21.15.105:0/3630018554 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fd09c0cef70 con 0x7fd0a81036e0 2024-09-16T02:03:27.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.169+0000 7fd0a7fff640 1 --2- 172.21.15.105:0/3630018554 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fd0900a32c0 0x7fd0900a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:27.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.169+0000 7fd0b0a8f740 1 -- start start 2024-09-16T02:03:27.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.169+0000 7fd0b0a8f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a15c192b40 0x55a15c191090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:27.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.169+0000 7fd0b0a8f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a15c1915d0 0x55a15c191a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:27.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.169+0000 7fd0b0a8f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a15c1a0780 0x55a15c1a2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:27.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.169+0000 7fd0b0a8f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a15c0f72f0 con 0x55a15c192b40 2024-09-16T02:03:27.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.169+0000 7fd0b0a8f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a15c09ecf0 con 0x55a15c1a0780 2024-09-16T02:03:27.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.169+0000 7fd0b0a8f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a15c0f32b0 con 0x55a15c1915d0 2024-09-16T02:03:27.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.169+0000 7fd0a77fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a15c1915d0 0x55a15c191a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:27.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.169+0000 7fd0a77fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a15c1915d0 0x55a15c191a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.105:47806/0 (socket says 172.21.15.105:47806) 2024-09-16T02:03:27.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.169+0000 7fd0a77fe640 1 -- 172.21.15.105:0/1005616238 learned_addr learned my addr 172.21.15.105:0/1005616238 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:27.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.169+0000 7fd0acc53640 1 --2- 172.21.15.105:0/1005616238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a15c1a0780 0x55a15c1a2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:27.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.170+0000 7fd0acc53640 1 -- 172.21.15.105:0/1005616238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a15c1915d0 msgr2=0x55a15c191a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:27.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.170+0000 7fd0acc53640 1 --2- 172.21.15.105:0/1005616238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a15c1915d0 0x55a15c191a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:27.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.170+0000 7fd0acc53640 1 -- 172.21.15.105:0/1005616238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a15c192b40 msgr2=0x55a15c191090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:27.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.170+0000 7fd0acc53640 1 --2- 172.21.15.105:0/1005616238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a15c192b40 0x55a15c191090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:27.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.170+0000 7fd0acc53640 1 -- 172.21.15.105:0/1005616238 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a15c0833a0 con 0x55a15c1a0780 2024-09-16T02:03:27.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.170+0000 7fd0acc53640 1 --2- 172.21.15.105:0/1005616238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a15c1a0780 0x55a15c1a2b70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0006010 tx=0x7fd0a0005fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bca1ce17d33ea99e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:27.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.170+0000 7fd0a7fff640 1 --2- 172.21.15.105:0/3630018554 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fd0900a32c0 0x7fd0900a5780 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd098009740 tx=0x7fd098009b40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:27.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.170+0000 7fd08e7fc640 1 -- 172.21.15.105:0/1005616238 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0031460 con 0x55a15c1a0780 2024-09-16T02:03:27.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.170+0000 7fd08e7fc640 1 -- 172.21.15.105:0/1005616238 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0a0036030 con 0x55a15c1a0780 2024-09-16T02:03:27.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.170+0000 7fd08e7fc640 1 -- 172.21.15.105:0/1005616238 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0004440 con 0x55a15c1a0780 2024-09-16T02:03:27.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.171+0000 7fd0b0a8f740 1 -- 172.21.15.105:0/1005616238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a15c1a0780 msgr2=0x55a15c1a2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:27.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.171+0000 7fd0b0a8f740 1 --2- 172.21.15.105:0/1005616238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a15c1a0780 0x55a15c1a2b70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0006010 tx=0x7fd0a0005fa0 comp rx=0 tx=0).stop 2024-09-16T02:03:27.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.171+0000 7fd0b0a8f740 1 -- 172.21.15.105:0/1005616238 shutdown_connections 2024-09-16T02:03:27.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.171+0000 7fd0b0a8f740 1 --2- 172.21.15.105:0/1005616238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a15c1915d0 0x55a15c191a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:27.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.171+0000 7fd0b0a8f740 1 --2- 172.21.15.105:0/1005616238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a15c1a0780 0x55a15c1a2b70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:27.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.171+0000 7fd0b0a8f740 1 --2- 172.21.15.105:0/1005616238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a15c192b40 0x55a15c191090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:27.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.171+0000 7fd0b0a8f740 1 -- 172.21.15.105:0/1005616238 >> 172.21.15.105:0/1005616238 conn(0x55a15c15a310 msgr2=0x55a15c18dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:27.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.171+0000 7fd0b0a8f740 1 -- 172.21.15.105:0/1005616238 shutdown_connections 2024-09-16T02:03:27.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.171+0000 7fd0b0a8f740 1 -- 172.21.15.105:0/1005616238 wait complete. 2024-09-16T02:03:27.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.172+0000 7fd0b0a8f740 1 -- start start 2024-09-16T02:03:27.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.172+0000 7fd0b0a8f740 1 Processor -- start 2024-09-16T02:03:27.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.172+0000 7fd0b0a8f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a15c1915d0 0x55a15c1a0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:27.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.172+0000 7fd0b0a8f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a15c192b40 0x55a15c19d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:27.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.172+0000 7fd0b0a8f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a15c1a0780 0x55a15c19d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:27.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.172+0000 7fd0acc53640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a15c1a0780 0x55a15c19d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:27.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.173+0000 7fd0acc53640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a15c1a0780 0x55a15c19d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36044/0 (socket says 172.21.15.105:36044) 2024-09-16T02:03:27.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.173+0000 7fd0acc53640 1 -- 172.21.15.105:0/3689590202 learned_addr learned my addr 172.21.15.105:0/3689590202 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:27.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.173+0000 7fd0b0a8f740 1 -- 172.21.15.105:0/3689590202 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a15c1a3420 con 0x55a15c1a0780 2024-09-16T02:03:27.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.173+0000 7fd0b0a8f740 1 -- 172.21.15.105:0/3689590202 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a15c0b2060 con 0x55a15c1915d0 2024-09-16T02:03:27.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.173+0000 7fd0a7fff640 1 --2- 172.21.15.105:0/3689590202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a15c1915d0 0x55a15c1a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:27.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.173+0000 7fd0b0a8f740 1 -- 172.21.15.105:0/3689590202 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a15c1a35a0 con 0x55a15c192b40 2024-09-16T02:03:27.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.173+0000 7fd0a77fe640 1 --2- 172.21.15.105:0/3689590202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a15c192b40 0x55a15c19d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:27.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.173+0000 7fd0a7fff640 1 -- 172.21.15.105:0/3689590202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a15c192b40 msgr2=0x55a15c19d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:27.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.173+0000 7fd0a7fff640 1 --2- 172.21.15.105:0/3689590202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a15c192b40 0x55a15c19d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:27.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.173+0000 7fd0a7fff640 1 -- 172.21.15.105:0/3689590202 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a15c1a0780 msgr2=0x55a15c19d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:27.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.173+0000 7fd0a7fff640 1 --2- 172.21.15.105:0/3689590202 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a15c1a0780 0x55a15c19d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:27.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.173+0000 7fd0a7fff640 1 -- 172.21.15.105:0/3689590202 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a15be683c0 con 0x55a15c1915d0 2024-09-16T02:03:27.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.173+0000 7fd0a77fe640 1 --2- 172.21.15.105:0/3689590202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a15c192b40 0x55a15c19d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:03:27.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.173+0000 7fd0b0a8f740 10 client.0 _ll_register_callbacks cb 0x55a15be058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:03:27.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.173+0000 7fd0a7fff640 1 --2- 172.21.15.105:0/3689590202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a15c1915d0 0x55a15c1a0110 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd098006fd0 tx=0x7fd0980046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2928fa4a5ded268e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:27.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.173+0000 7fd0877fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-16T02:03:27.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.173+0000 7fd0b0a8f740 1 -- 172.21.15.105:0/3689590202 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0a0036030 con 0x55a15c1915d0 2024-09-16T02:03:27.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.174+0000 7fd0877fe640 1 -- 172.21.15.105:0/3689590202 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09800f040 con 0x55a15c1915d0 2024-09-16T02:03:27.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.174+0000 7fd0877fe640 1 -- 172.21.15.105:0/3689590202 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fd098017de0 con 0x55a15c1915d0 2024-09-16T02:03:27.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.174+0000 7fd0877fe640 1 -- 172.21.15.105:0/3689590202 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd098015430 con 0x55a15c1915d0 2024-09-16T02:03:27.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.174+0000 7fd0b0a8f740 10 client.24839 fetch_fsmap learned FSMap version 28 2024-09-16T02:03:27.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.174+0000 7fd0b0a8f740 1 -- 172.21.15.105:0/3689590202 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55a15c1aed10 con 0x55a15c1915d0 2024-09-16T02:03:27.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.174+0000 7fd0877fe640 1 -- 172.21.15.105:0/3689590202 <== mon.2 v2:172.21.15.137:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd0980155d0 con 0x55a15c1915d0 2024-09-16T02:03:27.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.174+0000 7fd0b0a8f740 10 client.24839 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:03:27.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.174+0000 7fd0b0a8f740 10 client.24839 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T02:03:27.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.174+0000 7fd0b0a8f740 20 client.24839 populate_metadata read hostname 'smithi105' 2024-09-16T02:03:27.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.174+0000 7fd0b0a8f740 1 --2- 172.21.15.105:0/3689590202 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x55a15c1b0130 0x55a15c1b25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:27.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.174+0000 7fd0b0a8f740 4 client.24839 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:03:27.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.174+0000 7fd0b0a8f740 1 -- 172.21.15.105:0/3689590202 --> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a15c104860 con 0x55a15c1b0130 2024-09-16T02:03:27.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.175+0000 7fd0a77fe640 1 --2- 172.21.15.105:0/3689590202 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x55a15c1b0130 0x55a15c1b25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:27.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.175+0000 7fd0877fe640 10 client.24839 ms_handle_connect on v2:172.21.15.190:6834/1851042281 2024-09-16T02:03:27.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.175+0000 7fd0a77fe640 1 --2- 172.21.15.105:0/3689590202 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x55a15c1b0130 0x55a15c1b25c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1153a0c1cd701181 server_cookie=77fe1a77b700d204 in_seq=0 out_seq=0 2024-09-16T02:03:27.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.176+0000 7fd0877fe640 1 -- 172.21.15.105:0/3689590202 <== mds.4 v2:172.21.15.190:6834/1851042281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a15c104860 con 0x55a15c1b0130 2024-09-16T02:03:27.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.176+0000 7fd0877fe640 10 client.24839 handle_command_reply: tid=0 2024-09-16T02:03:27.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.176+0000 7fd0b0a8f740 1 client.24839 shutdown 2024-09-16T02:03:27.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.176+0000 7fd0b0a8f740 1 -- 172.21.15.105:0/3689590202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a15c1915d0 msgr2=0x55a15c1a0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:27.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.176+0000 7fd0b0a8f740 1 --2- 172.21.15.105:0/3689590202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a15c1915d0 0x55a15c1a0110 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd098006fd0 tx=0x7fd0980046c0 comp rx=0 tx=0).stop 2024-09-16T02:03:27.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.176+0000 7fd0b0a8f740 1 -- 172.21.15.105:0/3689590202 shutdown_connections 2024-09-16T02:03:27.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.176+0000 7fd0b0a8f740 1 --2- 172.21.15.105:0/3689590202 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x55a15c1b0130 0x55a15c1b25c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:27.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.176+0000 7fd0877fe640 0 client.24839 ms_handle_reset on v2:172.21.15.190:6834/1851042281 2024-09-16T02:03:27.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.176+0000 7fd0b0a8f740 1 --2- 172.21.15.105:0/3689590202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a15c192b40 0x55a15c19d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:27.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.176+0000 7fd0b0a8f740 1 --2- 172.21.15.105:0/3689590202 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a15c1a0780 0x55a15c19d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:27.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.176+0000 7fd0b0a8f740 1 --2- 172.21.15.105:0/3689590202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a15c1915d0 0x55a15c1a0110 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:27.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.176+0000 7fd0b0a8f740 1 -- 172.21.15.105:0/3689590202 >> 172.21.15.105:0/3689590202 conn(0x55a15c15a310 msgr2=0x55a15c193960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:27.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.176+0000 7fd0b0a8f740 1 -- 172.21.15.105:0/3689590202 shutdown_connections 2024-09-16T02:03:27.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.176+0000 7fd0b0a8f740 1 -- 172.21.15.105:0/3689590202 wait complete. 2024-09-16T02:03:27.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.176+0000 7fd0b0a8f740 20 client.24839 trim_cache size 0 max 16384 2024-09-16T02:03:27.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.189+0000 7fd0b0a8f740 1 Processor -- start 2024-09-16T02:03:27.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.189+0000 7fd0b0a8f740 1 -- start start 2024-09-16T02:03:27.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.189+0000 7fd0b0a8f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a15c1915d0 0x55a15c1ac6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:27.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.189+0000 7fd0b0a8f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a15c192b40 0x55a15c19d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:27.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.189+0000 7fd0b0a8f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a15c1a0780 0x55a15c1b7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:27.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.190+0000 7fd0b0a8f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a81ab4a0 con 0x55a15c192b40 2024-09-16T02:03:27.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.190+0000 7fd0b0a8f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a81acfe0 con 0x55a15c1a0780 2024-09-16T02:03:27.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.190+0000 7fd0b0a8f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a81abb70 con 0x55a15c1915d0 2024-09-16T02:03:27.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.190+0000 7fd0a77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a15c192b40 0x55a15c19d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:27.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.190+0000 7fd0a77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a15c192b40 0x55a15c19d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36050/0 (socket says 172.21.15.105:36050) 2024-09-16T02:03:27.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.190+0000 7fd0a77fe640 1 -- 172.21.15.105:0/2706178526 learned_addr learned my addr 172.21.15.105:0/2706178526 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:27.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.190+0000 7fd0a7fff640 1 --2- 172.21.15.105:0/2706178526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a15c1915d0 0x55a15c1ac6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:27.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.190+0000 7fd0acc53640 1 --2- 172.21.15.105:0/2706178526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a15c1a0780 0x55a15c1b7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:27.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.190+0000 7fd0a77fe640 1 -- 172.21.15.105:0/2706178526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a15c1915d0 msgr2=0x55a15c1ac6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:27.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.190+0000 7fd0a77fe640 1 --2- 172.21.15.105:0/2706178526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a15c1915d0 0x55a15c1ac6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:27.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.190+0000 7fd0a77fe640 1 -- 172.21.15.105:0/2706178526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a15c1a0780 msgr2=0x55a15c1b7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:27.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.190+0000 7fd0a77fe640 1 --2- 172.21.15.105:0/2706178526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a15c1a0780 0x55a15c1b7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:27.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.190+0000 7fd0a77fe640 1 -- 172.21.15.105:0/2706178526 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a15c18e960 con 0x55a15c192b40 2024-09-16T02:03:27.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.190+0000 7fd0acc53640 1 --2- 172.21.15.105:0/2706178526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a15c1a0780 0x55a15c1b7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:03:27.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.191+0000 7fd0a7fff640 1 --2- 172.21.15.105:0/2706178526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a15c1915d0 0x55a15c1ac6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:03:27.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.191+0000 7fd0a77fe640 1 --2- 172.21.15.105:0/2706178526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a15c192b40 0x55a15c19d010 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd09c097f00 tx=0x7fd09c0cc8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e9571a5305b9fe6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:27.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.191+0000 7fd08dffb640 1 -- 172.21.15.105:0/2706178526 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c0bb040 con 0x55a15c192b40 2024-09-16T02:03:27.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.192+0000 7fd08dffb640 1 -- 172.21.15.105:0/2706178526 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd09c09f8e0 con 0x55a15c192b40 2024-09-16T02:03:27.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.192+0000 7fd08dffb640 1 -- 172.21.15.105:0/2706178526 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c0b4270 con 0x55a15c192b40 2024-09-16T02:03:27.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.192+0000 7fd0b0a8f740 1 -- 172.21.15.105:0/2706178526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a15c192b40 msgr2=0x55a15c19d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:27.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.192+0000 7fd0b0a8f740 1 --2- 172.21.15.105:0/2706178526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a15c192b40 0x55a15c19d010 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd09c097f00 tx=0x7fd09c0cc8e0 comp rx=0 tx=0).stop 2024-09-16T02:03:27.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.192+0000 7fd0b0a8f740 1 -- 172.21.15.105:0/2706178526 shutdown_connections 2024-09-16T02:03:27.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.192+0000 7fd0b0a8f740 1 --2- 172.21.15.105:0/2706178526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a15c1a0780 0x55a15c1b7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:27.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.192+0000 7fd0b0a8f740 1 --2- 172.21.15.105:0/2706178526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a15c192b40 0x55a15c19d010 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:27.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.192+0000 7fd0b0a8f740 1 --2- 172.21.15.105:0/2706178526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a15c1915d0 0x55a15c1ac6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:27.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.192+0000 7fd0b0a8f740 1 -- 172.21.15.105:0/2706178526 >> 172.21.15.105:0/2706178526 conn(0x55a15c15a310 msgr2=0x55a15c19c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:27.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.193+0000 7fd0b0a8f740 1 -- 172.21.15.105:0/2706178526 shutdown_connections 2024-09-16T02:03:27.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.193+0000 7fd0b0a8f740 1 -- 172.21.15.105:0/2706178526 wait complete. 2024-09-16T02:03:27.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.193+0000 7fd0b0a8f740 1 -- start start 2024-09-16T02:03:27.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.193+0000 7fd0b0a8f740 1 Processor -- start 2024-09-16T02:03:27.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.194+0000 7fd0b0a8f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a15c1915d0 0x55a15c197da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:27.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.194+0000 7fd0b0a8f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a15c192b40 0x55a15c1a8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:27.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.194+0000 7fd0b0a8f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a15c1a0780 0x55a15c1a8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:27.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.194+0000 7fd0b0a8f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a81ac910 con 0x55a15c1a0780 2024-09-16T02:03:27.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.194+0000 7fd0b0a8f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a81ac240 con 0x55a15c1915d0 2024-09-16T02:03:27.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.194+0000 7fd0b0a8f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a81a9480 con 0x55a15c192b40 2024-09-16T02:03:27.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.194+0000 7fd0b0a8f740 10 client.0 _ll_register_callbacks cb 0x7fd098006c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:03:27.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.194+0000 7fd0a77fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a15c192b40 0x55a15c1a8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:27.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.194+0000 7fd0a77fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a15c192b40 0x55a15c1a8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.105:47824/0 (socket says 172.21.15.105:47824) 2024-09-16T02:03:27.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.194+0000 7fd0a77fe640 1 -- 172.21.15.105:0/2102425711 learned_addr learned my addr 172.21.15.105:0/2102425711 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:27.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.194+0000 7fd0acc53640 1 --2- 172.21.15.105:0/2102425711 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a15c1a0780 0x55a15c1a8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:27.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.195+0000 7fd0a77fe640 1 -- 172.21.15.105:0/2102425711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a15c1915d0 msgr2=0x55a15c197da0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T02:03:27.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.195+0000 7fd0a77fe640 1 --2- 172.21.15.105:0/2102425711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a15c1915d0 0x55a15c197da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:27.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.195+0000 7fd0a77fe640 1 -- 172.21.15.105:0/2102425711 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a15c1a0780 msgr2=0x55a15c1a8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:27.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.195+0000 7fd0a77fe640 1 --2- 172.21.15.105:0/2102425711 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a15c1a0780 0x55a15c1a8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:27.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.195+0000 7fd0a77fe640 1 -- 172.21.15.105:0/2102425711 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a15c191be0 con 0x55a15c192b40 2024-09-16T02:03:27.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.195+0000 7fd0acc53640 1 --2- 172.21.15.105:0/2102425711 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a15c1a0780 0x55a15c1a8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:03:27.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.195+0000 7fd0a77fe640 1 --2- 172.21.15.105:0/2102425711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a15c192b40 0x55a15c1a8830 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd09c09b4c0 tx=0x7fd09c099a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26a7e6a6179048ef server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:27.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.195+0000 7fd0877fe640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-16T02:03:27.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.195+0000 7fd0877fe640 1 -- 172.21.15.105:0/2102425711 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c0b8070 con 0x55a15c192b40 2024-09-16T02:03:27.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.196+0000 7fd0877fe640 1 -- 172.21.15.105:0/2102425711 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fd09c0b4240 con 0x55a15c192b40 2024-09-16T02:03:27.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.196+0000 7fd0b0a8f740 1 -- 172.21.15.105:0/2102425711 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd09c09f8e0 con 0x55a15c192b40 2024-09-16T02:03:27.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.196+0000 7fd0877fe640 1 -- 172.21.15.105:0/2102425711 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd09c0bb040 con 0x55a15c192b40 2024-09-16T02:03:27.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.196+0000 7fd0b0a8f740 10 client.24820 fetch_fsmap learned FSMap version 28 2024-09-16T02:03:27.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.196+0000 7fd0b0a8f740 1 -- 172.21.15.105:0/2102425711 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a15c195e10 con 0x55a15c192b40 2024-09-16T02:03:27.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.196+0000 7fd0877fe640 1 -- 172.21.15.105:0/2102425711 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd09c0995d0 con 0x55a15c192b40 2024-09-16T02:03:27.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.197+0000 7fd0877fe640 1 -- 172.21.15.105:0/2102425711 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd09c097280 con 0x55a15c192b40 2024-09-16T02:03:27.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.197+0000 7fd0b0a8f740 10 client.24820 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:03:27.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.197+0000 7fd0b0a8f740 10 client.24820 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T02:03:27.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.197+0000 7fd0b0a8f740 20 client.24820 populate_metadata read hostname 'smithi105' 2024-09-16T02:03:27.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.197+0000 7fd0b0a8f740 1 --2- 172.21.15.105:0/2102425711 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x55a15c1b0130 0x55a15c221a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:27.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.197+0000 7fd0b0a8f740 4 client.24820 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T02:03:27.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.197+0000 7fd0b0a8f740 1 -- 172.21.15.105:0/2102425711 --> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fd09c09f8e0 con 0x55a15c1b0130 2024-09-16T02:03:27.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.198+0000 7fd0a7fff640 1 --2- 172.21.15.105:0/2102425711 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x55a15c1b0130 0x55a15c221a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:27.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.198+0000 7fd0877fe640 10 client.24820 ms_handle_connect on v2:172.21.15.190:6834/1851042281 2024-09-16T02:03:27.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.198+0000 7fd0a7fff640 1 --2- 172.21.15.105:0/2102425711 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x55a15c1b0130 0x55a15c221a80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6f92fe2878907c7 server_cookie=a86b0300717ad997 in_seq=0 out_seq=0 2024-09-16T02:03:27.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.198+0000 7fd0877fe640 1 -- 172.21.15.105:0/2102425711 <== mds.4 v2:172.21.15.190:6834/1851042281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fd09c09f8e0 con 0x55a15c1b0130 2024-09-16T02:03:27.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.198+0000 7fd0877fe640 10 client.24820 handle_command_reply: tid=0 2024-09-16T02:03:27.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.198+0000 7fd0b0a8f740 1 client.24820 shutdown 2024-09-16T02:03:27.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.199+0000 7fd0b0a8f740 1 -- 172.21.15.105:0/2102425711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a15c192b40 msgr2=0x55a15c1a8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:27.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.199+0000 7fd0b0a8f740 1 --2- 172.21.15.105:0/2102425711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a15c192b40 0x55a15c1a8830 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd09c09b4c0 tx=0x7fd09c099a20 comp rx=0 tx=0).stop 2024-09-16T02:03:27.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.199+0000 7fd0b0a8f740 1 -- 172.21.15.105:0/2102425711 shutdown_connections 2024-09-16T02:03:27.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.199+0000 7fd0b0a8f740 1 --2- 172.21.15.105:0/2102425711 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x55a15c1b0130 0x55a15c221a80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:27.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.199+0000 7fd0b0a8f740 1 --2- 172.21.15.105:0/2102425711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a15c192b40 0x55a15c1a8830 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:27.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.199+0000 7fd0877fe640 0 client.24820 ms_handle_reset on v2:172.21.15.190:6834/1851042281 2024-09-16T02:03:27.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.199+0000 7fd0b0a8f740 1 --2- 172.21.15.105:0/2102425711 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a15c1a0780 0x55a15c1a8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:27.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.199+0000 7fd0b0a8f740 1 --2- 172.21.15.105:0/2102425711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a15c1915d0 0x55a15c197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:27.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.199+0000 7fd0b0a8f740 1 -- 172.21.15.105:0/2102425711 >> 172.21.15.105:0/2102425711 conn(0x55a15c15a310 msgr2=0x55a15c19d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:27.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.199+0000 7fd0b0a8f740 1 -- 172.21.15.105:0/2102425711 shutdown_connections 2024-09-16T02:03:27.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.199+0000 7fd0b0a8f740 1 -- 172.21.15.105:0/2102425711 wait complete. 2024-09-16T02:03:27.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.199+0000 7fd0b0a8f740 20 client.24820 trim_cache size 0 max 16384 2024-09-16T02:03:27.217 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-09-16T02:03:27.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.200+0000 7fd0ae6b6640 1 -- 172.21.15.105:0/3630018554 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fd0900a32c0 msgr2=0x7fd0900a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:27.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.200+0000 7fd0ae6b6640 1 --2- 172.21.15.105:0/3630018554 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fd0900a32c0 0x7fd0900a5780 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd098009740 tx=0x7fd098009b40 comp rx=0 tx=0).stop 2024-09-16T02:03:27.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.200+0000 7fd0ae6b6640 1 -- 172.21.15.105:0/3630018554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0a81036e0 msgr2=0x7fd0a8198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:27.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.200+0000 7fd0ae6b6640 1 --2- 172.21.15.105:0/3630018554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0a81036e0 0x7fd0a8198b20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd0a80697d0 tx=0x7fd09c00b850 comp rx=0 tx=0).stop 2024-09-16T02:03:27.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.200+0000 7fd0ae6b6640 1 -- 172.21.15.105:0/3630018554 shutdown_connections 2024-09-16T02:03:27.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.200+0000 7fd0ae6b6640 1 --2- 172.21.15.105:0/3630018554 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fd0900a32c0 0x7fd0900a5780 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:27.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.200+0000 7fd0ae6b6640 1 --2- 172.21.15.105:0/3630018554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0a81082a0 0x7fd0a819ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:27.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.200+0000 7fd0ae6b6640 1 --2- 172.21.15.105:0/3630018554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0a81036e0 0x7fd0a8198b20 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:27.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.200+0000 7fd0ae6b6640 1 --2- 172.21.15.105:0/3630018554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0a8102d30 0x7fd0a81985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:27.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.200+0000 7fd0ae6b6640 1 -- 172.21.15.105:0/3630018554 >> 172.21.15.105:0/3630018554 conn(0x7fd0a80fc7d0 msgr2=0x7fd0a8105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:27.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.201+0000 7fd0ae6b6640 1 -- 172.21.15.105:0/3630018554 shutdown_connections 2024-09-16T02:03:27.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:27.201+0000 7fd0ae6b6640 1 -- 172.21.15.105:0/3630018554 wait complete. 2024-09-16T02:03:27.899 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-16T02:03:28.266 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:03:28.664 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:28 smithi105 ceph-mon[31197]: pgmap v367: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T02:03:28.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.764+0000 7f10167de640 1 -- 172.21.15.105:0/604286472 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f101010c150 msgr2=0x7f101010e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:28.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.764+0000 7f10167de640 1 --2- 172.21.15.105:0/604286472 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f101010c150 0x7f101010e5f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1008009ca0 tx=0x7f100802a950 comp rx=0 tx=0).stop 2024-09-16T02:03:28.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.764+0000 7f10167de640 1 -- 172.21.15.105:0/604286472 shutdown_connections 2024-09-16T02:03:28.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.764+0000 7f10167de640 1 --2- 172.21.15.105:0/604286472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1010103640 0x7f101010bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:28.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.764+0000 7f10167de640 1 --2- 172.21.15.105:0/604286472 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f101010c150 0x7f101010e5f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:28.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.764+0000 7f10167de640 1 --2- 172.21.15.105:0/604286472 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1010102d20 0x7f1010103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:28.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.764+0000 7f10167de640 1 -- 172.21.15.105:0/604286472 >> 172.21.15.105:0/604286472 conn(0x7f10100fc780 msgr2=0x7f10100feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:28.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.764+0000 7f10167de640 1 -- 172.21.15.105:0/604286472 shutdown_connections 2024-09-16T02:03:28.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.765+0000 7f10167de640 1 -- 172.21.15.105:0/604286472 wait complete. 2024-09-16T02:03:28.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.765+0000 7f10167de640 1 Processor -- start 2024-09-16T02:03:28.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.765+0000 7f10167de640 1 -- start start 2024-09-16T02:03:28.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.766+0000 7f10167de640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1010102d20 0x7f1010198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:28.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.766+0000 7f10167de640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1010103640 0x7f10101986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:28.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.766+0000 7f10167de640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f101010c150 0x7f101019ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:28.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.766+0000 7f10167de640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1010111e30 con 0x7f1010103640 2024-09-16T02:03:28.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.766+0000 7f10167de640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1010111cb0 con 0x7f1010102d20 2024-09-16T02:03:28.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.766+0000 7f10167de640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1010111fb0 con 0x7f101010c150 2024-09-16T02:03:28.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.766+0000 7f100f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1010103640 0x7f10101986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:28.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.766+0000 7f100f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1010103640 0x7f10101986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49720/0 (socket says 172.21.15.105:49720) 2024-09-16T02:03:28.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.766+0000 7f100f7fe640 1 -- 172.21.15.105:0/1803946385 learned_addr learned my addr 172.21.15.105:0/1803946385 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:28.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.766+0000 7f100ffff640 1 --2- 172.21.15.105:0/1803946385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1010102d20 0x7f1010198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:28.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.766+0000 7f1014d7b640 1 --2- 172.21.15.105:0/1803946385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f101010c150 0x7f101019ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:28.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.766+0000 7f100f7fe640 1 -- 172.21.15.105:0/1803946385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f101010c150 msgr2=0x7f101019ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:28.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.766+0000 7f100f7fe640 1 --2- 172.21.15.105:0/1803946385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f101010c150 0x7f101019ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:28.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.766+0000 7f100f7fe640 1 -- 172.21.15.105:0/1803946385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1010102d20 msgr2=0x7f1010198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:28.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.766+0000 7f100f7fe640 1 --2- 172.21.15.105:0/1803946385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1010102d20 0x7f1010198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:28.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.766+0000 7f100f7fe640 1 -- 172.21.15.105:0/1803946385 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f101019d140 con 0x7f1010103640 2024-09-16T02:03:28.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.767+0000 7f100f7fe640 1 --2- 172.21.15.105:0/1803946385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1010103640 0x7f10101986d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f100400c970 tx=0x7f100400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:28.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.767+0000 7f100d7fa640 1 -- 172.21.15.105:0/1803946385 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1004007bb0 con 0x7f1010103640 2024-09-16T02:03:28.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.767+0000 7f100d7fa640 1 -- 172.21.15.105:0/1803946385 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1004007d50 con 0x7f1010103640 2024-09-16T02:03:28.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.767+0000 7f100d7fa640 1 -- 172.21.15.105:0/1803946385 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1004004e10 con 0x7f1010103640 2024-09-16T02:03:28.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.767+0000 7f10167de640 1 -- 172.21.15.105:0/1803946385 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f101019d430 con 0x7f1010103640 2024-09-16T02:03:28.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.767+0000 7f10167de640 1 -- 172.21.15.105:0/1803946385 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10101008b0 con 0x7f1010103640 2024-09-16T02:03:28.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.770+0000 7f1018bb7740 1 Processor -- start 2024-09-16T02:03:28.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.770+0000 7f1018bb7740 1 -- start start 2024-09-16T02:03:28.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.770+0000 7f1018bb7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f787413b40 0x55f787412090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:28.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.770+0000 7f1018bb7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f7874125d0 0x55f787412a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:28.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.770+0000 7f1018bb7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f78741d370 0x55f78741f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:28.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.770+0000 7f1018bb7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f7873782f0 con 0x55f78741d370 2024-09-16T02:03:28.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.770+0000 7f1018bb7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f78731fcf0 con 0x55f7874125d0 2024-09-16T02:03:28.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.770+0000 7f1018bb7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f7873742b0 con 0x55f787413b40 2024-09-16T02:03:28.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.770+0000 7f1014d7b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f787413b40 0x55f787412090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:28.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.770+0000 7f100ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f7874125d0 0x55f787412a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:28.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.770+0000 7f100ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f7874125d0 0x55f787412a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.105:49346/0 (socket says 172.21.15.105:49346) 2024-09-16T02:03:28.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.770+0000 7f100ffff640 1 -- 172.21.15.105:0/985038218 learned_addr learned my addr 172.21.15.105:0/985038218 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:28.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.771+0000 7f100ffff640 1 -- 172.21.15.105:0/985038218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f787413b40 msgr2=0x55f787412090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:28.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.771+0000 7f100ffff640 1 --2- 172.21.15.105:0/985038218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f787413b40 0x55f787412090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:28.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.771+0000 7f100ffff640 1 -- 172.21.15.105:0/985038218 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f78741d370 msgr2=0x55f78741f760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T02:03:28.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.771+0000 7f100ffff640 1 --2- 172.21.15.105:0/985038218 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f78741d370 0x55f78741f760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:28.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.771+0000 7f100ffff640 1 -- 172.21.15.105:0/985038218 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7873043a0 con 0x55f7874125d0 2024-09-16T02:03:28.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.771+0000 7f100d7fa640 1 -- 172.21.15.105:0/1803946385 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f1004005020 con 0x7f1010103640 2024-09-16T02:03:28.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.771+0000 7f100ffff640 1 --2- 172.21.15.105:0/985038218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f7874125d0 0x55f787412a10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f100000bd10 tx=0x7f10000107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de0199b27e20394f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:28.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.771+0000 7f0ff67fc640 1 -- 172.21.15.105:0/985038218 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1000004030 con 0x55f7874125d0 2024-09-16T02:03:28.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.771+0000 7f0ff67fc640 1 -- 172.21.15.105:0/985038218 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10000035f0 con 0x55f7874125d0 2024-09-16T02:03:28.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.772+0000 7f0ff67fc640 1 -- 172.21.15.105:0/985038218 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1000011770 con 0x55f7874125d0 2024-09-16T02:03:28.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.772+0000 7f100d7fa640 1 --2- 172.21.15.105:0/1803946385 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f0ff80a32c0 0x7f0ff80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:28.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.772+0000 7f100d7fa640 1 -- 172.21.15.105:0/1803946385 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f10040cf670 con 0x7f1010103640 2024-09-16T02:03:28.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.772+0000 7f1014d7b640 1 --2- 172.21.15.105:0/1803946385 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f0ff80a32c0 0x7f0ff80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:28.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.772+0000 7f1018bb7740 1 -- 172.21.15.105:0/985038218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f7874125d0 msgr2=0x55f787412a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:28.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.772+0000 7f1018bb7740 1 --2- 172.21.15.105:0/985038218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f7874125d0 0x55f787412a10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f100000bd10 tx=0x7f10000107a0 comp rx=0 tx=0).stop 2024-09-16T02:03:28.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.773+0000 7f1018bb7740 1 -- 172.21.15.105:0/985038218 shutdown_connections 2024-09-16T02:03:28.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.773+0000 7f1018bb7740 1 --2- 172.21.15.105:0/985038218 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f78741d370 0x55f78741f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:28.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.773+0000 7f1018bb7740 1 --2- 172.21.15.105:0/985038218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f7874125d0 0x55f787412a10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:28.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.773+0000 7f1018bb7740 1 --2- 172.21.15.105:0/985038218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f787413b40 0x55f787412090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:28.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.773+0000 7f1018bb7740 1 -- 172.21.15.105:0/985038218 >> 172.21.15.105:0/985038218 conn(0x55f7873db310 msgr2=0x55f78740ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:28.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.773+0000 7f1018bb7740 1 -- 172.21.15.105:0/985038218 shutdown_connections 2024-09-16T02:03:28.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.773+0000 7f1018bb7740 1 -- 172.21.15.105:0/985038218 wait complete. 2024-09-16T02:03:28.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.774+0000 7f1018bb7740 1 -- start start 2024-09-16T02:03:28.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.774+0000 7f1014d7b640 1 --2- 172.21.15.105:0/1803946385 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f0ff80a32c0 0x7f0ff80a5780 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f100802af40 tx=0x7f1008009580 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:28.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.774+0000 7f1018bb7740 1 Processor -- start 2024-09-16T02:03:28.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.774+0000 7f1018bb7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f7874125d0 0x55f78741cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:28.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.774+0000 7f1018bb7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f787413b40 0x55f78741a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:28.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.774+0000 7f1018bb7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f78741d370 0x55f78741a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:28.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.775+0000 7f1018bb7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f7874200c0 con 0x55f7874125d0 2024-09-16T02:03:28.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.775+0000 7f1018bb7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f787333060 con 0x55f787413b40 2024-09-16T02:03:28.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.775+0000 7f1018bb7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f787420240 con 0x55f78741d370 2024-09-16T02:03:28.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.775+0000 7f1018bb7740 10 client.0 _ll_register_callbacks cb 0x55f7870868c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:03:28.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.775+0000 7f1014d7b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f78741d370 0x55f78741a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:28.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.775+0000 7f1014d7b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f78741d370 0x55f78741a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.105:51586/0 (socket says 172.21.15.105:51586) 2024-09-16T02:03:28.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.775+0000 7f1014d7b640 1 -- 172.21.15.105:0/3859187303 learned_addr learned my addr 172.21.15.105:0/3859187303 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:28.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.775+0000 7f100f7fe640 1 --2- 172.21.15.105:0/3859187303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f787413b40 0x55f78741a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:28.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.775+0000 7f100ffff640 1 --2- 172.21.15.105:0/3859187303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f7874125d0 0x55f78741cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:28.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.775+0000 7f1014d7b640 1 -- 172.21.15.105:0/3859187303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f787413b40 msgr2=0x55f78741a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:28.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.775+0000 7f1014d7b640 1 --2- 172.21.15.105:0/3859187303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f787413b40 0x55f78741a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:28.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.775+0000 7f1014d7b640 1 -- 172.21.15.105:0/3859187303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f7874125d0 msgr2=0x55f78741cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:28.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.775+0000 7f1014d7b640 1 --2- 172.21.15.105:0/3859187303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f7874125d0 0x55f78741cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:28.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.775+0000 7f1014d7b640 1 -- 172.21.15.105:0/3859187303 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7870e93c0 con 0x55f78741d370 2024-09-16T02:03:28.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.775+0000 7f1014d7b640 1 --2- 172.21.15.105:0/3859187303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f78741d370 0x55f78741a640 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f100802f810 tx=0x7f100802a950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f08f2e934e3cba07 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:28.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.775+0000 7f0fef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-16T02:03:28.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.776+0000 7f100f7fe640 1 --2- 172.21.15.105:0/3859187303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f787413b40 0x55f78741a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:03:28.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.776+0000 7f0fef7fe640 1 -- 172.21.15.105:0/3859187303 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1008034040 con 0x55f78741d370 2024-09-16T02:03:28.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.776+0000 7f1018bb7740 1 -- 172.21.15.105:0/3859187303 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10000035f0 con 0x55f78741d370 2024-09-16T02:03:28.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.776+0000 7f0fef7fe640 1 -- 172.21.15.105:0/3859187303 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f1008003fa0 con 0x55f78741d370 2024-09-16T02:03:28.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.776+0000 7f0fef7fe640 1 -- 172.21.15.105:0/3859187303 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f10080041a0 con 0x55f78741d370 2024-09-16T02:03:28.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.776+0000 7f1018bb7740 10 client.24829 fetch_fsmap learned FSMap version 28 2024-09-16T02:03:28.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.776+0000 7f1018bb7740 1 -- 172.21.15.105:0/3859187303 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f787434420 con 0x55f78741d370 2024-09-16T02:03:28.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.776+0000 7f0fef7fe640 1 -- 172.21.15.105:0/3859187303 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10000035f0 con 0x55f78741d370 2024-09-16T02:03:28.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.777+0000 7f0fef7fe640 1 -- 172.21.15.105:0/3859187303 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1008052ac0 con 0x55f78741d370 2024-09-16T02:03:28.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.777+0000 7f1018bb7740 10 client.24829 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:03:28.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.777+0000 7f1018bb7740 10 client.24829 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T02:03:28.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.777+0000 7f1018bb7740 20 client.24829 populate_metadata read hostname 'smithi105' 2024-09-16T02:03:28.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.777+0000 7f1018bb7740 1 --2- 172.21.15.105:0/3859187303 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x55f787435830 0x55f787437c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:28.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.777+0000 7f1018bb7740 4 client.24829 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:03:28.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.777+0000 7f1018bb7740 1 -- 172.21.15.105:0/3859187303 --> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f787385860 con 0x55f787435830 2024-09-16T02:03:28.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.778+0000 7f100ffff640 1 --2- 172.21.15.105:0/3859187303 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x55f787435830 0x55f787437c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:28.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.779+0000 7f100ffff640 1 --2- 172.21.15.105:0/3859187303 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x55f787435830 0x55f787437c20 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dc8edbe3dbd4aea2 server_cookie=22cb76a72c886843 in_seq=0 out_seq=0 2024-09-16T02:03:28.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.779+0000 7f0fef7fe640 10 client.24829 ms_handle_connect on v2:172.21.15.190:6836/3181257549 2024-09-16T02:03:28.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.780+0000 7f0fef7fe640 1 -- 172.21.15.105:0/3859187303 <== mds.3 v2:172.21.15.190:6836/3181257549 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f787385860 con 0x55f787435830 2024-09-16T02:03:28.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.780+0000 7f0fef7fe640 10 client.24829 handle_command_reply: tid=0 2024-09-16T02:03:28.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.780+0000 7f1018bb7740 1 client.24829 shutdown 2024-09-16T02:03:28.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.780+0000 7f1018bb7740 1 -- 172.21.15.105:0/3859187303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f78741d370 msgr2=0x55f78741a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:28.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.780+0000 7f1018bb7740 1 --2- 172.21.15.105:0/3859187303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f78741d370 0x55f78741a640 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f100802f810 tx=0x7f100802a950 comp rx=0 tx=0).stop 2024-09-16T02:03:28.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.781+0000 7f1018bb7740 1 -- 172.21.15.105:0/3859187303 shutdown_connections 2024-09-16T02:03:28.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.781+0000 7f1018bb7740 1 --2- 172.21.15.105:0/3859187303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f78741d370 0x55f78741a640 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:28.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.781+0000 7f1018bb7740 1 --2- 172.21.15.105:0/3859187303 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x55f787435830 0x55f787437c20 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:28.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.781+0000 7f1018bb7740 1 --2- 172.21.15.105:0/3859187303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f787413b40 0x55f78741a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:28.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.781+0000 7f0fef7fe640 0 client.24829 ms_handle_reset on v2:172.21.15.190:6836/3181257549 2024-09-16T02:03:28.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.781+0000 7f1018bb7740 1 --2- 172.21.15.105:0/3859187303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f7874125d0 0x55f78741cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:28.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.781+0000 7f1018bb7740 1 -- 172.21.15.105:0/3859187303 >> 172.21.15.105:0/3859187303 conn(0x55f7873db310 msgr2=0x55f787414960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:28.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.783+0000 7f1018bb7740 1 -- 172.21.15.105:0/3859187303 shutdown_connections 2024-09-16T02:03:28.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.783+0000 7f1018bb7740 1 -- 172.21.15.105:0/3859187303 wait complete. 2024-09-16T02:03:28.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.783+0000 7f1018bb7740 20 client.24829 trim_cache size 0 max 16384 2024-09-16T02:03:28.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.807+0000 7f1018bb7740 1 Processor -- start 2024-09-16T02:03:28.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.807+0000 7f1018bb7740 1 -- start start 2024-09-16T02:03:28.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.807+0000 7f1018bb7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7874125d0 0x55f78741a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:28.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.807+0000 7f1018bb7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f787413b40 0x55f787431c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:28.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.807+0000 7f1018bb7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f78741d370 0x55f78743d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:28.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.807+0000 7f1018bb7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f10101ab510 con 0x55f78741d370 2024-09-16T02:03:28.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.807+0000 7f1018bb7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f10101ad050 con 0x55f787413b40 2024-09-16T02:03:28.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.808+0000 7f1018bb7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f10101abbe0 con 0x55f7874125d0 2024-09-16T02:03:28.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.808+0000 7f100ffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7874125d0 0x55f78741a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:28.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.808+0000 7f100f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f787413b40 0x55f787431c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:28.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.808+0000 7f100f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f787413b40 0x55f787431c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.105:49372/0 (socket says 172.21.15.105:49372) 2024-09-16T02:03:28.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.808+0000 7f100ffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7874125d0 0x55f78741a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.105:51594/0 (socket says 172.21.15.105:51594) 2024-09-16T02:03:28.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.808+0000 7f100f7fe640 1 -- 172.21.15.105:0/2609172077 learned_addr learned my addr 172.21.15.105:0/2609172077 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:28.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.808+0000 7f1014d7b640 1 --2- 172.21.15.105:0/2609172077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f78741d370 0x55f78743d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:28.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.808+0000 7f100f7fe640 1 -- 172.21.15.105:0/2609172077 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7874125d0 msgr2=0x55f78741a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:28.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.808+0000 7f100f7fe640 1 --2- 172.21.15.105:0/2609172077 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7874125d0 0x55f78741a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:28.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.808+0000 7f100f7fe640 1 -- 172.21.15.105:0/2609172077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f78741d370 msgr2=0x55f78743d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:28.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.808+0000 7f100f7fe640 1 --2- 172.21.15.105:0/2609172077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f78741d370 0x55f78743d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:28.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.808+0000 7f100f7fe640 1 -- 172.21.15.105:0/2609172077 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f787438280 con 0x55f787413b40 2024-09-16T02:03:28.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.808+0000 7f1014d7b640 1 --2- 172.21.15.105:0/2609172077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f78741d370 0x55f78743d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:03:28.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.808+0000 7f100f7fe640 1 --2- 172.21.15.105:0/2609172077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f787413b40 0x55f787431c20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f10040cd160 tx=0x7f1004005900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75864c93b2039f94 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:28.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.808+0000 7f0ff5ffb640 1 -- 172.21.15.105:0/2609172077 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100401d040 con 0x55f787413b40 2024-09-16T02:03:28.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.809+0000 7f0ff5ffb640 1 -- 172.21.15.105:0/2609172077 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f100409bc30 con 0x55f787413b40 2024-09-16T02:03:28.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.809+0000 7f0ff5ffb640 1 -- 172.21.15.105:0/2609172077 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10040860d0 con 0x55f787413b40 2024-09-16T02:03:28.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.810+0000 7f1018bb7740 1 -- 172.21.15.105:0/2609172077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f787413b40 msgr2=0x55f787431c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:28.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.810+0000 7f1018bb7740 1 --2- 172.21.15.105:0/2609172077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f787413b40 0x55f787431c20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f10040cd160 tx=0x7f1004005900 comp rx=0 tx=0).stop 2024-09-16T02:03:28.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.810+0000 7f1018bb7740 1 -- 172.21.15.105:0/2609172077 shutdown_connections 2024-09-16T02:03:28.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.810+0000 7f1018bb7740 1 --2- 172.21.15.105:0/2609172077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f78741d370 0x55f78743d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:28.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.810+0000 7f1018bb7740 1 --2- 172.21.15.105:0/2609172077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f787413b40 0x55f787431c20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:28.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.810+0000 7f1018bb7740 1 --2- 172.21.15.105:0/2609172077 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7874125d0 0x55f78741a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:28.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.810+0000 7f1018bb7740 1 -- 172.21.15.105:0/2609172077 >> 172.21.15.105:0/2609172077 conn(0x55f7873db310 msgr2=0x55f7874198f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:28.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.810+0000 7f1018bb7740 1 -- 172.21.15.105:0/2609172077 shutdown_connections 2024-09-16T02:03:28.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.810+0000 7f1018bb7740 1 -- 172.21.15.105:0/2609172077 wait complete. 2024-09-16T02:03:28.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.810+0000 7f1018bb7740 1 -- start start 2024-09-16T02:03:28.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.811+0000 7f1018bb7740 1 Processor -- start 2024-09-16T02:03:28.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.811+0000 7f1018bb7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7874125d0 0x55f78742e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:28.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.811+0000 7f1018bb7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f787413b40 0x55f7874299e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:28.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.811+0000 7f1018bb7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f78741d370 0x55f78742a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:28.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.811+0000 7f1018bb7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f10101ac980 con 0x55f78741d370 2024-09-16T02:03:28.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.811+0000 7f1018bb7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f10101ac2b0 con 0x55f787413b40 2024-09-16T02:03:28.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.811+0000 7f1018bb7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f10101a94f0 con 0x55f7874125d0 2024-09-16T02:03:28.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.811+0000 7f1018bb7740 10 client.0 _ll_register_callbacks cb 0x7f1008001040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:03:28.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.811+0000 7f1014d7b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f78741d370 0x55f78742a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:28.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.811+0000 7f100f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f787413b40 0x55f7874299e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:28.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.811+0000 7f100f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f787413b40 0x55f7874299e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.105:49376/0 (socket says 172.21.15.105:49376) 2024-09-16T02:03:28.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.811+0000 7f100f7fe640 1 -- 172.21.15.105:0/4160793267 learned_addr learned my addr 172.21.15.105:0/4160793267 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:28.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.811+0000 7f100ffff640 1 --2- 172.21.15.105:0/4160793267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7874125d0 0x55f78742e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:28.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.812+0000 7f100ffff640 1 -- 172.21.15.105:0/4160793267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f787413b40 msgr2=0x55f7874299e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:28.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.812+0000 7f100ffff640 1 --2- 172.21.15.105:0/4160793267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f787413b40 0x55f7874299e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:28.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.812+0000 7f100ffff640 1 -- 172.21.15.105:0/4160793267 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f78741d370 msgr2=0x55f78742a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:28.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.812+0000 7f100ffff640 1 --2- 172.21.15.105:0/4160793267 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f78741d370 0x55f78742a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:28.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.812+0000 7f100ffff640 1 -- 172.21.15.105:0/4160793267 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f787415120 con 0x55f7874125d0 2024-09-16T02:03:28.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.812+0000 7f1014d7b640 1 --2- 172.21.15.105:0/4160793267 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f78741d370 0x55f78742a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:03:28.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.812+0000 7f100ffff640 1 --2- 172.21.15.105:0/4160793267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7874125d0 0x55f78742e660 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1000000c00 tx=0x7f1000005ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdc1595b6aa10a7e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:28.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.812+0000 7f0fef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-16T02:03:28.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.812+0000 7f0fef7fe640 1 -- 172.21.15.105:0/4160793267 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1000011770 con 0x55f7874125d0 2024-09-16T02:03:28.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.812+0000 7f0fef7fe640 1 -- 172.21.15.105:0/4160793267 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f1000011b20 con 0x55f7874125d0 2024-09-16T02:03:28.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.812+0000 7f1018bb7740 1 -- 172.21.15.105:0/4160793267 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f100409bc30 con 0x55f7874125d0 2024-09-16T02:03:28.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.813+0000 7f0fef7fe640 1 -- 172.21.15.105:0/4160793267 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1000011d20 con 0x55f7874125d0 2024-09-16T02:03:28.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.813+0000 7f1018bb7740 10 client.24835 fetch_fsmap learned FSMap version 28 2024-09-16T02:03:28.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.813+0000 7f1018bb7740 1 -- 172.21.15.105:0/4160793267 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f7874a0440 con 0x55f7874125d0 2024-09-16T02:03:28.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.813+0000 7f0fef7fe640 1 -- 172.21.15.105:0/4160793267 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f100002b070 con 0x55f7874125d0 2024-09-16T02:03:28.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.813+0000 7f0fef7fe640 1 -- 172.21.15.105:0/4160793267 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1000003960 con 0x55f7874125d0 2024-09-16T02:03:28.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.813+0000 7f1018bb7740 10 client.24835 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:03:28.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.813+0000 7f1018bb7740 10 client.24835 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T02:03:28.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.813+0000 7f1018bb7740 20 client.24835 populate_metadata read hostname 'smithi105' 2024-09-16T02:03:28.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.813+0000 7f1018bb7740 1 --2- 172.21.15.105:0/4160793267 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x55f7874a0a70 0x55f7874a2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:28.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.813+0000 7f1018bb7740 4 client.24835 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T02:03:28.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.813+0000 7f1018bb7740 1 -- 172.21.15.105:0/4160793267 --> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f100409bc30 con 0x55f7874a0a70 2024-09-16T02:03:28.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.814+0000 7f100f7fe640 1 --2- 172.21.15.105:0/4160793267 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x55f7874a0a70 0x55f7874a2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:28.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.814+0000 7f100f7fe640 1 --2- 172.21.15.105:0/4160793267 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x55f7874a0a70 0x55f7874a2e60 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4966aa7ec0ed55f4 server_cookie=5e92fb430d11939f in_seq=0 out_seq=0 2024-09-16T02:03:28.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.814+0000 7f0fef7fe640 10 client.24835 ms_handle_connect on v2:172.21.15.190:6836/3181257549 2024-09-16T02:03:28.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.814+0000 7f0fef7fe640 1 -- 172.21.15.105:0/4160793267 <== mds.3 v2:172.21.15.190:6836/3181257549 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f100409bc30 con 0x55f7874a0a70 2024-09-16T02:03:28.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.814+0000 7f0fef7fe640 10 client.24835 handle_command_reply: tid=0 2024-09-16T02:03:28.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.814+0000 7f1018bb7740 1 client.24835 shutdown 2024-09-16T02:03:28.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.815+0000 7f1018bb7740 1 -- 172.21.15.105:0/4160793267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7874125d0 msgr2=0x55f78742e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:28.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.815+0000 7f1018bb7740 1 --2- 172.21.15.105:0/4160793267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7874125d0 0x55f78742e660 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1000000c00 tx=0x7f1000005ce0 comp rx=0 tx=0).stop 2024-09-16T02:03:28.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.815+0000 7f1018bb7740 1 -- 172.21.15.105:0/4160793267 shutdown_connections 2024-09-16T02:03:28.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.815+0000 7f1018bb7740 1 --2- 172.21.15.105:0/4160793267 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x55f7874a0a70 0x55f7874a2e60 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:28.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.815+0000 7f1018bb7740 1 --2- 172.21.15.105:0/4160793267 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f78741d370 0x55f78742a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:28.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.815+0000 7f1018bb7740 1 --2- 172.21.15.105:0/4160793267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f787413b40 0x55f7874299e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:28.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.815+0000 7f1018bb7740 1 --2- 172.21.15.105:0/4160793267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7874125d0 0x55f78742e660 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:28.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.815+0000 7f1018bb7740 1 -- 172.21.15.105:0/4160793267 >> 172.21.15.105:0/4160793267 conn(0x55f7873db310 msgr2=0x55f78741b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:28.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.815+0000 7f0fef7fe640 0 client.24835 ms_handle_reset on v2:172.21.15.190:6836/3181257549 2024-09-16T02:03:28.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.815+0000 7f1018bb7740 1 -- 172.21.15.105:0/4160793267 shutdown_connections 2024-09-16T02:03:28.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.815+0000 7f1018bb7740 1 -- 172.21.15.105:0/4160793267 wait complete. 2024-09-16T02:03:28.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.815+0000 7f1018bb7740 20 client.24835 trim_cache size 0 max 16384 2024-09-16T02:03:28.821 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-09-16T02:03:28.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.816+0000 7f10167de640 1 -- 172.21.15.105:0/1803946385 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f0ff80a32c0 msgr2=0x7f0ff80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:28.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.816+0000 7f10167de640 1 --2- 172.21.15.105:0/1803946385 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f0ff80a32c0 0x7f0ff80a5780 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f100802af40 tx=0x7f1008009580 comp rx=0 tx=0).stop 2024-09-16T02:03:28.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.816+0000 7f10167de640 1 -- 172.21.15.105:0/1803946385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1010103640 msgr2=0x7f10101986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:28.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.816+0000 7f10167de640 1 --2- 172.21.15.105:0/1803946385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1010103640 0x7f10101986d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f100400c970 tx=0x7f100400ce30 comp rx=0 tx=0).stop 2024-09-16T02:03:28.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.816+0000 7f10167de640 1 -- 172.21.15.105:0/1803946385 shutdown_connections 2024-09-16T02:03:28.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.816+0000 7f10167de640 1 --2- 172.21.15.105:0/1803946385 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f0ff80a32c0 0x7f0ff80a5780 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:28.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.816+0000 7f10167de640 1 --2- 172.21.15.105:0/1803946385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f101010c150 0x7f101019ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:28.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.816+0000 7f10167de640 1 --2- 172.21.15.105:0/1803946385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1010103640 0x7f10101986d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:28.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.816+0000 7f10167de640 1 --2- 172.21.15.105:0/1803946385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1010102d20 0x7f1010198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:28.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.816+0000 7f10167de640 1 -- 172.21.15.105:0/1803946385 >> 172.21.15.105:0/1803946385 conn(0x7f10100fc780 msgr2=0x7f10100fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:28.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.816+0000 7f10167de640 1 -- 172.21.15.105:0/1803946385 shutdown_connections 2024-09-16T02:03:28.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:28.816+0000 7f10167de640 1 -- 172.21.15.105:0/1803946385 wait complete. 2024-09-16T02:03:28.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:28 smithi190 ceph-mon[34762]: pgmap v367: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T02:03:28.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:28 smithi137 ceph-mon[34099]: pgmap v367: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T02:03:29.522 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T02:03:29.909 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:03:30.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.458+0000 7f5512cef640 1 -- 172.21.15.105:0/1017792826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f550c101fc0 msgr2=0x7f550c1056d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:30.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.458+0000 7f5512cef640 1 --2- 172.21.15.105:0/1017792826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f550c101fc0 0x7f550c1056d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f54f8009900 tx=0x7f54f802a410 comp rx=0 tx=0).stop 2024-09-16T02:03:30.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.458+0000 7f5512cef640 1 -- 172.21.15.105:0/1017792826 shutdown_connections 2024-09-16T02:03:30.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.458+0000 7f5512cef640 1 --2- 172.21.15.105:0/1017792826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f550c105e00 0x7f550c10d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:30.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.458+0000 7f5512cef640 1 --2- 172.21.15.105:0/1017792826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f550c101fc0 0x7f550c1056d0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:30.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.458+0000 7f5512cef640 1 --2- 172.21.15.105:0/1017792826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f550c101610 0x7f550c1019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:30.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.458+0000 7f5512cef640 1 -- 172.21.15.105:0/1017792826 >> 172.21.15.105:0/1017792826 conn(0x7f550c0fc5c0 msgr2=0x7f550c0fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:30.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.459+0000 7f5512cef640 1 -- 172.21.15.105:0/1017792826 shutdown_connections 2024-09-16T02:03:30.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.459+0000 7f5512cef640 1 -- 172.21.15.105:0/1017792826 wait complete. 2024-09-16T02:03:30.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.460+0000 7f5512cef640 1 Processor -- start 2024-09-16T02:03:30.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.460+0000 7f5512cef640 1 -- start start 2024-09-16T02:03:30.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.460+0000 7f5512cef640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f550c101610 0x7f550c1a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:30.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.460+0000 7f5512cef640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f550c101fc0 0x7f550c1a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:30.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.460+0000 7f5512cef640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f550c105e00 0x7f550c19ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:30.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.460+0000 7f5510a8b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f550c101610 0x7f550c1a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:30.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.460+0000 7f5510a8b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f550c101610 0x7f550c1a4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.105:51624/0 (socket says 172.21.15.105:51624) 2024-09-16T02:03:30.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.460+0000 7f550bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f550c101fc0 0x7f550c1a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:30.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.460+0000 7f550bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f550c101fc0 0x7f550c1a4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49754/0 (socket says 172.21.15.105:49754) 2024-09-16T02:03:30.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.460+0000 7f5510a8b640 1 -- 172.21.15.105:0/2041628654 learned_addr learned my addr 172.21.15.105:0/2041628654 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:30.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.460+0000 7f5512cef640 1 -- 172.21.15.105:0/2041628654 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f550c110d60 con 0x7f550c101fc0 2024-09-16T02:03:30.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.461+0000 7f5512cef640 1 -- 172.21.15.105:0/2041628654 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f550c110be0 con 0x7f550c105e00 2024-09-16T02:03:30.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.461+0000 7f5512cef640 1 -- 172.21.15.105:0/2041628654 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f550c110ee0 con 0x7f550c101610 2024-09-16T02:03:30.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.461+0000 7f551128c640 1 --2- 172.21.15.105:0/2041628654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f550c105e00 0x7f550c19ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:30.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.461+0000 7f551128c640 1 -- 172.21.15.105:0/2041628654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f550c101610 msgr2=0x7f550c1a4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:30.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.461+0000 7f551128c640 1 --2- 172.21.15.105:0/2041628654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f550c101610 0x7f550c1a4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:30.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.461+0000 7f551128c640 1 -- 172.21.15.105:0/2041628654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f550c101fc0 msgr2=0x7f550c1a4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:30.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.461+0000 7f551128c640 1 --2- 172.21.15.105:0/2041628654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f550c101fc0 0x7f550c1a4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:30.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.461+0000 7f551128c640 1 -- 172.21.15.105:0/2041628654 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f550c19f1a0 con 0x7f550c105e00 2024-09-16T02:03:30.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.462+0000 7f550bfff640 1 --2- 172.21.15.105:0/2041628654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f550c101fc0 0x7f550c1a4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:03:30.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.462+0000 7f551128c640 1 --2- 172.21.15.105:0/2041628654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f550c105e00 0x7f550c19ea00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f550400ca30 tx=0x7f550400cef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:30.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.462+0000 7f5509ffb640 1 -- 172.21.15.105:0/2041628654 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5504007d90 con 0x7f550c105e00 2024-09-16T02:03:30.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.462+0000 7f5509ffb640 1 -- 172.21.15.105:0/2041628654 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5504005020 con 0x7f550c105e00 2024-09-16T02:03:30.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.462+0000 7f5509ffb640 1 -- 172.21.15.105:0/2041628654 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550400f690 con 0x7f550c105e00 2024-09-16T02:03:30.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.463+0000 7f5512cef640 1 -- 172.21.15.105:0/2041628654 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f550c19f490 con 0x7f550c105e00 2024-09-16T02:03:30.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.463+0000 7f5512cef640 1 -- 172.21.15.105:0/2041628654 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f550c1ab490 con 0x7f550c105e00 2024-09-16T02:03:30.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.466+0000 7f5509ffb640 1 -- 172.21.15.105:0/2041628654 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f55040051d0 con 0x7f550c105e00 2024-09-16T02:03:30.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.467+0000 7f55150c8740 1 Processor -- start 2024-09-16T02:03:30.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.467+0000 7f55150c8740 1 -- start start 2024-09-16T02:03:30.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.467+0000 7f55150c8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564666db2b40 0x564666db1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:30.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.467+0000 7f55150c8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564666db15d0 0x564666db1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:30.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.467+0000 7f55150c8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564666dbc370 0x564666dbe760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:30.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.467+0000 7f55150c8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564666d172f0 con 0x564666db2b40 2024-09-16T02:03:30.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.467+0000 7f55150c8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564666cbecf0 con 0x564666dbc370 2024-09-16T02:03:30.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.467+0000 7f55150c8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564666d132b0 con 0x564666db15d0 2024-09-16T02:03:30.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.467+0000 7f550bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564666db2b40 0x564666db1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:30.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.467+0000 7f550bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564666db2b40 0x564666db1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49764/0 (socket says 172.21.15.105:49764) 2024-09-16T02:03:30.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.467+0000 7f550bfff640 1 -- 172.21.15.105:0/229630940 learned_addr learned my addr 172.21.15.105:0/229630940 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:30.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.468+0000 7f5510a8b640 1 --2- 172.21.15.105:0/229630940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564666db15d0 0x564666db1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:30.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.468+0000 7f550bfff640 1 -- 172.21.15.105:0/229630940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564666db15d0 msgr2=0x564666db1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:30.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.468+0000 7f550bfff640 1 --2- 172.21.15.105:0/229630940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564666db15d0 0x564666db1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:30.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.468+0000 7f550bfff640 1 -- 172.21.15.105:0/229630940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564666dbc370 msgr2=0x564666dbe760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:30.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.468+0000 7f550bfff640 1 --2- 172.21.15.105:0/229630940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564666dbc370 0x564666dbe760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:30.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.468+0000 7f550bfff640 1 -- 172.21.15.105:0/229630940 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564666ca33a0 con 0x564666db2b40 2024-09-16T02:03:30.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.468+0000 7f550bfff640 1 --2- 172.21.15.105:0/229630940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564666db2b40 0x564666db1090 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f550c0745b0 tx=0x7f54f8004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff923519b1842b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:30.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.468+0000 7f5509ffb640 1 --2- 172.21.15.105:0/2041628654 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f54fc0a3340 0x7f54fc0a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:30.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.468+0000 7f5509ffb640 1 -- 172.21.15.105:0/2041628654 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f55040cfab0 con 0x7f550c105e00 2024-09-16T02:03:30.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.468+0000 7f5510a8b640 1 --2- 172.21.15.105:0/2041628654 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f54fc0a3340 0x7f54fc0a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:30.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.469+0000 7f54f6ffd640 1 -- 172.21.15.105:0/229630940 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f8035070 con 0x564666db2b40 2024-09-16T02:03:30.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.469+0000 7f54f6ffd640 1 -- 172.21.15.105:0/229630940 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54f8041030 con 0x564666db2b40 2024-09-16T02:03:30.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.469+0000 7f54f6ffd640 1 -- 172.21.15.105:0/229630940 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f803cd60 con 0x564666db2b40 2024-09-16T02:03:30.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.470+0000 7f5510a8b640 1 --2- 172.21.15.105:0/2041628654 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f54fc0a3340 0x7f54fc0a5800 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5500009640 tx=0x7f5500005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:30.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.470+0000 7f55150c8740 1 -- 172.21.15.105:0/229630940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564666db2b40 msgr2=0x564666db1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:30.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.470+0000 7f55150c8740 1 --2- 172.21.15.105:0/229630940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564666db2b40 0x564666db1090 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f550c0745b0 tx=0x7f54f8004610 comp rx=0 tx=0).stop 2024-09-16T02:03:30.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.470+0000 7f55150c8740 1 -- 172.21.15.105:0/229630940 shutdown_connections 2024-09-16T02:03:30.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.470+0000 7f55150c8740 1 --2- 172.21.15.105:0/229630940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564666db15d0 0x564666db1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:30.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.470+0000 7f55150c8740 1 --2- 172.21.15.105:0/229630940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564666dbc370 0x564666dbe760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:30.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.470+0000 7f55150c8740 1 --2- 172.21.15.105:0/229630940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564666db2b40 0x564666db1090 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:30.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.470+0000 7f55150c8740 1 -- 172.21.15.105:0/229630940 >> 172.21.15.105:0/229630940 conn(0x564666d7a310 msgr2=0x564666dadc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:30.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.470+0000 7f55150c8740 1 -- 172.21.15.105:0/229630940 shutdown_connections 2024-09-16T02:03:30.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.470+0000 7f55150c8740 1 -- 172.21.15.105:0/229630940 wait complete. 2024-09-16T02:03:30.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.471+0000 7f55150c8740 1 -- start start 2024-09-16T02:03:30.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.471+0000 7f55150c8740 1 Processor -- start 2024-09-16T02:03:30.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.471+0000 7f55150c8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564666db15d0 0x564666dbbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:30.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.472+0000 7f55150c8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564666db2b40 0x564666db9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:30.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.472+0000 7f5510a8b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564666db15d0 0x564666dbbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:30.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.472+0000 7f5510a8b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564666db15d0 0x564666dbbd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49766/0 (socket says 172.21.15.105:49766) 2024-09-16T02:03:30.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.472+0000 7f5510a8b640 1 -- 172.21.15.105:0/1249361235 learned_addr learned my addr 172.21.15.105:0/1249361235 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:30.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.472+0000 7f55150c8740 1 --2- 172.21.15.105:0/1249361235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564666dbc370 0x564666db96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:30.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.472+0000 7f55150c8740 1 -- 172.21.15.105:0/1249361235 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564666dbf010 con 0x564666db15d0 2024-09-16T02:03:30.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.472+0000 7f550bfff640 1 --2- 172.21.15.105:0/1249361235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564666db2b40 0x564666db9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:30.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.472+0000 7f55150c8740 1 -- 172.21.15.105:0/1249361235 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564666cd2060 con 0x564666db2b40 2024-09-16T02:03:30.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.472+0000 7f551128c640 1 --2- 172.21.15.105:0/1249361235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564666dbc370 0x564666db96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:30.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.472+0000 7f55150c8740 1 -- 172.21.15.105:0/1249361235 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564666dbf190 con 0x564666dbc370 2024-09-16T02:03:30.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.472+0000 7f55150c8740 10 client.0 _ll_register_callbacks cb 0x564666a258c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:03:30.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.473+0000 7f551128c640 1 -- 172.21.15.105:0/1249361235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564666db2b40 msgr2=0x564666db9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:30.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.473+0000 7f551128c640 1 --2- 172.21.15.105:0/1249361235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564666db2b40 0x564666db9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:30.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.473+0000 7f551128c640 1 -- 172.21.15.105:0/1249361235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564666db15d0 msgr2=0x564666dbbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:30.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.473+0000 7f551128c640 1 --2- 172.21.15.105:0/1249361235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564666db15d0 0x564666dbbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:30.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.473+0000 7f551128c640 1 -- 172.21.15.105:0/1249361235 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564666a883c0 con 0x564666dbc370 2024-09-16T02:03:30.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.473+0000 7f5510a8b640 1 --2- 172.21.15.105:0/1249361235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564666db15d0 0x564666dbbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:03:30.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.473+0000 7f551128c640 1 --2- 172.21.15.105:0/1249361235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564666dbc370 0x564666db96a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f550409bdc0 tx=0x7f55040a0480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c43d23cb03009376 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:30.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.473+0000 7f54ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-16T02:03:30.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.473+0000 7f54ebfff640 1 -- 172.21.15.105:0/1249361235 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550401d030 con 0x564666dbc370 2024-09-16T02:03:30.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.473+0000 7f54ebfff640 1 -- 172.21.15.105:0/1249361235 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f550409c320 con 0x564666dbc370 2024-09-16T02:03:30.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.473+0000 7f55150c8740 1 -- 172.21.15.105:0/1249361235 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54f8041030 con 0x564666dbc370 2024-09-16T02:03:30.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.474+0000 7f54ebfff640 1 -- 172.21.15.105:0/1249361235 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f550409c520 con 0x564666dbc370 2024-09-16T02:03:30.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.474+0000 7f55150c8740 10 client.24847 fetch_fsmap learned FSMap version 28 2024-09-16T02:03:30.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.474+0000 7f55150c8740 1 -- 172.21.15.105:0/1249361235 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564666dcab50 con 0x564666dbc370 2024-09-16T02:03:30.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.474+0000 7f54ebfff640 1 -- 172.21.15.105:0/1249361235 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54f8041030 con 0x564666dbc370 2024-09-16T02:03:30.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.475+0000 7f54ebfff640 1 -- 172.21.15.105:0/1249361235 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5504098010 con 0x564666dbc370 2024-09-16T02:03:30.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.475+0000 7f55150c8740 10 client.24847 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:03:30.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.475+0000 7f55150c8740 10 client.24847 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T02:03:30.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.475+0000 7f55150c8740 20 client.24847 populate_metadata read hostname 'smithi105' 2024-09-16T02:03:30.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.475+0000 7f55150c8740 1 --2- 172.21.15.105:0/1249361235 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x564666dcbf60 0x564666dce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:30.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.475+0000 7f55150c8740 4 client.24847 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:03:30.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.476+0000 7f55150c8740 1 -- 172.21.15.105:0/1249361235 --> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564666d24860 con 0x564666dcbf60 2024-09-16T02:03:30.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.476+0000 7f5510a8b640 1 --2- 172.21.15.105:0/1249361235 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x564666dcbf60 0x564666dce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:30.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.476+0000 7f5510a8b640 1 --2- 172.21.15.105:0/1249361235 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x564666dcbf60 0x564666dce350 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6380039844d9803e server_cookie=cc2aeab07a838a6f in_seq=0 out_seq=0 2024-09-16T02:03:30.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.476+0000 7f54ebfff640 10 client.24847 ms_handle_connect on v2:172.21.15.137:6836/2788541030 2024-09-16T02:03:30.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.477+0000 7f54ebfff640 1 -- 172.21.15.105:0/1249361235 <== mds.2 v2:172.21.15.137:6836/2788541030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564666d24860 con 0x564666dcbf60 2024-09-16T02:03:30.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.477+0000 7f54ebfff640 10 client.24847 handle_command_reply: tid=0 2024-09-16T02:03:30.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.477+0000 7f55150c8740 1 client.24847 shutdown 2024-09-16T02:03:30.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.477+0000 7f55150c8740 1 -- 172.21.15.105:0/1249361235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564666dbc370 msgr2=0x564666db96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:30.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.477+0000 7f55150c8740 1 --2- 172.21.15.105:0/1249361235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564666dbc370 0x564666db96a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f550409bdc0 tx=0x7f55040a0480 comp rx=0 tx=0).stop 2024-09-16T02:03:30.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.477+0000 7f55150c8740 1 -- 172.21.15.105:0/1249361235 shutdown_connections 2024-09-16T02:03:30.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.477+0000 7f55150c8740 1 --2- 172.21.15.105:0/1249361235 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x564666dcbf60 0x564666dce350 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:30.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.477+0000 7f54ebfff640 0 client.24847 ms_handle_reset on v2:172.21.15.137:6836/2788541030 2024-09-16T02:03:30.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.477+0000 7f55150c8740 1 --2- 172.21.15.105:0/1249361235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564666dbc370 0x564666db96a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:30.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.477+0000 7f55150c8740 1 --2- 172.21.15.105:0/1249361235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564666db2b40 0x564666db9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:30.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.477+0000 7f55150c8740 1 --2- 172.21.15.105:0/1249361235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564666db15d0 0x564666dbbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:30.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.477+0000 7f55150c8740 1 -- 172.21.15.105:0/1249361235 >> 172.21.15.105:0/1249361235 conn(0x564666d7a310 msgr2=0x564666db3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:30.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.478+0000 7f55150c8740 1 -- 172.21.15.105:0/1249361235 shutdown_connections 2024-09-16T02:03:30.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.478+0000 7f55150c8740 1 -- 172.21.15.105:0/1249361235 wait complete. 2024-09-16T02:03:30.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.478+0000 7f55150c8740 20 client.24847 trim_cache size 0 max 16384 2024-09-16T02:03:30.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.503+0000 7f55150c8740 1 Processor -- start 2024-09-16T02:03:30.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.503+0000 7f55150c8740 1 -- start start 2024-09-16T02:03:30.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.503+0000 7f55150c8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564666db15d0 0x564666dade10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:30.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.504+0000 7f55150c8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564666db2b40 0x564666dc82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:30.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.504+0000 7f55150c8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564666dbc370 0x564666dd3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:30.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.504+0000 7f5510a8b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564666db15d0 0x564666dade10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:30.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.504+0000 7f5510a8b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564666db15d0 0x564666dade10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.105:49424/0 (socket says 172.21.15.105:49424) 2024-09-16T02:03:30.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.504+0000 7f551128c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564666dbc370 0x564666dd3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:30.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.504+0000 7f551128c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564666dbc370 0x564666dd3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49770/0 (socket says 172.21.15.105:49770) 2024-09-16T02:03:30.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.504+0000 7f55150c8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f550c1b3bc0 con 0x564666dbc370 2024-09-16T02:03:30.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.504+0000 7f5510a8b640 1 -- 172.21.15.105:0/4148262195 learned_addr learned my addr 172.21.15.105:0/4148262195 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:30.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.504+0000 7f55150c8740 1 -- 172.21.15.105:0/4148262195 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f550c1b5700 con 0x564666db15d0 2024-09-16T02:03:30.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.504+0000 7f55150c8740 1 -- 172.21.15.105:0/4148262195 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f550c1b4290 con 0x564666db2b40 2024-09-16T02:03:30.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.504+0000 7f550bfff640 1 --2- 172.21.15.105:0/4148262195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564666db2b40 0x564666dc82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:30.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.504+0000 7f550bfff640 1 -- 172.21.15.105:0/4148262195 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564666db15d0 msgr2=0x564666dade10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:30.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.505+0000 7f550bfff640 1 --2- 172.21.15.105:0/4148262195 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564666db15d0 0x564666dade10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:30.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.505+0000 7f550bfff640 1 -- 172.21.15.105:0/4148262195 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564666dbc370 msgr2=0x564666dd3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:30.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.505+0000 7f550bfff640 1 --2- 172.21.15.105:0/4148262195 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564666dbc370 0x564666dd3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:30.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.505+0000 7f550bfff640 1 -- 172.21.15.105:0/4148262195 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564666dbc0b0 con 0x564666db2b40 2024-09-16T02:03:30.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.505+0000 7f551128c640 1 --2- 172.21.15.105:0/4148262195 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564666dbc370 0x564666dd3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:03:30.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.505+0000 7f550bfff640 1 --2- 172.21.15.105:0/4148262195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564666db2b40 0x564666dc82e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f54f803ca40 tx=0x7f54f802a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc3f0b3cccfc96fb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:30.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.505+0000 7f54f67fc640 1 -- 172.21.15.105:0/4148262195 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f8035070 con 0x564666db2b40 2024-09-16T02:03:30.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.505+0000 7f54f67fc640 1 -- 172.21.15.105:0/4148262195 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54f8041050 con 0x564666db2b40 2024-09-16T02:03:30.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.505+0000 7f54f67fc640 1 -- 172.21.15.105:0/4148262195 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f8003a10 con 0x564666db2b40 2024-09-16T02:03:30.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.506+0000 7f55150c8740 1 -- 172.21.15.105:0/4148262195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564666db2b40 msgr2=0x564666dc82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:30.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.506+0000 7f55150c8740 1 --2- 172.21.15.105:0/4148262195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564666db2b40 0x564666dc82e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f54f803ca40 tx=0x7f54f802a410 comp rx=0 tx=0).stop 2024-09-16T02:03:30.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.506+0000 7f55150c8740 1 -- 172.21.15.105:0/4148262195 shutdown_connections 2024-09-16T02:03:30.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.506+0000 7f55150c8740 1 --2- 172.21.15.105:0/4148262195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564666db2b40 0x564666dc82e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:30.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.506+0000 7f55150c8740 1 --2- 172.21.15.105:0/4148262195 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564666dbc370 0x564666dd3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:30.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.507+0000 7f55150c8740 1 --2- 172.21.15.105:0/4148262195 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564666db15d0 0x564666dade10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:30.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.507+0000 7f55150c8740 1 -- 172.21.15.105:0/4148262195 >> 172.21.15.105:0/4148262195 conn(0x564666d7a310 msgr2=0x564666dad700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:30.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.507+0000 7f55150c8740 1 -- 172.21.15.105:0/4148262195 shutdown_connections 2024-09-16T02:03:30.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.507+0000 7f55150c8740 1 -- 172.21.15.105:0/4148262195 wait complete. 2024-09-16T02:03:30.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.507+0000 7f55150c8740 1 -- start start 2024-09-16T02:03:30.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.507+0000 7f55150c8740 1 Processor -- start 2024-09-16T02:03:30.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.508+0000 7f55150c8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564666db15d0 0x564666dc4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:30.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.508+0000 7f55150c8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564666db2b40 0x564666dc4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:30.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.508+0000 7f55150c8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564666dbc370 0x564666db5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:30.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.508+0000 7f55150c8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f550c1b5030 con 0x564666db2b40 2024-09-16T02:03:30.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.508+0000 7f5510a8b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564666db15d0 0x564666dc4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:30.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.508+0000 7f550bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564666db2b40 0x564666dc4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:30.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.508+0000 7f550bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564666db2b40 0x564666dc4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49780/0 (socket says 172.21.15.105:49780) 2024-09-16T02:03:30.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.508+0000 7f550bfff640 1 -- 172.21.15.105:0/816107075 learned_addr learned my addr 172.21.15.105:0/816107075 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:30.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.508+0000 7f551128c640 1 --2- 172.21.15.105:0/816107075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564666dbc370 0x564666db5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:30.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.508+0000 7f5510a8b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564666db15d0 0x564666dc4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.105:51660/0 (socket says 172.21.15.105:51660) 2024-09-16T02:03:30.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.508+0000 7f55150c8740 1 -- 172.21.15.105:0/816107075 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f550c1b4960 con 0x564666dbc370 2024-09-16T02:03:30.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.508+0000 7f55150c8740 1 -- 172.21.15.105:0/816107075 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f550c1b1ba0 con 0x564666db15d0 2024-09-16T02:03:30.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.509+0000 7f55150c8740 10 client.0 _ll_register_callbacks cb 0x7f5504098360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:03:30.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.509+0000 7f550bfff640 1 -- 172.21.15.105:0/816107075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564666db15d0 msgr2=0x564666dc4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:30.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.509+0000 7f550bfff640 1 --2- 172.21.15.105:0/816107075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564666db15d0 0x564666dc4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:30.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.509+0000 7f550bfff640 1 -- 172.21.15.105:0/816107075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564666dbc370 msgr2=0x564666db5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:30.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.509+0000 7f550bfff640 1 --2- 172.21.15.105:0/816107075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564666dbc370 0x564666db5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:30.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.509+0000 7f550bfff640 1 -- 172.21.15.105:0/816107075 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564666dc5050 con 0x564666db2b40 2024-09-16T02:03:30.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.509+0000 7f550bfff640 1 --2- 172.21.15.105:0/816107075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564666db2b40 0x564666dc4990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f54f8005740 tx=0x7f54f8004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b184cb0c4451b9c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:30.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.509+0000 7f54ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:03:30.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.509+0000 7f54ebfff640 1 -- 172.21.15.105:0/816107075 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f8058060 con 0x564666db2b40 2024-09-16T02:03:30.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.509+0000 7f55150c8740 1 -- 172.21.15.105:0/816107075 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54f8041050 con 0x564666db2b40 2024-09-16T02:03:30.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.509+0000 7f54ebfff640 1 -- 172.21.15.105:0/816107075 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f54f803cc80 con 0x564666db2b40 2024-09-16T02:03:30.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.510+0000 7f54ebfff640 1 -- 172.21.15.105:0/816107075 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f54f8035040 con 0x564666db2b40 2024-09-16T02:03:30.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.510+0000 7f55150c8740 10 client.15420 fetch_fsmap learned FSMap version 28 2024-09-16T02:03:30.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.510+0000 7f55150c8740 1 -- 172.21.15.105:0/816107075 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564666e36150 con 0x564666db2b40 2024-09-16T02:03:30.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.510+0000 7f54ebfff640 1 -- 172.21.15.105:0/816107075 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54f8041050 con 0x564666db2b40 2024-09-16T02:03:30.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.510+0000 7f54ebfff640 1 -- 172.21.15.105:0/816107075 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f54f803d7c0 con 0x564666db2b40 2024-09-16T02:03:30.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.510+0000 7f55150c8740 10 client.15420 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:03:30.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.510+0000 7f55150c8740 10 client.15420 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T02:03:30.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.510+0000 7f55150c8740 20 client.15420 populate_metadata read hostname 'smithi105' 2024-09-16T02:03:30.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.510+0000 7f55150c8740 1 --2- 172.21.15.105:0/816107075 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x564666dcbf60 0x564666e387e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:30.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.510+0000 7f55150c8740 4 client.15420 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T02:03:30.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.510+0000 7f55150c8740 1 -- 172.21.15.105:0/816107075 --> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564666d1ed70 con 0x564666dcbf60 2024-09-16T02:03:30.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.511+0000 7f5510a8b640 1 --2- 172.21.15.105:0/816107075 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x564666dcbf60 0x564666e387e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:30.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.511+0000 7f54ebfff640 10 client.15420 ms_handle_connect on v2:172.21.15.137:6836/2788541030 2024-09-16T02:03:30.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.512+0000 7f5510a8b640 1 --2- 172.21.15.105:0/816107075 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x564666dcbf60 0x564666e387e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=de1571b52b663342 server_cookie=58168eef81ab40af in_seq=0 out_seq=0 2024-09-16T02:03:30.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.512+0000 7f54ebfff640 1 -- 172.21.15.105:0/816107075 <== mds.2 v2:172.21.15.137:6836/2788541030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564666d1ed70 con 0x564666dcbf60 2024-09-16T02:03:30.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.512+0000 7f54ebfff640 10 client.15420 handle_command_reply: tid=0 2024-09-16T02:03:30.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.512+0000 7f55150c8740 1 client.15420 shutdown 2024-09-16T02:03:30.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.512+0000 7f55150c8740 1 -- 172.21.15.105:0/816107075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564666db2b40 msgr2=0x564666dc4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:30.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.512+0000 7f55150c8740 1 --2- 172.21.15.105:0/816107075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564666db2b40 0x564666dc4990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f54f8005740 tx=0x7f54f8004610 comp rx=0 tx=0).stop 2024-09-16T02:03:30.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.512+0000 7f55150c8740 1 -- 172.21.15.105:0/816107075 shutdown_connections 2024-09-16T02:03:30.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.512+0000 7f55150c8740 1 --2- 172.21.15.105:0/816107075 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x564666dcbf60 0x564666e387e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:30.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.512+0000 7f55150c8740 1 --2- 172.21.15.105:0/816107075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564666dbc370 0x564666db5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:30.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.512+0000 7f54ebfff640 0 client.15420 ms_handle_reset on v2:172.21.15.137:6836/2788541030 2024-09-16T02:03:30.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.512+0000 7f55150c8740 1 --2- 172.21.15.105:0/816107075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564666db2b40 0x564666dc4990 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:30.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.512+0000 7f55150c8740 1 --2- 172.21.15.105:0/816107075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564666db15d0 0x564666dc4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:30.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.513+0000 7f55150c8740 1 -- 172.21.15.105:0/816107075 >> 172.21.15.105:0/816107075 conn(0x564666d7a310 msgr2=0x564666db7930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:30.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.513+0000 7f55150c8740 1 -- 172.21.15.105:0/816107075 shutdown_connections 2024-09-16T02:03:30.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.513+0000 7f55150c8740 1 -- 172.21.15.105:0/816107075 wait complete. 2024-09-16T02:03:30.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.513+0000 7f55150c8740 20 client.15420 trim_cache size 0 max 16384 2024-09-16T02:03:30.518 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-09-16T02:03:30.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.514+0000 7f5512cef640 1 -- 172.21.15.105:0/2041628654 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f54fc0a3340 msgr2=0x7f54fc0a5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:30.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.514+0000 7f5512cef640 1 --2- 172.21.15.105:0/2041628654 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f54fc0a3340 0x7f54fc0a5800 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5500009640 tx=0x7f5500005f90 comp rx=0 tx=0).stop 2024-09-16T02:03:30.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.514+0000 7f5512cef640 1 -- 172.21.15.105:0/2041628654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f550c105e00 msgr2=0x7f550c19ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:30.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.514+0000 7f5512cef640 1 --2- 172.21.15.105:0/2041628654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f550c105e00 0x7f550c19ea00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f550400ca30 tx=0x7f550400cef0 comp rx=0 tx=0).stop 2024-09-16T02:03:30.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.514+0000 7f5512cef640 1 -- 172.21.15.105:0/2041628654 shutdown_connections 2024-09-16T02:03:30.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.514+0000 7f5512cef640 1 --2- 172.21.15.105:0/2041628654 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f54fc0a3340 0x7f54fc0a5800 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:30.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.514+0000 7f5512cef640 1 --2- 172.21.15.105:0/2041628654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f550c105e00 0x7f550c19ea00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:30.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.514+0000 7f5512cef640 1 --2- 172.21.15.105:0/2041628654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f550c101fc0 0x7f550c1a4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:30.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.514+0000 7f5512cef640 1 --2- 172.21.15.105:0/2041628654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f550c101610 0x7f550c1a4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:30.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.514+0000 7f5512cef640 1 -- 172.21.15.105:0/2041628654 >> 172.21.15.105:0/2041628654 conn(0x7f550c0fc5c0 msgr2=0x7f550c1046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:30.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.514+0000 7f5512cef640 1 -- 172.21.15.105:0/2041628654 shutdown_connections 2024-09-16T02:03:30.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:30.514+0000 7f5512cef640 1 -- 172.21.15.105:0/2041628654 wait complete. 2024-09-16T02:03:30.582 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:30 smithi105 ceph-mon[31197]: pgmap v368: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-16T02:03:30.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:30 smithi190 ceph-mon[34762]: pgmap v368: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-16T02:03:30.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:30 smithi137 ceph-mon[34099]: pgmap v368: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-16T02:03:31.372 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T02:03:31.643 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:03:32.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.232+0000 7f04baf17640 1 -- 172.21.15.105:0/387155416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04b4101430 msgr2=0x7f04b410d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:32.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.232+0000 7f04baf17640 1 --2- 172.21.15.105:0/387155416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04b4101430 0x7f04b410d960 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f04ac009cd0 tx=0x7f04ac02a8e0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.233+0000 7f04baf17640 1 -- 172.21.15.105:0/387155416 shutdown_connections 2024-09-16T02:03:32.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.233+0000 7f04baf17640 1 --2- 172.21.15.105:0/387155416 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04b4100a90 0x7f04b4100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.233+0000 7f04baf17640 1 --2- 172.21.15.105:0/387155416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04b4101430 0x7f04b410d960 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.233+0000 7f04baf17640 1 --2- 172.21.15.105:0/387155416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04b4106a90 0x7f04b4106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.233+0000 7f04baf17640 1 -- 172.21.15.105:0/387155416 >> 172.21.15.105:0/387155416 conn(0x7f04b40fc7d0 msgr2=0x7f04b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:32.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.233+0000 7f04baf17640 1 -- 172.21.15.105:0/387155416 shutdown_connections 2024-09-16T02:03:32.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.233+0000 7f04baf17640 1 -- 172.21.15.105:0/387155416 wait complete. 2024-09-16T02:03:32.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.234+0000 7f04baf17640 1 Processor -- start 2024-09-16T02:03:32.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.234+0000 7f04baf17640 1 -- start start 2024-09-16T02:03:32.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.234+0000 7f04baf17640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04b4100a90 0x7f04b410e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:32.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.234+0000 7f04baf17640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04b4101430 0x7f04b410e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:32.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.234+0000 7f04baf17640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04b4106a90 0x7f04b410ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:32.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.234+0000 7f04baf17640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f04b4111e90 con 0x7f04b4101430 2024-09-16T02:03:32.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.234+0000 7f04baf17640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f04b4111d10 con 0x7f04b4100a90 2024-09-16T02:03:32.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.234+0000 7f04baf17640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f04b4112010 con 0x7f04b4106a90 2024-09-16T02:03:32.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.234+0000 7f04b3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04b4101430 0x7f04b410e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:32.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.234+0000 7f04b8cb3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04b4100a90 0x7f04b410e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:32.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.234+0000 7f04b8cb3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04b4100a90 0x7f04b410e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.105:49444/0 (socket says 172.21.15.105:49444) 2024-09-16T02:03:32.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.234+0000 7f04b8cb3640 1 -- 172.21.15.105:0/2245804527 learned_addr learned my addr 172.21.15.105:0/2245804527 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:32.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.234+0000 7f04b94b4640 1 --2- 172.21.15.105:0/2245804527 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04b4106a90 0x7f04b410ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:32.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.235+0000 7f04b8cb3640 1 -- 172.21.15.105:0/2245804527 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04b4106a90 msgr2=0x7f04b410ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:32.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.235+0000 7f04b8cb3640 1 --2- 172.21.15.105:0/2245804527 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04b4106a90 0x7f04b410ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.235+0000 7f04b8cb3640 1 -- 172.21.15.105:0/2245804527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04b4101430 msgr2=0x7f04b410e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:32.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.235+0000 7f04b8cb3640 1 --2- 172.21.15.105:0/2245804527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04b4101430 0x7f04b410e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.235+0000 7f04b8cb3640 1 -- 172.21.15.105:0/2245804527 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04b41aab20 con 0x7f04b4100a90 2024-09-16T02:03:32.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.235+0000 7f04b3fff640 1 --2- 172.21.15.105:0/2245804527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04b4101430 0x7f04b410e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:03:32.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.235+0000 7f04b8cb3640 1 --2- 172.21.15.105:0/2245804527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04b4100a90 0x7f04b410e270 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f04a800d950 tx=0x7f04a800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:32.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.235+0000 7f04b1ffb640 1 -- 172.21.15.105:0/2245804527 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a8017070 con 0x7f04b4100a90 2024-09-16T02:03:32.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.235+0000 7f04b1ffb640 1 -- 172.21.15.105:0/2245804527 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04a8009c30 con 0x7f04b4100a90 2024-09-16T02:03:32.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.235+0000 7f04baf17640 1 -- 172.21.15.105:0/2245804527 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04b41aad50 con 0x7f04b4100a90 2024-09-16T02:03:32.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.235+0000 7f04b1ffb640 1 -- 172.21.15.105:0/2245804527 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a800f620 con 0x7f04b4100a90 2024-09-16T02:03:32.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.236+0000 7f04baf17640 1 -- 172.21.15.105:0/2245804527 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04b41ab180 con 0x7f04b4100a90 2024-09-16T02:03:32.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.239+0000 7f04b1ffb640 1 -- 172.21.15.105:0/2245804527 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f04a80109a0 con 0x7f04b4100a90 2024-09-16T02:03:32.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.239+0000 7f04bd2f0740 1 Processor -- start 2024-09-16T02:03:32.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.239+0000 7f04bd2f0740 1 -- start start 2024-09-16T02:03:32.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.240+0000 7f04b1ffb640 1 --2- 172.21.15.105:0/2245804527 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f04a40a3240 0x7f04a40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:32.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.240+0000 7f04b94b4640 1 --2- 172.21.15.105:0/2245804527 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f04a40a3240 0x7f04a40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:32.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.240+0000 7f04b1ffb640 1 -- 172.21.15.105:0/2245804527 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f04a80cf4f0 con 0x7f04b4100a90 2024-09-16T02:03:32.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.240+0000 7f04bd2f0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f04564b40 0x563f04563090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:32.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.240+0000 7f04b8cb3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f04564b40 0x563f04563090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:32.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.240+0000 7f04b8cb3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f04564b40 0x563f04563090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49798/0 (socket says 172.21.15.105:49798) 2024-09-16T02:03:32.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.240+0000 7f04bd2f0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f045635d0 0x563f04563a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:32.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.240+0000 7f04b8cb3640 1 -- 172.21.15.105:0/674399653 learned_addr learned my addr 172.21.15.105:0/674399653 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:32.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.240+0000 7f04bd2f0740 1 --2- 172.21.15.105:0/674399653 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f0456e370 0x563f04570760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:32.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.240+0000 7f04bd2f0740 1 -- 172.21.15.105:0/674399653 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563f044c92f0 con 0x563f04564b40 2024-09-16T02:03:32.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.240+0000 7f04bd2f0740 1 -- 172.21.15.105:0/674399653 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563f04470cf0 con 0x563f0456e370 2024-09-16T02:03:32.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.240+0000 7f04bd2f0740 1 -- 172.21.15.105:0/674399653 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x563f044c52b0 con 0x563f045635d0 2024-09-16T02:03:32.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.241+0000 7f04b3fff640 1 --2- 172.21.15.105:0/674399653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f045635d0 0x563f04563a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:32.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.241+0000 7f04b8cb3640 1 -- 172.21.15.105:0/674399653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f045635d0 msgr2=0x563f04563a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:32.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.241+0000 7f04b8cb3640 1 --2- 172.21.15.105:0/674399653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f045635d0 0x563f04563a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.241+0000 7f04b8cb3640 1 -- 172.21.15.105:0/674399653 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f0456e370 msgr2=0x563f04570760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:32.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.241+0000 7f04b8cb3640 1 --2- 172.21.15.105:0/674399653 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f0456e370 0x563f04570760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.241+0000 7f04b8cb3640 1 -- 172.21.15.105:0/674399653 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f044553a0 con 0x563f04564b40 2024-09-16T02:03:32.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.241+0000 7f04b94b4640 1 --2- 172.21.15.105:0/2245804527 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f04a40a3240 0x7f04a40a5700 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f04ac005e30 tx=0x7f04ac000fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:32.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.241+0000 7f04b8cb3640 1 --2- 172.21.15.105:0/674399653 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f04564b40 0x563f04563090 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f04a80a4190 tx=0x7f04a809b520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd966c3f66f5615 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:32.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.242+0000 7f049effd640 1 -- 172.21.15.105:0/674399653 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a8020040 con 0x563f04564b40 2024-09-16T02:03:32.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.242+0000 7f049effd640 1 -- 172.21.15.105:0/674399653 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04a8084e50 con 0x563f04564b40 2024-09-16T02:03:32.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.242+0000 7f049effd640 1 -- 172.21.15.105:0/674399653 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a80982e0 con 0x563f04564b40 2024-09-16T02:03:32.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.243+0000 7f04bd2f0740 1 -- 172.21.15.105:0/674399653 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f04564b40 msgr2=0x563f04563090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:32.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.243+0000 7f04bd2f0740 1 --2- 172.21.15.105:0/674399653 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f04564b40 0x563f04563090 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f04a80a4190 tx=0x7f04a809b520 comp rx=0 tx=0).stop 2024-09-16T02:03:32.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.243+0000 7f04bd2f0740 1 -- 172.21.15.105:0/674399653 shutdown_connections 2024-09-16T02:03:32.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.243+0000 7f04bd2f0740 1 --2- 172.21.15.105:0/674399653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f045635d0 0x563f04563a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.243+0000 7f04bd2f0740 1 --2- 172.21.15.105:0/674399653 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f0456e370 0x563f04570760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.243+0000 7f04bd2f0740 1 --2- 172.21.15.105:0/674399653 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f04564b40 0x563f04563090 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.243+0000 7f04bd2f0740 1 -- 172.21.15.105:0/674399653 >> 172.21.15.105:0/674399653 conn(0x563f0452c310 msgr2=0x563f0455fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:32.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.243+0000 7f04bd2f0740 1 -- 172.21.15.105:0/674399653 shutdown_connections 2024-09-16T02:03:32.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.243+0000 7f04bd2f0740 1 -- 172.21.15.105:0/674399653 wait complete. 2024-09-16T02:03:32.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.243+0000 7f04bd2f0740 1 -- start start 2024-09-16T02:03:32.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.243+0000 7f04bd2f0740 1 Processor -- start 2024-09-16T02:03:32.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.244+0000 7f04bd2f0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f045635d0 0x563f0456b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:32.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.244+0000 7f04bd2f0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f0456e370 0x563f0456b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:32.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.244+0000 7f04b3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f0456e370 0x563f0456b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:32.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.244+0000 7f04b3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f0456e370 0x563f0456b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49810/0 (socket says 172.21.15.105:49810) 2024-09-16T02:03:32.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.244+0000 7f04b8cb3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f045635d0 0x563f0456b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:32.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.244+0000 7f04b3fff640 1 -- 172.21.15.105:0/1812330432 learned_addr learned my addr 172.21.15.105:0/1812330432 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:32.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.244+0000 7f04bd2f0740 1 --2- 172.21.15.105:0/1812330432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f0456bd70 0x563f0457a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:32.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.244+0000 7f04bd2f0740 1 -- 172.21.15.105:0/1812330432 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563f04571070 con 0x563f0456e370 2024-09-16T02:03:32.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.244+0000 7f04bd2f0740 1 -- 172.21.15.105:0/1812330432 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563f04484060 con 0x563f0456bd70 2024-09-16T02:03:32.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.244+0000 7f04bd2f0740 1 -- 172.21.15.105:0/1812330432 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x563f045711f0 con 0x563f045635d0 2024-09-16T02:03:32.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.244+0000 7f04b94b4640 1 --2- 172.21.15.105:0/1812330432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f0456bd70 0x563f0457a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:32.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.244+0000 7f04b8cb3640 1 -- 172.21.15.105:0/1812330432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f0456bd70 msgr2=0x563f0457a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:32.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.245+0000 7f04b8cb3640 1 --2- 172.21.15.105:0/1812330432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f0456bd70 0x563f0457a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.245+0000 7f04b8cb3640 1 -- 172.21.15.105:0/1812330432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f0456e370 msgr2=0x563f0456b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:32.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.245+0000 7f04b8cb3640 1 --2- 172.21.15.105:0/1812330432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f0456e370 0x563f0456b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.245+0000 7f04bd2f0740 10 client.0 _ll_register_callbacks cb 0x563f041d78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:03:32.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.245+0000 7f04b8cb3640 1 -- 172.21.15.105:0/1812330432 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f0423a3c0 con 0x563f045635d0 2024-09-16T02:03:32.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.245+0000 7f04b3fff640 1 --2- 172.21.15.105:0/1812330432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f0456e370 0x563f0456b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:03:32.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.245+0000 7f04b94b4640 1 --2- 172.21.15.105:0/1812330432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f0456bd70 0x563f0457a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:03:32.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.245+0000 7f04bd2f0740 1 -- 172.21.15.105:0/1812330432 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04a8084e50 con 0x563f045635d0 2024-09-16T02:03:32.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.245+0000 7f04b8cb3640 1 --2- 172.21.15.105:0/1812330432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f045635d0 0x563f0456b2f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f04a80ce000 tx=0x7f04a80a46c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cd492b1fc85d974 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:32.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.245+0000 7f0493fff640 10 client.24865 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-16T02:03:32.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.245+0000 7f0493fff640 1 -- 172.21.15.105:0/1812330432 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a8020070 con 0x563f045635d0 2024-09-16T02:03:32.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.245+0000 7f0493fff640 1 -- 172.21.15.105:0/1812330432 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f04a80cf0d0 con 0x563f045635d0 2024-09-16T02:03:32.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.246+0000 7f0493fff640 1 -- 172.21.15.105:0/1812330432 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f04a80cf2d0 con 0x563f045635d0 2024-09-16T02:03:32.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.246+0000 7f04bd2f0740 10 client.24865 fetch_fsmap learned FSMap version 28 2024-09-16T02:03:32.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.246+0000 7f04bd2f0740 1 -- 172.21.15.105:0/1812330432 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({fsmap=28}) -- 0x563f04571110 con 0x563f045635d0 2024-09-16T02:03:32.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.246+0000 7f0493fff640 1 -- 172.21.15.105:0/1812330432 <== mon.1 v2:172.21.15.190:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f04a80860c0 con 0x563f045635d0 2024-09-16T02:03:32.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.246+0000 7f04bd2f0740 10 client.24865 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:03:32.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.246+0000 7f04bd2f0740 10 client.24865 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T02:03:32.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.246+0000 7f04bd2f0740 20 client.24865 populate_metadata read hostname 'smithi105' 2024-09-16T02:03:32.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.246+0000 7f04bd2f0740 1 --2- 172.21.15.105:0/1812330432 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x563f0457e1a0 0x563f04580590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:32.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.246+0000 7f04bd2f0740 4 client.24865 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:03:32.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.246+0000 7f04bd2f0740 1 -- 172.21.15.105:0/1812330432 --> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f044d6860 con 0x563f0457e1a0 2024-09-16T02:03:32.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.246+0000 7f04b3fff640 1 --2- 172.21.15.105:0/1812330432 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x563f0457e1a0 0x563f04580590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:32.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.247+0000 7f04b3fff640 1 --2- 172.21.15.105:0/1812330432 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x563f0457e1a0 0x563f04580590 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b0e6ff10574deeeb server_cookie=3924a06dc2af67b6 in_seq=0 out_seq=0 2024-09-16T02:03:32.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.247+0000 7f0493fff640 10 client.24865 ms_handle_connect on v2:172.21.15.190:6838/1222573537 2024-09-16T02:03:32.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.248+0000 7f0493fff640 1 -- 172.21.15.105:0/1812330432 <== mds.1 v2:172.21.15.190:6838/1222573537 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563f044d6860 con 0x563f0457e1a0 2024-09-16T02:03:32.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.248+0000 7f0493fff640 10 client.24865 handle_command_reply: tid=0 2024-09-16T02:03:32.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.248+0000 7f04bd2f0740 1 client.24865 shutdown 2024-09-16T02:03:32.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.248+0000 7f04bd2f0740 1 -- 172.21.15.105:0/1812330432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f045635d0 msgr2=0x563f0456b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:32.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.248+0000 7f04bd2f0740 1 --2- 172.21.15.105:0/1812330432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f045635d0 0x563f0456b2f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f04a80ce000 tx=0x7f04a80a46c0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.248+0000 7f04bd2f0740 1 -- 172.21.15.105:0/1812330432 shutdown_connections 2024-09-16T02:03:32.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.248+0000 7f04bd2f0740 1 --2- 172.21.15.105:0/1812330432 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x563f0457e1a0 0x563f04580590 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.248+0000 7f04bd2f0740 1 --2- 172.21.15.105:0/1812330432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f0456bd70 0x563f0457a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.248+0000 7f04bd2f0740 1 --2- 172.21.15.105:0/1812330432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f0456e370 0x563f0456b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.248+0000 7f04bd2f0740 1 --2- 172.21.15.105:0/1812330432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f045635d0 0x563f0456b2f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.248+0000 7f04bd2f0740 1 -- 172.21.15.105:0/1812330432 >> 172.21.15.105:0/1812330432 conn(0x563f0452c310 msgr2=0x563f04565990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:32.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.248+0000 7f0493fff640 0 client.24865 ms_handle_reset on v2:172.21.15.190:6838/1222573537 2024-09-16T02:03:32.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.249+0000 7f04bd2f0740 1 -- 172.21.15.105:0/1812330432 shutdown_connections 2024-09-16T02:03:32.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.249+0000 7f04bd2f0740 1 -- 172.21.15.105:0/1812330432 wait complete. 2024-09-16T02:03:32.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.249+0000 7f04bd2f0740 20 client.24865 trim_cache size 0 max 16384 2024-09-16T02:03:32.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.274+0000 7f04bd2f0740 1 Processor -- start 2024-09-16T02:03:32.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.274+0000 7f04bd2f0740 1 -- start start 2024-09-16T02:03:32.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.274+0000 7f04bd2f0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f045635d0 0x563f0456b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:32.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.274+0000 7f04bd2f0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f0456bd70 0x563f04585c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:32.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.274+0000 7f04bd2f0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f0456e370 0x563f04586190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:32.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.274+0000 7f04bd2f0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f04b41b3d10 con 0x563f0456e370 2024-09-16T02:03:32.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.274+0000 7f04bd2f0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f04b41b5850 con 0x563f0456bd70 2024-09-16T02:03:32.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.274+0000 7f04bd2f0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f04b41b43e0 con 0x563f045635d0 2024-09-16T02:03:32.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.274+0000 7f04b3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f0456bd70 0x563f04585c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:32.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.274+0000 7f04b8cb3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f045635d0 0x563f0456b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:32.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.274+0000 7f04b3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f0456bd70 0x563f04585c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.105:49464/0 (socket says 172.21.15.105:49464) 2024-09-16T02:03:32.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.274+0000 7f04b3fff640 1 -- 172.21.15.105:0/3042445849 learned_addr learned my addr 172.21.15.105:0/3042445849 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:32.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.274+0000 7f04b94b4640 1 --2- 172.21.15.105:0/3042445849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f0456e370 0x563f04586190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:32.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.275+0000 7f04b3fff640 1 -- 172.21.15.105:0/3042445849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f045635d0 msgr2=0x563f0456b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:32.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.275+0000 7f04b3fff640 1 --2- 172.21.15.105:0/3042445849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f045635d0 0x563f0456b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.275+0000 7f04b3fff640 1 -- 172.21.15.105:0/3042445849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f0456e370 msgr2=0x563f04586190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:32.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.275+0000 7f04b3fff640 1 --2- 172.21.15.105:0/3042445849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f0456e370 0x563f04586190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.275+0000 7f04b3fff640 1 -- 172.21.15.105:0/3042445849 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f0456db60 con 0x563f0456bd70 2024-09-16T02:03:32.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.275+0000 7f04b94b4640 1 --2- 172.21.15.105:0/3042445849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f0456e370 0x563f04586190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:03:32.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.275+0000 7f04b3fff640 1 --2- 172.21.15.105:0/3042445849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f0456bd70 0x563f04585c50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f04a0009fd0 tx=0x7f04a0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac970d482f786d09 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:32.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.275+0000 7f049e7fc640 1 -- 172.21.15.105:0/3042445849 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a0004030 con 0x563f0456bd70 2024-09-16T02:03:32.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.276+0000 7f049e7fc640 1 -- 172.21.15.105:0/3042445849 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04a000fde0 con 0x563f0456bd70 2024-09-16T02:03:32.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.276+0000 7f049e7fc640 1 -- 172.21.15.105:0/3042445849 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a0005620 con 0x563f0456bd70 2024-09-16T02:03:32.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.277+0000 7f04bd2f0740 1 -- 172.21.15.105:0/3042445849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f0456bd70 msgr2=0x563f04585c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:32.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.277+0000 7f04bd2f0740 1 --2- 172.21.15.105:0/3042445849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f0456bd70 0x563f04585c50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f04a0009fd0 tx=0x7f04a0005f90 comp rx=0 tx=0).stop 2024-09-16T02:03:32.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.277+0000 7f04bd2f0740 1 -- 172.21.15.105:0/3042445849 shutdown_connections 2024-09-16T02:03:32.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.277+0000 7f04bd2f0740 1 --2- 172.21.15.105:0/3042445849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f0456e370 0x563f04586190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.277+0000 7f04bd2f0740 1 --2- 172.21.15.105:0/3042445849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f0456bd70 0x563f04585c50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.277+0000 7f04bd2f0740 1 --2- 172.21.15.105:0/3042445849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f045635d0 0x563f0456b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.277+0000 7f04bd2f0740 1 -- 172.21.15.105:0/3042445849 >> 172.21.15.105:0/3042445849 conn(0x563f0452c310 msgr2=0x563f0455ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:32.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.277+0000 7f04bd2f0740 1 -- 172.21.15.105:0/3042445849 shutdown_connections 2024-09-16T02:03:32.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.277+0000 7f04bd2f0740 1 -- 172.21.15.105:0/3042445849 wait complete. 2024-09-16T02:03:32.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.277+0000 7f04bd2f0740 1 -- start start 2024-09-16T02:03:32.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.277+0000 7f04bd2f0740 1 Processor -- start 2024-09-16T02:03:32.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.278+0000 7f04bd2f0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f045635d0 0x563f0456f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:32.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.278+0000 7f04bd2f0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f0456bd70 0x563f0456fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:32.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.278+0000 7f04bd2f0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f0456e370 0x563f045700c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:32.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.278+0000 7f04bd2f0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f04b41b5180 con 0x563f0456bd70 2024-09-16T02:03:32.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.278+0000 7f04bd2f0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f04b41b4ab0 con 0x563f0456e370 2024-09-16T02:03:32.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.278+0000 7f04bd2f0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f04b41b1cf0 con 0x563f045635d0 2024-09-16T02:03:32.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.278+0000 7f04bd2f0740 10 client.0 _ll_register_callbacks cb 0x7f04a809b640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:03:32.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.278+0000 7f04b3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f0456bd70 0x563f0456fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:32.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.278+0000 7f04b3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f0456bd70 0x563f0456fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49820/0 (socket says 172.21.15.105:49820) 2024-09-16T02:03:32.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.278+0000 7f04b3fff640 1 -- 172.21.15.105:0/2335199789 learned_addr learned my addr 172.21.15.105:0/2335199789 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:32.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.279+0000 7f04b8cb3640 1 --2- 172.21.15.105:0/2335199789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f045635d0 0x563f0456f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:32.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.279+0000 7f04b3fff640 1 -- 172.21.15.105:0/2335199789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f045635d0 msgr2=0x563f0456f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:32.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.279+0000 7f04b3fff640 1 --2- 172.21.15.105:0/2335199789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f045635d0 0x563f0456f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.279+0000 7f04b3fff640 1 -- 172.21.15.105:0/2335199789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f0456e370 msgr2=0x563f045700c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:32.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.279+0000 7f04b3fff640 1 --2- 172.21.15.105:0/2335199789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f0456e370 0x563f045700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.279+0000 7f04b3fff640 1 -- 172.21.15.105:0/2335199789 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f04574e70 con 0x563f0456bd70 2024-09-16T02:03:32.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.279+0000 7f04b3fff640 1 --2- 172.21.15.105:0/2335199789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f0456bd70 0x563f0456fb80 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x563f0456c540 tx=0x7f04a00121f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa5c58dba0389a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:32.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.279+0000 7f0493fff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:03:32.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.279+0000 7f0493fff640 1 -- 172.21.15.105:0/2335199789 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a0035060 con 0x563f0456bd70 2024-09-16T02:03:32.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.280+0000 7f0493fff640 1 -- 172.21.15.105:0/2335199789 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f04a000f930 con 0x563f0456bd70 2024-09-16T02:03:32.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.280+0000 7f04bd2f0740 1 -- 172.21.15.105:0/2335199789 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04a000fde0 con 0x563f0456bd70 2024-09-16T02:03:32.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.280+0000 7f0493fff640 1 -- 172.21.15.105:0/2335199789 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f04a000fb30 con 0x563f0456bd70 2024-09-16T02:03:32.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.280+0000 7f04bd2f0740 10 client.15438 fetch_fsmap learned FSMap version 28 2024-09-16T02:03:32.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.280+0000 7f04bd2f0740 1 -- 172.21.15.105:0/2335199789 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563f045e90d0 con 0x563f0456bd70 2024-09-16T02:03:32.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.280+0000 7f0493fff640 1 -- 172.21.15.105:0/2335199789 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04a0004030 con 0x563f0456bd70 2024-09-16T02:03:32.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.280+0000 7f0493fff640 1 -- 172.21.15.105:0/2335199789 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f04a0039030 con 0x563f0456bd70 2024-09-16T02:03:32.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.280+0000 7f04bd2f0740 10 client.15438 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:03:32.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.280+0000 7f04bd2f0740 10 client.15438 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T02:03:32.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.280+0000 7f04bd2f0740 20 client.15438 populate_metadata read hostname 'smithi105' 2024-09-16T02:03:32.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.280+0000 7f04bd2f0740 1 --2- 172.21.15.105:0/2335199789 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x563f045e9690 0x563f045ebab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:32.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.281+0000 7f04bd2f0740 4 client.15438 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T02:03:32.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.281+0000 7f04bd2f0740 1 -- 172.21.15.105:0/2335199789 --> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f04a000fde0 con 0x563f045e9690 2024-09-16T02:03:32.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.281+0000 7f04b8cb3640 1 --2- 172.21.15.105:0/2335199789 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x563f045e9690 0x563f045ebab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:32.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.281+0000 7f04b8cb3640 1 --2- 172.21.15.105:0/2335199789 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x563f045e9690 0x563f045ebab0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3d849a1fa333afa7 server_cookie=c65d09eb72fbbc89 in_seq=0 out_seq=0 2024-09-16T02:03:32.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.281+0000 7f0493fff640 10 client.15438 ms_handle_connect on v2:172.21.15.190:6838/1222573537 2024-09-16T02:03:32.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.282+0000 7f0493fff640 1 -- 172.21.15.105:0/2335199789 <== mds.1 v2:172.21.15.190:6838/1222573537 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f04a000fde0 con 0x563f045e9690 2024-09-16T02:03:32.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.282+0000 7f0493fff640 10 client.15438 handle_command_reply: tid=0 2024-09-16T02:03:32.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.282+0000 7f04bd2f0740 1 client.15438 shutdown 2024-09-16T02:03:32.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.282+0000 7f04bd2f0740 1 -- 172.21.15.105:0/2335199789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f0456bd70 msgr2=0x563f0456fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:32.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.282+0000 7f04bd2f0740 1 --2- 172.21.15.105:0/2335199789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f0456bd70 0x563f0456fb80 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x563f0456c540 tx=0x7f04a00121f0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.282+0000 7f04bd2f0740 1 -- 172.21.15.105:0/2335199789 shutdown_connections 2024-09-16T02:03:32.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.282+0000 7f04bd2f0740 1 --2- 172.21.15.105:0/2335199789 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x563f045e9690 0x563f045ebab0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.282+0000 7f0493fff640 0 client.15438 ms_handle_reset on v2:172.21.15.190:6838/1222573537 2024-09-16T02:03:32.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.282+0000 7f04bd2f0740 1 --2- 172.21.15.105:0/2335199789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f0456e370 0x563f045700c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.282+0000 7f04bd2f0740 1 --2- 172.21.15.105:0/2335199789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f0456bd70 0x563f0456fb80 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.282+0000 7f04bd2f0740 1 --2- 172.21.15.105:0/2335199789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f045635d0 0x563f0456f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.282+0000 7f04bd2f0740 1 -- 172.21.15.105:0/2335199789 >> 172.21.15.105:0/2335199789 conn(0x563f0452c310 msgr2=0x563f0455e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:32.286 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-09-16T02:03:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.282+0000 7f04bd2f0740 1 -- 172.21.15.105:0/2335199789 shutdown_connections 2024-09-16T02:03:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.282+0000 7f04bd2f0740 1 -- 172.21.15.105:0/2335199789 wait complete. 2024-09-16T02:03:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.282+0000 7f04bd2f0740 20 client.15438 trim_cache size 0 max 16384 2024-09-16T02:03:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.283+0000 7f04baf17640 1 -- 172.21.15.105:0/2245804527 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f04a40a3240 msgr2=0x7f04a40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.283+0000 7f04baf17640 1 --2- 172.21.15.105:0/2245804527 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f04a40a3240 0x7f04a40a5700 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f04ac005e30 tx=0x7f04ac000fb0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.283+0000 7f04baf17640 1 -- 172.21.15.105:0/2245804527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04b4100a90 msgr2=0x7f04b410e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.283+0000 7f04baf17640 1 --2- 172.21.15.105:0/2245804527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04b4100a90 0x7f04b410e270 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f04a800d950 tx=0x7f04a800de10 comp rx=0 tx=0).stop 2024-09-16T02:03:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.283+0000 7f04baf17640 1 -- 172.21.15.105:0/2245804527 shutdown_connections 2024-09-16T02:03:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.283+0000 7f04baf17640 1 --2- 172.21.15.105:0/2245804527 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f04a40a3240 0x7f04a40a5700 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.283+0000 7f04baf17640 1 --2- 172.21.15.105:0/2245804527 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04b4106a90 0x7f04b410ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.283+0000 7f04baf17640 1 --2- 172.21.15.105:0/2245804527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04b4101430 0x7f04b410e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.283+0000 7f04baf17640 1 --2- 172.21.15.105:0/2245804527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04b4100a90 0x7f04b410e270 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.283+0000 7f04baf17640 1 -- 172.21.15.105:0/2245804527 >> 172.21.15.105:0/2245804527 conn(0x7f04b40fc7d0 msgr2=0x7f04b40fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:32.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.283+0000 7f04baf17640 1 -- 172.21.15.105:0/2245804527 shutdown_connections 2024-09-16T02:03:32.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:32.284+0000 7f04baf17640 1 -- 172.21.15.105:0/2245804527 wait complete. 2024-09-16T02:03:32.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:32 smithi190 ceph-mon[34762]: pgmap v369: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T02:03:32.923 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:32 smithi137 ceph-mon[34099]: pgmap v369: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T02:03:33.599 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:32 smithi105 ceph-mon[31197]: pgmap v369: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T02:03:33.600 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T02:03:33.863 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:03:34.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.387+0000 7f863bfff640 1 -- 172.21.15.105:0/2144265297 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8634031880 con 0x7f8644100a80 2024-09-16T02:03:34.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.388+0000 7f864b6a7640 1 -- 172.21.15.105:0/2144265297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8644100a80 msgr2=0x7f8644100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:34.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.388+0000 7f864b6a7640 1 --2- 172.21.15.105:0/2144265297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8644100a80 0x7f8644100ee0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8634009900 tx=0x7f863402a410 comp rx=0 tx=0).stop 2024-09-16T02:03:34.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.388+0000 7f864b6a7640 1 -- 172.21.15.105:0/2144265297 shutdown_connections 2024-09-16T02:03:34.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.388+0000 7f864b6a7640 1 --2- 172.21.15.105:0/2144265297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8644101420 0x7f864410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:34.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.388+0000 7f864b6a7640 1 --2- 172.21.15.105:0/2144265297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8644100a80 0x7f8644100ee0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:34.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.388+0000 7f864b6a7640 1 --2- 172.21.15.105:0/2144265297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8644106a80 0x7f8644106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:34.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.388+0000 7f864b6a7640 1 -- 172.21.15.105:0/2144265297 >> 172.21.15.105:0/2144265297 conn(0x7f86440fc7a0 msgr2=0x7f86440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:34.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.388+0000 7f864b6a7640 1 -- 172.21.15.105:0/2144265297 shutdown_connections 2024-09-16T02:03:34.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.388+0000 7f864b6a7640 1 -- 172.21.15.105:0/2144265297 wait complete. 2024-09-16T02:03:34.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.390+0000 7f864b6a7640 1 Processor -- start 2024-09-16T02:03:34.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.390+0000 7f864b6a7640 1 -- start start 2024-09-16T02:03:34.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.390+0000 7f864b6a7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8644100a80 0x7f864410e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:34.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.390+0000 7f864b6a7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8644101420 0x7f864410e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:34.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.390+0000 7f864b6a7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8644106a80 0x7f864410eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:34.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.390+0000 7f864b6a7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8644111e40 con 0x7f8644100a80 2024-09-16T02:03:34.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.390+0000 7f864b6a7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8644111cc0 con 0x7f8644101420 2024-09-16T02:03:34.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.390+0000 7f864b6a7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8644111fc0 con 0x7f8644106a80 2024-09-16T02:03:34.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.390+0000 7f8649443640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8644100a80 0x7f864410e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:34.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.390+0000 7f8649443640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8644100a80 0x7f864410e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49844/0 (socket says 172.21.15.105:49844) 2024-09-16T02:03:34.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.390+0000 7f8649443640 1 -- 172.21.15.105:0/166820150 learned_addr learned my addr 172.21.15.105:0/166820150 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:34.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.391+0000 7f8648c42640 1 --2- 172.21.15.105:0/166820150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8644101420 0x7f864410e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:34.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.391+0000 7f8649c44640 1 --2- 172.21.15.105:0/166820150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8644106a80 0x7f864410eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:34.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.391+0000 7f8649443640 1 -- 172.21.15.105:0/166820150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8644106a80 msgr2=0x7f864410eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:34.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.391+0000 7f8649443640 1 --2- 172.21.15.105:0/166820150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8644106a80 0x7f864410eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:34.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.391+0000 7f8649443640 1 -- 172.21.15.105:0/166820150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8644101420 msgr2=0x7f864410e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:34.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.391+0000 7f8649443640 1 --2- 172.21.15.105:0/166820150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8644101420 0x7f864410e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:34.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.391+0000 7f8649443640 1 -- 172.21.15.105:0/166820150 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86441a6800 con 0x7f8644100a80 2024-09-16T02:03:34.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.391+0000 7f8649443640 1 --2- 172.21.15.105:0/166820150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8644100a80 0x7f864410e0e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f864000e9f0 tx=0x7f864000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:34.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.391+0000 7f863a7fc640 1 -- 172.21.15.105:0/166820150 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864000cd80 con 0x7f8644100a80 2024-09-16T02:03:34.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.391+0000 7f864b6a7640 1 -- 172.21.15.105:0/166820150 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86441a6af0 con 0x7f8644100a80 2024-09-16T02:03:34.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.391+0000 7f864b6a7640 1 -- 172.21.15.105:0/166820150 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86441a7050 con 0x7f8644100a80 2024-09-16T02:03:34.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.391+0000 7f863a7fc640 1 -- 172.21.15.105:0/166820150 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86400108e0 con 0x7f8644100a80 2024-09-16T02:03:34.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.391+0000 7f863a7fc640 1 -- 172.21.15.105:0/166820150 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8640018920 con 0x7f8644100a80 2024-09-16T02:03:34.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.395+0000 7f864da80740 1 Processor -- start 2024-09-16T02:03:34.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.395+0000 7f864da80740 1 -- start start 2024-09-16T02:03:34.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.395+0000 7f864da80740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598c5189b40 0x5598c5188090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:34.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.395+0000 7f864da80740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598c51885d0 0x5598c5188a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:34.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.395+0000 7f864da80740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598c5193370 0x5598c5195760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:34.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.395+0000 7f864da80740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5598c50ee2f0 con 0x5598c5193370 2024-09-16T02:03:34.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.395+0000 7f864da80740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5598c5095cf0 con 0x5598c5189b40 2024-09-16T02:03:34.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.395+0000 7f864da80740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5598c50ea2b0 con 0x5598c51885d0 2024-09-16T02:03:34.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.396+0000 7f8648c42640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598c5193370 0x5598c5195760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:34.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.396+0000 7f8648c42640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598c5193370 0x5598c5195760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49846/0 (socket says 172.21.15.105:49846) 2024-09-16T02:03:34.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.396+0000 7f8648c42640 1 -- 172.21.15.105:0/1562255745 learned_addr learned my addr 172.21.15.105:0/1562255745 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:34.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.396+0000 7f8649c44640 1 --2- 172.21.15.105:0/1562255745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598c5189b40 0x5598c5188090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:34.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.396+0000 7f8648c42640 1 -- 172.21.15.105:0/1562255745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598c51885d0 msgr2=0x5598c5188a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T02:03:34.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.396+0000 7f8648c42640 1 --2- 172.21.15.105:0/1562255745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598c51885d0 0x5598c5188a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:34.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.396+0000 7f8648c42640 1 -- 172.21.15.105:0/1562255745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598c5189b40 msgr2=0x5598c5188090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:34.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.396+0000 7f8648c42640 1 --2- 172.21.15.105:0/1562255745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598c5189b40 0x5598c5188090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:34.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.396+0000 7f8648c42640 1 -- 172.21.15.105:0/1562255745 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598c507a3a0 con 0x5598c5193370 2024-09-16T02:03:34.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.396+0000 7f8649c44640 1 --2- 172.21.15.105:0/1562255745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598c5189b40 0x5598c5188090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:03:34.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.396+0000 7f8648c42640 1 --2- 172.21.15.105:0/1562255745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598c5193370 0x5598c5195760 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f863402a930 tx=0x7f8634032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=222234d0deea9c14 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:34.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.397+0000 7f862f7fe640 1 -- 172.21.15.105:0/1562255745 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863403f070 con 0x5598c5193370 2024-09-16T02:03:34.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.397+0000 7f863a7fc640 1 -- 172.21.15.105:0/166820150 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f8640010430 con 0x7f8644100a80 2024-09-16T02:03:34.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.397+0000 7f862f7fe640 1 -- 172.21.15.105:0/1562255745 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8634004030 con 0x5598c5193370 2024-09-16T02:03:34.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.397+0000 7f862f7fe640 1 -- 172.21.15.105:0/1562255745 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8634035070 con 0x5598c5193370 2024-09-16T02:03:34.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.398+0000 7f863a7fc640 1 --2- 172.21.15.105:0/166820150 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f86300a32c0 0x7f86300a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:34.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.398+0000 7f8649c44640 1 --2- 172.21.15.105:0/166820150 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f86300a32c0 0x7f86300a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:34.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.398+0000 7f863a7fc640 1 -- 172.21.15.105:0/166820150 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f86400cfbc0 con 0x7f8644100a80 2024-09-16T02:03:34.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.398+0000 7f8649c44640 1 --2- 172.21.15.105:0/166820150 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f86300a32c0 0x7f86300a5780 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f863c0083e0 tx=0x7f863c008370 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:34.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.398+0000 7f864da80740 1 -- 172.21.15.105:0/1562255745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598c5193370 msgr2=0x5598c5195760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:34.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.399+0000 7f864da80740 1 --2- 172.21.15.105:0/1562255745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598c5193370 0x5598c5195760 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f863402a930 tx=0x7f8634032c10 comp rx=0 tx=0).stop 2024-09-16T02:03:34.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.399+0000 7f864da80740 1 -- 172.21.15.105:0/1562255745 shutdown_connections 2024-09-16T02:03:34.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.399+0000 7f864da80740 1 --2- 172.21.15.105:0/1562255745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598c51885d0 0x5598c5188a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:34.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.399+0000 7f864da80740 1 --2- 172.21.15.105:0/1562255745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598c5193370 0x5598c5195760 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:34.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.399+0000 7f864da80740 1 --2- 172.21.15.105:0/1562255745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598c5189b40 0x5598c5188090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:34.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.399+0000 7f864da80740 1 -- 172.21.15.105:0/1562255745 >> 172.21.15.105:0/1562255745 conn(0x5598c5151310 msgr2=0x5598c5184c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:34.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.399+0000 7f864da80740 1 -- 172.21.15.105:0/1562255745 shutdown_connections 2024-09-16T02:03:34.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.399+0000 7f864da80740 1 -- 172.21.15.105:0/1562255745 wait complete. 2024-09-16T02:03:34.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.400+0000 7f864da80740 1 -- start start 2024-09-16T02:03:34.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.400+0000 7f864da80740 1 Processor -- start 2024-09-16T02:03:34.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.401+0000 7f864da80740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598c51885d0 0x5598c5192e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:34.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.401+0000 7f864da80740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598c5189b40 0x5598c51900d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:34.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.401+0000 7f864da80740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598c5193370 0x5598c5190860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:34.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.401+0000 7f864da80740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5598c5196010 con 0x5598c5193370 2024-09-16T02:03:34.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.401+0000 7f8649c44640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598c5193370 0x5598c5190860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:34.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.401+0000 7f8649c44640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598c5193370 0x5598c5190860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49858/0 (socket says 172.21.15.105:49858) 2024-09-16T02:03:34.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.401+0000 7f8649c44640 1 -- 172.21.15.105:0/2138278353 learned_addr learned my addr 172.21.15.105:0/2138278353 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:34.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.401+0000 7f864da80740 1 -- 172.21.15.105:0/2138278353 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5598c50a9060 con 0x5598c5189b40 2024-09-16T02:03:34.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.401+0000 7f864da80740 1 -- 172.21.15.105:0/2138278353 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5598c5196190 con 0x5598c51885d0 2024-09-16T02:03:34.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.401+0000 7f864da80740 10 client.0 _ll_register_callbacks cb 0x5598c4dfc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:03:34.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.402+0000 7f8649443640 1 --2- 172.21.15.105:0/2138278353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598c51885d0 0x5598c5192e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:34.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.402+0000 7f8649c44640 1 -- 172.21.15.105:0/2138278353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598c51885d0 msgr2=0x5598c5192e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:34.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.402+0000 7f8649c44640 1 --2- 172.21.15.105:0/2138278353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598c51885d0 0x5598c5192e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:34.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.402+0000 7f8648c42640 1 --2- 172.21.15.105:0/2138278353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598c5189b40 0x5598c51900d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:34.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.402+0000 7f8649c44640 1 -- 172.21.15.105:0/2138278353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598c5189b40 msgr2=0x5598c51900d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:34.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.402+0000 7f8649c44640 1 --2- 172.21.15.105:0/2138278353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598c5189b40 0x5598c51900d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:34.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.402+0000 7f8649c44640 1 -- 172.21.15.105:0/2138278353 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598c4e5f3c0 con 0x5598c5193370 2024-09-16T02:03:34.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.402+0000 7f8649c44640 1 --2- 172.21.15.105:0/2138278353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598c5193370 0x5598c5190860 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f863c011e30 tx=0x7f863c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25de69d1c2717533 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:34.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.402+0000 7f862cff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:03:34.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.402+0000 7f862cff9640 1 -- 172.21.15.105:0/2138278353 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863c01e420 con 0x5598c5193370 2024-09-16T02:03:34.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.402+0000 7f864da80740 1 -- 172.21.15.105:0/2138278353 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8634004030 con 0x5598c5193370 2024-09-16T02:03:34.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.402+0000 7f862cff9640 1 -- 172.21.15.105:0/2138278353 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f863c01e5c0 con 0x5598c5193370 2024-09-16T02:03:34.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.402+0000 7f862cff9640 1 -- 172.21.15.105:0/2138278353 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f863c01e7c0 con 0x5598c5193370 2024-09-16T02:03:34.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.402+0000 7f864da80740 10 client.15450 fetch_fsmap learned FSMap version 28 2024-09-16T02:03:34.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.403+0000 7f864da80740 1 -- 172.21.15.105:0/2138278353 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5598c51a1ca0 con 0x5598c5193370 2024-09-16T02:03:34.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.403+0000 7f862cff9640 1 -- 172.21.15.105:0/2138278353 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f863c01ee10 con 0x5598c5193370 2024-09-16T02:03:34.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.403+0000 7f862cff9640 1 -- 172.21.15.105:0/2138278353 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f863c030c00 con 0x5598c5193370 2024-09-16T02:03:34.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.403+0000 7f864da80740 10 client.15450 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:03:34.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.403+0000 7f864da80740 10 client.15450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:03:34.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.403+0000 7f864da80740 20 client.15450 populate_metadata read hostname 'smithi105' 2024-09-16T02:03:34.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.403+0000 7f864da80740 1 --2- 172.21.15.105:0/2138278353 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x5598c51a35e0 0x5598c51a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:34.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.403+0000 7f864da80740 4 client.15450 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:03:34.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.403+0000 7f864da80740 1 -- 172.21.15.105:0/2138278353 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8634004030 con 0x5598c51a35e0 2024-09-16T02:03:34.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.404+0000 7f8649443640 1 --2- 172.21.15.105:0/2138278353 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x5598c51a35e0 0x5598c51a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:34.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.404+0000 7f862cff9640 10 client.15450 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:03:34.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.404+0000 7f8649443640 1 --2- 172.21.15.105:0/2138278353 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x5598c51a35e0 0x5598c51a5a40 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6094adde0020616 server_cookie=4e1211d5d05b008a in_seq=0 out_seq=0 2024-09-16T02:03:34.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.405+0000 7f862cff9640 1 -- 172.21.15.105:0/2138278353 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f8634004030 con 0x5598c51a35e0 2024-09-16T02:03:34.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.405+0000 7f862cff9640 10 client.15450 handle_command_reply: tid=0 2024-09-16T02:03:34.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.405+0000 7f864da80740 1 client.15450 shutdown 2024-09-16T02:03:34.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.405+0000 7f864da80740 1 -- 172.21.15.105:0/2138278353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598c5193370 msgr2=0x5598c5190860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:34.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.405+0000 7f864da80740 1 --2- 172.21.15.105:0/2138278353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598c5193370 0x5598c5190860 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f863c011e30 tx=0x7f863c004a40 comp rx=0 tx=0).stop 2024-09-16T02:03:34.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.405+0000 7f864da80740 1 -- 172.21.15.105:0/2138278353 shutdown_connections 2024-09-16T02:03:34.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.405+0000 7f864da80740 1 --2- 172.21.15.105:0/2138278353 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x5598c51a35e0 0x5598c51a5a40 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:34.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.405+0000 7f864da80740 1 --2- 172.21.15.105:0/2138278353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598c5193370 0x5598c5190860 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:34.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.405+0000 7f864da80740 1 --2- 172.21.15.105:0/2138278353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598c5189b40 0x5598c51900d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:34.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.405+0000 7f864da80740 1 --2- 172.21.15.105:0/2138278353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598c51885d0 0x5598c5192e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:34.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.405+0000 7f864da80740 1 -- 172.21.15.105:0/2138278353 >> 172.21.15.105:0/2138278353 conn(0x5598c5151310 msgr2=0x5598c518a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:34.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.405+0000 7f862cff9640 0 client.15450 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:03:34.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.406+0000 7f864da80740 1 -- 172.21.15.105:0/2138278353 shutdown_connections 2024-09-16T02:03:34.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.406+0000 7f864da80740 1 -- 172.21.15.105:0/2138278353 wait complete. 2024-09-16T02:03:34.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.406+0000 7f864da80740 20 client.15450 trim_cache size 0 max 16384 2024-09-16T02:03:34.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.425+0000 7f864da80740 1 Processor -- start 2024-09-16T02:03:34.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.425+0000 7f864da80740 1 -- start start 2024-09-16T02:03:34.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.426+0000 7f864da80740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598c51885d0 0x5598c518e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:34.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.426+0000 7f864da80740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598c5189b40 0x5598c518eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:34.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.426+0000 7f864da80740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598c5193370 0x5598c51ab350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:34.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.426+0000 7f864da80740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f862800e6b0 con 0x5598c51885d0 2024-09-16T02:03:34.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.426+0000 7f864da80740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f86280101f0 con 0x5598c5193370 2024-09-16T02:03:34.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.426+0000 7f864da80740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f862800ed80 con 0x5598c5189b40 2024-09-16T02:03:34.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.426+0000 7f8648c42640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598c5189b40 0x5598c518eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:34.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.426+0000 7f8649443640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598c51885d0 0x5598c518e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:34.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.426+0000 7f8649443640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598c51885d0 0x5598c518e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49862/0 (socket says 172.21.15.105:49862) 2024-09-16T02:03:34.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.426+0000 7f8649443640 1 -- 172.21.15.105:0/1228345054 learned_addr learned my addr 172.21.15.105:0/1228345054 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:34.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.426+0000 7f8648c42640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598c5189b40 0x5598c518eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.105:51786/0 (socket says 172.21.15.105:51786) 2024-09-16T02:03:34.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.426+0000 7f8649c44640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598c5193370 0x5598c51ab350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:34.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.427+0000 7f8649443640 1 -- 172.21.15.105:0/1228345054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598c5189b40 msgr2=0x5598c518eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:34.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.427+0000 7f8649443640 1 --2- 172.21.15.105:0/1228345054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598c5189b40 0x5598c518eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:34.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.427+0000 7f8649443640 1 -- 172.21.15.105:0/1228345054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598c5193370 msgr2=0x5598c51ab350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:34.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.427+0000 7f8649443640 1 --2- 172.21.15.105:0/1228345054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598c5193370 0x5598c51ab350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:34.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.427+0000 7f8649443640 1 -- 172.21.15.105:0/1228345054 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598c518b120 con 0x5598c51885d0 2024-09-16T02:03:34.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.427+0000 7f8649c44640 1 --2- 172.21.15.105:0/1228345054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598c5193370 0x5598c51ab350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:03:34.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.428+0000 7f8649443640 1 --2- 172.21.15.105:0/1228345054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598c51885d0 0x5598c518e9a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f864000e9c0 tx=0x7f86400cd530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=422ebc830bfad15a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:34.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.428+0000 7f862effd640 1 -- 172.21.15.105:0/1228345054 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86400a0530 con 0x5598c51885d0 2024-09-16T02:03:34.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.428+0000 7f862effd640 1 -- 172.21.15.105:0/1228345054 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86400a4df0 con 0x5598c51885d0 2024-09-16T02:03:34.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.428+0000 7f862effd640 1 -- 172.21.15.105:0/1228345054 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86400a4960 con 0x5598c51885d0 2024-09-16T02:03:34.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.429+0000 7f864da80740 1 -- 172.21.15.105:0/1228345054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598c51885d0 msgr2=0x5598c518e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:34.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.429+0000 7f864da80740 1 --2- 172.21.15.105:0/1228345054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598c51885d0 0x5598c518e9a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f864000e9c0 tx=0x7f86400cd530 comp rx=0 tx=0).stop 2024-09-16T02:03:34.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.429+0000 7f864da80740 1 -- 172.21.15.105:0/1228345054 shutdown_connections 2024-09-16T02:03:34.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.429+0000 7f864da80740 1 --2- 172.21.15.105:0/1228345054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598c5189b40 0x5598c518eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:34.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.429+0000 7f864da80740 1 --2- 172.21.15.105:0/1228345054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598c5193370 0x5598c51ab350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:34.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.429+0000 7f864da80740 1 --2- 172.21.15.105:0/1228345054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598c51885d0 0x5598c518e9a0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:34.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.429+0000 7f864da80740 1 -- 172.21.15.105:0/1228345054 >> 172.21.15.105:0/1228345054 conn(0x5598c5151310 msgr2=0x5598c518b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:34.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.429+0000 7f864da80740 1 -- 172.21.15.105:0/1228345054 shutdown_connections 2024-09-16T02:03:34.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.429+0000 7f864da80740 1 -- 172.21.15.105:0/1228345054 wait complete. 2024-09-16T02:03:34.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.430+0000 7f864da80740 1 -- start start 2024-09-16T02:03:34.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.430+0000 7f864da80740 1 Processor -- start 2024-09-16T02:03:34.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.430+0000 7f864da80740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598c51885d0 0x5598c519b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:34.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.430+0000 7f864da80740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598c5189b40 0x5598c519bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:34.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.431+0000 7f864da80740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598c5193370 0x5598c5196c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:34.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.431+0000 7f864da80740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f862800fb20 con 0x5598c5193370 2024-09-16T02:03:34.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.431+0000 7f864da80740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f862800f450 con 0x5598c5189b40 2024-09-16T02:03:34.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.431+0000 7f864da80740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f862800c690 con 0x5598c51885d0 2024-09-16T02:03:34.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.431+0000 7f864da80740 10 client.0 _ll_register_callbacks cb 0x7f863c008f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:03:34.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.431+0000 7f8648c42640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598c5189b40 0x5598c519bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:34.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.431+0000 7f8648c42640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598c5189b40 0x5598c519bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.105:49526/0 (socket says 172.21.15.105:49526) 2024-09-16T02:03:34.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.431+0000 7f8648c42640 1 -- 172.21.15.105:0/3829252933 learned_addr learned my addr 172.21.15.105:0/3829252933 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:34.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.431+0000 7f8649c44640 1 --2- 172.21.15.105:0/3829252933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598c5193370 0x5598c5196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:34.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.431+0000 7f8649443640 1 --2- 172.21.15.105:0/3829252933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598c51885d0 0x5598c519b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:34.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.431+0000 7f8649443640 1 -- 172.21.15.105:0/3829252933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598c5189b40 msgr2=0x5598c519bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:34.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.431+0000 7f8649443640 1 --2- 172.21.15.105:0/3829252933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598c5189b40 0x5598c519bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:34.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.431+0000 7f8649443640 1 -- 172.21.15.105:0/3829252933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598c5193370 msgr2=0x5598c5196c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:34.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.432+0000 7f8649443640 1 --2- 172.21.15.105:0/3829252933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598c5193370 0x5598c5196c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:34.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.432+0000 7f8649443640 1 -- 172.21.15.105:0/3829252933 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598c5197450 con 0x5598c51885d0 2024-09-16T02:03:34.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.432+0000 7f8649443640 1 --2- 172.21.15.105:0/3829252933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598c51885d0 0x5598c519b7d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f86400985e0 tx=0x7f8640005b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46a84e728ccc93ac server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:34.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.432+0000 7f862cff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-16T02:03:34.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.432+0000 7f862cff9640 1 -- 172.21.15.105:0/3829252933 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86400af070 con 0x5598c51885d0 2024-09-16T02:03:34.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.432+0000 7f862cff9640 1 -- 172.21.15.105:0/3829252933 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f86400987f0 con 0x5598c51885d0 2024-09-16T02:03:34.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.432+0000 7f864da80740 1 -- 172.21.15.105:0/3829252933 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86400a4df0 con 0x5598c51885d0 2024-09-16T02:03:34.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.433+0000 7f862cff9640 1 -- 172.21.15.105:0/3829252933 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f86400a0530 con 0x5598c51885d0 2024-09-16T02:03:34.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.433+0000 7f864da80740 10 client.24883 fetch_fsmap learned FSMap version 28 2024-09-16T02:03:34.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.433+0000 7f864da80740 1 -- 172.21.15.105:0/3829252933 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5598c5216730 con 0x5598c51885d0 2024-09-16T02:03:34.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.433+0000 7f862cff9640 1 -- 172.21.15.105:0/3829252933 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86400a0170 con 0x5598c51885d0 2024-09-16T02:03:34.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.434+0000 7f862cff9640 1 -- 172.21.15.105:0/3829252933 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f86400cdb20 con 0x5598c51885d0 2024-09-16T02:03:34.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.434+0000 7f864da80740 10 client.24883 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:03:34.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.434+0000 7f864da80740 10 client.24883 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:03:34.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.434+0000 7f864da80740 20 client.24883 populate_metadata read hostname 'smithi105' 2024-09-16T02:03:34.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.434+0000 7f864da80740 1 --2- 172.21.15.105:0/3829252933 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x5598c5216d00 0x5598c52190f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:34.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.434+0000 7f864da80740 4 client.24883 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T02:03:34.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.434+0000 7f864da80740 1 -- 172.21.15.105:0/3829252933 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f86400a4df0 con 0x5598c5216d00 2024-09-16T02:03:34.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.435+0000 7f8648c42640 1 --2- 172.21.15.105:0/3829252933 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x5598c5216d00 0x5598c52190f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:34.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.435+0000 7f862cff9640 10 client.24883 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:03:34.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.435+0000 7f8648c42640 1 --2- 172.21.15.105:0/3829252933 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x5598c5216d00 0x5598c52190f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11b29e5b84aaecc1 server_cookie=db3d4bbfbd8020d in_seq=0 out_seq=0 2024-09-16T02:03:34.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.435+0000 7f862cff9640 1 -- 172.21.15.105:0/3829252933 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f86400a4df0 con 0x5598c5216d00 2024-09-16T02:03:34.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.435+0000 7f862cff9640 10 client.24883 handle_command_reply: tid=0 2024-09-16T02:03:34.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.435+0000 7f864da80740 1 client.24883 shutdown 2024-09-16T02:03:34.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.436+0000 7f864da80740 1 -- 172.21.15.105:0/3829252933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598c51885d0 msgr2=0x5598c519b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:34.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.436+0000 7f864da80740 1 --2- 172.21.15.105:0/3829252933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598c51885d0 0x5598c519b7d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f86400985e0 tx=0x7f8640005b20 comp rx=0 tx=0).stop 2024-09-16T02:03:34.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.436+0000 7f864da80740 1 -- 172.21.15.105:0/3829252933 shutdown_connections 2024-09-16T02:03:34.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.436+0000 7f864da80740 1 --2- 172.21.15.105:0/3829252933 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x5598c5216d00 0x5598c52190f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:34.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.436+0000 7f864da80740 1 --2- 172.21.15.105:0/3829252933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598c5193370 0x5598c5196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:34.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.436+0000 7f864da80740 1 --2- 172.21.15.105:0/3829252933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598c5189b40 0x5598c519bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:34.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.436+0000 7f864da80740 1 --2- 172.21.15.105:0/3829252933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598c51885d0 0x5598c519b7d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:34.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.436+0000 7f864da80740 1 -- 172.21.15.105:0/3829252933 >> 172.21.15.105:0/3829252933 conn(0x5598c5151310 msgr2=0x5598c518e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:34.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.436+0000 7f862cff9640 0 client.24883 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:03:34.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.436+0000 7f864da80740 1 -- 172.21.15.105:0/3829252933 shutdown_connections 2024-09-16T02:03:34.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.436+0000 7f864da80740 1 -- 172.21.15.105:0/3829252933 wait complete. 2024-09-16T02:03:34.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.436+0000 7f864da80740 20 client.24883 trim_cache size 0 max 16384 2024-09-16T02:03:34.450 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-09-16T02:03:34.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.437+0000 7f864b6a7640 1 -- 172.21.15.105:0/166820150 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f86300a32c0 msgr2=0x7f86300a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:34.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.437+0000 7f864b6a7640 1 --2- 172.21.15.105:0/166820150 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f86300a32c0 0x7f86300a5780 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f863c0083e0 tx=0x7f863c008370 comp rx=0 tx=0).stop 2024-09-16T02:03:34.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.437+0000 7f864b6a7640 1 -- 172.21.15.105:0/166820150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8644100a80 msgr2=0x7f864410e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:34.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.437+0000 7f864b6a7640 1 --2- 172.21.15.105:0/166820150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8644100a80 0x7f864410e0e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f864000e9f0 tx=0x7f864000eeb0 comp rx=0 tx=0).stop 2024-09-16T02:03:34.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.437+0000 7f864b6a7640 1 -- 172.21.15.105:0/166820150 shutdown_connections 2024-09-16T02:03:34.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.438+0000 7f864b6a7640 1 --2- 172.21.15.105:0/166820150 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f86300a32c0 0x7f86300a5780 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:34.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.438+0000 7f864b6a7640 1 --2- 172.21.15.105:0/166820150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8644106a80 0x7f864410eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:34.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.438+0000 7f864b6a7640 1 --2- 172.21.15.105:0/166820150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8644101420 0x7f864410e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:34.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.438+0000 7f864b6a7640 1 --2- 172.21.15.105:0/166820150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8644100a80 0x7f864410e0e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:34.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.438+0000 7f864b6a7640 1 -- 172.21.15.105:0/166820150 >> 172.21.15.105:0/166820150 conn(0x7f86440fc7a0 msgr2=0x7f86440fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:34.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.438+0000 7f864b6a7640 1 -- 172.21.15.105:0/166820150 shutdown_connections 2024-09-16T02:03:34.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:34.438+0000 7f864b6a7640 1 -- 172.21.15.105:0/166820150 wait complete. 2024-09-16T02:03:34.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:34 smithi190 ceph-mon[34762]: pgmap v370: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-16T02:03:34.863 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:34 smithi105 ceph-mon[31197]: pgmap v370: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-16T02:03:34.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:34 smithi137 ceph-mon[34099]: pgmap v370: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-16T02:03:36.264 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T02:03:36.265 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T02:03:36.562 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:03:36.590 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:36 smithi105 ceph-mon[31197]: pgmap v371: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T02:03:36.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:36 smithi190 ceph-mon[34762]: pgmap v371: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T02:03:36.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:36 smithi137 ceph-mon[34099]: pgmap v371: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T02:03:37.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.135+0000 7f515cf1e640 1 -- 172.21.15.105:0/1356214016 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5158106f20 msgr2=0x7f515810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:37.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.135+0000 7f515cf1e640 1 --2- 172.21.15.105:0/1356214016 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5158106f20 0x7f515810eaa0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5150009ca0 tx=0x7f515002a8c0 comp rx=0 tx=0).stop 2024-09-16T02:03:37.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.135+0000 7f515cf1e640 1 -- 172.21.15.105:0/1356214016 shutdown_connections 2024-09-16T02:03:37.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.135+0000 7f515cf1e640 1 --2- 172.21.15.105:0/1356214016 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5158106f20 0x7f515810eaa0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:37.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.135+0000 7f515cf1e640 1 --2- 172.21.15.105:0/1356214016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51581027a0 0x7f51581067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:37.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.135+0000 7f515cf1e640 1 --2- 172.21.15.105:0/1356214016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5158101df0 0x7f51581021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:37.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.135+0000 7f515cf1e640 1 -- 172.21.15.105:0/1356214016 >> 172.21.15.105:0/1356214016 conn(0x7f51580fd8f0 msgr2=0x7f51580ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:37.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.135+0000 7f515cf1e640 1 -- 172.21.15.105:0/1356214016 shutdown_connections 2024-09-16T02:03:37.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.135+0000 7f515cf1e640 1 -- 172.21.15.105:0/1356214016 wait complete. 2024-09-16T02:03:37.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.136+0000 7f515cf1e640 1 Processor -- start 2024-09-16T02:03:37.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.136+0000 7f515cf1e640 1 -- start start 2024-09-16T02:03:37.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.136+0000 7f515cf1e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5158101df0 0x7f5158198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:37.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.136+0000 7f515cf1e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51581027a0 0x7f5158198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:37.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.137+0000 7f515cf1e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5158106f20 0x7f515819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:37.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.137+0000 7f515cf1e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5158111e30 con 0x7f5158106f20 2024-09-16T02:03:37.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.137+0000 7f515cf1e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5158111cb0 con 0x7f5158101df0 2024-09-16T02:03:37.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.137+0000 7f515cf1e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5158111fb0 con 0x7f51581027a0 2024-09-16T02:03:37.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.137+0000 7f5156d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5158106f20 0x7f515819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:37.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.137+0000 7f5156d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5158106f20 0x7f515819cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49896/0 (socket says 172.21.15.105:49896) 2024-09-16T02:03:37.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.137+0000 7f5156d9d640 1 -- 172.21.15.105:0/2129161363 learned_addr learned my addr 172.21.15.105:0/2129161363 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:37.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.137+0000 7f515659c640 1 --2- 172.21.15.105:0/2129161363 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5158101df0 0x7f5158198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:37.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.137+0000 7f5156d9d640 1 -- 172.21.15.105:0/2129161363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51581027a0 msgr2=0x7f5158198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:37.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.137+0000 7f5156d9d640 1 --2- 172.21.15.105:0/2129161363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51581027a0 0x7f5158198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:37.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.137+0000 7f5156d9d640 1 -- 172.21.15.105:0/2129161363 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5158101df0 msgr2=0x7f5158198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:37.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.137+0000 7f5156d9d640 1 --2- 172.21.15.105:0/2129161363 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5158101df0 0x7f5158198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:37.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.137+0000 7f5156d9d640 1 -- 172.21.15.105:0/2129161363 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f515819d300 con 0x7f5158106f20 2024-09-16T02:03:37.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.137+0000 7f5156d9d640 1 --2- 172.21.15.105:0/2129161363 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5158106f20 0x7f515819cc20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5150009dc0 tx=0x7f5150032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:37.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.138+0000 7f51477fe640 1 -- 172.21.15.105:0/2129161363 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5150003cf0 con 0x7f5158106f20 2024-09-16T02:03:37.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.138+0000 7f51477fe640 1 -- 172.21.15.105:0/2129161363 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51500044f0 con 0x7f5158106f20 2024-09-16T02:03:37.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.138+0000 7f51477fe640 1 -- 172.21.15.105:0/2129161363 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5150030bf0 con 0x7f5158106f20 2024-09-16T02:03:37.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.138+0000 7f515cf1e640 1 -- 172.21.15.105:0/2129161363 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f515819d590 con 0x7f5158106f20 2024-09-16T02:03:37.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.138+0000 7f515cf1e640 1 -- 172.21.15.105:0/2129161363 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f515819d9c0 con 0x7f5158106f20 2024-09-16T02:03:37.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.140+0000 7f51477fe640 1 -- 172.21.15.105:0/2129161363 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f515002f4c0 con 0x7f5158106f20 2024-09-16T02:03:37.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.140+0000 7f515f2f7740 1 Processor -- start 2024-09-16T02:03:37.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.141+0000 7f515f2f7740 1 -- start start 2024-09-16T02:03:37.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.141+0000 7f515f2f7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ffa31cb60 0x563ffa31cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:37.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.141+0000 7f515f2f7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ffa31d480 0x563ffa327800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:37.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.141+0000 7f51477fe640 1 --2- 172.21.15.105:0/2129161363 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f51400a3390 0x7f51400a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:37.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.141+0000 7f515f2f7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ffa327d40 0x563ffa32a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:37.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.142+0000 7f51477fe640 1 -- 172.21.15.105:0/2129161363 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f51500f0f80 con 0x7f5158106f20 2024-09-16T02:03:37.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.142+0000 7f5155d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ffa327d40 0x563ffa32a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:37.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.142+0000 7f5155d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ffa327d40 0x563ffa32a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49902/0 (socket says 172.21.15.105:49902) 2024-09-16T02:03:37.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.142+0000 7f5155d9b640 1 -- 172.21.15.105:0/1242660470 learned_addr learned my addr 172.21.15.105:0/1242660470 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:37.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.142+0000 7f515f2f7740 1 -- 172.21.15.105:0/1242660470 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563ffa282280 con 0x563ffa327d40 2024-09-16T02:03:37.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.142+0000 7f515f2f7740 1 -- 172.21.15.105:0/1242660470 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563ffa229b10 con 0x563ffa31d480 2024-09-16T02:03:37.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.142+0000 7f515f2f7740 1 -- 172.21.15.105:0/1242660470 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x563ffa27e240 con 0x563ffa31cb60 2024-09-16T02:03:37.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.142+0000 7f515659c640 1 --2- 172.21.15.105:0/1242660470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ffa31d480 0x563ffa327800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:37.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.142+0000 7f5156d9d640 1 --2- 172.21.15.105:0/2129161363 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f51400a3390 0x7f51400a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:37.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.142+0000 7f5155d9b640 1 --2- 172.21.15.105:0/1242660470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ffa31cb60 0x563ffa31cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:37.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.142+0000 7f5155d9b640 1 -- 172.21.15.105:0/1242660470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ffa31cb60 msgr2=0x563ffa31cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:37.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.142+0000 7f5155d9b640 1 --2- 172.21.15.105:0/1242660470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ffa31cb60 0x563ffa31cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:37.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.142+0000 7f5155d9b640 1 -- 172.21.15.105:0/1242660470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ffa31d480 msgr2=0x563ffa327800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:37.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.142+0000 7f5155d9b640 1 --2- 172.21.15.105:0/1242660470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ffa31d480 0x563ffa327800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:37.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.143+0000 7f5155d9b640 1 -- 172.21.15.105:0/1242660470 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ffa2361f0 con 0x563ffa327d40 2024-09-16T02:03:37.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.143+0000 7f5156d9d640 1 --2- 172.21.15.105:0/2129161363 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f51400a3390 0x7f51400a5850 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5150006010 tx=0x7f515003e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:37.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.143+0000 7f5155d9b640 1 --2- 172.21.15.105:0/1242660470 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ffa327d40 0x563ffa32a130 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f514c00bf50 tx=0x7f514c00b2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78cd4edffab730e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:37.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.143+0000 7f5144ff9640 1 -- 172.21.15.105:0/1242660470 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c0125f0 con 0x563ffa327d40 2024-09-16T02:03:37.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.143+0000 7f5144ff9640 1 -- 172.21.15.105:0/1242660470 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f514c012c40 con 0x563ffa327d40 2024-09-16T02:03:37.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.143+0000 7f5144ff9640 1 -- 172.21.15.105:0/1242660470 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c00eaa0 con 0x563ffa327d40 2024-09-16T02:03:37.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.144+0000 7f515f2f7740 1 -- 172.21.15.105:0/1242660470 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ffa327d40 msgr2=0x563ffa32a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:37.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.144+0000 7f515f2f7740 1 --2- 172.21.15.105:0/1242660470 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ffa327d40 0x563ffa32a130 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f514c00bf50 tx=0x7f514c00b2d0 comp rx=0 tx=0).stop 2024-09-16T02:03:37.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.144+0000 7f515f2f7740 1 -- 172.21.15.105:0/1242660470 shutdown_connections 2024-09-16T02:03:37.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.144+0000 7f515f2f7740 1 --2- 172.21.15.105:0/1242660470 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ffa327d40 0x563ffa32a130 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:37.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.144+0000 7f515f2f7740 1 --2- 172.21.15.105:0/1242660470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ffa31d480 0x563ffa327800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:37.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.144+0000 7f515f2f7740 1 --2- 172.21.15.105:0/1242660470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ffa31cb60 0x563ffa31cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:37.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.144+0000 7f515f2f7740 1 -- 172.21.15.105:0/1242660470 >> 172.21.15.105:0/1242660470 conn(0x563ffa3190f0 msgr2=0x563ffa3194d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:37.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.145+0000 7f515f2f7740 1 -- 172.21.15.105:0/1242660470 shutdown_connections 2024-09-16T02:03:37.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.145+0000 7f515f2f7740 1 -- 172.21.15.105:0/1242660470 wait complete. 2024-09-16T02:03:37.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.145+0000 7f515f2f7740 1 -- start start 2024-09-16T02:03:37.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.145+0000 7f515f2f7740 1 Processor -- start 2024-09-16T02:03:37.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.146+0000 7f515f2f7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ffa31cb60 0x563ffa324c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:37.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.146+0000 7f515f2f7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ffa31d480 0x563ffa325180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:37.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.146+0000 7f515f2f7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ffa327d40 0x563ffa3256c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:37.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.146+0000 7f515f2f7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563ffa32a910 con 0x563ffa31d480 2024-09-16T02:03:37.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.146+0000 7f515f2f7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563ffa23cf10 con 0x563ffa327d40 2024-09-16T02:03:37.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.146+0000 7f515f2f7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x563ffa32aa90 con 0x563ffa31cb60 2024-09-16T02:03:37.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.146+0000 7f515f2f7740 10 client.0 _ll_register_callbacks cb 0x563ffa205a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:03:37.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.146+0000 7f515659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ffa31d480 0x563ffa325180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:37.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.146+0000 7f515659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ffa31d480 0x563ffa325180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49918/0 (socket says 172.21.15.105:49918) 2024-09-16T02:03:37.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.146+0000 7f515659c640 1 -- 172.21.15.105:0/396138723 learned_addr learned my addr 172.21.15.105:0/396138723 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:37.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.146+0000 7f5155d9b640 1 --2- 172.21.15.105:0/396138723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ffa327d40 0x563ffa3256c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:37.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.146+0000 7f5155d9b640 1 --2- 172.21.15.105:0/396138723 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ffa31cb60 0x563ffa324c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:37.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.146+0000 7f515659c640 1 -- 172.21.15.105:0/396138723 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ffa31cb60 msgr2=0x563ffa324c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:37.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.146+0000 7f515659c640 1 --2- 172.21.15.105:0/396138723 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ffa31cb60 0x563ffa324c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:37.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.146+0000 7f515659c640 1 -- 172.21.15.105:0/396138723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ffa327d40 msgr2=0x563ffa3256c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:37.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.146+0000 7f515659c640 1 --2- 172.21.15.105:0/396138723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ffa327d40 0x563ffa3256c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:37.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.146+0000 7f515659c640 1 -- 172.21.15.105:0/396138723 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ff9ff36a0 con 0x563ffa31d480 2024-09-16T02:03:37.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.146+0000 7f515659c640 1 --2- 172.21.15.105:0/396138723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ffa31d480 0x563ffa325180 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f514800bf10 tx=0x7f5148012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=240c9f55612f35d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:37.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.146+0000 7f513dffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:03:37.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.147+0000 7f513dffb640 1 -- 172.21.15.105:0/396138723 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5148004230 con 0x563ffa31d480 2024-09-16T02:03:37.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.147+0000 7f513dffb640 1 -- 172.21.15.105:0/396138723 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f51480043d0 con 0x563ffa31d480 2024-09-16T02:03:37.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.147+0000 7f515f2f7740 1 -- 172.21.15.105:0/396138723 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f514c012c40 con 0x563ffa31d480 2024-09-16T02:03:37.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.147+0000 7f513dffb640 1 -- 172.21.15.105:0/396138723 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f51480106a0 con 0x563ffa31d480 2024-09-16T02:03:37.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.147+0000 7f515f2f7740 10 client.15468 fetch_fsmap learned FSMap version 28 2024-09-16T02:03:37.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.147+0000 7f515f2f7740 1 -- 172.21.15.105:0/396138723 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563ffa3365a0 con 0x563ffa31d480 2024-09-16T02:03:37.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.147+0000 7f513dffb640 1 -- 172.21.15.105:0/396138723 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f514c012c40 con 0x563ffa31d480 2024-09-16T02:03:37.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.148+0000 7f513dffb640 1 -- 172.21.15.105:0/396138723 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5148019070 con 0x563ffa31d480 2024-09-16T02:03:37.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.148+0000 7f515f2f7740 10 client.15468 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:03:37.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.148+0000 7f515f2f7740 10 client.15468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:03:37.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.149+0000 7f515f2f7740 20 client.15468 populate_metadata read hostname 'smithi105' 2024-09-16T02:03:37.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.149+0000 7f515f2f7740 1 --2- 172.21.15.105:0/396138723 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x563ffa337a50 0x563ffa339e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:37.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.149+0000 7f515f2f7740 4 client.15468 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:03:37.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.149+0000 7f515f2f7740 1 -- 172.21.15.105:0/396138723 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ffa28f7f0 con 0x563ffa337a50 2024-09-16T02:03:37.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.149+0000 7f5155d9b640 1 --2- 172.21.15.105:0/396138723 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x563ffa337a50 0x563ffa339e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:37.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.150+0000 7f5155d9b640 1 --2- 172.21.15.105:0/396138723 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x563ffa337a50 0x563ffa339e70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59462e23b7e72494 server_cookie=3d7802ef2e322a15 in_seq=0 out_seq=0 2024-09-16T02:03:37.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.150+0000 7f513dffb640 10 client.15468 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:03:37.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.150+0000 7f513dffb640 1 -- 172.21.15.105:0/396138723 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563ffa28f7f0 con 0x563ffa337a50 2024-09-16T02:03:37.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.150+0000 7f513dffb640 10 client.15468 handle_command_reply: tid=0 2024-09-16T02:03:37.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.150+0000 7f515f2f7740 1 client.15468 shutdown 2024-09-16T02:03:37.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.151+0000 7f515f2f7740 1 -- 172.21.15.105:0/396138723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ffa31d480 msgr2=0x563ffa325180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:37.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.151+0000 7f515f2f7740 1 --2- 172.21.15.105:0/396138723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ffa31d480 0x563ffa325180 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f514800bf10 tx=0x7f5148012820 comp rx=0 tx=0).stop 2024-09-16T02:03:37.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.151+0000 7f515f2f7740 1 -- 172.21.15.105:0/396138723 shutdown_connections 2024-09-16T02:03:37.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.151+0000 7f515f2f7740 1 --2- 172.21.15.105:0/396138723 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x563ffa337a50 0x563ffa339e70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:37.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.151+0000 7f515f2f7740 1 --2- 172.21.15.105:0/396138723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ffa327d40 0x563ffa3256c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:37.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.151+0000 7f513dffb640 0 client.15468 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:03:37.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.151+0000 7f515f2f7740 1 --2- 172.21.15.105:0/396138723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ffa31d480 0x563ffa325180 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:37.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.151+0000 7f515f2f7740 1 --2- 172.21.15.105:0/396138723 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ffa31cb60 0x563ffa324c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:37.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.151+0000 7f515f2f7740 1 -- 172.21.15.105:0/396138723 >> 172.21.15.105:0/396138723 conn(0x563ffa3190f0 msgr2=0x563ffa318270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:37.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.151+0000 7f515f2f7740 1 -- 172.21.15.105:0/396138723 shutdown_connections 2024-09-16T02:03:37.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.151+0000 7f515f2f7740 1 -- 172.21.15.105:0/396138723 wait complete. 2024-09-16T02:03:37.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.151+0000 7f515f2f7740 20 client.15468 trim_cache size 0 max 16384 2024-09-16T02:03:37.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.184+0000 7f515f2f7740 1 Processor -- start 2024-09-16T02:03:37.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.184+0000 7f515f2f7740 1 -- start start 2024-09-16T02:03:37.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.184+0000 7f515f2f7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ffa31cb60 0x563ffa33f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:37.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.184+0000 7f515f2f7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ffa31d480 0x563ffa33fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:37.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.184+0000 7f515f2f7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ffa327d40 0x563ffa321f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:37.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.184+0000 7f515f2f7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f51581af7a0 con 0x563ffa31d480 2024-09-16T02:03:37.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.184+0000 7f515f2f7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f51581b12e0 con 0x563ffa327d40 2024-09-16T02:03:37.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.184+0000 7f515f2f7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f51581afe70 con 0x563ffa31cb60 2024-09-16T02:03:37.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.184+0000 7f515659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ffa31d480 0x563ffa33fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:37.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.184+0000 7f515659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ffa31d480 0x563ffa33fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49932/0 (socket says 172.21.15.105:49932) 2024-09-16T02:03:37.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.184+0000 7f515659c640 1 -- 172.21.15.105:0/1344101684 learned_addr learned my addr 172.21.15.105:0/1344101684 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:37.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.185+0000 7f515659c640 1 -- 172.21.15.105:0/1344101684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ffa31cb60 msgr2=0x563ffa33f740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T02:03:37.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.185+0000 7f515659c640 1 --2- 172.21.15.105:0/1344101684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ffa31cb60 0x563ffa33f740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:37.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.185+0000 7f515659c640 1 -- 172.21.15.105:0/1344101684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ffa327d40 msgr2=0x563ffa321f20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T02:03:37.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.185+0000 7f515659c640 1 --2- 172.21.15.105:0/1344101684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ffa327d40 0x563ffa321f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:37.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.185+0000 7f515659c640 1 -- 172.21.15.105:0/1344101684 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ffa31a280 con 0x563ffa31d480 2024-09-16T02:03:37.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.185+0000 7f515659c640 1 --2- 172.21.15.105:0/1344101684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ffa31d480 0x563ffa33fc80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5148003cb0 tx=0x7f5148005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a8719c40d3e8495 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:37.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.185+0000 7f513ffff640 1 -- 172.21.15.105:0/1344101684 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514802a030 con 0x563ffa31d480 2024-09-16T02:03:37.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.185+0000 7f513ffff640 1 -- 172.21.15.105:0/1344101684 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5148003ea0 con 0x563ffa31d480 2024-09-16T02:03:37.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.185+0000 7f513ffff640 1 -- 172.21.15.105:0/1344101684 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5148010bc0 con 0x563ffa31d480 2024-09-16T02:03:37.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.186+0000 7f515f2f7740 1 -- 172.21.15.105:0/1344101684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ffa31d480 msgr2=0x563ffa33fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:37.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.186+0000 7f515f2f7740 1 --2- 172.21.15.105:0/1344101684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ffa31d480 0x563ffa33fc80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5148003cb0 tx=0x7f5148005850 comp rx=0 tx=0).stop 2024-09-16T02:03:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.186+0000 7f515f2f7740 1 -- 172.21.15.105:0/1344101684 shutdown_connections 2024-09-16T02:03:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.186+0000 7f515f2f7740 1 --2- 172.21.15.105:0/1344101684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ffa327d40 0x563ffa321f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.186+0000 7f515f2f7740 1 --2- 172.21.15.105:0/1344101684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ffa31d480 0x563ffa33fc80 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.186+0000 7f515f2f7740 1 --2- 172.21.15.105:0/1344101684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ffa31cb60 0x563ffa33f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.186+0000 7f515f2f7740 1 -- 172.21.15.105:0/1344101684 >> 172.21.15.105:0/1344101684 conn(0x563ffa3190f0 msgr2=0x563ffa326bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.186+0000 7f515f2f7740 1 -- 172.21.15.105:0/1344101684 shutdown_connections 2024-09-16T02:03:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.186+0000 7f515f2f7740 1 -- 172.21.15.105:0/1344101684 wait complete. 2024-09-16T02:03:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.187+0000 7f515f2f7740 1 -- start start 2024-09-16T02:03:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.187+0000 7f515f2f7740 1 Processor -- start 2024-09-16T02:03:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.187+0000 7f515f2f7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ffa31cb60 0x563ffa330430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.187+0000 7f515f2f7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ffa31d480 0x563ffa32ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.187+0000 7f515f2f7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ffa327d40 0x563ffa32d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.188+0000 7f515659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ffa31d480 0x563ffa32ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.188+0000 7f515659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ffa31d480 0x563ffa32ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49934/0 (socket says 172.21.15.105:49934) 2024-09-16T02:03:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.188+0000 7f5155d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ffa31cb60 0x563ffa330430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.188+0000 7f5155d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ffa31cb60 0x563ffa330430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.105:51860/0 (socket says 172.21.15.105:51860) 2024-09-16T02:03:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.188+0000 7f515659c640 1 -- 172.21.15.105:0/1053332340 learned_addr learned my addr 172.21.15.105:0/1053332340 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.188+0000 7f515f2f7740 1 -- 172.21.15.105:0/1053332340 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f51581b0c10 con 0x563ffa31d480 2024-09-16T02:03:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.188+0000 7f515f2f7740 1 -- 172.21.15.105:0/1053332340 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f51581b0540 con 0x563ffa327d40 2024-09-16T02:03:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.188+0000 7f515f2f7740 1 -- 172.21.15.105:0/1053332340 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f51581ad780 con 0x563ffa31cb60 2024-09-16T02:03:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.188+0000 7f515659c640 1 -- 172.21.15.105:0/1053332340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ffa31cb60 msgr2=0x563ffa330430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.188+0000 7f515659c640 1 --2- 172.21.15.105:0/1053332340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ffa31cb60 0x563ffa330430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.188+0000 7f515659c640 1 -- 172.21.15.105:0/1053332340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ffa327d40 msgr2=0x563ffa32d210 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T02:03:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.188+0000 7f515659c640 1 --2- 172.21.15.105:0/1053332340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ffa327d40 0x563ffa32d210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.188+0000 7f515659c640 1 -- 172.21.15.105:0/1053332340 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ffa32da40 con 0x563ffa31d480 2024-09-16T02:03:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.188+0000 7f5155d9b640 1 --2- 172.21.15.105:0/1053332340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ffa31cb60 0x563ffa330430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:03:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.188+0000 7f515f2f7740 10 client.0 _ll_register_callbacks cb 0x7f514800b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:03:37.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.188+0000 7f515f2f7740 1 -- 172.21.15.105:0/1053332340 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5148003ea0 con 0x563ffa31d480 2024-09-16T02:03:37.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.188+0000 7f515659c640 1 --2- 172.21.15.105:0/1053332340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ffa31d480 0x563ffa32ccb0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5148003c30 tx=0x7f5148004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=723a75df6f392e85 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:37.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.189+0000 7f513dffb640 10 client.15474 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:03:37.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.189+0000 7f513dffb640 1 -- 172.21.15.105:0/1053332340 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514802a040 con 0x563ffa31d480 2024-09-16T02:03:37.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.189+0000 7f513dffb640 1 -- 172.21.15.105:0/1053332340 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f5148010960 con 0x563ffa31d480 2024-09-16T02:03:37.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.189+0000 7f513dffb640 1 -- 172.21.15.105:0/1053332340 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5148010ba0 con 0x563ffa31d480 2024-09-16T02:03:37.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.189+0000 7f515f2f7740 10 client.15474 fetch_fsmap learned FSMap version 28 2024-09-16T02:03:37.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.189+0000 7f515f2f7740 1 -- 172.21.15.105:0/1053332340 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({fsmap=28}) -- 0x563ffa3abc60 con 0x563ffa31d480 2024-09-16T02:03:37.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.189+0000 7f513dffb640 1 -- 172.21.15.105:0/1053332340 <== mon.0 v2:172.21.15.105:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5148019070 con 0x563ffa31d480 2024-09-16T02:03:37.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.190+0000 7f515f2f7740 10 client.15474 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:03:37.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.190+0000 7f515f2f7740 10 client.15474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:03:37.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.190+0000 7f515f2f7740 20 client.15474 populate_metadata read hostname 'smithi105' 2024-09-16T02:03:37.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.190+0000 7f515f2f7740 1 --2- 172.21.15.105:0/1053332340 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x563ffa337a50 0x563ffa3ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:37.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.190+0000 7f515f2f7740 4 client.15474 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T02:03:37.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.190+0000 7f515f2f7740 1 -- 172.21.15.105:0/1053332340 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563ffa289d00 con 0x563ffa337a50 2024-09-16T02:03:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.190+0000 7f5155d9b640 1 --2- 172.21.15.105:0/1053332340 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x563ffa337a50 0x563ffa3ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.190+0000 7f5155d9b640 1 --2- 172.21.15.105:0/1053332340 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x563ffa337a50 0x563ffa3ae3f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d41d63afda854af1 server_cookie=7511caa371d7fe17 in_seq=0 out_seq=0 2024-09-16T02:03:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.190+0000 7f513dffb640 10 client.15474 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:03:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.192+0000 7f513dffb640 1 -- 172.21.15.105:0/1053332340 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563ffa289d00 con 0x563ffa337a50 2024-09-16T02:03:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.192+0000 7f513dffb640 10 client.15474 handle_command_reply: tid=0 2024-09-16T02:03:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.192+0000 7f515f2f7740 1 client.15474 shutdown 2024-09-16T02:03:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.192+0000 7f515f2f7740 1 -- 172.21.15.105:0/1053332340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ffa31d480 msgr2=0x563ffa32ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.192+0000 7f515f2f7740 1 --2- 172.21.15.105:0/1053332340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ffa31d480 0x563ffa32ccb0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5148003c30 tx=0x7f5148004280 comp rx=0 tx=0).stop 2024-09-16T02:03:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.192+0000 7f515f2f7740 1 -- 172.21.15.105:0/1053332340 shutdown_connections 2024-09-16T02:03:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.192+0000 7f515f2f7740 1 --2- 172.21.15.105:0/1053332340 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x563ffa337a50 0x563ffa3ae3f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.192+0000 7f515f2f7740 1 --2- 172.21.15.105:0/1053332340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ffa327d40 0x563ffa32d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.192+0000 7f515f2f7740 1 --2- 172.21.15.105:0/1053332340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ffa31d480 0x563ffa32ccb0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.192+0000 7f515f2f7740 1 --2- 172.21.15.105:0/1053332340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ffa31cb60 0x563ffa330430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.192+0000 7f515f2f7740 1 -- 172.21.15.105:0/1053332340 >> 172.21.15.105:0/1053332340 conn(0x563ffa3190f0 msgr2=0x563ffa324be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.192+0000 7f513dffb640 0 client.15474 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:03:37.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.192+0000 7f515f2f7740 1 -- 172.21.15.105:0/1053332340 shutdown_connections 2024-09-16T02:03:37.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.192+0000 7f515f2f7740 1 -- 172.21.15.105:0/1053332340 wait complete. 2024-09-16T02:03:37.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.192+0000 7f515f2f7740 20 client.15474 trim_cache size 0 max 16384 2024-09-16T02:03:37.196 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-16T02:03:37.196 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-09-16T02:03:37.196 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "cbabab3c-4462-4a7b-bd58-200f0b911172", 2024-09-16T02:03:37.196 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-09-16T02:03:37.196 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-16T02:03:37.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.193+0000 7f515cf1e640 1 -- 172.21.15.105:0/2129161363 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f51400a3390 msgr2=0x7f51400a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:37.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.193+0000 7f515cf1e640 1 --2- 172.21.15.105:0/2129161363 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f51400a3390 0x7f51400a5850 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5150006010 tx=0x7f515003e040 comp rx=0 tx=0).stop 2024-09-16T02:03:37.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.193+0000 7f515cf1e640 1 -- 172.21.15.105:0/2129161363 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5158106f20 msgr2=0x7f515819cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:37.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.193+0000 7f515cf1e640 1 --2- 172.21.15.105:0/2129161363 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5158106f20 0x7f515819cc20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5150009dc0 tx=0x7f5150032ff0 comp rx=0 tx=0).stop 2024-09-16T02:03:37.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.193+0000 7f515cf1e640 1 -- 172.21.15.105:0/2129161363 shutdown_connections 2024-09-16T02:03:37.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.193+0000 7f515cf1e640 1 --2- 172.21.15.105:0/2129161363 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f51400a3390 0x7f51400a5850 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:37.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.193+0000 7f515cf1e640 1 --2- 172.21.15.105:0/2129161363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51581027a0 0x7f5158198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:37.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.193+0000 7f515cf1e640 1 --2- 172.21.15.105:0/2129161363 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5158106f20 0x7f515819cc20 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:37.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.193+0000 7f515cf1e640 1 --2- 172.21.15.105:0/2129161363 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5158101df0 0x7f5158198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:37.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.193+0000 7f515cf1e640 1 -- 172.21.15.105:0/2129161363 >> 172.21.15.105:0/2129161363 conn(0x7f51580fd8f0 msgr2=0x7f51580fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:37.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.193+0000 7f515cf1e640 1 -- 172.21.15.105:0/2129161363 shutdown_connections 2024-09-16T02:03:37.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:37.193+0000 7f515cf1e640 1 -- 172.21.15.105:0/2129161363 wait complete. 2024-09-16T02:03:38.121 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T02:03:38.406 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:03:38.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:38 smithi105 ceph-mon[31197]: scrub queued for path: / 2024-09-16T02:03:38.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:38 smithi105 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-16T02:03:38.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:38 smithi105 ceph-mon[31197]: scrub summary: active paths [/] 2024-09-16T02:03:38.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:38 smithi105 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-16T02:03:38.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:38 smithi105 ceph-mon[31197]: pgmap v372: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T02:03:38.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:38 smithi137 ceph-mon[34099]: scrub queued for path: / 2024-09-16T02:03:38.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:38 smithi137 ceph-mon[34099]: scrub summary: idle+waiting paths [/] 2024-09-16T02:03:38.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:38 smithi137 ceph-mon[34099]: scrub summary: active paths [/] 2024-09-16T02:03:38.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:38 smithi137 ceph-mon[34099]: scrub summary: idle+waiting paths [/] 2024-09-16T02:03:38.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:38 smithi137 ceph-mon[34099]: pgmap v372: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T02:03:38.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.942+0000 7f8959f13640 1 -- 172.21.15.105:0/1500708972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8954102d30 msgr2=0x7f8954103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:38.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.942+0000 7f8959f13640 1 --2- 172.21.15.105:0/1500708972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8954102d30 0x7f8954103110 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8944009900 tx=0x7f894402a480 comp rx=0 tx=0).stop 2024-09-16T02:03:38.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.942+0000 7f8959f13640 1 -- 172.21.15.105:0/1500708972 shutdown_connections 2024-09-16T02:03:38.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.942+0000 7f8959f13640 1 --2- 172.21.15.105:0/1500708972 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8954103650 0x7f895410bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:38.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.942+0000 7f8959f13640 1 --2- 172.21.15.105:0/1500708972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f895410c160 0x7f895410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:38.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.942+0000 7f8959f13640 1 --2- 172.21.15.105:0/1500708972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8954102d30 0x7f8954103110 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:38.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.942+0000 7f8959f13640 1 -- 172.21.15.105:0/1500708972 >> 172.21.15.105:0/1500708972 conn(0x7f89540fc7d0 msgr2=0x7f89540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:38.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.942+0000 7f8959f13640 1 -- 172.21.15.105:0/1500708972 shutdown_connections 2024-09-16T02:03:38.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.942+0000 7f8959f13640 1 -- 172.21.15.105:0/1500708972 wait complete. 2024-09-16T02:03:38.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.943+0000 7f8959f13640 1 Processor -- start 2024-09-16T02:03:38.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.943+0000 7f8959f13640 1 -- start start 2024-09-16T02:03:38.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.943+0000 7f8959f13640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8954102d30 0x7f8954198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:38.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.943+0000 7f8959f13640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8954103650 0x7f8954198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:38.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.943+0000 7f8959f13640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f895410c160 0x7f895419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:38.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.944+0000 7f8959f13640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8954111e10 con 0x7f895410c160 2024-09-16T02:03:38.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.944+0000 7f8959f13640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8954111c90 con 0x7f8954102d30 2024-09-16T02:03:38.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.944+0000 7f8959f13640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8954111f90 con 0x7f8954103650 2024-09-16T02:03:38.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.944+0000 7f8952ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8954103650 0x7f8954198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:38.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.944+0000 7f89537fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8954102d30 0x7f8954198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:38.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.944+0000 7f8952ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8954103650 0x7f8954198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.105:43056/0 (socket says 172.21.15.105:43056) 2024-09-16T02:03:38.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.944+0000 7f8952ffd640 1 -- 172.21.15.105:0/2141854149 learned_addr learned my addr 172.21.15.105:0/2141854149 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:38.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.944+0000 7f8953fff640 1 --2- 172.21.15.105:0/2141854149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f895410c160 0x7f895419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:38.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.944+0000 7f8953fff640 1 -- 172.21.15.105:0/2141854149 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8954103650 msgr2=0x7f8954198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:38.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.944+0000 7f8953fff640 1 --2- 172.21.15.105:0/2141854149 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8954103650 0x7f8954198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:38.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.944+0000 7f8953fff640 1 -- 172.21.15.105:0/2141854149 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8954102d30 msgr2=0x7f8954198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:38.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.944+0000 7f8953fff640 1 --2- 172.21.15.105:0/2141854149 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8954102d30 0x7f8954198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:38.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.944+0000 7f8953fff640 1 -- 172.21.15.105:0/2141854149 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f895419d300 con 0x7f895410c160 2024-09-16T02:03:38.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.944+0000 7f8952ffd640 1 --2- 172.21.15.105:0/2141854149 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8954103650 0x7f8954198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:03:38.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.945+0000 7f8953fff640 1 --2- 172.21.15.105:0/2141854149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f895410c160 0x7f895419cc20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f894c007db0 tx=0x7f894c010420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:38.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.945+0000 7f8950ff9640 1 -- 172.21.15.105:0/2141854149 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894c010d40 con 0x7f895410c160 2024-09-16T02:03:38.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.945+0000 7f8950ff9640 1 -- 172.21.15.105:0/2141854149 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f894c00c640 con 0x7f895410c160 2024-09-16T02:03:38.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.945+0000 7f8950ff9640 1 -- 172.21.15.105:0/2141854149 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894c017650 con 0x7f895410c160 2024-09-16T02:03:38.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.945+0000 7f8959f13640 1 -- 172.21.15.105:0/2141854149 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f895419d5f0 con 0x7f895410c160 2024-09-16T02:03:38.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.945+0000 7f8959f13640 1 -- 172.21.15.105:0/2141854149 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f895419da20 con 0x7f895410c160 2024-09-16T02:03:38.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.948+0000 7f895c2ec740 1 Processor -- start 2024-09-16T02:03:38.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.948+0000 7f8950ff9640 1 -- 172.21.15.105:0/2141854149 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f894c00c7f0 con 0x7f895410c160 2024-09-16T02:03:38.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.949+0000 7f895c2ec740 1 -- start start 2024-09-16T02:03:38.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.949+0000 7f895c2ec740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596909e34a0 0x5596909e3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:38.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.949+0000 7f895c2ec740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596909e3dc0 0x5596909ee140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:38.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.949+0000 7f895c2ec740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596909ee680 0x5596909f0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:38.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.949+0000 7f895c2ec740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5596909492e0 con 0x5596909ee680 2024-09-16T02:03:38.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.949+0000 7f895c2ec740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5596908f0ce0 con 0x5596909e3dc0 2024-09-16T02:03:38.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.949+0000 7f895c2ec740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5596909452a0 con 0x5596909e34a0 2024-09-16T02:03:38.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.949+0000 7f89537fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596909e3dc0 0x5596909ee140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:38.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.949+0000 7f89537fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596909e3dc0 0x5596909ee140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.105:43098/0 (socket says 172.21.15.105:43098) 2024-09-16T02:03:38.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.949+0000 7f89537fe640 1 -- 172.21.15.105:0/3478022660 learned_addr learned my addr 172.21.15.105:0/3478022660 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:38.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.949+0000 7f8952ffd640 1 --2- 172.21.15.105:0/3478022660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596909ee680 0x5596909f0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:38.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.949+0000 7f89537fe640 1 -- 172.21.15.105:0/3478022660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596909e34a0 msgr2=0x5596909e3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:38.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.949+0000 7f89537fe640 1 --2- 172.21.15.105:0/3478022660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596909e34a0 0x5596909e3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:38.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.949+0000 7f89537fe640 1 -- 172.21.15.105:0/3478022660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596909ee680 msgr2=0x5596909f0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:38.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.950+0000 7f89537fe640 1 --2- 172.21.15.105:0/3478022660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596909ee680 0x5596909f0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:38.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.950+0000 7f89537fe640 1 -- 172.21.15.105:0/3478022660 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596908d5440 con 0x5596909e3dc0 2024-09-16T02:03:38.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.950+0000 7f8952ffd640 1 --2- 172.21.15.105:0/3478022660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596909ee680 0x5596909f0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:03:38.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.950+0000 7f8950ff9640 1 --2- 172.21.15.105:0/2141854149 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f89400a32c0 0x7f89400a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:38.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.950+0000 7f89537fe640 1 --2- 172.21.15.105:0/3478022660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596909e3dc0 0x5596909ee140 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8944009900 tx=0x7f8944032e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89e459a235e227d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:38.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.950+0000 7f8950ff9640 1 -- 172.21.15.105:0/2141854149 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f894c0d3640 con 0x7f895410c160 2024-09-16T02:03:38.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.950+0000 7f8952ffd640 1 --2- 172.21.15.105:0/2141854149 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f89400a32c0 0x7f89400a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:38.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.950+0000 7f8949ffb640 1 -- 172.21.15.105:0/3478022660 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8944040070 con 0x5596909e3dc0 2024-09-16T02:03:38.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.951+0000 7f8949ffb640 1 -- 172.21.15.105:0/3478022660 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89440316a0 con 0x5596909e3dc0 2024-09-16T02:03:38.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.951+0000 7f8949ffb640 1 -- 172.21.15.105:0/3478022660 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8944035040 con 0x5596909e3dc0 2024-09-16T02:03:38.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.951+0000 7f895c2ec740 1 -- 172.21.15.105:0/3478022660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596909e3dc0 msgr2=0x5596909ee140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:38.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.951+0000 7f8952ffd640 1 --2- 172.21.15.105:0/2141854149 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f89400a32c0 0x7f89400a5780 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f893c00b9c0 tx=0x7f893c00b950 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:38.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.951+0000 7f895c2ec740 1 --2- 172.21.15.105:0/3478022660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596909e3dc0 0x5596909ee140 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8944009900 tx=0x7f8944032e40 comp rx=0 tx=0).stop 2024-09-16T02:03:38.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.952+0000 7f895c2ec740 1 -- 172.21.15.105:0/3478022660 shutdown_connections 2024-09-16T02:03:38.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.952+0000 7f895c2ec740 1 --2- 172.21.15.105:0/3478022660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596909ee680 0x5596909f0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:38.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.952+0000 7f895c2ec740 1 --2- 172.21.15.105:0/3478022660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596909e3dc0 0x5596909ee140 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:38.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.952+0000 7f895c2ec740 1 --2- 172.21.15.105:0/3478022660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596909e34a0 0x5596909e3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:38.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.952+0000 7f895c2ec740 1 -- 172.21.15.105:0/3478022660 >> 172.21.15.105:0/3478022660 conn(0x5596909dfa30 msgr2=0x5596909dfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:38.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.952+0000 7f895c2ec740 1 -- 172.21.15.105:0/3478022660 shutdown_connections 2024-09-16T02:03:38.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.952+0000 7f895c2ec740 1 -- 172.21.15.105:0/3478022660 wait complete. 2024-09-16T02:03:38.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.952+0000 7f895c2ec740 1 -- start start 2024-09-16T02:03:38.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.952+0000 7f895c2ec740 1 Processor -- start 2024-09-16T02:03:38.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.953+0000 7f895c2ec740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596909e34a0 0x5596909edfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:38.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.953+0000 7f895c2ec740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596909e3dc0 0x5596909eb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:38.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.953+0000 7f895c2ec740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596909ee680 0x5596909eb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:38.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.953+0000 7f895c2ec740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5596909f1250 con 0x5596909ee680 2024-09-16T02:03:38.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.953+0000 7f895c2ec740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559690904050 con 0x5596909e3dc0 2024-09-16T02:03:38.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.953+0000 7f895c2ec740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5596909f13d0 con 0x5596909e34a0 2024-09-16T02:03:38.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.953+0000 7f8953fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596909ee680 0x5596909eb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:38.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.953+0000 7f8953fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596909ee680 0x5596909eb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48696/0 (socket says 172.21.15.105:48696) 2024-09-16T02:03:38.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.953+0000 7f8953fff640 1 -- 172.21.15.105:0/738971753 learned_addr learned my addr 172.21.15.105:0/738971753 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:38.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.953+0000 7f8952ffd640 1 --2- 172.21.15.105:0/738971753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596909e3dc0 0x5596909eb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:38.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.953+0000 7f8953fff640 1 -- 172.21.15.105:0/738971753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596909e34a0 msgr2=0x5596909edfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:38.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.953+0000 7f8953fff640 1 --2- 172.21.15.105:0/738971753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596909e34a0 0x5596909edfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:38.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.953+0000 7f8953fff640 1 -- 172.21.15.105:0/738971753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596909e3dc0 msgr2=0x5596909eb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:38.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.953+0000 7f8953fff640 1 --2- 172.21.15.105:0/738971753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596909e3dc0 0x5596909eb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:38.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.953+0000 7f8953fff640 1 -- 172.21.15.105:0/738971753 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596906ba3c0 con 0x5596909ee680 2024-09-16T02:03:38.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.953+0000 7f8952ffd640 1 --2- 172.21.15.105:0/738971753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596909e3dc0 0x5596909eb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:03:38.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.954+0000 7f895c2ec740 10 client.0 _ll_register_callbacks cb 0x559690944160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:03:38.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.954+0000 7f8953fff640 1 --2- 172.21.15.105:0/738971753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596909ee680 0x5596909eb790 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f894c09f900 tx=0x7f894c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d61fb0601f34e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:38.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.954+0000 7f895c2ec740 1 -- 172.21.15.105:0/738971753 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89440316a0 con 0x5596909ee680 2024-09-16T02:03:38.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.954+0000 7f8932ffd640 10 client.15486 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:03:38.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.954+0000 7f8932ffd640 1 -- 172.21.15.105:0/738971753 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894c0a3fb0 con 0x5596909ee680 2024-09-16T02:03:38.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.954+0000 7f8932ffd640 1 -- 172.21.15.105:0/738971753 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f894c09f050 con 0x5596909ee680 2024-09-16T02:03:38.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.954+0000 7f8932ffd640 1 -- 172.21.15.105:0/738971753 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f894c09f250 con 0x5596909ee680 2024-09-16T02:03:38.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.954+0000 7f895c2ec740 10 client.15486 fetch_fsmap learned FSMap version 28 2024-09-16T02:03:38.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.954+0000 7f895c2ec740 1 -- 172.21.15.105:0/738971753 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({fsmap=28}) -- 0x5596909fcb50 con 0x5596909ee680 2024-09-16T02:03:38.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.955+0000 7f8932ffd640 1 -- 172.21.15.105:0/738971753 <== mon.0 v2:172.21.15.105:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f894c016070 con 0x5596909ee680 2024-09-16T02:03:38.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.955+0000 7f895c2ec740 10 client.15486 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:03:38.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.955+0000 7f895c2ec740 10 client.15486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:03:38.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.955+0000 7f895c2ec740 20 client.15486 populate_metadata read hostname 'smithi105' 2024-09-16T02:03:38.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.955+0000 7f895c2ec740 1 --2- 172.21.15.105:0/738971753 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x5596909fdfc0 0x559690a00450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:38.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.955+0000 7f895c2ec740 4 client.15486 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:03:38.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.955+0000 7f895c2ec740 1 -- 172.21.15.105:0/738971753 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559690956850 con 0x5596909fdfc0 2024-09-16T02:03:38.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.955+0000 7f89537fe640 1 --2- 172.21.15.105:0/738971753 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x5596909fdfc0 0x559690a00450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:38.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.956+0000 7f89537fe640 1 --2- 172.21.15.105:0/738971753 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x5596909fdfc0 0x559690a00450 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc4ad12c76adaed1 server_cookie=655ba4363186c9d2 in_seq=0 out_seq=0 2024-09-16T02:03:38.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.956+0000 7f8932ffd640 10 client.15486 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:03:38.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.956+0000 7f8932ffd640 1 -- 172.21.15.105:0/738971753 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559690956850 con 0x5596909fdfc0 2024-09-16T02:03:38.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.956+0000 7f8932ffd640 10 client.15486 handle_command_reply: tid=0 2024-09-16T02:03:38.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.956+0000 7f895c2ec740 1 client.15486 shutdown 2024-09-16T02:03:38.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.957+0000 7f895c2ec740 1 -- 172.21.15.105:0/738971753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596909ee680 msgr2=0x5596909eb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:38.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.957+0000 7f895c2ec740 1 --2- 172.21.15.105:0/738971753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596909ee680 0x5596909eb790 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f894c09f900 tx=0x7f894c00cc20 comp rx=0 tx=0).stop 2024-09-16T02:03:38.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.957+0000 7f895c2ec740 1 -- 172.21.15.105:0/738971753 shutdown_connections 2024-09-16T02:03:38.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.957+0000 7f895c2ec740 1 --2- 172.21.15.105:0/738971753 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x5596909fdfc0 0x559690a00450 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:38.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.957+0000 7f895c2ec740 1 --2- 172.21.15.105:0/738971753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596909ee680 0x5596909eb790 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:38.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.957+0000 7f895c2ec740 1 --2- 172.21.15.105:0/738971753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596909e3dc0 0x5596909eb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:38.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.957+0000 7f895c2ec740 1 --2- 172.21.15.105:0/738971753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596909e34a0 0x5596909edfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:38.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.957+0000 7f895c2ec740 1 -- 172.21.15.105:0/738971753 >> 172.21.15.105:0/738971753 conn(0x5596909dfa30 msgr2=0x5596909e5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:38.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.957+0000 7f8932ffd640 0 client.15486 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:03:38.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.957+0000 7f895c2ec740 1 -- 172.21.15.105:0/738971753 shutdown_connections 2024-09-16T02:03:38.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.957+0000 7f895c2ec740 1 -- 172.21.15.105:0/738971753 wait complete. 2024-09-16T02:03:38.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.957+0000 7f895c2ec740 20 client.15486 trim_cache size 0 max 16384 2024-09-16T02:03:38.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.990+0000 7f895c2ec740 1 Processor -- start 2024-09-16T02:03:38.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.990+0000 7f895c2ec740 1 -- start start 2024-09-16T02:03:38.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.990+0000 7f895c2ec740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596909e34a0 0x559690a05d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:38.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.990+0000 7f895c2ec740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596909e3dc0 0x559690a062a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:38.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.991+0000 7f895c2ec740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596909ee680 0x5596909e84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:38.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.991+0000 7f895c2ec740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f89541ab440 con 0x5596909ee680 2024-09-16T02:03:38.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.991+0000 7f895c2ec740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f89541acf80 con 0x5596909e34a0 2024-09-16T02:03:38.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.991+0000 7f895c2ec740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f89541abb10 con 0x5596909e3dc0 2024-09-16T02:03:38.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.991+0000 7f8952ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596909e3dc0 0x559690a062a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:38.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.991+0000 7f8952ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596909e3dc0 0x559690a062a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.105:43078/0 (socket says 172.21.15.105:43078) 2024-09-16T02:03:38.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.991+0000 7f8952ffd640 1 -- 172.21.15.105:0/656922545 learned_addr learned my addr 172.21.15.105:0/656922545 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:38.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.991+0000 7f89537fe640 1 --2- 172.21.15.105:0/656922545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596909e34a0 0x559690a05d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:38.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.991+0000 7f8953fff640 1 --2- 172.21.15.105:0/656922545 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596909ee680 0x5596909e84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:38.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.991+0000 7f8952ffd640 1 -- 172.21.15.105:0/656922545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596909e34a0 msgr2=0x559690a05d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:38.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.991+0000 7f8952ffd640 1 --2- 172.21.15.105:0/656922545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596909e34a0 0x559690a05d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:38.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.991+0000 7f8952ffd640 1 -- 172.21.15.105:0/656922545 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596909ee680 msgr2=0x5596909e84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:38.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.991+0000 7f8952ffd640 1 --2- 172.21.15.105:0/656922545 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596909ee680 0x5596909e84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:38.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.991+0000 7f8952ffd640 1 -- 172.21.15.105:0/656922545 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596909e3bf0 con 0x5596909e3dc0 2024-09-16T02:03:38.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.991+0000 7f8953fff640 1 --2- 172.21.15.105:0/656922545 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596909ee680 0x5596909e84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:03:38.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.991+0000 7f89537fe640 1 --2- 172.21.15.105:0/656922545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596909e34a0 0x559690a05d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:03:38.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.992+0000 7f8952ffd640 1 --2- 172.21.15.105:0/656922545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596909e3dc0 0x559690a062a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f89540fee40 tx=0x7f893c015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b63d06c0e81b7ee9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:38.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.992+0000 7f89497fa640 1 -- 172.21.15.105:0/656922545 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893c01c780 con 0x5596909e3dc0 2024-09-16T02:03:38.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.992+0000 7f89497fa640 1 -- 172.21.15.105:0/656922545 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f893c01cdd0 con 0x5596909e3dc0 2024-09-16T02:03:38.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.992+0000 7f89497fa640 1 -- 172.21.15.105:0/656922545 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893c01dda0 con 0x5596909e3dc0 2024-09-16T02:03:38.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.993+0000 7f895c2ec740 1 -- 172.21.15.105:0/656922545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596909e3dc0 msgr2=0x559690a062a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:38.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.993+0000 7f895c2ec740 1 --2- 172.21.15.105:0/656922545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596909e3dc0 0x559690a062a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f89540fee40 tx=0x7f893c015000 comp rx=0 tx=0).stop 2024-09-16T02:03:38.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.993+0000 7f895c2ec740 1 -- 172.21.15.105:0/656922545 shutdown_connections 2024-09-16T02:03:38.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.993+0000 7f895c2ec740 1 --2- 172.21.15.105:0/656922545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596909e3dc0 0x559690a062a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:38.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.993+0000 7f895c2ec740 1 --2- 172.21.15.105:0/656922545 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596909ee680 0x5596909e84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:38.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.993+0000 7f895c2ec740 1 --2- 172.21.15.105:0/656922545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596909e34a0 0x559690a05d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:38.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.993+0000 7f895c2ec740 1 -- 172.21.15.105:0/656922545 >> 172.21.15.105:0/656922545 conn(0x5596909dfa30 msgr2=0x5596909dea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:38.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.993+0000 7f895c2ec740 1 -- 172.21.15.105:0/656922545 shutdown_connections 2024-09-16T02:03:38.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.993+0000 7f895c2ec740 1 -- 172.21.15.105:0/656922545 wait complete. 2024-09-16T02:03:38.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.993+0000 7f895c2ec740 1 -- start start 2024-09-16T02:03:38.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.994+0000 7f895c2ec740 1 Processor -- start 2024-09-16T02:03:38.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.994+0000 7f895c2ec740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596909e34a0 0x5596909f4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:38.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.994+0000 7f895c2ec740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596909e3dc0 0x5596909f4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:38.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.994+0000 7f895c2ec740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596909ee680 0x559690a70550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:38.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.994+0000 7f895c2ec740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f89541ac8b0 con 0x5596909e3dc0 2024-09-16T02:03:38.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.994+0000 7f895c2ec740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f89541ac1e0 con 0x5596909e34a0 2024-09-16T02:03:38.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.994+0000 7f895c2ec740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f89541a9420 con 0x5596909ee680 2024-09-16T02:03:38.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.994+0000 7f895c2ec740 10 client.0 _ll_register_callbacks cb 0x7f894c09b780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:03:38.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.994+0000 7f8952ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596909e3dc0 0x5596909f4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:38.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.994+0000 7f8952ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596909e3dc0 0x5596909f4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48710/0 (socket says 172.21.15.105:48710) 2024-09-16T02:03:38.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.994+0000 7f8952ffd640 1 -- 172.21.15.105:0/1108747737 learned_addr learned my addr 172.21.15.105:0/1108747737 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:03:38.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.994+0000 7f89537fe640 1 --2- 172.21.15.105:0/1108747737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596909e34a0 0x5596909f4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:38.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.994+0000 7f8953fff640 1 --2- 172.21.15.105:0/1108747737 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596909ee680 0x559690a70550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:38.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.994+0000 7f8952ffd640 1 -- 172.21.15.105:0/1108747737 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596909ee680 msgr2=0x559690a70550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:38.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.994+0000 7f8952ffd640 1 --2- 172.21.15.105:0/1108747737 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596909ee680 0x559690a70550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:38.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.994+0000 7f8952ffd640 1 -- 172.21.15.105:0/1108747737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596909e34a0 msgr2=0x5596909f4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:03:38.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.994+0000 7f8952ffd640 1 --2- 172.21.15.105:0/1108747737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596909e34a0 0x5596909f4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:38.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.994+0000 7f8952ffd640 1 -- 172.21.15.105:0/1108747737 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559690a70a90 con 0x5596909e3dc0 2024-09-16T02:03:38.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.994+0000 7f8953fff640 1 --2- 172.21.15.105:0/1108747737 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596909ee680 0x559690a70550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:03:38.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.995+0000 7f8952ffd640 1 --2- 172.21.15.105:0/1108747737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596909e3dc0 0x5596909f4fd0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x5596909ee5e0 tx=0x7f893c01d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb9bc66431b66df server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:39.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.995+0000 7f8932ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:03:39.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.995+0000 7f8932ffd640 1 -- 172.21.15.105:0/1108747737 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893c034020 con 0x5596909e3dc0 2024-09-16T02:03:39.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.995+0000 7f8932ffd640 1 -- 172.21.15.105:0/1108747737 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f893c01c780 con 0x5596909e3dc0 2024-09-16T02:03:39.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.995+0000 7f895c2ec740 1 -- 172.21.15.105:0/1108747737 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f893c01cdd0 con 0x5596909e3dc0 2024-09-16T02:03:39.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.995+0000 7f8932ffd640 1 -- 172.21.15.105:0/1108747737 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f893c03f050 con 0x5596909e3dc0 2024-09-16T02:03:39.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.995+0000 7f895c2ec740 10 client.15492 fetch_fsmap learned FSMap version 28 2024-09-16T02:03:39.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.995+0000 7f895c2ec740 1 -- 172.21.15.105:0/1108747737 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559690a71d40 con 0x5596909e3dc0 2024-09-16T02:03:39.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.996+0000 7f8932ffd640 1 -- 172.21.15.105:0/1108747737 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f893c01c980 con 0x5596909e3dc0 2024-09-16T02:03:39.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.996+0000 7f8932ffd640 1 -- 172.21.15.105:0/1108747737 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f893c02f030 con 0x5596909e3dc0 2024-09-16T02:03:39.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.996+0000 7f895c2ec740 10 client.15492 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:03:39.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.996+0000 7f895c2ec740 10 client.15492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:03:39.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.996+0000 7f895c2ec740 20 client.15492 populate_metadata read hostname 'smithi105' 2024-09-16T02:03:39.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.996+0000 7f895c2ec740 1 --2- 172.21.15.105:0/1108747737 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x559690a72450 0x559690a74870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:39.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.996+0000 7f895c2ec740 4 client.15492 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T02:03:39.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.996+0000 7f895c2ec740 1 -- 172.21.15.105:0/1108747737 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f893c01cdd0 con 0x559690a72450 2024-09-16T02:03:39.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.996+0000 7f89537fe640 1 --2- 172.21.15.105:0/1108747737 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x559690a72450 0x559690a74870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:39.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.997+0000 7f8932ffd640 10 client.15492 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:03:39.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.997+0000 7f89537fe640 1 --2- 172.21.15.105:0/1108747737 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x559690a72450 0x559690a74870 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de20f3eb12636d server_cookie=1dfdd6f2105e9a04 in_seq=0 out_seq=0 2024-09-16T02:03:39.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.997+0000 7f8932ffd640 1 -- 172.21.15.105:0/1108747737 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f893c01cdd0 con 0x559690a72450 2024-09-16T02:03:39.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.997+0000 7f8932ffd640 10 client.15492 handle_command_reply: tid=0 2024-09-16T02:03:39.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.997+0000 7f895c2ec740 1 client.15492 shutdown 2024-09-16T02:03:39.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.997+0000 7f895c2ec740 1 -- 172.21.15.105:0/1108747737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596909e3dc0 msgr2=0x5596909f4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:39.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.997+0000 7f895c2ec740 1 --2- 172.21.15.105:0/1108747737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596909e3dc0 0x5596909f4fd0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x5596909ee5e0 tx=0x7f893c01d6b0 comp rx=0 tx=0).stop 2024-09-16T02:03:39.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.997+0000 7f895c2ec740 1 -- 172.21.15.105:0/1108747737 shutdown_connections 2024-09-16T02:03:39.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.997+0000 7f895c2ec740 1 --2- 172.21.15.105:0/1108747737 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x559690a72450 0x559690a74870 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:39.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.997+0000 7f8932ffd640 0 client.15492 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:03:39.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.997+0000 7f895c2ec740 1 --2- 172.21.15.105:0/1108747737 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596909ee680 0x559690a70550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:39.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.997+0000 7f895c2ec740 1 --2- 172.21.15.105:0/1108747737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596909e3dc0 0x5596909f4fd0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:39.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.997+0000 7f895c2ec740 1 --2- 172.21.15.105:0/1108747737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596909e34a0 0x5596909f4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:39.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.997+0000 7f895c2ec740 1 -- 172.21.15.105:0/1108747737 >> 172.21.15.105:0/1108747737 conn(0x5596909dfa30 msgr2=0x5596909ecab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:39.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.997+0000 7f895c2ec740 1 -- 172.21.15.105:0/1108747737 shutdown_connections 2024-09-16T02:03:39.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.997+0000 7f895c2ec740 1 -- 172.21.15.105:0/1108747737 wait complete. 2024-09-16T02:03:39.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.998+0000 7f895c2ec740 20 client.15492 trim_cache size 0 max 16384 2024-09-16T02:03:39.002 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-16T02:03:39.002 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T02:03:39.003 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-09-16T02:03:39.003 INFO:teuthology.orchestra.run.smithi105.stdout: "cbabab3c-4462-4a7b-bd58-200f0b911172": { 2024-09-16T02:03:39.003 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-09-16T02:03:39.003 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "cbabab3c-4462-4a7b-bd58-200f0b911172", 2024-09-16T02:03:39.003 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-09-16T02:03:39.003 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:03:39.003 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:03:39.003 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-16T02:03:39.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.998+0000 7f8959f13640 1 -- 172.21.15.105:0/2141854149 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f89400a32c0 msgr2=0x7f89400a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:39.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.998+0000 7f8959f13640 1 --2- 172.21.15.105:0/2141854149 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f89400a32c0 0x7f89400a5780 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f893c00b9c0 tx=0x7f893c00b950 comp rx=0 tx=0).stop 2024-09-16T02:03:39.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.998+0000 7f8959f13640 1 -- 172.21.15.105:0/2141854149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f895410c160 msgr2=0x7f895419cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:39.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.998+0000 7f8959f13640 1 --2- 172.21.15.105:0/2141854149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f895410c160 0x7f895419cc20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f894c007db0 tx=0x7f894c010420 comp rx=0 tx=0).stop 2024-09-16T02:03:39.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.998+0000 7f8959f13640 1 -- 172.21.15.105:0/2141854149 shutdown_connections 2024-09-16T02:03:39.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.998+0000 7f8959f13640 1 --2- 172.21.15.105:0/2141854149 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f89400a32c0 0x7f89400a5780 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:39.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.998+0000 7f8959f13640 1 --2- 172.21.15.105:0/2141854149 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8954103650 0x7f8954198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:39.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.998+0000 7f8959f13640 1 --2- 172.21.15.105:0/2141854149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f895410c160 0x7f895419cc20 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:39.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.998+0000 7f8959f13640 1 --2- 172.21.15.105:0/2141854149 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8954102d30 0x7f8954198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:39.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.998+0000 7f8959f13640 1 -- 172.21.15.105:0/2141854149 >> 172.21.15.105:0/2141854149 conn(0x7f89540fc7d0 msgr2=0x7f89540fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:39.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.999+0000 7f8959f13640 1 -- 172.21.15.105:0/2141854149 shutdown_connections 2024-09-16T02:03:39.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:03:38.999+0000 7f8959f13640 1 -- 172.21.15.105:0/2141854149 wait complete. 2024-09-16T02:03:39.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:38 smithi190 ceph-mon[34762]: scrub queued for path: / 2024-09-16T02:03:39.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:38 smithi190 ceph-mon[34762]: scrub summary: idle+waiting paths [/] 2024-09-16T02:03:39.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:38 smithi190 ceph-mon[34762]: scrub summary: active paths [/] 2024-09-16T02:03:39.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:38 smithi190 ceph-mon[34762]: scrub summary: idle+waiting paths [/] 2024-09-16T02:03:39.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:38 smithi190 ceph-mon[34762]: pgmap v372: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T02:03:39.889 INFO:tasks.cephfs.filesystem:scrub status for tag:cbabab3c-4462-4a7b-bd58-200f0b911172 - {'path': '/', 'tag': 'cbabab3c-4462-4a7b-bd58-200f0b911172', 'options': 'recursive,force'} 2024-09-16T02:03:40.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:40 smithi137 ceph-mon[34099]: pgmap v373: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-09-16T02:03:40.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:40 smithi137 ceph-mon[34099]: scrub summary: idle 2024-09-16T02:03:41.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:40 smithi105 ceph-mon[31197]: pgmap v373: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-09-16T02:03:41.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:40 smithi105 ceph-mon[31197]: scrub summary: idle 2024-09-16T02:03:41.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:40 smithi190 ceph-mon[34762]: pgmap v373: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-09-16T02:03:41.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:40 smithi190 ceph-mon[34762]: scrub summary: idle 2024-09-16T02:03:42.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:42 smithi137 ceph-mon[34099]: pgmap v374: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-09-16T02:03:43.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:42 smithi105 ceph-mon[31197]: pgmap v374: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-09-16T02:03:43.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:42 smithi190 ceph-mon[34762]: pgmap v374: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-09-16T02:03:44.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:44 smithi137 ceph-mon[34099]: pgmap v375: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-16T02:03:45.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:44 smithi105 ceph-mon[31197]: pgmap v375: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-16T02:03:45.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:44 smithi190 ceph-mon[34762]: pgmap v375: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-16T02:03:45.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:45 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:03:45.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:45 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:03:45.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:45 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:03:45.921 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:45 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:03:45.921 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:45 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:03:45.921 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:45 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:03:46.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:45 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:03:46.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:45 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:03:46.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:45 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:03:46.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:46 smithi137 ceph-mon[34099]: pgmap v376: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-16T02:03:46.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:46 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:03:46.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:46 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:03:46.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:46 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:03:47.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:46 smithi105 ceph-mon[31197]: pgmap v376: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-16T02:03:47.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:46 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:03:47.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:46 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:03:47.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:46 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:03:47.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:46 smithi190 ceph-mon[34762]: pgmap v376: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-16T02:03:47.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:46 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:03:47.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:46 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:03:47.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:46 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:03:47.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:47 smithi137 ceph-mon[34099]: pgmap v377: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-16T02:03:47.980 INFO:tasks.workunit.client.0.smithi105.stderr:Note: switching to 'ad35db06e8924912f01e32b35576297c0f1352b4'. 2024-09-16T02:03:47.980 INFO:tasks.workunit.client.0.smithi105.stderr: 2024-09-16T02:03:47.980 INFO:tasks.workunit.client.0.smithi105.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-16T02:03:47.980 INFO:tasks.workunit.client.0.smithi105.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-16T02:03:47.980 INFO:tasks.workunit.client.0.smithi105.stderr:state without impacting any branches by switching back to a branch. 2024-09-16T02:03:47.980 INFO:tasks.workunit.client.0.smithi105.stderr: 2024-09-16T02:03:47.980 INFO:tasks.workunit.client.0.smithi105.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-16T02:03:47.980 INFO:tasks.workunit.client.0.smithi105.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-16T02:03:47.980 INFO:tasks.workunit.client.0.smithi105.stderr: 2024-09-16T02:03:47.981 INFO:tasks.workunit.client.0.smithi105.stderr: git switch -c 2024-09-16T02:03:47.981 INFO:tasks.workunit.client.0.smithi105.stderr: 2024-09-16T02:03:47.981 INFO:tasks.workunit.client.0.smithi105.stderr:Or undo this operation with: 2024-09-16T02:03:47.981 INFO:tasks.workunit.client.0.smithi105.stderr: 2024-09-16T02:03:47.981 INFO:tasks.workunit.client.0.smithi105.stderr: git switch - 2024-09-16T02:03:47.981 INFO:tasks.workunit.client.0.smithi105.stderr: 2024-09-16T02:03:47.981 INFO:tasks.workunit.client.0.smithi105.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-16T02:03:47.981 INFO:tasks.workunit.client.0.smithi105.stderr: 2024-09-16T02:03:47.981 INFO:tasks.workunit.client.0.smithi105.stderr:HEAD is now at ad35db06e89 Merge pull request #59794 from zdover23/wip-doc-2024-09-15-backport-59613-to-squid 2024-09-16T02:03:47.985 DEBUG:teuthology.orchestra.run.smithi105:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-16T02:03:48.017 INFO:tasks.workunit.client.0.smithi105.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-16T02:03:48.018 INFO:tasks.workunit.client.0.smithi105.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-16T02:03:48.019 INFO:tasks.workunit.client.0.smithi105.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-16T02:03:48.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:47 smithi105 ceph-mon[31197]: pgmap v377: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-16T02:03:48.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:47 smithi190 ceph-mon[34762]: pgmap v377: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-16T02:03:48.695 INFO:tasks.workunit.client.0.smithi105.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-16T02:03:48.773 INFO:tasks.workunit.client.0.smithi105.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-16T02:03:48.815 INFO:tasks.workunit.client.0.smithi105.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-16T02:03:48.817 INFO:tasks.workunit.client.0.smithi105.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-16T02:03:48.817 INFO:tasks.workunit.client.0.smithi105.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-16T02:03:48.856 INFO:tasks.workunit.client.0.smithi105.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-16T02:03:48.863 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-16T02:03:48.863 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-16T02:03:48.921 INFO:tasks.workunit:Running workunits matching suites/iozone.sh on client.0... 2024-09-16T02:03:48.922 INFO:tasks.workunit:Running workunit suites/iozone.sh... 2024-09-16T02:03:48.923 DEBUG:teuthology.orchestra.run.smithi105:workunit test suites/iozone.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ad35db06e8924912f01e32b35576297c0f1352b4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh 2024-09-16T02:03:48.986 INFO:tasks.workunit.client.0.smithi105.stderr:+ iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-09-16T02:03:49.056 INFO:tasks.workunit.client.0.smithi105.stdout: Iozone: Performance Test of File I/O 2024-09-16T02:03:49.056 INFO:tasks.workunit.client.0.smithi105.stdout: Version $Revision: 3.494 $ 2024-09-16T02:03:49.056 INFO:tasks.workunit.client.0.smithi105.stdout: Compiled for 64 bit mode. 2024-09-16T02:03:49.056 INFO:tasks.workunit.client.0.smithi105.stdout: Build: linux-AMD64 2024-09-16T02:03:49.056 INFO:tasks.workunit.client.0.smithi105.stdout: 2024-09-16T02:03:49.056 INFO:tasks.workunit.client.0.smithi105.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-09-16T02:03:49.057 INFO:tasks.workunit.client.0.smithi105.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-09-16T02:03:49.057 INFO:tasks.workunit.client.0.smithi105.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-09-16T02:03:49.057 INFO:tasks.workunit.client.0.smithi105.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-09-16T02:03:49.057 INFO:tasks.workunit.client.0.smithi105.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-09-16T02:03:49.057 INFO:tasks.workunit.client.0.smithi105.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-09-16T02:03:49.057 INFO:tasks.workunit.client.0.smithi105.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-09-16T02:03:49.057 INFO:tasks.workunit.client.0.smithi105.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-09-16T02:03:49.058 INFO:tasks.workunit.client.0.smithi105.stdout: Alexey Skidanov, Sudhir Kumar. 2024-09-16T02:03:49.058 INFO:tasks.workunit.client.0.smithi105.stdout: 2024-09-16T02:03:49.058 INFO:tasks.workunit.client.0.smithi105.stdout: Run began: Mon Sep 16 02:03:48 2024 2024-09-16T02:03:49.058 INFO:tasks.workunit.client.0.smithi105.stdout: 2024-09-16T02:03:49.058 INFO:tasks.workunit.client.0.smithi105.stdout: Include close in write timing 2024-09-16T02:03:49.058 INFO:tasks.workunit.client.0.smithi105.stdout: Include fsync in write timing 2024-09-16T02:03:49.058 INFO:tasks.workunit.client.0.smithi105.stdout: File size set to 1048576 kB 2024-09-16T02:03:49.058 INFO:tasks.workunit.client.0.smithi105.stdout: Record Size 16 kB 2024-09-16T02:03:49.058 INFO:tasks.workunit.client.0.smithi105.stdout: Command line used: iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-09-16T02:03:49.058 INFO:tasks.workunit.client.0.smithi105.stdout: Output is in kBytes/sec 2024-09-16T02:03:49.059 INFO:tasks.workunit.client.0.smithi105.stdout: Time Resolution = 0.000001 seconds. 2024-09-16T02:03:49.059 INFO:tasks.workunit.client.0.smithi105.stdout: Processor cache size set to 1024 kBytes. 2024-09-16T02:03:49.059 INFO:tasks.workunit.client.0.smithi105.stdout: Processor cache line size set to 32 bytes. 2024-09-16T02:03:49.059 INFO:tasks.workunit.client.0.smithi105.stdout: File stride size set to 17 * record size. 2024-09-16T02:03:49.059 INFO:tasks.workunit.client.0.smithi105.stdout: Throughput test with 1 process 2024-09-16T02:03:49.059 INFO:tasks.workunit.client.0.smithi105.stdout: Each process writes a 1048576 kByte file in 16 kByte records 2024-09-16T02:03:50.796 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:50 smithi190 ceph-mon[34762]: pgmap v378: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-16T02:03:50.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:50 smithi137 ceph-mon[34099]: pgmap v378: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-16T02:03:51.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:50 smithi105 ceph-mon[31197]: pgmap v378: 97 pgs: 97 active+clean; 660 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-16T02:03:52.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:52 smithi190 ceph-mon[34762]: pgmap v379: 97 pgs: 97 active+clean; 77 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 6.3 MiB/s wr, 11 op/s 2024-09-16T02:03:52.920 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:52 smithi137 ceph-mon[34099]: pgmap v379: 97 pgs: 97 active+clean; 77 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 6.3 MiB/s wr, 11 op/s 2024-09-16T02:03:53.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:52 smithi105 ceph-mon[31197]: pgmap v379: 97 pgs: 97 active+clean; 77 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 6.3 MiB/s wr, 11 op/s 2024-09-16T02:03:54.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:54 smithi190 ceph-mon[34762]: pgmap v380: 97 pgs: 97 active+clean; 161 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 13 MiB/s wr, 14 op/s 2024-09-16T02:03:54.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:54 smithi137 ceph-mon[34099]: pgmap v380: 97 pgs: 97 active+clean; 161 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 13 MiB/s wr, 14 op/s 2024-09-16T02:03:55.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:54 smithi105 ceph-mon[31197]: pgmap v380: 97 pgs: 97 active+clean; 161 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 13 MiB/s wr, 14 op/s 2024-09-16T02:03:56.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:56 smithi190 ceph-mon[34762]: pgmap v381: 97 pgs: 97 active+clean; 473 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 39 MiB/s wr, 21 op/s 2024-09-16T02:03:56.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:56 smithi137 ceph-mon[34099]: pgmap v381: 97 pgs: 97 active+clean; 473 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 39 MiB/s wr, 21 op/s 2024-09-16T02:03:57.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:56 smithi105 ceph-mon[31197]: pgmap v381: 97 pgs: 97 active+clean; 473 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 39 MiB/s wr, 21 op/s 2024-09-16T02:03:58.038 INFO:tasks.workunit.client.0.smithi105.stdout: 2024-09-16T02:03:58.038 INFO:tasks.workunit.client.0.smithi105.stdout: Children see throughput for 1 initial writers = 116764.29 kB/sec 2024-09-16T02:03:58.039 INFO:tasks.workunit.client.0.smithi105.stdout: Parent sees throughput for 1 initial writers = 116761.62 kB/sec 2024-09-16T02:03:58.039 INFO:tasks.workunit.client.0.smithi105.stdout: Min throughput per process = 116764.29 kB/sec 2024-09-16T02:03:58.039 INFO:tasks.workunit.client.0.smithi105.stdout: Max throughput per process = 116764.29 kB/sec 2024-09-16T02:03:58.039 INFO:tasks.workunit.client.0.smithi105.stdout: Avg throughput per process = 116764.29 kB/sec 2024-09-16T02:03:58.039 INFO:tasks.workunit.client.0.smithi105.stdout: Min xfer = 1048576.00 kB 2024-09-16T02:03:58.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:03:58 smithi190 ceph-mon[34762]: pgmap v382: 97 pgs: 97 active+clean; 613 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 51 MiB/s wr, 23 op/s 2024-09-16T02:03:58.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:03:58 smithi137 ceph-mon[34099]: pgmap v382: 97 pgs: 97 active+clean; 613 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 51 MiB/s wr, 23 op/s 2024-09-16T02:03:59.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:03:58 smithi105 ceph-mon[31197]: pgmap v382: 97 pgs: 97 active+clean; 613 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 51 MiB/s wr, 23 op/s 2024-09-16T02:04:00.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:00 smithi190 ceph-mon[34762]: pgmap v383: 97 pgs: 97 active+clean; 765 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 64 MiB/s wr, 28 op/s 2024-09-16T02:04:00.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:00 smithi137 ceph-mon[34099]: pgmap v383: 97 pgs: 97 active+clean; 765 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 64 MiB/s wr, 28 op/s 2024-09-16T02:04:01.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:00 smithi105 ceph-mon[31197]: pgmap v383: 97 pgs: 97 active+clean; 765 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 64 MiB/s wr, 28 op/s 2024-09-16T02:04:02.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:02 smithi190 ceph-mon[34762]: pgmap v384: 97 pgs: 97 active+clean; 1021 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 85 MiB/s wr, 33 op/s 2024-09-16T02:04:02.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:02 smithi137 ceph-mon[34099]: pgmap v384: 97 pgs: 97 active+clean; 1021 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 85 MiB/s wr, 33 op/s 2024-09-16T02:04:03.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:02 smithi105 ceph-mon[31197]: pgmap v384: 97 pgs: 97 active+clean; 1021 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 85 MiB/s wr, 33 op/s 2024-09-16T02:04:04.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:04 smithi190 ceph-mon[34762]: pgmap v385: 97 pgs: 97 active+clean; 1.0 GiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 79 MiB/s wr, 31 op/s 2024-09-16T02:04:04.846 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:04 smithi105 ceph-mon[31197]: pgmap v385: 97 pgs: 97 active+clean; 1.0 GiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 79 MiB/s wr, 31 op/s 2024-09-16T02:04:04.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:04 smithi137 ceph-mon[34099]: pgmap v385: 97 pgs: 97 active+clean; 1.0 GiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 79 MiB/s wr, 31 op/s 2024-09-16T02:04:06.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:06 smithi190 ceph-mon[34762]: pgmap v386: 97 pgs: 97 active+clean; 1.0 GiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 75 MiB/s wr, 29 op/s 2024-09-16T02:04:06.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:06 smithi137 ceph-mon[34099]: pgmap v386: 97 pgs: 97 active+clean; 1.0 GiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 75 MiB/s wr, 29 op/s 2024-09-16T02:04:07.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:06 smithi105 ceph-mon[31197]: pgmap v386: 97 pgs: 97 active+clean; 1.0 GiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 75 MiB/s wr, 29 op/s 2024-09-16T02:04:08.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:08 smithi190 ceph-mon[34762]: pgmap v387: 97 pgs: 97 active+clean; 1.0 GiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 54 MiB/s wr, 23 op/s 2024-09-16T02:04:08.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:08 smithi137 ceph-mon[34099]: pgmap v387: 97 pgs: 97 active+clean; 1.0 GiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 54 MiB/s wr, 23 op/s 2024-09-16T02:04:09.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:08 smithi105 ceph-mon[31197]: pgmap v387: 97 pgs: 97 active+clean; 1.0 GiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 54 MiB/s wr, 23 op/s 2024-09-16T02:04:09.890 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T02:04:10.183 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:10.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.764+0000 7fc9d1117640 1 -- 172.21.15.105:0/3342731796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9cc100e20 msgr2=0x7fc9cc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:10.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.764+0000 7fc9d1117640 1 --2- 172.21.15.105:0/3342731796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9cc100e20 0x7fc9cc101200 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc009900 tx=0x7fc9bc02a480 comp rx=0 tx=0).stop 2024-09-16T02:04:10.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.764+0000 7fc9d1117640 1 -- 172.21.15.105:0/3342731796 shutdown_connections 2024-09-16T02:04:10.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.764+0000 7fc9d1117640 1 --2- 172.21.15.105:0/3342731796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9cc102570 0x7fc9cc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.764+0000 7fc9d1117640 1 --2- 172.21.15.105:0/3342731796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9cc102ef0 0x7fc9cc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.764+0000 7fc9d1117640 1 --2- 172.21.15.105:0/3342731796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9cc100e20 0x7fc9cc101200 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.764+0000 7fc9d1117640 1 -- 172.21.15.105:0/3342731796 >> 172.21.15.105:0/3342731796 conn(0x7fc9cc0fc7d0 msgr2=0x7fc9cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:10.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.764+0000 7fc9d1117640 1 -- 172.21.15.105:0/3342731796 shutdown_connections 2024-09-16T02:04:10.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.764+0000 7fc9d1117640 1 -- 172.21.15.105:0/3342731796 wait complete. 2024-09-16T02:04:10.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.765+0000 7fc9d1117640 1 Processor -- start 2024-09-16T02:04:10.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.765+0000 7fc9d1117640 1 -- start start 2024-09-16T02:04:10.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.766+0000 7fc9d1117640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9cc100e20 0x7fc9cc198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:10.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.766+0000 7fc9d1117640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9cc102570 0x7fc9cc198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:10.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.766+0000 7fc9d1117640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9cc102ef0 0x7fc9cc19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:10.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.766+0000 7fc9d1117640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc9cc110e30 con 0x7fc9cc100e20 2024-09-16T02:04:10.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.766+0000 7fc9d1117640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc9cc110cb0 con 0x7fc9cc102570 2024-09-16T02:04:10.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.766+0000 7fc9d1117640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc9cc110fb0 con 0x7fc9cc102ef0 2024-09-16T02:04:10.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.766+0000 7fc9cad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9cc100e20 0x7fc9cc198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:10.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.766+0000 7fc9cad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9cc100e20 0x7fc9cc198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54332/0 (socket says 172.21.15.105:54332) 2024-09-16T02:04:10.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.766+0000 7fc9cad9d640 1 -- 172.21.15.105:0/3614901120 learned_addr learned my addr 172.21.15.105:0/3614901120 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:10.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.766+0000 7fc9ca59c640 1 --2- 172.21.15.105:0/3614901120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9cc102570 0x7fc9cc198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:10.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.766+0000 7fc9cb59e640 1 --2- 172.21.15.105:0/3614901120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9cc102ef0 0x7fc9cc19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:10.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.766+0000 7fc9cad9d640 1 -- 172.21.15.105:0/3614901120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9cc102ef0 msgr2=0x7fc9cc19ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:10.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.766+0000 7fc9cad9d640 1 --2- 172.21.15.105:0/3614901120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9cc102ef0 0x7fc9cc19ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.766+0000 7fc9cad9d640 1 -- 172.21.15.105:0/3614901120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9cc102570 msgr2=0x7fc9cc198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:10.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.766+0000 7fc9cad9d640 1 --2- 172.21.15.105:0/3614901120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9cc102570 0x7fc9cc198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.766+0000 7fc9cad9d640 1 -- 172.21.15.105:0/3614901120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9cc19d3d0 con 0x7fc9cc100e20 2024-09-16T02:04:10.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.766+0000 7fc9cb59e640 1 --2- 172.21.15.105:0/3614901120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9cc102ef0 0x7fc9cc19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:04:10.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.766+0000 7fc9cad9d640 1 --2- 172.21.15.105:0/3614901120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9cc100e20 0x7fc9cc198420 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc006010 tx=0x7fc9bc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:10.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.767+0000 7fc9b3fff640 1 -- 172.21.15.105:0/3614901120 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc03f030 con 0x7fc9cc100e20 2024-09-16T02:04:10.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.767+0000 7fc9b3fff640 1 -- 172.21.15.105:0/3614901120 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9bc003aa0 con 0x7fc9cc100e20 2024-09-16T02:04:10.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.767+0000 7fc9d1117640 1 -- 172.21.15.105:0/3614901120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9cc19d660 con 0x7fc9cc100e20 2024-09-16T02:04:10.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.767+0000 7fc9b3fff640 1 -- 172.21.15.105:0/3614901120 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc035070 con 0x7fc9cc100e20 2024-09-16T02:04:10.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.767+0000 7fc9d1117640 1 -- 172.21.15.105:0/3614901120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9cc1a4fa0 con 0x7fc9cc100e20 2024-09-16T02:04:10.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.770+0000 7fc9b3fff640 1 -- 172.21.15.105:0/3614901120 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fc9bc042030 con 0x7fc9cc100e20 2024-09-16T02:04:10.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.770+0000 7fc9d34f0740 1 Processor -- start 2024-09-16T02:04:10.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.770+0000 7fc9d34f0740 1 -- start start 2024-09-16T02:04:10.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.770+0000 7fc9d34f0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555630c744a0 0x555630c74880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:10.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.770+0000 7fc9d34f0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555630c74dc0 0x555630c7f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:10.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.770+0000 7fc9d34f0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555630c7f680 0x555630c81a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:10.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.770+0000 7fc9d34f0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555630bda2e0 con 0x555630c7f680 2024-09-16T02:04:10.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.770+0000 7fc9d34f0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555630b81ce0 con 0x555630c74dc0 2024-09-16T02:04:10.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.770+0000 7fc9d34f0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x555630bd62a0 con 0x555630c744a0 2024-09-16T02:04:10.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.771+0000 7fc9ca59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555630c7f680 0x555630c81a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:10.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.771+0000 7fc9ca59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555630c7f680 0x555630c81a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54348/0 (socket says 172.21.15.105:54348) 2024-09-16T02:04:10.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.771+0000 7fc9ca59c640 1 -- 172.21.15.105:0/536994307 learned_addr learned my addr 172.21.15.105:0/536994307 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:10.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.771+0000 7fc9cad9d640 1 --2- 172.21.15.105:0/536994307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555630c74dc0 0x555630c7f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:10.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.771+0000 7fc9cb59e640 1 --2- 172.21.15.105:0/536994307 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555630c744a0 0x555630c74880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:10.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.771+0000 7fc9cad9d640 1 -- 172.21.15.105:0/536994307 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555630c744a0 msgr2=0x555630c74880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:10.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.771+0000 7fc9cad9d640 1 --2- 172.21.15.105:0/536994307 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555630c744a0 0x555630c74880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.771+0000 7fc9cad9d640 1 -- 172.21.15.105:0/536994307 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555630c7f680 msgr2=0x555630c81a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:10.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.771+0000 7fc9cad9d640 1 --2- 172.21.15.105:0/536994307 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555630c7f680 0x555630c81a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.771+0000 7fc9cad9d640 1 -- 172.21.15.105:0/536994307 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555630b66440 con 0x555630c74dc0 2024-09-16T02:04:10.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.771+0000 7fc9cb59e640 1 --2- 172.21.15.105:0/536994307 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555630c744a0 0x555630c74880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:04:10.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.771+0000 7fc9ca59c640 1 --2- 172.21.15.105:0/536994307 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555630c7f680 0x555630c81a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:04:10.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.771+0000 7fc9b3fff640 1 --2- 172.21.15.105:0/3614901120 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc9b40a32c0 0x7fc9b40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:10.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.771+0000 7fc9cb59e640 1 --2- 172.21.15.105:0/3614901120 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc9b40a32c0 0x7fc9b40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:10.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.771+0000 7fc9cad9d640 1 --2- 172.21.15.105:0/536994307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555630c74dc0 0x555630c7f140 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc0ed8f0 tx=0x7fc9bc0f81a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fbd60d0f54b4e66 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:10.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.771+0000 7fc9b3fff640 1 -- 172.21.15.105:0/3614901120 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fc9bc0eff10 con 0x7fc9cc100e20 2024-09-16T02:04:10.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.772+0000 7fc9b17fa640 1 -- 172.21.15.105:0/536994307 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc0b8710 con 0x555630c74dc0 2024-09-16T02:04:10.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.772+0000 7fc9b17fa640 1 -- 172.21.15.105:0/536994307 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9bc0bcb70 con 0x555630c74dc0 2024-09-16T02:04:10.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.772+0000 7fc9b17fa640 1 -- 172.21.15.105:0/536994307 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc0bbdd0 con 0x555630c74dc0 2024-09-16T02:04:10.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.772+0000 7fc9cb59e640 1 --2- 172.21.15.105:0/3614901120 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc9b40a32c0 0x7fc9b40a5780 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc9c40095f0 tx=0x7fc9c4009580 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:10.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.772+0000 7fc9d34f0740 1 -- 172.21.15.105:0/536994307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555630c74dc0 msgr2=0x555630c7f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:10.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.772+0000 7fc9d34f0740 1 --2- 172.21.15.105:0/536994307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555630c74dc0 0x555630c7f140 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc0ed8f0 tx=0x7fc9bc0f81a0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.773+0000 7fc9d34f0740 1 -- 172.21.15.105:0/536994307 shutdown_connections 2024-09-16T02:04:10.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.773+0000 7fc9d34f0740 1 --2- 172.21.15.105:0/536994307 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555630c7f680 0x555630c81a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.773+0000 7fc9d34f0740 1 --2- 172.21.15.105:0/536994307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555630c74dc0 0x555630c7f140 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.773+0000 7fc9d34f0740 1 --2- 172.21.15.105:0/536994307 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555630c744a0 0x555630c74880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.773+0000 7fc9d34f0740 1 -- 172.21.15.105:0/536994307 >> 172.21.15.105:0/536994307 conn(0x555630c70a30 msgr2=0x555630c70e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:10.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.773+0000 7fc9d34f0740 1 -- 172.21.15.105:0/536994307 shutdown_connections 2024-09-16T02:04:10.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.773+0000 7fc9d34f0740 1 -- 172.21.15.105:0/536994307 wait complete. 2024-09-16T02:04:10.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.773+0000 7fc9d34f0740 1 -- start start 2024-09-16T02:04:10.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.773+0000 7fc9d34f0740 1 Processor -- start 2024-09-16T02:04:10.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.773+0000 7fc9d34f0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555630c744a0 0x555630c7ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:10.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.773+0000 7fc9d34f0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555630c74dc0 0x555630c7c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:10.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.773+0000 7fc9d34f0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555630c7f680 0x555630c7c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:10.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.773+0000 7fc9d34f0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555630c86420 con 0x555630c74dc0 2024-09-16T02:04:10.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.773+0000 7fc9d34f0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555630b95050 con 0x555630c7f680 2024-09-16T02:04:10.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.773+0000 7fc9d34f0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x555630c865a0 con 0x555630c744a0 2024-09-16T02:04:10.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.773+0000 7fc9d34f0740 10 client.0 _ll_register_callbacks cb 0x555630bd5160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:04:10.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.774+0000 7fc9ca59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555630c74dc0 0x555630c7c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:10.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.774+0000 7fc9ca59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555630c74dc0 0x555630c7c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54364/0 (socket says 172.21.15.105:54364) 2024-09-16T02:04:10.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.774+0000 7fc9ca59c640 1 -- 172.21.15.105:0/2965498427 learned_addr learned my addr 172.21.15.105:0/2965498427 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:10.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.774+0000 7fc9cad9d640 1 --2- 172.21.15.105:0/2965498427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555630c744a0 0x555630c7ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:10.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.774+0000 7fc9cb59e640 1 --2- 172.21.15.105:0/2965498427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555630c7f680 0x555630c7c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:10.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.774+0000 7fc9ca59c640 1 -- 172.21.15.105:0/2965498427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555630c744a0 msgr2=0x555630c7ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:10.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.774+0000 7fc9ca59c640 1 --2- 172.21.15.105:0/2965498427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555630c744a0 0x555630c7ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.774+0000 7fc9ca59c640 1 -- 172.21.15.105:0/2965498427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555630c7f680 msgr2=0x555630c7c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:10.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.774+0000 7fc9ca59c640 1 --2- 172.21.15.105:0/2965498427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555630c7f680 0x555630c7c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.774+0000 7fc9ca59c640 1 -- 172.21.15.105:0/2965498427 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55563094b3c0 con 0x555630c74dc0 2024-09-16T02:04:10.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.774+0000 7fc9cad9d640 1 --2- 172.21.15.105:0/2965498427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555630c744a0 0x555630c7ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:04:10.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.774+0000 7fc9ca59c640 1 --2- 172.21.15.105:0/2965498427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555630c74dc0 0x555630c7c1f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc9c00126e0 tx=0x7fc9c0012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25df87a6d50813c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:10.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.774+0000 7fc9aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:04:10.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.774+0000 7fc9aeff5640 1 -- 172.21.15.105:0/2965498427 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c0012110 con 0x555630c74dc0 2024-09-16T02:04:10.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.774+0000 7fc9aeff5640 1 -- 172.21.15.105:0/2965498427 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fc9c0004050 con 0x555630c74dc0 2024-09-16T02:04:10.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.774+0000 7fc9d34f0740 1 -- 172.21.15.105:0/2965498427 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9bc0bcb70 con 0x555630c74dc0 2024-09-16T02:04:10.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.774+0000 7fc9aeff5640 1 -- 172.21.15.105:0/2965498427 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc9c00122b0 con 0x555630c74dc0 2024-09-16T02:04:10.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.775+0000 7fc9d34f0740 10 client.15504 fetch_fsmap learned FSMap version 28 2024-09-16T02:04:10.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.775+0000 7fc9d34f0740 1 -- 172.21.15.105:0/2965498427 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555630c91d40 con 0x555630c74dc0 2024-09-16T02:04:10.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.775+0000 7fc9aeff5640 1 -- 172.21.15.105:0/2965498427 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9bc0bcb70 con 0x555630c74dc0 2024-09-16T02:04:10.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.775+0000 7fc9aeff5640 1 -- 172.21.15.105:0/2965498427 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc9c0003940 con 0x555630c74dc0 2024-09-16T02:04:10.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.775+0000 7fc9d34f0740 10 client.15504 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:04:10.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.775+0000 7fc9d34f0740 10 client.15504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:04:10.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.775+0000 7fc9d34f0740 20 client.15504 populate_metadata read hostname 'smithi105' 2024-09-16T02:04:10.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.775+0000 7fc9d34f0740 1 --2- 172.21.15.105:0/2965498427 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x555630c93250 0x555630c956d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:10.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.775+0000 7fc9d34f0740 4 client.15504 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:04:10.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.775+0000 7fc9d34f0740 1 -- 172.21.15.105:0/2965498427 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555630be7850 con 0x555630c93250 2024-09-16T02:04:10.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.775+0000 7fc9cad9d640 1 --2- 172.21.15.105:0/2965498427 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x555630c93250 0x555630c956d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:10.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.776+0000 7fc9cad9d640 1 --2- 172.21.15.105:0/2965498427 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x555630c93250 0x555630c956d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2dcbabd275f764a server_cookie=96f4e295f2bea7bf in_seq=0 out_seq=0 2024-09-16T02:04:10.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.776+0000 7fc9aeff5640 10 client.15504 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:04:10.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.776+0000 7fc9aeff5640 1 -- 172.21.15.105:0/2965498427 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555630be7850 con 0x555630c93250 2024-09-16T02:04:10.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.776+0000 7fc9aeff5640 10 client.15504 handle_command_reply: tid=0 2024-09-16T02:04:10.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.776+0000 7fc9d34f0740 1 client.15504 shutdown 2024-09-16T02:04:10.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.777+0000 7fc9d34f0740 1 -- 172.21.15.105:0/2965498427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555630c74dc0 msgr2=0x555630c7c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:10.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.777+0000 7fc9d34f0740 1 --2- 172.21.15.105:0/2965498427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555630c74dc0 0x555630c7c1f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc9c00126e0 tx=0x7fc9c0012ba0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.777+0000 7fc9d34f0740 1 -- 172.21.15.105:0/2965498427 shutdown_connections 2024-09-16T02:04:10.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.777+0000 7fc9d34f0740 1 --2- 172.21.15.105:0/2965498427 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x555630c93250 0x555630c956d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.777+0000 7fc9d34f0740 1 --2- 172.21.15.105:0/2965498427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555630c7f680 0x555630c7c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.777+0000 7fc9d34f0740 1 --2- 172.21.15.105:0/2965498427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555630c74dc0 0x555630c7c1f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.777+0000 7fc9aeff5640 0 client.15504 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:04:10.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.777+0000 7fc9d34f0740 1 --2- 172.21.15.105:0/2965498427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555630c744a0 0x555630c7ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.777+0000 7fc9d34f0740 1 -- 172.21.15.105:0/2965498427 >> 172.21.15.105:0/2965498427 conn(0x555630c70a30 msgr2=0x555630c76710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:10.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.777+0000 7fc9d34f0740 1 -- 172.21.15.105:0/2965498427 shutdown_connections 2024-09-16T02:04:10.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.777+0000 7fc9d34f0740 1 -- 172.21.15.105:0/2965498427 wait complete. 2024-09-16T02:04:10.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.777+0000 7fc9d34f0740 20 client.15504 trim_cache size 0 max 16384 2024-09-16T02:04:10.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.795+0000 7fc9d34f0740 1 Processor -- start 2024-09-16T02:04:10.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.795+0000 7fc9d34f0740 1 -- start start 2024-09-16T02:04:10.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.795+0000 7fc9d34f0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555630c70a30 0x555630c8f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:10.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.795+0000 7fc9d34f0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555630c744a0 0x555630c77900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:10.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.795+0000 7fc9d34f0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555630c74dc0 0x555630c7b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:10.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.795+0000 7fc9d34f0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc9cc1ab3c0 con 0x555630c744a0 2024-09-16T02:04:10.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.795+0000 7fc9d34f0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc9cc1acf00 con 0x555630c70a30 2024-09-16T02:04:10.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.795+0000 7fc9d34f0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc9cc1aba90 con 0x555630c74dc0 2024-09-16T02:04:10.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.795+0000 7fc9ca59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555630c744a0 0x555630c77900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:10.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.795+0000 7fc9ca59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555630c744a0 0x555630c77900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54380/0 (socket says 172.21.15.105:54380) 2024-09-16T02:04:10.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.795+0000 7fc9ca59c640 1 -- 172.21.15.105:0/1768935362 learned_addr learned my addr 172.21.15.105:0/1768935362 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:10.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.795+0000 7fc9ca59c640 1 -- 172.21.15.105:0/1768935362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555630c74dc0 msgr2=0x555630c7b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:10.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.795+0000 7fc9ca59c640 1 --2- 172.21.15.105:0/1768935362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555630c74dc0 0x555630c7b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.795+0000 7fc9ca59c640 1 -- 172.21.15.105:0/1768935362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555630c70a30 msgr2=0x555630c8f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:10.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.795+0000 7fc9ca59c640 1 --2- 172.21.15.105:0/1768935362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555630c70a30 0x555630c8f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.795+0000 7fc9ca59c640 1 -- 172.21.15.105:0/1768935362 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555630c704d0 con 0x555630c744a0 2024-09-16T02:04:10.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.796+0000 7fc9ca59c640 1 --2- 172.21.15.105:0/1768935362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555630c744a0 0x555630c77900 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0003de0 tx=0x7fc9c0009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=785e8abb2c53d06a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:10.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.796+0000 7fc9b0ff9640 1 -- 172.21.15.105:0/1768935362 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c0012110 con 0x555630c744a0 2024-09-16T02:04:10.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.796+0000 7fc9b0ff9640 1 -- 172.21.15.105:0/1768935362 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9c000fdd0 con 0x555630c744a0 2024-09-16T02:04:10.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.796+0000 7fc9b0ff9640 1 -- 172.21.15.105:0/1768935362 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c0005460 con 0x555630c744a0 2024-09-16T02:04:10.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.797+0000 7fc9d34f0740 1 -- 172.21.15.105:0/1768935362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555630c744a0 msgr2=0x555630c77900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:10.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.797+0000 7fc9d34f0740 1 --2- 172.21.15.105:0/1768935362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555630c744a0 0x555630c77900 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0003de0 tx=0x7fc9c0009f00 comp rx=0 tx=0).stop 2024-09-16T02:04:10.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.797+0000 7fc9d34f0740 1 -- 172.21.15.105:0/1768935362 shutdown_connections 2024-09-16T02:04:10.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.797+0000 7fc9d34f0740 1 --2- 172.21.15.105:0/1768935362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555630c74dc0 0x555630c7b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.797+0000 7fc9d34f0740 1 --2- 172.21.15.105:0/1768935362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555630c744a0 0x555630c77900 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.797+0000 7fc9d34f0740 1 --2- 172.21.15.105:0/1768935362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555630c70a30 0x555630c8f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.797+0000 7fc9d34f0740 1 -- 172.21.15.105:0/1768935362 >> 172.21.15.105:0/1768935362 conn(0x555630c7c890 msgr2=0x555630c7aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:10.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.797+0000 7fc9d34f0740 1 -- 172.21.15.105:0/1768935362 shutdown_connections 2024-09-16T02:04:10.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.797+0000 7fc9d34f0740 1 -- 172.21.15.105:0/1768935362 wait complete. 2024-09-16T02:04:10.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.797+0000 7fc9d34f0740 1 -- start start 2024-09-16T02:04:10.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.798+0000 7fc9d34f0740 1 Processor -- start 2024-09-16T02:04:10.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.798+0000 7fc9d34f0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555630c70a30 0x555630c8b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:10.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.798+0000 7fc9d34f0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555630c744a0 0x555630c8be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:10.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.798+0000 7fc9d34f0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555630c74dc0 0x555630c86cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:10.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.798+0000 7fc9d34f0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc9cc1ac830 con 0x555630c70a30 2024-09-16T02:04:10.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.798+0000 7fc9d34f0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc9cc1ac160 con 0x555630c74dc0 2024-09-16T02:04:10.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.798+0000 7fc9d34f0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc9cc1a93a0 con 0x555630c744a0 2024-09-16T02:04:10.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.798+0000 7fc9d34f0740 10 client.0 _ll_register_callbacks cb 0x7fc9c0009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:04:10.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.798+0000 7fc9cad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555630c70a30 0x555630c8b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:10.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.798+0000 7fc9cad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555630c70a30 0x555630c8b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54394/0 (socket says 172.21.15.105:54394) 2024-09-16T02:04:10.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.798+0000 7fc9cad9d640 1 -- 172.21.15.105:0/1219176357 learned_addr learned my addr 172.21.15.105:0/1219176357 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:10.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.799+0000 7fc9cad9d640 1 -- 172.21.15.105:0/1219176357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555630c744a0 msgr2=0x555630c8be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:10.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.799+0000 7fc9cad9d640 1 --2- 172.21.15.105:0/1219176357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555630c744a0 0x555630c8be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.799+0000 7fc9cad9d640 1 -- 172.21.15.105:0/1219176357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555630c74dc0 msgr2=0x555630c86cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:10.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.799+0000 7fc9cad9d640 1 --2- 172.21.15.105:0/1219176357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555630c74dc0 0x555630c86cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.799+0000 7fc9cad9d640 1 -- 172.21.15.105:0/1219176357 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555630c874e0 con 0x555630c70a30 2024-09-16T02:04:10.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.799+0000 7fc9cad9d640 1 --2- 172.21.15.105:0/1219176357 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555630c70a30 0x555630c8b8f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc005d00 tx=0x7fc9bc009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a36ed276f520d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:10.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.799+0000 7fc9aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:04:10.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.799+0000 7fc9aeff5640 1 -- 172.21.15.105:0/1219176357 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc0a5070 con 0x555630c70a30 2024-09-16T02:04:10.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.799+0000 7fc9aeff5640 1 -- 172.21.15.105:0/1219176357 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fc9bc0f3a40 con 0x555630c70a30 2024-09-16T02:04:10.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.799+0000 7fc9d34f0740 1 -- 172.21.15.105:0/1219176357 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9c000fdd0 con 0x555630c70a30 2024-09-16T02:04:10.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.799+0000 7fc9aeff5640 1 -- 172.21.15.105:0/1219176357 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc9bc049040 con 0x555630c70a30 2024-09-16T02:04:10.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.799+0000 7fc9d34f0740 10 client.15510 fetch_fsmap learned FSMap version 28 2024-09-16T02:04:10.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.799+0000 7fc9d34f0740 1 -- 172.21.15.105:0/1219176357 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555630cfdae0 con 0x555630c70a30 2024-09-16T02:04:10.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.800+0000 7fc9aeff5640 1 -- 172.21.15.105:0/1219176357 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9c000fdd0 con 0x555630c70a30 2024-09-16T02:04:10.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.800+0000 7fc9aeff5640 1 -- 172.21.15.105:0/1219176357 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc9bc0b8ae0 con 0x555630c70a30 2024-09-16T02:04:10.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.800+0000 7fc9d34f0740 10 client.15510 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:04:10.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.800+0000 7fc9d34f0740 10 client.15510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:04:10.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.800+0000 7fc9d34f0740 20 client.15510 populate_metadata read hostname 'smithi105' 2024-09-16T02:04:10.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.800+0000 7fc9d34f0740 1 --2- 172.21.15.105:0/1219176357 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x555630c7f680 0x555630cffea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:10.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.800+0000 7fc9d34f0740 4 client.15510 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T02:04:10.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.800+0000 7fc9d34f0740 1 -- 172.21.15.105:0/1219176357 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555630be1d60 con 0x555630c7f680 2024-09-16T02:04:10.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.800+0000 7fc9ca59c640 1 --2- 172.21.15.105:0/1219176357 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x555630c7f680 0x555630cffea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:10.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.800+0000 7fc9aeff5640 10 client.15510 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:04:10.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.801+0000 7fc9ca59c640 1 --2- 172.21.15.105:0/1219176357 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x555630c7f680 0x555630cffea0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d620e4cbba99f31a server_cookie=fb9dfdfe94a8bcf2 in_seq=0 out_seq=0 2024-09-16T02:04:10.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.801+0000 7fc9aeff5640 1 -- 172.21.15.105:0/1219176357 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555630be1d60 con 0x555630c7f680 2024-09-16T02:04:10.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.801+0000 7fc9aeff5640 10 client.15510 handle_command_reply: tid=0 2024-09-16T02:04:10.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.801+0000 7fc9d34f0740 1 client.15510 shutdown 2024-09-16T02:04:10.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.801+0000 7fc9d34f0740 1 -- 172.21.15.105:0/1219176357 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555630c70a30 msgr2=0x555630c8b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:10.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.801+0000 7fc9d34f0740 1 --2- 172.21.15.105:0/1219176357 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555630c70a30 0x555630c8b8f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc005d00 tx=0x7fc9bc009680 comp rx=0 tx=0).stop 2024-09-16T02:04:10.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.801+0000 7fc9d34f0740 1 -- 172.21.15.105:0/1219176357 shutdown_connections 2024-09-16T02:04:10.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.801+0000 7fc9d34f0740 1 --2- 172.21.15.105:0/1219176357 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x555630c7f680 0x555630cffea0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.801+0000 7fc9d34f0740 1 --2- 172.21.15.105:0/1219176357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555630c744a0 0x555630c8be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.801+0000 7fc9d34f0740 1 --2- 172.21.15.105:0/1219176357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555630c74dc0 0x555630c86cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.801+0000 7fc9d34f0740 1 --2- 172.21.15.105:0/1219176357 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555630c70a30 0x555630c8b8f0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.801+0000 7fc9d34f0740 1 -- 172.21.15.105:0/1219176357 >> 172.21.15.105:0/1219176357 conn(0x555630c7c890 msgr2=0x555630c7b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:10.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.801+0000 7fc9aeff5640 0 client.15510 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:04:10.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.801+0000 7fc9d34f0740 1 -- 172.21.15.105:0/1219176357 shutdown_connections 2024-09-16T02:04:10.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.801+0000 7fc9d34f0740 1 -- 172.21.15.105:0/1219176357 wait complete. 2024-09-16T02:04:10.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.802+0000 7fc9d34f0740 20 client.15510 trim_cache size 0 max 16384 2024-09-16T02:04:10.808 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-16T02:04:10.808 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-09-16T02:04:10.808 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-09-16T02:04:10.808 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-16T02:04:10.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.802+0000 7fc9d1117640 1 -- 172.21.15.105:0/3614901120 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc9b40a32c0 msgr2=0x7fc9b40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:10.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.802+0000 7fc9d1117640 1 --2- 172.21.15.105:0/3614901120 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc9b40a32c0 0x7fc9b40a5780 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc9c40095f0 tx=0x7fc9c4009580 comp rx=0 tx=0).stop 2024-09-16T02:04:10.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.802+0000 7fc9d1117640 1 -- 172.21.15.105:0/3614901120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9cc100e20 msgr2=0x7fc9cc198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:10.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.802+0000 7fc9d1117640 1 --2- 172.21.15.105:0/3614901120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9cc100e20 0x7fc9cc198420 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc006010 tx=0x7fc9bc004270 comp rx=0 tx=0).stop 2024-09-16T02:04:10.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.802+0000 7fc9d1117640 1 -- 172.21.15.105:0/3614901120 shutdown_connections 2024-09-16T02:04:10.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.802+0000 7fc9d1117640 1 --2- 172.21.15.105:0/3614901120 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc9b40a32c0 0x7fc9b40a5780 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.802+0000 7fc9d1117640 1 --2- 172.21.15.105:0/3614901120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9cc102ef0 0x7fc9cc19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.802+0000 7fc9d1117640 1 --2- 172.21.15.105:0/3614901120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9cc102570 0x7fc9cc198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.802+0000 7fc9d1117640 1 --2- 172.21.15.105:0/3614901120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9cc100e20 0x7fc9cc198420 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.802+0000 7fc9d1117640 1 -- 172.21.15.105:0/3614901120 >> 172.21.15.105:0/3614901120 conn(0x7fc9cc0fc7d0 msgr2=0x7fc9cc069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:10.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.802+0000 7fc9d1117640 1 -- 172.21.15.105:0/3614901120 shutdown_connections 2024-09-16T02:04:10.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:10.802+0000 7fc9d1117640 1 -- 172.21.15.105:0/3614901120 wait complete. 2024-09-16T02:04:10.837 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:10 smithi105 ceph-mon[31197]: pgmap v388: 97 pgs: 97 active+clean; 1.0 GiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 59 MiB/s wr, 26 op/s 2024-09-16T02:04:10.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:10 smithi190 ceph-mon[34762]: pgmap v388: 97 pgs: 97 active+clean; 1.0 GiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 59 MiB/s wr, 26 op/s 2024-09-16T02:04:10.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:10 smithi137 ceph-mon[34099]: pgmap v388: 97 pgs: 97 active+clean; 1.0 GiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 59 MiB/s wr, 26 op/s 2024-09-16T02:04:11.739 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T02:04:11.739 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:04:11.895 INFO:tasks.workunit.client.0.smithi105.stdout: 2024-09-16T02:04:11.895 INFO:tasks.workunit.client.0.smithi105.stdout: Children see throughput for 1 rewriters = 130897.02 kB/sec 2024-09-16T02:04:11.895 INFO:tasks.workunit.client.0.smithi105.stdout: Parent sees throughput for 1 rewriters = 130893.13 kB/sec 2024-09-16T02:04:11.895 INFO:tasks.workunit.client.0.smithi105.stdout: Min throughput per process = 130897.02 kB/sec 2024-09-16T02:04:11.895 INFO:tasks.workunit.client.0.smithi105.stdout: Max throughput per process = 130897.02 kB/sec 2024-09-16T02:04:11.895 INFO:tasks.workunit.client.0.smithi105.stdout: Avg throughput per process = 130897.02 kB/sec 2024-09-16T02:04:11.895 INFO:tasks.workunit.client.0.smithi105.stdout: Min xfer = 1048576.00 kB 2024-09-16T02:04:12.010 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:12.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:12 smithi190 ceph-mon[34762]: pgmap v389: 97 pgs: 97 active+clean; 1.0 GiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 83 MiB/s wr, 31 op/s 2024-09-16T02:04:12.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:12 smithi137 ceph-mon[34099]: pgmap v389: 97 pgs: 97 active+clean; 1.0 GiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 83 MiB/s wr, 31 op/s 2024-09-16T02:04:13.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:12 smithi105 ceph-mon[31197]: pgmap v389: 97 pgs: 97 active+clean; 1.0 GiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 83 MiB/s wr, 31 op/s 2024-09-16T02:04:14.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.338+0000 7f00aacd9640 1 -- 172.21.15.105:0/2593391474 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00a4102ef0 msgr2=0x7f00a410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:14.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.338+0000 7f00aacd9640 1 --2- 172.21.15.105:0/2593391474 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00a4102ef0 0x7f00a410da40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f009c009ca0 tx=0x7f009c02a950 comp rx=0 tx=0).stop 2024-09-16T02:04:14.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.338+0000 7f00aacd9640 1 -- 172.21.15.105:0/2593391474 shutdown_connections 2024-09-16T02:04:14.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.338+0000 7f00aacd9640 1 --2- 172.21.15.105:0/2593391474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00a4102570 0x7f00a41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:14.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.338+0000 7f00aacd9640 1 --2- 172.21.15.105:0/2593391474 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00a4102ef0 0x7f00a410da40 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:14.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.338+0000 7f00aacd9640 1 --2- 172.21.15.105:0/2593391474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00a4100e20 0x7f00a4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:14.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.338+0000 7f00aacd9640 1 -- 172.21.15.105:0/2593391474 >> 172.21.15.105:0/2593391474 conn(0x7f00a40fc7d0 msgr2=0x7f00a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:14.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.338+0000 7f00aacd9640 1 -- 172.21.15.105:0/2593391474 shutdown_connections 2024-09-16T02:04:14.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.338+0000 7f00aacd9640 1 -- 172.21.15.105:0/2593391474 wait complete. 2024-09-16T02:04:14.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.338+0000 7f00aacd9640 1 Processor -- start 2024-09-16T02:04:14.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.339+0000 7f00aacd9640 1 -- start start 2024-09-16T02:04:14.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.339+0000 7f00aacd9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00a4100e20 0x7f00a4198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:14.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.339+0000 7f00aacd9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00a4102570 0x7f00a4198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:14.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.339+0000 7f00aacd9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00a4102ef0 0x7f00a419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:14.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.339+0000 7f00aacd9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f00a4110e50 con 0x7f00a4100e20 2024-09-16T02:04:14.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.339+0000 7f00aacd9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f00a4110cd0 con 0x7f00a4102ef0 2024-09-16T02:04:14.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.339+0000 7f00aacd9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f00a4110fd0 con 0x7f00a4102570 2024-09-16T02:04:14.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.339+0000 7f00a8a75640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00a4100e20 0x7f00a4198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:14.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.339+0000 7f00a8a75640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00a4100e20 0x7f00a4198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54412/0 (socket says 172.21.15.105:54412) 2024-09-16T02:04:14.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.339+0000 7f00a8a75640 1 -- 172.21.15.105:0/1575327262 learned_addr learned my addr 172.21.15.105:0/1575327262 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:14.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.339+0000 7f00a9276640 1 --2- 172.21.15.105:0/1575327262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00a4102ef0 0x7f00a419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:14.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.339+0000 7f00a8a75640 1 -- 172.21.15.105:0/1575327262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00a4102570 msgr2=0x7f00a4198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:14.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.339+0000 7f00a3fff640 1 --2- 172.21.15.105:0/1575327262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00a4102570 0x7f00a4198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:14.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.339+0000 7f00a8a75640 1 --2- 172.21.15.105:0/1575327262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00a4102570 0x7f00a4198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:14.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.339+0000 7f00a8a75640 1 -- 172.21.15.105:0/1575327262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00a4102ef0 msgr2=0x7f00a419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:14.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.339+0000 7f00a8a75640 1 --2- 172.21.15.105:0/1575327262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00a4102ef0 0x7f00a419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:14.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.339+0000 7f00a8a75640 1 -- 172.21.15.105:0/1575327262 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00a419d3e0 con 0x7f00a4100e20 2024-09-16T02:04:14.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.339+0000 7f00a9276640 1 --2- 172.21.15.105:0/1575327262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00a4102ef0 0x7f00a419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:04:14.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.339+0000 7f00a8a75640 1 --2- 172.21.15.105:0/1575327262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00a4100e20 0x7f00a4198430 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f009800c920 tx=0x7f009800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:14.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.339+0000 7f00a1ffb640 1 -- 172.21.15.105:0/1575327262 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0098007c70 con 0x7f00a4100e20 2024-09-16T02:04:14.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.339+0000 7f00aacd9640 1 -- 172.21.15.105:0/1575327262 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00a419d6d0 con 0x7f00a4100e20 2024-09-16T02:04:14.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.340+0000 7f00a1ffb640 1 -- 172.21.15.105:0/1575327262 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0098007e10 con 0x7f00a4100e20 2024-09-16T02:04:14.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.340+0000 7f00a1ffb640 1 -- 172.21.15.105:0/1575327262 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0098018820 con 0x7f00a4100e20 2024-09-16T02:04:14.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.340+0000 7f00aacd9640 1 -- 172.21.15.105:0/1575327262 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00a41a4fb0 con 0x7f00a4100e20 2024-09-16T02:04:14.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.341+0000 7f00aacd9640 1 -- 172.21.15.105:0/1575327262 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00a4101200 con 0x7f00a4100e20 2024-09-16T02:04:14.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.342+0000 7f00a1ffb640 1 -- 172.21.15.105:0/1575327262 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f00980189c0 con 0x7f00a4100e20 2024-09-16T02:04:14.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.342+0000 7f00a1ffb640 1 --2- 172.21.15.105:0/1575327262 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f00940a3240 0x7f00940a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:14.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.343+0000 7f00a1ffb640 1 -- 172.21.15.105:0/1575327262 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f00980cfe10 con 0x7f00a4100e20 2024-09-16T02:04:14.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.343+0000 7f00a3fff640 1 --2- 172.21.15.105:0/1575327262 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f00940a3240 0x7f00940a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:14.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.343+0000 7f00a3fff640 1 --2- 172.21.15.105:0/1575327262 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f00940a3240 0x7f00940a5700 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f00a40fef90 tx=0x7f0090008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:14.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.346+0000 7f00a1ffb640 1 -- 172.21.15.105:0/1575327262 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0098014030 con 0x7f00a4100e20 2024-09-16T02:04:14.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.549+0000 7f00aacd9640 1 -- 172.21.15.105:0/1575327262 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00a410a0d0 con 0x7f00a4100e20 2024-09-16T02:04:14.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.550+0000 7f00a1ffb640 1 -- 172.21.15.105:0/1575327262 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11991 (secure 0 0 0) 0x7f009809cb60 con 0x7f00a4100e20 2024-09-16T02:04:14.552 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:04:14.552 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":28,"btime":"2024-09-16T02:02:41:554178+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2057699920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2057699920},{"type":"v1","addr":"172.21.15.105:6835","nonce":2057699920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1764886948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1764886948},{"type":"v1","addr":"172.21.15.105:6837","nonce":1764886948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T02:00:35.112266+0000","modified":"2024-09-16T02:02:40.885614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24469,"mds_2":24461,"mds_3":24457,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/4112062361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":4112062361},{"type":"v1","addr":"172.21.15.105:6839","nonce":4112062361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6841/163156514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":163156514},{"type":"v1","addr":"172.21.15.105:6841","nonce":163156514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/683519097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":683519097},{"type":"v1","addr":"172.21.15.190:6833","nonce":683519097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3025442862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3025442862},{"type":"v1","addr":"172.21.15.137:6833","nonce":3025442862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/763783602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":763783602},{"type":"v1","addr":"172.21.15.137:6835","nonce":763783602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.190:6835/1851042281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1851042281},{"type":"v1","addr":"172.21.15.190:6835","nonce":1851042281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/3181257549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3181257549},{"type":"v1","addr":"172.21.15.190:6837","nonce":3181257549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6837/2788541030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2788541030},{"type":"v1","addr":"172.21.15.137:6837","nonce":2788541030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.190:6839/1222573537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":1222573537},{"type":"v1","addr":"172.21.15.190:6839","nonce":1222573537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/3454924009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3454924009},{"type":"v1","addr":"172.21.15.137:6839","nonce":3454924009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24445,24457,24461,24469,24482]},"id":1}]} 2024-09-16T02:04:14.552 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 28 2024-09-16T02:04:14.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.552+0000 7f00aacd9640 1 -- 172.21.15.105:0/1575327262 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f00940a3240 msgr2=0x7f00940a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:14.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.552+0000 7f00aacd9640 1 --2- 172.21.15.105:0/1575327262 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f00940a3240 0x7f00940a5700 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f00a40fef90 tx=0x7f0090008040 comp rx=0 tx=0).stop 2024-09-16T02:04:14.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.552+0000 7f00aacd9640 1 -- 172.21.15.105:0/1575327262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00a4100e20 msgr2=0x7f00a4198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:14.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.552+0000 7f00aacd9640 1 --2- 172.21.15.105:0/1575327262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00a4100e20 0x7f00a4198430 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f009800c920 tx=0x7f009800cde0 comp rx=0 tx=0).stop 2024-09-16T02:04:14.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.552+0000 7f00aacd9640 1 -- 172.21.15.105:0/1575327262 shutdown_connections 2024-09-16T02:04:14.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.552+0000 7f00aacd9640 1 --2- 172.21.15.105:0/1575327262 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f00940a3240 0x7f00940a5700 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:14.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.552+0000 7f00aacd9640 1 --2- 172.21.15.105:0/1575327262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00a4102570 0x7f00a4198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:14.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.552+0000 7f00aacd9640 1 --2- 172.21.15.105:0/1575327262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00a4102ef0 0x7f00a419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:14.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.552+0000 7f00aacd9640 1 --2- 172.21.15.105:0/1575327262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00a4100e20 0x7f00a4198430 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:14.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.552+0000 7f00aacd9640 1 -- 172.21.15.105:0/1575327262 >> 172.21.15.105:0/1575327262 conn(0x7f00a40fc7d0 msgr2=0x7f00a406a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:14.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.552+0000 7f00aacd9640 1 -- 172.21.15.105:0/1575327262 shutdown_connections 2024-09-16T02:04:14.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:14.552+0000 7f00aacd9640 1 -- 172.21.15.105:0/1575327262 wait complete. 2024-09-16T02:04:14.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:14 smithi190 ceph-mon[34762]: pgmap v390: 97 pgs: 97 active+clean; 1.0 GiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 70 MiB/s wr, 28 op/s 2024-09-16T02:04:14.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:14 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/1575327262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:04:14.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:14 smithi137 ceph-mon[34099]: pgmap v390: 97 pgs: 97 active+clean; 1.0 GiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 70 MiB/s wr, 28 op/s 2024-09-16T02:04:14.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:14 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/1575327262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:04:14.952 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:14 smithi105 ceph-mon[31197]: pgmap v390: 97 pgs: 97 active+clean; 1.0 GiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 70 MiB/s wr, 28 op/s 2024-09-16T02:04:14.952 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:14 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/1575327262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:04:15.301 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-16T02:04:15.548 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:16.091 INFO:tasks.workunit.client.0.smithi105.stdout: 2024-09-16T02:04:16.092 INFO:tasks.workunit.client.0.smithi105.stdout: Children see throughput for 1 readers = 7414619.00 kB/sec 2024-09-16T02:04:16.092 INFO:tasks.workunit.client.0.smithi105.stdout: Parent sees throughput for 1 readers = 7404619.76 kB/sec 2024-09-16T02:04:16.092 INFO:tasks.workunit.client.0.smithi105.stdout: Min throughput per process = 7414619.00 kB/sec 2024-09-16T02:04:16.092 INFO:tasks.workunit.client.0.smithi105.stdout: Max throughput per process = 7414619.00 kB/sec 2024-09-16T02:04:16.092 INFO:tasks.workunit.client.0.smithi105.stdout: Avg throughput per process = 7414619.00 kB/sec 2024-09-16T02:04:16.092 INFO:tasks.workunit.client.0.smithi105.stdout: Min xfer = 1048576.00 kB 2024-09-16T02:04:16.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.072+0000 7f1ad24df640 1 -- 172.21.15.105:0/3587806473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1acc108290 msgr2=0x7f1acc10ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:16.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.072+0000 7f1ad24df640 1 --2- 172.21.15.105:0/3587806473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1acc108290 0x7f1acc10ec30 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4009cd0 tx=0x7f1ac402a910 comp rx=0 tx=0).stop 2024-09-16T02:04:16.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.072+0000 7f1ad24df640 1 -- 172.21.15.105:0/3587806473 shutdown_connections 2024-09-16T02:04:16.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.072+0000 7f1ad24df640 1 --2- 172.21.15.105:0/3587806473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1acc108290 0x7f1acc10ec30 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:16.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.072+0000 7f1ad24df640 1 --2- 172.21.15.105:0/3587806473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1acc103740 0x7f1acc107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:16.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.072+0000 7f1ad24df640 1 --2- 172.21.15.105:0/3587806473 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1acc102d90 0x7f1acc103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:16.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.072+0000 7f1ad24df640 1 -- 172.21.15.105:0/3587806473 >> 172.21.15.105:0/3587806473 conn(0x7f1acc0fc7d0 msgr2=0x7f1acc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:16.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.072+0000 7f1ad24df640 1 -- 172.21.15.105:0/3587806473 shutdown_connections 2024-09-16T02:04:16.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.072+0000 7f1ad24df640 1 -- 172.21.15.105:0/3587806473 wait complete. 2024-09-16T02:04:16.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.073+0000 7f1ad24df640 1 Processor -- start 2024-09-16T02:04:16.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.073+0000 7f1ad24df640 1 -- start start 2024-09-16T02:04:16.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.073+0000 7f1ad24df640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1acc102d90 0x7f1acc198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:16.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.073+0000 7f1ad24df640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1acc103740 0x7f1acc198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:16.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.073+0000 7f1ad24df640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1acc108290 0x7f1acc19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:16.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.073+0000 7f1ad24df640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc111fa0 con 0x7f1acc102d90 2024-09-16T02:04:16.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.073+0000 7f1ad24df640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc111e20 con 0x7f1acc108290 2024-09-16T02:04:16.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.073+0000 7f1ad24df640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc112120 con 0x7f1acc103740 2024-09-16T02:04:16.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.073+0000 7f1acbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1acc102d90 0x7f1acc198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:16.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.074+0000 7f1acbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1acc102d90 0x7f1acc198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54426/0 (socket says 172.21.15.105:54426) 2024-09-16T02:04:16.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.074+0000 7f1acbfff640 1 -- 172.21.15.105:0/1439911302 learned_addr learned my addr 172.21.15.105:0/1439911302 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:16.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.074+0000 7f1ac3fff640 1 --2- 172.21.15.105:0/1439911302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1acc103740 0x7f1acc198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:16.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.074+0000 7f1ad0a7c640 1 --2- 172.21.15.105:0/1439911302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1acc108290 0x7f1acc19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:16.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.074+0000 7f1acbfff640 1 -- 172.21.15.105:0/1439911302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1acc103740 msgr2=0x7f1acc198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:16.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.074+0000 7f1acbfff640 1 --2- 172.21.15.105:0/1439911302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1acc103740 0x7f1acc198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:16.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.074+0000 7f1acbfff640 1 -- 172.21.15.105:0/1439911302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1acc108290 msgr2=0x7f1acc19cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:16.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.074+0000 7f1acbfff640 1 --2- 172.21.15.105:0/1439911302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1acc108290 0x7f1acc19cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:16.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.074+0000 7f1acbfff640 1 -- 172.21.15.105:0/1439911302 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1acc19d4b0 con 0x7f1acc102d90 2024-09-16T02:04:16.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.074+0000 7f1ac3fff640 1 --2- 172.21.15.105:0/1439911302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1acc103740 0x7f1acc198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:04:16.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.074+0000 7f1ad0a7c640 1 --2- 172.21.15.105:0/1439911302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1acc108290 0x7f1acc19cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:04:16.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.074+0000 7f1acbfff640 1 --2- 172.21.15.105:0/1439911302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1acc102d90 0x7f1acc198500 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1abc00c970 tx=0x7f1abc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:16.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.074+0000 7f1ac9ffb640 1 -- 172.21.15.105:0/1439911302 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1abc007bb0 con 0x7f1acc102d90 2024-09-16T02:04:16.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.074+0000 7f1ad24df640 1 -- 172.21.15.105:0/1439911302 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1acc19d7a0 con 0x7f1acc102d90 2024-09-16T02:04:16.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.074+0000 7f1ac9ffb640 1 -- 172.21.15.105:0/1439911302 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1abc007d50 con 0x7f1acc102d90 2024-09-16T02:04:16.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.074+0000 7f1ac9ffb640 1 -- 172.21.15.105:0/1439911302 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1abc00f450 con 0x7f1acc102d90 2024-09-16T02:04:16.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.074+0000 7f1ad24df640 1 -- 172.21.15.105:0/1439911302 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1acc1a5080 con 0x7f1acc102d90 2024-09-16T02:04:16.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.077+0000 7f1ad48b8740 1 Processor -- start 2024-09-16T02:04:16.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.078+0000 7f1ad48b8740 1 -- start start 2024-09-16T02:04:16.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.078+0000 7f1ac9ffb640 1 -- 172.21.15.105:0/1439911302 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f1abc004f20 con 0x7f1acc102d90 2024-09-16T02:04:16.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.078+0000 7f1ad48b8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9c8b06b40 0x55b9c8b05090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:16.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.078+0000 7f1ad48b8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9c8b055d0 0x55b9c8b05a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:16.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.078+0000 7f1ad48b8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c8b14780 0x55b9c8b16b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:16.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.078+0000 7f1ad48b8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b9c8a6b2f0 con 0x55b9c8b06b40 2024-09-16T02:04:16.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.078+0000 7f1ad48b8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b9c8a12cf0 con 0x55b9c8b055d0 2024-09-16T02:04:16.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.078+0000 7f1ad48b8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b9c8a672b0 con 0x55b9c8b14780 2024-09-16T02:04:16.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.078+0000 7f1ad0a7c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9c8b06b40 0x55b9c8b05090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:16.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.078+0000 7f1ad0a7c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9c8b06b40 0x55b9c8b05090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54436/0 (socket says 172.21.15.105:54436) 2024-09-16T02:04:16.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.078+0000 7f1ad0a7c640 1 -- 172.21.15.105:0/2469575945 learned_addr learned my addr 172.21.15.105:0/2469575945 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:16.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.078+0000 7f1acbfff640 1 --2- 172.21.15.105:0/2469575945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9c8b055d0 0x55b9c8b05a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:16.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.078+0000 7f1ac3fff640 1 --2- 172.21.15.105:0/2469575945 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c8b14780 0x55b9c8b16b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:16.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.078+0000 7f1ad0a7c640 1 -- 172.21.15.105:0/2469575945 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c8b14780 msgr2=0x55b9c8b16b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:16.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.078+0000 7f1ad0a7c640 1 --2- 172.21.15.105:0/2469575945 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c8b14780 0x55b9c8b16b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:16.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.078+0000 7f1ad0a7c640 1 -- 172.21.15.105:0/2469575945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9c8b055d0 msgr2=0x55b9c8b05a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:16.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.078+0000 7f1ad0a7c640 1 --2- 172.21.15.105:0/2469575945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9c8b055d0 0x55b9c8b05a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:16.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.078+0000 7f1ad0a7c640 1 -- 172.21.15.105:0/2469575945 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9c89f73a0 con 0x55b9c8b06b40 2024-09-16T02:04:16.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.078+0000 7f1ac3fff640 1 --2- 172.21.15.105:0/2469575945 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c8b14780 0x55b9c8b16b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:04:16.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.078+0000 7f1acbfff640 1 --2- 172.21.15.105:0/2469575945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9c8b055d0 0x55b9c8b05a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:04:16.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.079+0000 7f1ad0a7c640 1 --2- 172.21.15.105:0/2469575945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9c8b06b40 0x55b9c8b05090 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1ac40099f0 tx=0x7f1ac4009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ffe6be030fad08 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:16.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.079+0000 7f1ac9ffb640 1 --2- 172.21.15.105:0/1439911302 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f1ab80a3240 0x7f1ab80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:16.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.079+0000 7f1ac27fc640 1 -- 172.21.15.105:0/2469575945 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4031e00 con 0x55b9c8b06b40 2024-09-16T02:04:16.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.079+0000 7f1ac27fc640 1 -- 172.21.15.105:0/2469575945 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ac4036070 con 0x55b9c8b06b40 2024-09-16T02:04:16.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.079+0000 7f1ac27fc640 1 -- 172.21.15.105:0/2469575945 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac40043b0 con 0x55b9c8b06b40 2024-09-16T02:04:16.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.079+0000 7f1ac9ffb640 1 -- 172.21.15.105:0/1439911302 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f1abc0cf7a0 con 0x7f1acc102d90 2024-09-16T02:04:16.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.079+0000 7f1acbfff640 1 --2- 172.21.15.105:0/1439911302 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f1ab80a3240 0x7f1ab80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:16.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.080+0000 7f1ad48b8740 1 -- 172.21.15.105:0/2469575945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9c8b06b40 msgr2=0x55b9c8b05090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:16.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.080+0000 7f1ad48b8740 1 --2- 172.21.15.105:0/2469575945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9c8b06b40 0x55b9c8b05090 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1ac40099f0 tx=0x7f1ac4009a20 comp rx=0 tx=0).stop 2024-09-16T02:04:16.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.080+0000 7f1acbfff640 1 --2- 172.21.15.105:0/1439911302 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f1ab80a3240 0x7f1ab80a5700 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1abc0cd5b0 tx=0x7f1abc004920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:16.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.080+0000 7f1ad48b8740 1 -- 172.21.15.105:0/2469575945 shutdown_connections 2024-09-16T02:04:16.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.080+0000 7f1ad48b8740 1 --2- 172.21.15.105:0/2469575945 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c8b14780 0x55b9c8b16b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:16.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.080+0000 7f1ad48b8740 1 --2- 172.21.15.105:0/2469575945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9c8b055d0 0x55b9c8b05a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:16.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.080+0000 7f1ad48b8740 1 --2- 172.21.15.105:0/2469575945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9c8b06b40 0x55b9c8b05090 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:16.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.080+0000 7f1ad48b8740 1 -- 172.21.15.105:0/2469575945 >> 172.21.15.105:0/2469575945 conn(0x55b9c8ace310 msgr2=0x55b9c8b01c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:16.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.080+0000 7f1ad48b8740 1 -- 172.21.15.105:0/2469575945 shutdown_connections 2024-09-16T02:04:16.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.080+0000 7f1ad48b8740 1 -- 172.21.15.105:0/2469575945 wait complete. 2024-09-16T02:04:16.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.080+0000 7f1ad48b8740 1 -- start start 2024-09-16T02:04:16.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.081+0000 7f1ad48b8740 1 Processor -- start 2024-09-16T02:04:16.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.081+0000 7f1ad48b8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9c8b055d0 0x55b9c8b14120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:16.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.081+0000 7f1ad48b8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c8b06b40 0x55b9c8b113d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:16.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.081+0000 7f1ad48b8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9c8b14780 0x55b9c8b11970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:16.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.081+0000 7f1ad48b8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b9c8b17420 con 0x55b9c8b055d0 2024-09-16T02:04:16.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.081+0000 7f1ad48b8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b9c8a26060 con 0x55b9c8b14780 2024-09-16T02:04:16.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.081+0000 7f1ad48b8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b9c8b175a0 con 0x55b9c8b06b40 2024-09-16T02:04:16.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.081+0000 7f1ad48b8740 10 client.0 _ll_register_callbacks cb 0x55b9c87798c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:04:16.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.081+0000 7f1ad0a7c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9c8b055d0 0x55b9c8b14120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:16.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.081+0000 7f1ad0a7c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9c8b055d0 0x55b9c8b14120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54450/0 (socket says 172.21.15.105:54450) 2024-09-16T02:04:16.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.081+0000 7f1ad0a7c640 1 -- 172.21.15.105:0/2877655345 learned_addr learned my addr 172.21.15.105:0/2877655345 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:16.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.081+0000 7f1ad0a7c640 1 --2- 172.21.15.105:0/2877655345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9c8b14780 0x55b9c8b11970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:16.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.081+0000 7f1ad0a7c640 1 -- 172.21.15.105:0/2877655345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c8b06b40 msgr2=0x55b9c8b113d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:16.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.081+0000 7f1ad0a7c640 1 --2- 172.21.15.105:0/2877655345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c8b06b40 0x55b9c8b113d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:16.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.081+0000 7f1ad0a7c640 1 -- 172.21.15.105:0/2877655345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9c8b14780 msgr2=0x55b9c8b11970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:16.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.081+0000 7f1ad0a7c640 1 --2- 172.21.15.105:0/2877655345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9c8b14780 0x55b9c8b11970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:16.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.081+0000 7f1ad0a7c640 1 -- 172.21.15.105:0/2877655345 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9c87dc3c0 con 0x55b9c8b055d0 2024-09-16T02:04:16.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.081+0000 7f1ad0a7c640 1 --2- 172.21.15.105:0/2877655345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9c8b055d0 0x55b9c8b14120 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1ac40316b0 tx=0x7f1ac4003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67d02abd080cc45d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:16.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.081+0000 7f1aab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:04:16.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.082+0000 7f1aab7fe640 1 -- 172.21.15.105:0/2877655345 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4003d80 con 0x55b9c8b055d0 2024-09-16T02:04:16.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.082+0000 7f1aab7fe640 1 -- 172.21.15.105:0/2877655345 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f1ac40040b0 con 0x55b9c8b055d0 2024-09-16T02:04:16.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.082+0000 7f1ad48b8740 1 -- 172.21.15.105:0/2877655345 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ac4036070 con 0x55b9c8b055d0 2024-09-16T02:04:16.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.082+0000 7f1aab7fe640 1 -- 172.21.15.105:0/2877655345 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1ac4031810 con 0x55b9c8b055d0 2024-09-16T02:04:16.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.082+0000 7f1ad48b8740 10 client.15528 fetch_fsmap learned FSMap version 28 2024-09-16T02:04:16.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.082+0000 7f1ad48b8740 1 -- 172.21.15.105:0/2877655345 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b9c8b22e70 con 0x55b9c8b055d0 2024-09-16T02:04:16.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.082+0000 7f1aab7fe640 1 -- 172.21.15.105:0/2877655345 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ac4036070 con 0x55b9c8b055d0 2024-09-16T02:04:16.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.082+0000 7f1aab7fe640 1 -- 172.21.15.105:0/2877655345 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1ac4030c30 con 0x55b9c8b055d0 2024-09-16T02:04:16.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.082+0000 7f1ad48b8740 10 client.15528 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:04:16.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.082+0000 7f1ad48b8740 10 client.15528 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T02:04:16.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.082+0000 7f1ad48b8740 20 client.15528 populate_metadata read hostname 'smithi105' 2024-09-16T02:04:16.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.082+0000 7f1ad48b8740 1 --2- 172.21.15.105:0/2877655345 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x55b9c8b241c0 0x55b9c8b265b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:16.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.082+0000 7f1ad48b8740 4 client.15528 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:04:16.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.082+0000 7f1ad48b8740 1 -- 172.21.15.105:0/2877655345 --> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9c8a78860 con 0x55b9c8b241c0 2024-09-16T02:04:16.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.083+0000 7f1ac3fff640 1 --2- 172.21.15.105:0/2877655345 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x55b9c8b241c0 0x55b9c8b265b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:16.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.083+0000 7f1ac3fff640 1 --2- 172.21.15.105:0/2877655345 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x55b9c8b241c0 0x55b9c8b265b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=46d0e540ddb39561 server_cookie=c036037a3367c4ea in_seq=0 out_seq=0 2024-09-16T02:04:16.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.083+0000 7f1aab7fe640 10 client.15528 ms_handle_connect on v2:172.21.15.190:6834/1851042281 2024-09-16T02:04:16.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.084+0000 7f1aab7fe640 1 -- 172.21.15.105:0/2877655345 <== mds.4 v2:172.21.15.190:6834/1851042281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b9c8a78860 con 0x55b9c8b241c0 2024-09-16T02:04:16.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.084+0000 7f1aab7fe640 10 client.15528 handle_command_reply: tid=0 2024-09-16T02:04:16.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.084+0000 7f1ad48b8740 1 client.15528 shutdown 2024-09-16T02:04:16.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.084+0000 7f1ad48b8740 1 -- 172.21.15.105:0/2877655345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9c8b055d0 msgr2=0x55b9c8b14120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:16.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.084+0000 7f1ad48b8740 1 --2- 172.21.15.105:0/2877655345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9c8b055d0 0x55b9c8b14120 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1ac40316b0 tx=0x7f1ac4003f90 comp rx=0 tx=0).stop 2024-09-16T02:04:16.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.084+0000 7f1ad48b8740 1 -- 172.21.15.105:0/2877655345 shutdown_connections 2024-09-16T02:04:16.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.084+0000 7f1ad48b8740 1 --2- 172.21.15.105:0/2877655345 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x55b9c8b241c0 0x55b9c8b265b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:16.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.084+0000 7f1ad48b8740 1 --2- 172.21.15.105:0/2877655345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c8b06b40 0x55b9c8b113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:16.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.084+0000 7f1ad48b8740 1 --2- 172.21.15.105:0/2877655345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9c8b14780 0x55b9c8b11970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:16.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.084+0000 7f1ad48b8740 1 --2- 172.21.15.105:0/2877655345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9c8b055d0 0x55b9c8b14120 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:16.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.084+0000 7f1ad48b8740 1 -- 172.21.15.105:0/2877655345 >> 172.21.15.105:0/2877655345 conn(0x55b9c8ace310 msgr2=0x55b9c8b07930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:16.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.084+0000 7f1aab7fe640 0 client.15528 ms_handle_reset on v2:172.21.15.190:6834/1851042281 2024-09-16T02:04:16.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.084+0000 7f1ad48b8740 1 -- 172.21.15.105:0/2877655345 shutdown_connections 2024-09-16T02:04:16.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.084+0000 7f1ad48b8740 1 -- 172.21.15.105:0/2877655345 wait complete. 2024-09-16T02:04:16.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.084+0000 7f1ad48b8740 20 client.15528 trim_cache size 0 max 16384 2024-09-16T02:04:16.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.097+0000 7f1ad48b8740 1 Processor -- start 2024-09-16T02:04:16.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.097+0000 7f1ad48b8740 1 -- start start 2024-09-16T02:04:16.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.098+0000 7f1ad48b8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9c8b055d0 0x55b9c8b01510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:16.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.098+0000 7f1ad48b8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9c8b06b40 0x55b9c8b01a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:16.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.098+0000 7f1ad48b8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c8b14780 0x55b9c8b2bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:16.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.098+0000 7f1ad48b8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc1ab440 con 0x55b9c8b055d0 2024-09-16T02:04:16.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.098+0000 7f1ad48b8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc1acf80 con 0x55b9c8b06b40 2024-09-16T02:04:16.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.098+0000 7f1ad48b8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc1abb10 con 0x55b9c8b14780 2024-09-16T02:04:16.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.098+0000 7f1ad0a7c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9c8b055d0 0x55b9c8b01510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:16.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.098+0000 7f1ad0a7c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9c8b055d0 0x55b9c8b01510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54454/0 (socket says 172.21.15.105:54454) 2024-09-16T02:04:16.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.098+0000 7f1ad0a7c640 1 -- 172.21.15.105:0/3942237200 learned_addr learned my addr 172.21.15.105:0/3942237200 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:16.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.098+0000 7f1ad0a7c640 1 -- 172.21.15.105:0/3942237200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c8b14780 msgr2=0x55b9c8b2bb50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T02:04:16.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.098+0000 7f1ac3fff640 1 --2- 172.21.15.105:0/3942237200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9c8b06b40 0x55b9c8b01a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:16.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.098+0000 7f1ad0a7c640 1 --2- 172.21.15.105:0/3942237200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c8b14780 0x55b9c8b2bb50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:16.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.098+0000 7f1ad0a7c640 1 -- 172.21.15.105:0/3942237200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9c8b06b40 msgr2=0x55b9c8b01a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:16.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.098+0000 7f1ad0a7c640 1 --2- 172.21.15.105:0/3942237200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9c8b06b40 0x55b9c8b01a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:16.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.098+0000 7f1ad0a7c640 1 -- 172.21.15.105:0/3942237200 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9c8b0f920 con 0x55b9c8b055d0 2024-09-16T02:04:16.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.098+0000 7f1ac3fff640 1 --2- 172.21.15.105:0/3942237200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9c8b06b40 0x55b9c8b01a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:04:16.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.098+0000 7f1ad0a7c640 1 --2- 172.21.15.105:0/3942237200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9c8b055d0 0x55b9c8b01510 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4031040 tx=0x7f1ac4032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28409127acf1d881 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:16.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.098+0000 7f1ac1ffb640 1 -- 172.21.15.105:0/3942237200 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac40314d0 con 0x55b9c8b055d0 2024-09-16T02:04:16.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.098+0000 7f1ac1ffb640 1 -- 172.21.15.105:0/3942237200 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ac4036070 con 0x55b9c8b055d0 2024-09-16T02:04:16.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.098+0000 7f1ac1ffb640 1 -- 172.21.15.105:0/3942237200 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac40308c0 con 0x55b9c8b055d0 2024-09-16T02:04:16.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.099+0000 7f1ad48b8740 1 -- 172.21.15.105:0/3942237200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9c8b055d0 msgr2=0x55b9c8b01510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:16.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.099+0000 7f1ad48b8740 1 --2- 172.21.15.105:0/3942237200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9c8b055d0 0x55b9c8b01510 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4031040 tx=0x7f1ac4032f60 comp rx=0 tx=0).stop 2024-09-16T02:04:16.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.099+0000 7f1ad48b8740 1 -- 172.21.15.105:0/3942237200 shutdown_connections 2024-09-16T02:04:16.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.099+0000 7f1ad48b8740 1 --2- 172.21.15.105:0/3942237200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c8b14780 0x55b9c8b2bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:16.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.099+0000 7f1ad48b8740 1 --2- 172.21.15.105:0/3942237200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9c8b06b40 0x55b9c8b01a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:16.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.099+0000 7f1ad48b8740 1 --2- 172.21.15.105:0/3942237200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9c8b055d0 0x55b9c8b01510 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:16.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.099+0000 7f1ad48b8740 1 -- 172.21.15.105:0/3942237200 >> 172.21.15.105:0/3942237200 conn(0x55b9c8ace310 msgr2=0x55b9c8b07930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:16.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.099+0000 7f1ad48b8740 1 -- 172.21.15.105:0/3942237200 shutdown_connections 2024-09-16T02:04:16.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.099+0000 7f1ad48b8740 1 -- 172.21.15.105:0/3942237200 wait complete. 2024-09-16T02:04:16.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.100+0000 7f1ad48b8740 1 -- start start 2024-09-16T02:04:16.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.100+0000 7f1ad48b8740 1 Processor -- start 2024-09-16T02:04:16.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.101+0000 7f1ad48b8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9c8b055d0 0x55b9c8b1c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:16.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.101+0000 7f1ad48b8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9c8b06b40 0x55b9c8b1cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:16.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.101+0000 7f1ad48b8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c8b14780 0x55b9c8b19800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:16.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.101+0000 7f1ad48b8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc1ac8b0 con 0x55b9c8b055d0 2024-09-16T02:04:16.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.101+0000 7f1ad48b8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc1ac1e0 con 0x55b9c8b06b40 2024-09-16T02:04:16.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.101+0000 7f1ad48b8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc1a9420 con 0x55b9c8b14780 2024-09-16T02:04:16.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.101+0000 7f1ad48b8740 10 client.0 _ll_register_callbacks cb 0x7f1ac4009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:04:16.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.101+0000 7f1ac3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9c8b06b40 0x55b9c8b1cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:16.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.101+0000 7f1ac3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9c8b06b40 0x55b9c8b1cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.105:57034/0 (socket says 172.21.15.105:57034) 2024-09-16T02:04:16.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.101+0000 7f1ac3fff640 1 -- 172.21.15.105:0/732648004 learned_addr learned my addr 172.21.15.105:0/732648004 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:16.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.101+0000 7f1ad0a7c640 1 --2- 172.21.15.105:0/732648004 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9c8b055d0 0x55b9c8b1c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:16.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.101+0000 7f1ad0a7c640 1 --2- 172.21.15.105:0/732648004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c8b14780 0x55b9c8b19800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:16.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.101+0000 7f1ac3fff640 1 -- 172.21.15.105:0/732648004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c8b14780 msgr2=0x55b9c8b19800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:16.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.101+0000 7f1ac3fff640 1 --2- 172.21.15.105:0/732648004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c8b14780 0x55b9c8b19800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:16.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.101+0000 7f1ac3fff640 1 -- 172.21.15.105:0/732648004 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9c8b055d0 msgr2=0x55b9c8b1c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:16.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.101+0000 7f1ac3fff640 1 --2- 172.21.15.105:0/732648004 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9c8b055d0 0x55b9c8b1c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:16.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.101+0000 7f1ac3fff640 1 -- 172.21.15.105:0/732648004 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9c8b1d400 con 0x55b9c8b06b40 2024-09-16T02:04:16.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.101+0000 7f1ad0a7c640 1 --2- 172.21.15.105:0/732648004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c8b14780 0x55b9c8b19800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:04:16.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.102+0000 7f1ac3fff640 1 --2- 172.21.15.105:0/732648004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9c8b06b40 0x55b9c8b1cd70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400b4c0 tx=0x7f1ab400bc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6e25fbdb471b64d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:16.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.102+0000 7f1aab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-16T02:04:16.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.102+0000 7f1aab7fe640 1 -- 172.21.15.105:0/732648004 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4021020 con 0x55b9c8b06b40 2024-09-16T02:04:16.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.102+0000 7f1aab7fe640 1 -- 172.21.15.105:0/732648004 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f1ab4017b20 con 0x55b9c8b06b40 2024-09-16T02:04:16.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.102+0000 7f1ad48b8740 1 -- 172.21.15.105:0/732648004 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ac4036070 con 0x55b9c8b06b40 2024-09-16T02:04:16.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.102+0000 7f1aab7fe640 1 -- 172.21.15.105:0/732648004 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1ab402c050 con 0x55b9c8b06b40 2024-09-16T02:04:16.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.102+0000 7f1ad48b8740 10 client.24941 fetch_fsmap learned FSMap version 28 2024-09-16T02:04:16.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.102+0000 7f1ad48b8740 1 -- 172.21.15.105:0/732648004 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b9c8b088a0 con 0x55b9c8b06b40 2024-09-16T02:04:16.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.102+0000 7f1aab7fe640 1 -- 172.21.15.105:0/732648004 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ac4036070 con 0x55b9c8b06b40 2024-09-16T02:04:16.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.103+0000 7f1aab7fe640 1 -- 172.21.15.105:0/732648004 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1ab4014400 con 0x55b9c8b06b40 2024-09-16T02:04:16.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.103+0000 7f1ad48b8740 10 client.24941 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:04:16.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.103+0000 7f1ad48b8740 10 client.24941 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T02:04:16.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.103+0000 7f1ad48b8740 20 client.24941 populate_metadata read hostname 'smithi105' 2024-09-16T02:04:16.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.103+0000 7f1ad48b8740 1 --2- 172.21.15.105:0/732648004 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x55b9c8b09050 0x55b9c8b0b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:16.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.103+0000 7f1ad48b8740 4 client.24941 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T02:04:16.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.103+0000 7f1ad48b8740 1 -- 172.21.15.105:0/732648004 --> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b9c8a72d70 con 0x55b9c8b09050 2024-09-16T02:04:16.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.103+0000 7f1ad0a7c640 1 --2- 172.21.15.105:0/732648004 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x55b9c8b09050 0x55b9c8b0b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:16.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.104+0000 7f1aab7fe640 10 client.24941 ms_handle_connect on v2:172.21.15.190:6834/1851042281 2024-09-16T02:04:16.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.104+0000 7f1ad0a7c640 1 --2- 172.21.15.105:0/732648004 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x55b9c8b09050 0x55b9c8b0b470 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=44b85adc33ca24ae server_cookie=a9279e09ab5545f2 in_seq=0 out_seq=0 2024-09-16T02:04:16.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.104+0000 7f1aab7fe640 1 -- 172.21.15.105:0/732648004 <== mds.4 v2:172.21.15.190:6834/1851042281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b9c8a72d70 con 0x55b9c8b09050 2024-09-16T02:04:16.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.104+0000 7f1aab7fe640 10 client.24941 handle_command_reply: tid=0 2024-09-16T02:04:16.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.104+0000 7f1ad48b8740 1 client.24941 shutdown 2024-09-16T02:04:16.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.104+0000 7f1ad48b8740 1 -- 172.21.15.105:0/732648004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9c8b06b40 msgr2=0x55b9c8b1cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:16.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.104+0000 7f1ad48b8740 1 --2- 172.21.15.105:0/732648004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9c8b06b40 0x55b9c8b1cd70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400b4c0 tx=0x7f1ab400bc10 comp rx=0 tx=0).stop 2024-09-16T02:04:16.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.104+0000 7f1ad48b8740 1 -- 172.21.15.105:0/732648004 shutdown_connections 2024-09-16T02:04:16.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.104+0000 7f1ad48b8740 1 --2- 172.21.15.105:0/732648004 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x55b9c8b09050 0x55b9c8b0b470 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:16.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.104+0000 7f1ad48b8740 1 --2- 172.21.15.105:0/732648004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c8b14780 0x55b9c8b19800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:16.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.104+0000 7f1ad48b8740 1 --2- 172.21.15.105:0/732648004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9c8b06b40 0x55b9c8b1cd70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:16.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.104+0000 7f1ad48b8740 1 --2- 172.21.15.105:0/732648004 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9c8b055d0 0x55b9c8b1c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:16.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.104+0000 7f1aab7fe640 0 client.24941 ms_handle_reset on v2:172.21.15.190:6834/1851042281 2024-09-16T02:04:16.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.104+0000 7f1ad48b8740 1 -- 172.21.15.105:0/732648004 >> 172.21.15.105:0/732648004 conn(0x55b9c8ace310 msgr2=0x55b9c8b12c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:16.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.104+0000 7f1ad48b8740 1 -- 172.21.15.105:0/732648004 shutdown_connections 2024-09-16T02:04:16.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.104+0000 7f1ad48b8740 1 -- 172.21.15.105:0/732648004 wait complete. 2024-09-16T02:04:16.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.104+0000 7f1ad48b8740 20 client.24941 trim_cache size 0 max 16384 2024-09-16T02:04:16.123 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-09-16T02:04:16.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.105+0000 7f1ad24df640 1 -- 172.21.15.105:0/1439911302 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f1ab80a3240 msgr2=0x7f1ab80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:16.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.105+0000 7f1ad24df640 1 --2- 172.21.15.105:0/1439911302 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f1ab80a3240 0x7f1ab80a5700 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1abc0cd5b0 tx=0x7f1abc004920 comp rx=0 tx=0).stop 2024-09-16T02:04:16.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.105+0000 7f1ad24df640 1 -- 172.21.15.105:0/1439911302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1acc102d90 msgr2=0x7f1acc198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:16.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.105+0000 7f1ad24df640 1 --2- 172.21.15.105:0/1439911302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1acc102d90 0x7f1acc198500 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1abc00c970 tx=0x7f1abc00ce30 comp rx=0 tx=0).stop 2024-09-16T02:04:16.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.105+0000 7f1ad24df640 1 -- 172.21.15.105:0/1439911302 shutdown_connections 2024-09-16T02:04:16.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.105+0000 7f1ad24df640 1 --2- 172.21.15.105:0/1439911302 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f1ab80a3240 0x7f1ab80a5700 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:16.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.105+0000 7f1ad24df640 1 --2- 172.21.15.105:0/1439911302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1acc103740 0x7f1acc198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:16.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.105+0000 7f1ad24df640 1 --2- 172.21.15.105:0/1439911302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1acc108290 0x7f1acc19cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:16.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.105+0000 7f1ad24df640 1 --2- 172.21.15.105:0/1439911302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1acc102d90 0x7f1acc198500 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:16.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.105+0000 7f1ad24df640 1 -- 172.21.15.105:0/1439911302 >> 172.21.15.105:0/1439911302 conn(0x7f1acc0fc7d0 msgr2=0x7f1acc105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:16.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.105+0000 7f1ad24df640 1 -- 172.21.15.105:0/1439911302 shutdown_connections 2024-09-16T02:04:16.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:16.105+0000 7f1ad24df640 1 -- 172.21.15.105:0/1439911302 wait complete. 2024-09-16T02:04:16.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:16 smithi190 ceph-mon[34762]: pgmap v391: 97 pgs: 97 active+clean; 1.0 GiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 81 MiB/s wr, 30 op/s 2024-09-16T02:04:16.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:16 smithi137 ceph-mon[34099]: pgmap v391: 97 pgs: 97 active+clean; 1.0 GiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 81 MiB/s wr, 30 op/s 2024-09-16T02:04:17.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:16 smithi105 ceph-mon[31197]: pgmap v391: 97 pgs: 97 active+clean; 1.0 GiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 81 MiB/s wr, 30 op/s 2024-09-16T02:04:17.123 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-16T02:04:17.370 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:17.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.849+0000 7f86a9468640 1 -- 172.21.15.105:0/1360895858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86a4102570 msgr2=0x7f86a41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:17.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.849+0000 7f86a9468640 1 --2- 172.21.15.105:0/1360895858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86a4102570 0x7f86a41029b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f868c009900 tx=0x7f868c02a480 comp rx=0 tx=0).stop 2024-09-16T02:04:17.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.849+0000 7f86a9468640 1 -- 172.21.15.105:0/1360895858 shutdown_connections 2024-09-16T02:04:17.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.849+0000 7f86a9468640 1 --2- 172.21.15.105:0/1360895858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86a4102ef0 0x7f86a410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.849+0000 7f86a9468640 1 --2- 172.21.15.105:0/1360895858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86a4102570 0x7f86a41029b0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.849+0000 7f86a9468640 1 --2- 172.21.15.105:0/1360895858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86a4100e20 0x7f86a4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.849+0000 7f86a9468640 1 -- 172.21.15.105:0/1360895858 >> 172.21.15.105:0/1360895858 conn(0x7f86a40fc7d0 msgr2=0x7f86a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:17.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.849+0000 7f86a9468640 1 -- 172.21.15.105:0/1360895858 shutdown_connections 2024-09-16T02:04:17.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.849+0000 7f86a9468640 1 -- 172.21.15.105:0/1360895858 wait complete. 2024-09-16T02:04:17.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.850+0000 7f86a9468640 1 Processor -- start 2024-09-16T02:04:17.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.850+0000 7f86a9468640 1 -- start start 2024-09-16T02:04:17.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.850+0000 7f86a9468640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86a4100e20 0x7f86a40fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:17.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.850+0000 7f86a9468640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86a4102570 0x7f86a40fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:17.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.850+0000 7f86a9468640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86a4102ef0 0x7f86a40fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:17.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.850+0000 7f86a9468640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f86a4110e30 con 0x7f86a4102ef0 2024-09-16T02:04:17.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.850+0000 7f86a9468640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f86a4110cb0 con 0x7f86a4102570 2024-09-16T02:04:17.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.850+0000 7f86a9468640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f86a4110fb0 con 0x7f86a4100e20 2024-09-16T02:04:17.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.850+0000 7f86a37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86a4102ef0 0x7f86a40fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:17.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.850+0000 7f86a37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86a4102ef0 0x7f86a40fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54482/0 (socket says 172.21.15.105:54482) 2024-09-16T02:04:17.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.850+0000 7f86a37fe640 1 -- 172.21.15.105:0/1602773668 learned_addr learned my addr 172.21.15.105:0/1602773668 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:17.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.851+0000 7f86a2ffd640 1 --2- 172.21.15.105:0/1602773668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86a4100e20 0x7f86a40fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:17.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.851+0000 7f86a37fe640 1 -- 172.21.15.105:0/1602773668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86a4100e20 msgr2=0x7f86a40fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:17.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.851+0000 7f869a7fc640 1 --2- 172.21.15.105:0/1602773668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86a4102570 0x7f86a40fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:17.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.851+0000 7f86a37fe640 1 --2- 172.21.15.105:0/1602773668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86a4100e20 0x7f86a40fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.851+0000 7f86a37fe640 1 -- 172.21.15.105:0/1602773668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86a4102570 msgr2=0x7f86a40fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:17.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.851+0000 7f86a37fe640 1 --2- 172.21.15.105:0/1602773668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86a4102570 0x7f86a40fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.851+0000 7f86a37fe640 1 -- 172.21.15.105:0/1602773668 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86a41a68c0 con 0x7f86a4102ef0 2024-09-16T02:04:17.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.851+0000 7f86a2ffd640 1 --2- 172.21.15.105:0/1602773668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86a4100e20 0x7f86a40fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:04:17.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.851+0000 7f86a37fe640 1 --2- 172.21.15.105:0/1602773668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86a4102ef0 0x7f86a40fd700 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f869c00cce0 tx=0x7f869c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:17.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.851+0000 7f86a0ff9640 1 -- 172.21.15.105:0/1602773668 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c00f450 con 0x7f86a4102ef0 2024-09-16T02:04:17.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.851+0000 7f86a0ff9640 1 -- 172.21.15.105:0/1602773668 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f869c00faa0 con 0x7f86a4102ef0 2024-09-16T02:04:17.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.851+0000 7f86a9468640 1 -- 172.21.15.105:0/1602773668 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86a41a6bb0 con 0x7f86a4102ef0 2024-09-16T02:04:17.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.851+0000 7f86a0ff9640 1 -- 172.21.15.105:0/1602773668 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c018b70 con 0x7f86a4102ef0 2024-09-16T02:04:17.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.851+0000 7f86a9468640 1 -- 172.21.15.105:0/1602773668 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86a41a70a0 con 0x7f86a4102ef0 2024-09-16T02:04:17.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.853+0000 7f86a0ff9640 1 -- 172.21.15.105:0/1602773668 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f869c018d10 con 0x7f86a4102ef0 2024-09-16T02:04:17.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.854+0000 7f86ab841740 1 Processor -- start 2024-09-16T02:04:17.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.854+0000 7f86ab841740 1 -- start start 2024-09-16T02:04:17.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.854+0000 7f86ab841740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef9ebdab40 0x55ef9ebd9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:17.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.854+0000 7f86ab841740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9ebd95d0 0x55ef9ebd9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:17.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.854+0000 7f86ab841740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef9ebe4370 0x55ef9ebe6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:17.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.854+0000 7f86ab841740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ef9eb3f2f0 con 0x55ef9ebd95d0 2024-09-16T02:04:17.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.854+0000 7f86ab841740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ef9eae6cf0 con 0x55ef9ebe4370 2024-09-16T02:04:17.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.854+0000 7f86ab841740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ef9eb3b2b0 con 0x55ef9ebdab40 2024-09-16T02:04:17.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.854+0000 7f86a2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9ebd95d0 0x55ef9ebd9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:17.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.854+0000 7f86a2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9ebd95d0 0x55ef9ebd9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54490/0 (socket says 172.21.15.105:54490) 2024-09-16T02:04:17.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.854+0000 7f86a2ffd640 1 -- 172.21.15.105:0/1958051597 learned_addr learned my addr 172.21.15.105:0/1958051597 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:17.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.854+0000 7f86a0ff9640 1 --2- 172.21.15.105:0/1602773668 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f86900a3240 0x7f86900a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:17.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.854+0000 7f869a7fc640 1 --2- 172.21.15.105:0/1958051597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef9ebe4370 0x55ef9ebe6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:17.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.854+0000 7f869a7fc640 1 --2- 172.21.15.105:0/1958051597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef9ebdab40 0x55ef9ebd9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:17.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.854+0000 7f86a2ffd640 1 -- 172.21.15.105:0/1958051597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef9ebdab40 msgr2=0x55ef9ebd9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:17.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.854+0000 7f86a0ff9640 1 -- 172.21.15.105:0/1602773668 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f869c0cfdf0 con 0x7f86a4102ef0 2024-09-16T02:04:17.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.854+0000 7f86a2ffd640 1 --2- 172.21.15.105:0/1958051597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef9ebdab40 0x55ef9ebd9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.854+0000 7f86a2ffd640 1 -- 172.21.15.105:0/1958051597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef9ebe4370 msgr2=0x55ef9ebe6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:17.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.854+0000 7f86a37fe640 1 --2- 172.21.15.105:0/1602773668 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f86900a3240 0x7f86900a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:17.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.854+0000 7f86a2ffd640 1 --2- 172.21.15.105:0/1958051597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef9ebe4370 0x55ef9ebe6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.855+0000 7f86a2ffd640 1 -- 172.21.15.105:0/1958051597 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef9eacb3a0 con 0x55ef9ebd95d0 2024-09-16T02:04:17.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.855+0000 7f869a7fc640 1 --2- 172.21.15.105:0/1958051597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef9ebdab40 0x55ef9ebd9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:04:17.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.855+0000 7f86a2ffd640 1 --2- 172.21.15.105:0/1958051597 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9ebd95d0 0x55ef9ebd9a10 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f869400cca0 tx=0x7f869400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1208d175b9f98c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:17.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.855+0000 7f86a37fe640 1 --2- 172.21.15.105:0/1602773668 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f86900a3240 0x7f86900a5700 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f869c00ab20 tx=0x7f869c00af30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:17.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.855+0000 7f86997fa640 1 -- 172.21.15.105:0/1958051597 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8694005220 con 0x55ef9ebd95d0 2024-09-16T02:04:17.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.855+0000 7f86997fa640 1 -- 172.21.15.105:0/1958051597 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8694004c50 con 0x55ef9ebd95d0 2024-09-16T02:04:17.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.855+0000 7f86997fa640 1 -- 172.21.15.105:0/1958051597 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8694010850 con 0x55ef9ebd95d0 2024-09-16T02:04:17.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.856+0000 7f86ab841740 1 -- 172.21.15.105:0/1958051597 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9ebd95d0 msgr2=0x55ef9ebd9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:17.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.856+0000 7f86ab841740 1 --2- 172.21.15.105:0/1958051597 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9ebd95d0 0x55ef9ebd9a10 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f869400cca0 tx=0x7f869400e720 comp rx=0 tx=0).stop 2024-09-16T02:04:17.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.856+0000 7f86ab841740 1 -- 172.21.15.105:0/1958051597 shutdown_connections 2024-09-16T02:04:17.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.856+0000 7f86ab841740 1 --2- 172.21.15.105:0/1958051597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef9ebe4370 0x55ef9ebe6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.856+0000 7f86ab841740 1 --2- 172.21.15.105:0/1958051597 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9ebd95d0 0x55ef9ebd9a10 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.856+0000 7f86ab841740 1 --2- 172.21.15.105:0/1958051597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef9ebdab40 0x55ef9ebd9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.856+0000 7f86ab841740 1 -- 172.21.15.105:0/1958051597 >> 172.21.15.105:0/1958051597 conn(0x55ef9eba2310 msgr2=0x55ef9ebd5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:17.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.856+0000 7f86ab841740 1 -- 172.21.15.105:0/1958051597 shutdown_connections 2024-09-16T02:04:17.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.856+0000 7f86ab841740 1 -- 172.21.15.105:0/1958051597 wait complete. 2024-09-16T02:04:17.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.856+0000 7f86ab841740 1 -- start start 2024-09-16T02:04:17.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.856+0000 7f86ab841740 1 Processor -- start 2024-09-16T02:04:17.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.856+0000 7f86ab841740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef9ebd95d0 0x55ef9ebe3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:17.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.856+0000 7f86ab841740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9ebdab40 0x55ef9ebe1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:17.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.856+0000 7f86ab841740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef9ebe4370 0x55ef9ebe1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:17.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.856+0000 7f86ab841740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ef9ebe7070 con 0x55ef9ebdab40 2024-09-16T02:04:17.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.856+0000 7f86ab841740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ef9eafa060 con 0x55ef9ebe4370 2024-09-16T02:04:17.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.856+0000 7f86ab841740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ef9ebe71f0 con 0x55ef9ebd95d0 2024-09-16T02:04:17.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.857+0000 7f86ab841740 10 client.0 _ll_register_callbacks cb 0x55ef9e84d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:04:17.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.857+0000 7f86a2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9ebdab40 0x55ef9ebe1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:17.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.857+0000 7f86a2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9ebdab40 0x55ef9ebe1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54492/0 (socket says 172.21.15.105:54492) 2024-09-16T02:04:17.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.857+0000 7f86a2ffd640 1 -- 172.21.15.105:0/2774661464 learned_addr learned my addr 172.21.15.105:0/2774661464 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:17.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.857+0000 7f869a7fc640 1 --2- 172.21.15.105:0/2774661464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef9ebd95d0 0x55ef9ebe3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:17.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.857+0000 7f869a7fc640 1 --2- 172.21.15.105:0/2774661464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef9ebe4370 0x55ef9ebe1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:17.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.857+0000 7f86a2ffd640 1 -- 172.21.15.105:0/2774661464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef9ebd95d0 msgr2=0x55ef9ebe3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:17.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.857+0000 7f86a2ffd640 1 --2- 172.21.15.105:0/2774661464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef9ebd95d0 0x55ef9ebe3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.857+0000 7f86a2ffd640 1 -- 172.21.15.105:0/2774661464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef9ebe4370 msgr2=0x55ef9ebe1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:17.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.857+0000 7f86a2ffd640 1 --2- 172.21.15.105:0/2774661464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef9ebe4370 0x55ef9ebe1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.857+0000 7f86a2ffd640 1 -- 172.21.15.105:0/2774661464 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef9e8b03c0 con 0x55ef9ebdab40 2024-09-16T02:04:17.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.857+0000 7f86a2ffd640 1 --2- 172.21.15.105:0/2774661464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9ebdab40 0x55ef9ebe1020 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f869400c9e0 tx=0x7f8694005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c912f92b5500662b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:17.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.857+0000 7f868a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:04:17.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.857+0000 7f868a7fc640 1 -- 172.21.15.105:0/2774661464 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869402a060 con 0x55ef9ebdab40 2024-09-16T02:04:17.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.857+0000 7f868a7fc640 1 -- 172.21.15.105:0/2774661464 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f8694005010 con 0x55ef9ebdab40 2024-09-16T02:04:17.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.857+0000 7f86ab841740 1 -- 172.21.15.105:0/2774661464 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8694004c50 con 0x55ef9ebdab40 2024-09-16T02:04:17.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.857+0000 7f868a7fc640 1 -- 172.21.15.105:0/2774661464 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8694005210 con 0x55ef9ebdab40 2024-09-16T02:04:17.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.857+0000 7f86ab841740 10 client.15546 fetch_fsmap learned FSMap version 28 2024-09-16T02:04:17.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.857+0000 7f86ab841740 1 -- 172.21.15.105:0/2774661464 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ef9ebf2ab0 con 0x55ef9ebdab40 2024-09-16T02:04:17.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.858+0000 7f868a7fc640 1 -- 172.21.15.105:0/2774661464 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8694004c50 con 0x55ef9ebdab40 2024-09-16T02:04:17.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.858+0000 7f868a7fc640 1 -- 172.21.15.105:0/2774661464 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f869402e030 con 0x55ef9ebdab40 2024-09-16T02:04:17.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.858+0000 7f86ab841740 10 client.15546 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:04:17.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.858+0000 7f86ab841740 10 client.15546 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T02:04:17.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.858+0000 7f86ab841740 20 client.15546 populate_metadata read hostname 'smithi105' 2024-09-16T02:04:17.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.858+0000 7f86ab841740 1 --2- 172.21.15.105:0/2774661464 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x55ef9ebf3e00 0x55ef9ebf61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:17.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.858+0000 7f86ab841740 4 client.15546 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:04:17.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.858+0000 7f86ab841740 1 -- 172.21.15.105:0/2774661464 --> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef9eb4c860 con 0x55ef9ebf3e00 2024-09-16T02:04:17.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.858+0000 7f869a7fc640 1 --2- 172.21.15.105:0/2774661464 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x55ef9ebf3e00 0x55ef9ebf61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:17.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.858+0000 7f868a7fc640 10 client.15546 ms_handle_connect on v2:172.21.15.190:6836/3181257549 2024-09-16T02:04:17.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.858+0000 7f869a7fc640 1 --2- 172.21.15.105:0/2774661464 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x55ef9ebf3e00 0x55ef9ebf61f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ddce6678bc1cfdfc server_cookie=218e22366e982b53 in_seq=0 out_seq=0 2024-09-16T02:04:17.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.859+0000 7f868a7fc640 1 -- 172.21.15.105:0/2774661464 <== mds.3 v2:172.21.15.190:6836/3181257549 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ef9eb4c860 con 0x55ef9ebf3e00 2024-09-16T02:04:17.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.859+0000 7f868a7fc640 10 client.15546 handle_command_reply: tid=0 2024-09-16T02:04:17.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.859+0000 7f86ab841740 1 client.15546 shutdown 2024-09-16T02:04:17.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.859+0000 7f86ab841740 1 -- 172.21.15.105:0/2774661464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9ebdab40 msgr2=0x55ef9ebe1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:17.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.859+0000 7f86ab841740 1 --2- 172.21.15.105:0/2774661464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9ebdab40 0x55ef9ebe1020 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f869400c9e0 tx=0x7f8694005540 comp rx=0 tx=0).stop 2024-09-16T02:04:17.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.859+0000 7f86ab841740 1 -- 172.21.15.105:0/2774661464 shutdown_connections 2024-09-16T02:04:17.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.859+0000 7f86ab841740 1 --2- 172.21.15.105:0/2774661464 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x55ef9ebf3e00 0x55ef9ebf61f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.859+0000 7f86ab841740 1 --2- 172.21.15.105:0/2774661464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef9ebe4370 0x55ef9ebe1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.859+0000 7f86ab841740 1 --2- 172.21.15.105:0/2774661464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9ebdab40 0x55ef9ebe1020 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.859+0000 7f868a7fc640 0 client.15546 ms_handle_reset on v2:172.21.15.190:6836/3181257549 2024-09-16T02:04:17.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.859+0000 7f86ab841740 1 --2- 172.21.15.105:0/2774661464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef9ebd95d0 0x55ef9ebe3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.859+0000 7f86ab841740 1 -- 172.21.15.105:0/2774661464 >> 172.21.15.105:0/2774661464 conn(0x55ef9eba2310 msgr2=0x55ef9ebdb930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:17.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.859+0000 7f86ab841740 1 -- 172.21.15.105:0/2774661464 shutdown_connections 2024-09-16T02:04:17.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.859+0000 7f86ab841740 1 -- 172.21.15.105:0/2774661464 wait complete. 2024-09-16T02:04:17.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.860+0000 7f86ab841740 20 client.15546 trim_cache size 0 max 16384 2024-09-16T02:04:17.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.872+0000 7f86ab841740 1 Processor -- start 2024-09-16T02:04:17.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.872+0000 7f86ab841740 1 -- start start 2024-09-16T02:04:17.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.873+0000 7f86ab841740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9ebd95d0 0x55ef9ebf0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:17.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.873+0000 7f86ab841740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef9ebdab40 0x55ef9ebe08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:17.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.873+0000 7f86ab841740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef9ebe4370 0x55ef9ebd4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:17.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.873+0000 7f86ab841740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f86a41af820 con 0x55ef9ebd95d0 2024-09-16T02:04:17.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.873+0000 7f86ab841740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f86a41b1360 con 0x55ef9ebdab40 2024-09-16T02:04:17.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.873+0000 7f86ab841740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f86a41afef0 con 0x55ef9ebe4370 2024-09-16T02:04:17.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.873+0000 7f869a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9ebd95d0 0x55ef9ebf0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:17.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.873+0000 7f869a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9ebd95d0 0x55ef9ebf0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54506/0 (socket says 172.21.15.105:54506) 2024-09-16T02:04:17.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.873+0000 7f869a7fc640 1 -- 172.21.15.105:0/330245956 learned_addr learned my addr 172.21.15.105:0/330245956 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:17.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.873+0000 7f869a7fc640 1 -- 172.21.15.105:0/330245956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef9ebe4370 msgr2=0x55ef9ebd4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:17.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.873+0000 7f869a7fc640 1 --2- 172.21.15.105:0/330245956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef9ebe4370 0x55ef9ebd4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.873+0000 7f869a7fc640 1 -- 172.21.15.105:0/330245956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef9ebdab40 msgr2=0x55ef9ebe08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:17.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.873+0000 7f869a7fc640 1 --2- 172.21.15.105:0/330245956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef9ebdab40 0x55ef9ebe08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.873+0000 7f869a7fc640 1 -- 172.21.15.105:0/330245956 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef9ebd6960 con 0x55ef9ebd95d0 2024-09-16T02:04:17.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.873+0000 7f869a7fc640 1 --2- 172.21.15.105:0/330245956 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9ebd95d0 0x55ef9ebf0340 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f868c0096c0 tx=0x7f868c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab8e71217b40f92 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:17.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.873+0000 7f8698ff9640 1 -- 172.21.15.105:0/330245956 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868c049070 con 0x55ef9ebd95d0 2024-09-16T02:04:17.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.873+0000 7f8698ff9640 1 -- 172.21.15.105:0/330245956 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f868c0318e0 con 0x55ef9ebd95d0 2024-09-16T02:04:17.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.874+0000 7f8698ff9640 1 -- 172.21.15.105:0/330245956 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868c035040 con 0x55ef9ebd95d0 2024-09-16T02:04:17.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.874+0000 7f86ab841740 1 -- 172.21.15.105:0/330245956 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9ebd95d0 msgr2=0x55ef9ebf0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:17.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.874+0000 7f86ab841740 1 --2- 172.21.15.105:0/330245956 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9ebd95d0 0x55ef9ebf0340 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f868c0096c0 tx=0x7f868c005fd0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.874+0000 7f86ab841740 1 -- 172.21.15.105:0/330245956 shutdown_connections 2024-09-16T02:04:17.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.874+0000 7f86ab841740 1 --2- 172.21.15.105:0/330245956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef9ebe4370 0x55ef9ebd4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.874+0000 7f86ab841740 1 --2- 172.21.15.105:0/330245956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef9ebdab40 0x55ef9ebe08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.874+0000 7f86ab841740 1 --2- 172.21.15.105:0/330245956 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9ebd95d0 0x55ef9ebf0340 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.874+0000 7f86ab841740 1 -- 172.21.15.105:0/330245956 >> 172.21.15.105:0/330245956 conn(0x55ef9eba2310 msgr2=0x55ef9ebd4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:17.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.874+0000 7f86ab841740 1 -- 172.21.15.105:0/330245956 shutdown_connections 2024-09-16T02:04:17.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.874+0000 7f86ab841740 1 -- 172.21.15.105:0/330245956 wait complete. 2024-09-16T02:04:17.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.874+0000 7f86ab841740 1 -- start start 2024-09-16T02:04:17.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.875+0000 7f86ab841740 1 Processor -- start 2024-09-16T02:04:17.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.875+0000 7f86ab841740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9ebd95d0 0x55ef9ebec760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:17.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.875+0000 7f86ab841740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef9ebdab40 0x55ef9ebecca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:17.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.875+0000 7f86ab841740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef9ebe4370 0x55ef9ebddac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:17.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.875+0000 7f86ab841740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f86a41b0c90 con 0x55ef9ebd95d0 2024-09-16T02:04:17.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.875+0000 7f86ab841740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f86a41b05c0 con 0x55ef9ebe4370 2024-09-16T02:04:17.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.875+0000 7f86ab841740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f86a41ad800 con 0x55ef9ebdab40 2024-09-16T02:04:17.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.875+0000 7f86ab841740 10 client.0 _ll_register_callbacks cb 0x7f8694014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:04:17.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.875+0000 7f869a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9ebd95d0 0x55ef9ebec760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:17.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.875+0000 7f869a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9ebd95d0 0x55ef9ebec760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54510/0 (socket says 172.21.15.105:54510) 2024-09-16T02:04:17.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.875+0000 7f869a7fc640 1 -- 172.21.15.105:0/4253788446 learned_addr learned my addr 172.21.15.105:0/4253788446 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:17.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.875+0000 7f86a2ffd640 1 --2- 172.21.15.105:0/4253788446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef9ebdab40 0x55ef9ebecca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:17.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.875+0000 7f869a7fc640 1 --2- 172.21.15.105:0/4253788446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef9ebe4370 0x55ef9ebddac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:17.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.875+0000 7f869a7fc640 1 -- 172.21.15.105:0/4253788446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef9ebdab40 msgr2=0x55ef9ebecca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:17.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.875+0000 7f869a7fc640 1 --2- 172.21.15.105:0/4253788446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef9ebdab40 0x55ef9ebecca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.875+0000 7f869a7fc640 1 -- 172.21.15.105:0/4253788446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef9ebe4370 msgr2=0x55ef9ebddac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:17.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.875+0000 7f869a7fc640 1 --2- 172.21.15.105:0/4253788446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef9ebe4370 0x55ef9ebddac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.875+0000 7f869a7fc640 1 -- 172.21.15.105:0/4253788446 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef9ebde2c0 con 0x55ef9ebd95d0 2024-09-16T02:04:17.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.875+0000 7f869a7fc640 1 --2- 172.21.15.105:0/4253788446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9ebd95d0 0x55ef9ebec760 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f868c02ac60 tx=0x7f868c009870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea724efe438e510 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:17.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.875+0000 7f868a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:04:17.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.876+0000 7f868a7fc640 1 -- 172.21.15.105:0/4253788446 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868c05a060 con 0x55ef9ebd95d0 2024-09-16T02:04:17.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.876+0000 7f868a7fc640 1 -- 172.21.15.105:0/4253788446 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f868c0039d0 con 0x55ef9ebd95d0 2024-09-16T02:04:17.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.876+0000 7f86ab841740 1 -- 172.21.15.105:0/4253788446 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ef9eb46d70 con 0x55ef9ebd95d0 2024-09-16T02:04:17.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.876+0000 7f868a7fc640 1 -- 172.21.15.105:0/4253788446 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f868c049070 con 0x55ef9ebd95d0 2024-09-16T02:04:17.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.876+0000 7f86ab841740 10 client.15552 fetch_fsmap learned FSMap version 28 2024-09-16T02:04:17.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.876+0000 7f86ab841740 1 -- 172.21.15.105:0/4253788446 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ef9ec5e3a0 con 0x55ef9ebd95d0 2024-09-16T02:04:17.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.876+0000 7f868a7fc640 1 -- 172.21.15.105:0/4253788446 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55ef9eb46d70 con 0x55ef9ebd95d0 2024-09-16T02:04:17.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.876+0000 7f868a7fc640 1 -- 172.21.15.105:0/4253788446 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f868c05e030 con 0x55ef9ebd95d0 2024-09-16T02:04:17.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.876+0000 7f86ab841740 10 client.15552 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:04:17.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.876+0000 7f86ab841740 10 client.15552 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T02:04:17.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.876+0000 7f86ab841740 20 client.15552 populate_metadata read hostname 'smithi105' 2024-09-16T02:04:17.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.876+0000 7f86ab841740 1 --2- 172.21.15.105:0/4253788446 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x55ef9ec5ea70 0x55ef9ec60e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:17.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.876+0000 7f86ab841740 4 client.15552 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T02:04:17.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.876+0000 7f86ab841740 1 -- 172.21.15.105:0/4253788446 --> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ef9eac3170 con 0x55ef9ec5ea70 2024-09-16T02:04:17.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.876+0000 7f86a2ffd640 1 --2- 172.21.15.105:0/4253788446 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x55ef9ec5ea70 0x55ef9ec60e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:17.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.877+0000 7f868a7fc640 10 client.15552 ms_handle_connect on v2:172.21.15.190:6836/3181257549 2024-09-16T02:04:17.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.877+0000 7f86a2ffd640 1 --2- 172.21.15.105:0/4253788446 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x55ef9ec5ea70 0x55ef9ec60e60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=754afe2947be750e server_cookie=5f23b4ca45d74a39 in_seq=0 out_seq=0 2024-09-16T02:04:17.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.877+0000 7f868a7fc640 1 -- 172.21.15.105:0/4253788446 <== mds.3 v2:172.21.15.190:6836/3181257549 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ef9eac3170 con 0x55ef9ec5ea70 2024-09-16T02:04:17.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.877+0000 7f868a7fc640 10 client.15552 handle_command_reply: tid=0 2024-09-16T02:04:17.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.877+0000 7f86ab841740 1 client.15552 shutdown 2024-09-16T02:04:17.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.877+0000 7f86ab841740 1 -- 172.21.15.105:0/4253788446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9ebd95d0 msgr2=0x55ef9ebec760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:17.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.877+0000 7f86ab841740 1 --2- 172.21.15.105:0/4253788446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9ebd95d0 0x55ef9ebec760 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f868c02ac60 tx=0x7f868c009870 comp rx=0 tx=0).stop 2024-09-16T02:04:17.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.877+0000 7f86ab841740 1 -- 172.21.15.105:0/4253788446 shutdown_connections 2024-09-16T02:04:17.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.877+0000 7f86ab841740 1 --2- 172.21.15.105:0/4253788446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef9ebdab40 0x55ef9ebecca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.877+0000 7f86ab841740 1 --2- 172.21.15.105:0/4253788446 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x55ef9ec5ea70 0x55ef9ec60e60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.877+0000 7f86ab841740 1 --2- 172.21.15.105:0/4253788446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef9ebe4370 0x55ef9ebddac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.877+0000 7f86ab841740 1 --2- 172.21.15.105:0/4253788446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9ebd95d0 0x55ef9ebec760 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.877+0000 7f86ab841740 1 -- 172.21.15.105:0/4253788446 >> 172.21.15.105:0/4253788446 conn(0x55ef9eba2310 msgr2=0x55ef9ebd44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:17.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.877+0000 7f868a7fc640 0 client.15552 ms_handle_reset on v2:172.21.15.190:6836/3181257549 2024-09-16T02:04:17.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.878+0000 7f86ab841740 1 -- 172.21.15.105:0/4253788446 shutdown_connections 2024-09-16T02:04:17.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.878+0000 7f86ab841740 1 -- 172.21.15.105:0/4253788446 wait complete. 2024-09-16T02:04:17.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.879+0000 7f86ab841740 20 client.15552 trim_cache size 0 max 16384 2024-09-16T02:04:17.898 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-09-16T02:04:17.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.879+0000 7f86a9468640 1 -- 172.21.15.105:0/1602773668 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f86900a3240 msgr2=0x7f86900a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:17.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.879+0000 7f86a9468640 1 --2- 172.21.15.105:0/1602773668 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f86900a3240 0x7f86900a5700 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f869c00ab20 tx=0x7f869c00af30 comp rx=0 tx=0).stop 2024-09-16T02:04:17.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.879+0000 7f86a9468640 1 -- 172.21.15.105:0/1602773668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86a4102ef0 msgr2=0x7f86a40fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:17.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.879+0000 7f86a9468640 1 --2- 172.21.15.105:0/1602773668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86a4102ef0 0x7f86a40fd700 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f869c00cce0 tx=0x7f869c00a620 comp rx=0 tx=0).stop 2024-09-16T02:04:17.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.879+0000 7f86a9468640 1 -- 172.21.15.105:0/1602773668 shutdown_connections 2024-09-16T02:04:17.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.879+0000 7f86a9468640 1 --2- 172.21.15.105:0/1602773668 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f86900a3240 0x7f86900a5700 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.879+0000 7f86a9468640 1 --2- 172.21.15.105:0/1602773668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86a4102ef0 0x7f86a40fd700 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.879+0000 7f86a9468640 1 --2- 172.21.15.105:0/1602773668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86a4102570 0x7f86a40fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.879+0000 7f86a9468640 1 --2- 172.21.15.105:0/1602773668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86a4100e20 0x7f86a40fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.879+0000 7f86a9468640 1 -- 172.21.15.105:0/1602773668 >> 172.21.15.105:0/1602773668 conn(0x7f86a40fc7d0 msgr2=0x7f86a406a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:17.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.879+0000 7f86a9468640 1 -- 172.21.15.105:0/1602773668 shutdown_connections 2024-09-16T02:04:17.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:17.880+0000 7f86a9468640 1 -- 172.21.15.105:0/1602773668 wait complete. 2024-09-16T02:04:18.464 INFO:tasks.workunit.client.0.smithi105.stdout: 2024-09-16T02:04:18.464 INFO:tasks.workunit.client.0.smithi105.stdout: Children see throughput for 1 re-readers = 7670750.50 kB/sec 2024-09-16T02:04:18.464 INFO:tasks.workunit.client.0.smithi105.stdout: Parent sees throughput for 1 re-readers = 7658982.23 kB/sec 2024-09-16T02:04:18.464 INFO:tasks.workunit.client.0.smithi105.stdout: Min throughput per process = 7670750.50 kB/sec 2024-09-16T02:04:18.464 INFO:tasks.workunit.client.0.smithi105.stdout: Max throughput per process = 7670750.50 kB/sec 2024-09-16T02:04:18.464 INFO:tasks.workunit.client.0.smithi105.stdout: Avg throughput per process = 7670750.50 kB/sec 2024-09-16T02:04:18.464 INFO:tasks.workunit.client.0.smithi105.stdout: Min xfer = 1048576.00 kB 2024-09-16T02:04:18.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:18 smithi105 ceph-mon[31197]: pgmap v392: 97 pgs: 97 active+clean; 1.0 GiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 82 MiB/s wr, 30 op/s 2024-09-16T02:04:18.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:18 smithi190 ceph-mon[34762]: pgmap v392: 97 pgs: 97 active+clean; 1.0 GiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 82 MiB/s wr, 30 op/s 2024-09-16T02:04:18.864 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T02:04:18.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:18 smithi137 ceph-mon[34099]: pgmap v392: 97 pgs: 97 active+clean; 1.0 GiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 82 MiB/s wr, 30 op/s 2024-09-16T02:04:19.107 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:19.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.594+0000 7f49e1ea3640 1 -- 172.21.15.105:0/3643732171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49dc106f10 msgr2=0x7f49dc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:19.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.594+0000 7f49e1ea3640 1 --2- 172.21.15.105:0/3643732171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49dc106f10 0x7f49dc10ea90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f49d4009ca0 tx=0x7f49d402a8c0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.594+0000 7f49e1ea3640 1 -- 172.21.15.105:0/3643732171 shutdown_connections 2024-09-16T02:04:19.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.594+0000 7f49e1ea3640 1 --2- 172.21.15.105:0/3643732171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49dc106f10 0x7f49dc10ea90 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.594+0000 7f49e1ea3640 1 --2- 172.21.15.105:0/3643732171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49dc102790 0x7f49dc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.594+0000 7f49e1ea3640 1 --2- 172.21.15.105:0/3643732171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49dc101de0 0x7f49dc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.594+0000 7f49e1ea3640 1 -- 172.21.15.105:0/3643732171 >> 172.21.15.105:0/3643732171 conn(0x7f49dc0fd8c0 msgr2=0x7f49dc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:19.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.594+0000 7f49e1ea3640 1 -- 172.21.15.105:0/3643732171 shutdown_connections 2024-09-16T02:04:19.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.594+0000 7f49e1ea3640 1 -- 172.21.15.105:0/3643732171 wait complete. 2024-09-16T02:04:19.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.594+0000 7f49e1ea3640 1 Processor -- start 2024-09-16T02:04:19.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.595+0000 7f49e1ea3640 1 -- start start 2024-09-16T02:04:19.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.595+0000 7f49e1ea3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49dc101de0 0x7f49dc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:19.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.595+0000 7f49e1ea3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49dc102790 0x7f49dc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:19.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.595+0000 7f49e1ea3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49dc106f10 0x7f49dc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:19.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.595+0000 7f49e1ea3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc111e20 con 0x7f49dc102790 2024-09-16T02:04:19.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.595+0000 7f49e1ea3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc111ca0 con 0x7f49dc101de0 2024-09-16T02:04:19.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.595+0000 7f49e1ea3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc111fa0 con 0x7f49dc106f10 2024-09-16T02:04:19.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.595+0000 7f49daffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49dc102790 0x7f49dc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:19.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.595+0000 7f49daffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49dc102790 0x7f49dc074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57090/0 (socket says 172.21.15.105:57090) 2024-09-16T02:04:19.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.595+0000 7f49daffd640 1 -- 172.21.15.105:0/2670952158 learned_addr learned my addr 172.21.15.105:0/2670952158 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:19.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.595+0000 7f49db7fe640 1 --2- 172.21.15.105:0/2670952158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49dc101de0 0x7f49dc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:19.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.595+0000 7f49dbfff640 1 --2- 172.21.15.105:0/2670952158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49dc106f10 0x7f49dc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:19.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.595+0000 7f49daffd640 1 -- 172.21.15.105:0/2670952158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49dc106f10 msgr2=0x7f49dc074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:19.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.595+0000 7f49daffd640 1 --2- 172.21.15.105:0/2670952158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49dc106f10 0x7f49dc074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.595+0000 7f49daffd640 1 -- 172.21.15.105:0/2670952158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49dc101de0 msgr2=0x7f49dc0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:19.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.595+0000 7f49daffd640 1 --2- 172.21.15.105:0/2670952158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49dc101de0 0x7f49dc0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.595+0000 7f49daffd640 1 -- 172.21.15.105:0/2670952158 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49dc1a6700 con 0x7f49dc102790 2024-09-16T02:04:19.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.595+0000 7f49dbfff640 1 --2- 172.21.15.105:0/2670952158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49dc106f10 0x7f49dc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:04:19.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.595+0000 7f49daffd640 1 --2- 172.21.15.105:0/2670952158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49dc102790 0x7f49dc074a30 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f49d000d950 tx=0x7f49d000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:19.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.596+0000 7f49d8ff9640 1 -- 172.21.15.105:0/2670952158 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d0017070 con 0x7f49dc102790 2024-09-16T02:04:19.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.596+0000 7f49d8ff9640 1 -- 172.21.15.105:0/2670952158 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49d000de80 con 0x7f49dc102790 2024-09-16T02:04:19.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.596+0000 7f49e1ea3640 1 -- 172.21.15.105:0/2670952158 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49dc1a69f0 con 0x7f49dc102790 2024-09-16T02:04:19.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.596+0000 7f49d8ff9640 1 -- 172.21.15.105:0/2670952158 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d000f620 con 0x7f49dc102790 2024-09-16T02:04:19.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.596+0000 7f49e1ea3640 1 -- 172.21.15.105:0/2670952158 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49dc1a6f80 con 0x7f49dc102790 2024-09-16T02:04:19.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.598+0000 7f49d8ff9640 1 -- 172.21.15.105:0/2670952158 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f49d0009c30 con 0x7f49dc102790 2024-09-16T02:04:19.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.598+0000 7f49e427c740 1 Processor -- start 2024-09-16T02:04:19.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.598+0000 7f49e427c740 1 -- start start 2024-09-16T02:04:19.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.598+0000 7f49e427c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e598907b40 0x55e598906090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:19.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.598+0000 7f49e427c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5989065d0 0x55e598906a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:19.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.598+0000 7f49e427c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e598911370 0x55e598913760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:19.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.598+0000 7f49e427c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e59886c2f0 con 0x55e5989065d0 2024-09-16T02:04:19.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.598+0000 7f49e427c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e598813cf0 con 0x55e598907b40 2024-09-16T02:04:19.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.598+0000 7f49e427c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e5988682b0 con 0x55e598911370 2024-09-16T02:04:19.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.598+0000 7f49d8ff9640 1 --2- 172.21.15.105:0/2670952158 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f49c40a32c0 0x7f49c40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:19.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.598+0000 7f49db7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5989065d0 0x55e598906a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:19.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.599+0000 7f49db7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5989065d0 0x55e598906a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57102/0 (socket says 172.21.15.105:57102) 2024-09-16T02:04:19.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.599+0000 7f49db7fe640 1 -- 172.21.15.105:0/2012197817 learned_addr learned my addr 172.21.15.105:0/2012197817 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:19.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.599+0000 7f49dbfff640 1 --2- 172.21.15.105:0/2670952158 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f49c40a32c0 0x7f49c40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:19.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.599+0000 7f49daffd640 1 --2- 172.21.15.105:0/2012197817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e598911370 0x55e598913760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:19.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.599+0000 7f49dbfff640 1 --2- 172.21.15.105:0/2012197817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e598907b40 0x55e598906090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:19.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.599+0000 7f49db7fe640 1 -- 172.21.15.105:0/2012197817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e598911370 msgr2=0x55e598913760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:19.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.599+0000 7f49d8ff9640 1 -- 172.21.15.105:0/2670952158 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f49d00cf530 con 0x7f49dc102790 2024-09-16T02:04:19.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.599+0000 7f49db7fe640 1 --2- 172.21.15.105:0/2012197817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e598911370 0x55e598913760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.599+0000 7f49db7fe640 1 -- 172.21.15.105:0/2012197817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e598907b40 msgr2=0x55e598906090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:19.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.599+0000 7f49db7fe640 1 --2- 172.21.15.105:0/2012197817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e598907b40 0x55e598906090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.599+0000 7f49db7fe640 1 -- 172.21.15.105:0/2012197817 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5987f83a0 con 0x55e5989065d0 2024-09-16T02:04:19.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.599+0000 7f49daffd640 1 --2- 172.21.15.105:0/2012197817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e598911370 0x55e598913760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:04:19.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.599+0000 7f49dbfff640 1 --2- 172.21.15.105:0/2012197817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e598907b40 0x55e598906090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:04:19.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.599+0000 7f49dbfff640 1 --2- 172.21.15.105:0/2670952158 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f49c40a32c0 0x7f49c40a5780 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f49d4005e30 tx=0x7f49d4005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:19.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.599+0000 7f49db7fe640 1 --2- 172.21.15.105:0/2012197817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5989065d0 0x55e598906a10 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f49cc00ad00 tx=0x7f49cc00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c79d943f58465b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:19.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.599+0000 7f49c1ffb640 1 -- 172.21.15.105:0/2012197817 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49cc01d070 con 0x55e5989065d0 2024-09-16T02:04:19.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.599+0000 7f49c1ffb640 1 -- 172.21.15.105:0/2012197817 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49cc018af0 con 0x55e5989065d0 2024-09-16T02:04:19.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.599+0000 7f49c1ffb640 1 -- 172.21.15.105:0/2012197817 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49cc010b40 con 0x55e5989065d0 2024-09-16T02:04:19.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.600+0000 7f49e427c740 1 -- 172.21.15.105:0/2012197817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5989065d0 msgr2=0x55e598906a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:19.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.600+0000 7f49e427c740 1 --2- 172.21.15.105:0/2012197817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5989065d0 0x55e598906a10 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f49cc00ad00 tx=0x7f49cc00f790 comp rx=0 tx=0).stop 2024-09-16T02:04:19.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.600+0000 7f49e427c740 1 -- 172.21.15.105:0/2012197817 shutdown_connections 2024-09-16T02:04:19.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.600+0000 7f49e427c740 1 --2- 172.21.15.105:0/2012197817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e598911370 0x55e598913760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.600+0000 7f49e427c740 1 --2- 172.21.15.105:0/2012197817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5989065d0 0x55e598906a10 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.600+0000 7f49e427c740 1 --2- 172.21.15.105:0/2012197817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e598907b40 0x55e598906090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.600+0000 7f49e427c740 1 -- 172.21.15.105:0/2012197817 >> 172.21.15.105:0/2012197817 conn(0x55e5988cf310 msgr2=0x55e598902c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:19.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.600+0000 7f49e427c740 1 -- 172.21.15.105:0/2012197817 shutdown_connections 2024-09-16T02:04:19.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.600+0000 7f49e427c740 1 -- 172.21.15.105:0/2012197817 wait complete. 2024-09-16T02:04:19.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.600+0000 7f49e427c740 1 -- start start 2024-09-16T02:04:19.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.601+0000 7f49e427c740 1 Processor -- start 2024-09-16T02:04:19.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.601+0000 7f49e427c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5989065d0 0x55e598910d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:19.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.601+0000 7f49e427c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e598907b40 0x55e59890dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:19.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.601+0000 7f49e427c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e598911370 0x55e59890e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:19.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.601+0000 7f49e427c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e598914070 con 0x55e598911370 2024-09-16T02:04:19.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.601+0000 7f49e427c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e598827060 con 0x55e598907b40 2024-09-16T02:04:19.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.601+0000 7f49e427c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e5989141f0 con 0x55e5989065d0 2024-09-16T02:04:19.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.601+0000 7f49e427c740 10 client.0 _ll_register_callbacks cb 0x55e59857a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:04:19.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.601+0000 7f49dbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e598911370 0x55e59890e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:19.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.601+0000 7f49dbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e598911370 0x55e59890e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57104/0 (socket says 172.21.15.105:57104) 2024-09-16T02:04:19.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.601+0000 7f49dbfff640 1 -- 172.21.15.105:0/3784328471 learned_addr learned my addr 172.21.15.105:0/3784328471 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:19.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.601+0000 7f49daffd640 1 --2- 172.21.15.105:0/3784328471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e598907b40 0x55e59890dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:19.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.601+0000 7f49db7fe640 1 --2- 172.21.15.105:0/3784328471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5989065d0 0x55e598910d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:19.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.601+0000 7f49dbfff640 1 -- 172.21.15.105:0/3784328471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5989065d0 msgr2=0x55e598910d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:19.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.601+0000 7f49dbfff640 1 --2- 172.21.15.105:0/3784328471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5989065d0 0x55e598910d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.601+0000 7f49dbfff640 1 -- 172.21.15.105:0/3784328471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e598907b40 msgr2=0x55e59890dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:19.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.601+0000 7f49dbfff640 1 --2- 172.21.15.105:0/3784328471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e598907b40 0x55e59890dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.601+0000 7f49dbfff640 1 -- 172.21.15.105:0/3784328471 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5985dd3c0 con 0x55e598911370 2024-09-16T02:04:19.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.602+0000 7f49dbfff640 1 --2- 172.21.15.105:0/3784328471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e598911370 0x55e59890e560 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f49d4003bf0 tx=0x7f49d402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f7e2128c56f4b76 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:19.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.602+0000 7f49baffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:04:19.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.602+0000 7f49baffd640 1 -- 172.21.15.105:0/3784328471 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d4004250 con 0x55e598911370 2024-09-16T02:04:19.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.602+0000 7f49baffd640 1 -- 172.21.15.105:0/3784328471 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f49d40043f0 con 0x55e598911370 2024-09-16T02:04:19.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.602+0000 7f49e427c740 1 -- 172.21.15.105:0/3784328471 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49cc018af0 con 0x55e598911370 2024-09-16T02:04:19.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.602+0000 7f49baffd640 1 -- 172.21.15.105:0/3784328471 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f49d40045f0 con 0x55e598911370 2024-09-16T02:04:19.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.602+0000 7f49e427c740 10 client.15564 fetch_fsmap learned FSMap version 28 2024-09-16T02:04:19.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.602+0000 7f49e427c740 1 -- 172.21.15.105:0/3784328471 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e59891fa60 con 0x55e598911370 2024-09-16T02:04:19.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.602+0000 7f49baffd640 1 -- 172.21.15.105:0/3784328471 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49cc018af0 con 0x55e598911370 2024-09-16T02:04:19.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.602+0000 7f49baffd640 1 -- 172.21.15.105:0/3784328471 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f49d402f550 con 0x55e598911370 2024-09-16T02:04:19.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.602+0000 7f49e427c740 10 client.15564 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:04:19.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.602+0000 7f49e427c740 10 client.15564 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T02:04:19.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.602+0000 7f49e427c740 20 client.15564 populate_metadata read hostname 'smithi105' 2024-09-16T02:04:19.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.602+0000 7f49e427c740 1 --2- 172.21.15.105:0/3784328471 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x55e598920e70 0x55e598923330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:19.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.602+0000 7f49e427c740 4 client.15564 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:04:19.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.602+0000 7f49e427c740 1 -- 172.21.15.105:0/3784328471 --> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e598879860 con 0x55e598920e70 2024-09-16T02:04:19.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.603+0000 7f49db7fe640 1 --2- 172.21.15.105:0/3784328471 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x55e598920e70 0x55e598923330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:19.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.604+0000 7f49db7fe640 1 --2- 172.21.15.105:0/3784328471 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x55e598920e70 0x55e598923330 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=933e94db16457ae2 server_cookie=d45b3f1893ad30e5 in_seq=0 out_seq=0 2024-09-16T02:04:19.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.604+0000 7f49baffd640 10 client.15564 ms_handle_connect on v2:172.21.15.137:6836/2788541030 2024-09-16T02:04:19.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.604+0000 7f49baffd640 1 -- 172.21.15.105:0/3784328471 <== mds.2 v2:172.21.15.137:6836/2788541030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e598879860 con 0x55e598920e70 2024-09-16T02:04:19.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.604+0000 7f49baffd640 10 client.15564 handle_command_reply: tid=0 2024-09-16T02:04:19.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.605+0000 7f49e427c740 1 client.15564 shutdown 2024-09-16T02:04:19.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.605+0000 7f49e427c740 1 -- 172.21.15.105:0/3784328471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e598911370 msgr2=0x55e59890e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:19.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.605+0000 7f49e427c740 1 --2- 172.21.15.105:0/3784328471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e598911370 0x55e59890e560 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f49d4003bf0 tx=0x7f49d402a8c0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.605+0000 7f49e427c740 1 -- 172.21.15.105:0/3784328471 shutdown_connections 2024-09-16T02:04:19.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.605+0000 7f49e427c740 1 --2- 172.21.15.105:0/3784328471 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x55e598920e70 0x55e598923330 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.605+0000 7f49e427c740 1 --2- 172.21.15.105:0/3784328471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e598911370 0x55e59890e560 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.605+0000 7f49e427c740 1 --2- 172.21.15.105:0/3784328471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e598907b40 0x55e59890dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.605+0000 7f49baffd640 0 client.15564 ms_handle_reset on v2:172.21.15.137:6836/2788541030 2024-09-16T02:04:19.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.605+0000 7f49e427c740 1 --2- 172.21.15.105:0/3784328471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5989065d0 0x55e598910d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.605+0000 7f49e427c740 1 -- 172.21.15.105:0/3784328471 >> 172.21.15.105:0/3784328471 conn(0x55e5988cf310 msgr2=0x55e598908990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:19.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.605+0000 7f49e427c740 1 -- 172.21.15.105:0/3784328471 shutdown_connections 2024-09-16T02:04:19.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.605+0000 7f49e427c740 1 -- 172.21.15.105:0/3784328471 wait complete. 2024-09-16T02:04:19.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.605+0000 7f49e427c740 20 client.15564 trim_cache size 0 max 16384 2024-09-16T02:04:19.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.618+0000 7f49e427c740 1 Processor -- start 2024-09-16T02:04:19.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.618+0000 7f49e427c740 1 -- start start 2024-09-16T02:04:19.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.618+0000 7f49e427c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5989065d0 0x55e598902e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:19.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.618+0000 7f49e427c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e598907b40 0x55e59891d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:19.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.618+0000 7f49e427c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e598911370 0x55e598928c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:19.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.618+0000 7f49e427c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc1af7b0 con 0x55e598907b40 2024-09-16T02:04:19.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.618+0000 7f49e427c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc1b12f0 con 0x55e598911370 2024-09-16T02:04:19.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.618+0000 7f49e427c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc1afe80 con 0x55e5989065d0 2024-09-16T02:04:19.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.618+0000 7f49daffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e598907b40 0x55e59891d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:19.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.618+0000 7f49daffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e598907b40 0x55e59891d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57112/0 (socket says 172.21.15.105:57112) 2024-09-16T02:04:19.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.618+0000 7f49daffd640 1 -- 172.21.15.105:0/1142691741 learned_addr learned my addr 172.21.15.105:0/1142691741 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:19.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.618+0000 7f49daffd640 1 -- 172.21.15.105:0/1142691741 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5989065d0 msgr2=0x55e598902e10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T02:04:19.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.618+0000 7f49daffd640 1 --2- 172.21.15.105:0/1142691741 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5989065d0 0x55e598902e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.618+0000 7f49daffd640 1 -- 172.21.15.105:0/1142691741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e598911370 msgr2=0x55e598928c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:19.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.618+0000 7f49daffd640 1 --2- 172.21.15.105:0/1142691741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e598911370 0x55e598928c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.618+0000 7f49daffd640 1 -- 172.21.15.105:0/1142691741 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5989239b0 con 0x55e598907b40 2024-09-16T02:04:19.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.619+0000 7f49daffd640 1 --2- 172.21.15.105:0/1142691741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e598907b40 0x55e59891d340 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f49d00058c0 tx=0x7f49d001e7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc9815eed74bd5f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:19.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.619+0000 7f49c17fa640 1 -- 172.21.15.105:0/1142691741 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d0020040 con 0x55e598907b40 2024-09-16T02:04:19.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.619+0000 7f49c17fa640 1 -- 172.21.15.105:0/1142691741 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49d00a4760 con 0x55e598907b40 2024-09-16T02:04:19.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.619+0000 7f49c17fa640 1 -- 172.21.15.105:0/1142691741 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d00a42d0 con 0x55e598907b40 2024-09-16T02:04:19.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.619+0000 7f49e427c740 1 -- 172.21.15.105:0/1142691741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e598907b40 msgr2=0x55e59891d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:19.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.619+0000 7f49e427c740 1 --2- 172.21.15.105:0/1142691741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e598907b40 0x55e59891d340 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f49d00058c0 tx=0x7f49d001e7b0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.619+0000 7f49e427c740 1 -- 172.21.15.105:0/1142691741 shutdown_connections 2024-09-16T02:04:19.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.619+0000 7f49e427c740 1 --2- 172.21.15.105:0/1142691741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e598911370 0x55e598928c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.619+0000 7f49e427c740 1 --2- 172.21.15.105:0/1142691741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e598907b40 0x55e59891d340 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.619+0000 7f49e427c740 1 --2- 172.21.15.105:0/1142691741 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5989065d0 0x55e598902e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.619+0000 7f49e427c740 1 -- 172.21.15.105:0/1142691741 >> 172.21.15.105:0/1142691741 conn(0x55e5988cf310 msgr2=0x55e598902700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:19.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.619+0000 7f49e427c740 1 -- 172.21.15.105:0/1142691741 shutdown_connections 2024-09-16T02:04:19.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.619+0000 7f49e427c740 1 -- 172.21.15.105:0/1142691741 wait complete. 2024-09-16T02:04:19.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.620+0000 7f49e427c740 1 -- start start 2024-09-16T02:04:19.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.620+0000 7f49e427c740 1 Processor -- start 2024-09-16T02:04:19.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.620+0000 7f49e427c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e5989065d0 0x55e5989194b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:19.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.620+0000 7f49e427c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e598907b40 0x55e5989199f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:19.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.620+0000 7f49e427c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e598911370 0x55e598916480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:19.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.620+0000 7f49e427c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc1b0c20 con 0x55e598911370 2024-09-16T02:04:19.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.620+0000 7f49e427c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc1b0550 con 0x55e5989065d0 2024-09-16T02:04:19.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.620+0000 7f49e427c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc1ad790 con 0x55e598907b40 2024-09-16T02:04:19.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.620+0000 7f49e427c740 10 client.0 _ll_register_callbacks cb 0x7f49d4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:04:19.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.620+0000 7f49dbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e598911370 0x55e598916480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:19.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.620+0000 7f49dbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e598911370 0x55e598916480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57116/0 (socket says 172.21.15.105:57116) 2024-09-16T02:04:19.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.620+0000 7f49dbfff640 1 -- 172.21.15.105:0/655066862 learned_addr learned my addr 172.21.15.105:0/655066862 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:19.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.620+0000 7f49daffd640 1 --2- 172.21.15.105:0/655066862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e598907b40 0x55e5989199f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:19.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.621+0000 7f49db7fe640 1 --2- 172.21.15.105:0/655066862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e5989065d0 0x55e5989194b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:19.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.621+0000 7f49dbfff640 1 -- 172.21.15.105:0/655066862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e598907b40 msgr2=0x55e5989199f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:19.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.621+0000 7f49dbfff640 1 --2- 172.21.15.105:0/655066862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e598907b40 0x55e5989199f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.621+0000 7f49dbfff640 1 -- 172.21.15.105:0/655066862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e5989065d0 msgr2=0x55e5989194b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:19.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.621+0000 7f49dbfff640 1 --2- 172.21.15.105:0/655066862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e5989065d0 0x55e5989194b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.621+0000 7f49dbfff640 1 -- 172.21.15.105:0/655066862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e59891a030 con 0x55e598911370 2024-09-16T02:04:19.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.621+0000 7f49db7fe640 1 --2- 172.21.15.105:0/655066862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e5989065d0 0x55e5989194b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:04:19.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.621+0000 7f49dbfff640 1 --2- 172.21.15.105:0/655066862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e598911370 0x55e598916480 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f49d4009ca0 tx=0x7f49d40095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e43784b160f6aa64 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:19.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.621+0000 7f49baffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:04:19.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.621+0000 7f49baffd640 1 -- 172.21.15.105:0/655066862 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d405c030 con 0x55e598911370 2024-09-16T02:04:19.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.621+0000 7f49e427c740 1 -- 172.21.15.105:0/655066862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49d00a4760 con 0x55e598911370 2024-09-16T02:04:19.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.621+0000 7f49baffd640 1 -- 172.21.15.105:0/655066862 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f49d402fc00 con 0x55e598911370 2024-09-16T02:04:19.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.621+0000 7f49baffd640 1 -- 172.21.15.105:0/655066862 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f49d402fe00 con 0x55e598911370 2024-09-16T02:04:19.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.621+0000 7f49e427c740 10 client.15570 fetch_fsmap learned FSMap version 28 2024-09-16T02:04:19.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.621+0000 7f49e427c740 1 -- 172.21.15.105:0/655066862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e59898c120 con 0x55e598911370 2024-09-16T02:04:19.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.621+0000 7f49baffd640 1 -- 172.21.15.105:0/655066862 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49d00a4760 con 0x55e598911370 2024-09-16T02:04:19.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.621+0000 7f49baffd640 1 -- 172.21.15.105:0/655066862 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f49d40549e0 con 0x55e598911370 2024-09-16T02:04:19.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.621+0000 7f49e427c740 10 client.15570 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:04:19.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.621+0000 7f49e427c740 10 client.15570 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T02:04:19.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.621+0000 7f49e427c740 20 client.15570 populate_metadata read hostname 'smithi105' 2024-09-16T02:04:19.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.621+0000 7f49e427c740 1 --2- 172.21.15.105:0/655066862 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x55e59898c7e0 0x55e59898ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:19.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.621+0000 7f49e427c740 4 client.15570 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T02:04:19.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.621+0000 7f49e427c740 1 -- 172.21.15.105:0/655066862 --> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e598873d70 con 0x55e59898c7e0 2024-09-16T02:04:19.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.622+0000 7f49db7fe640 1 --2- 172.21.15.105:0/655066862 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x55e59898c7e0 0x55e59898ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:19.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.622+0000 7f49baffd640 10 client.15570 ms_handle_connect on v2:172.21.15.137:6836/2788541030 2024-09-16T02:04:19.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.622+0000 7f49db7fe640 1 --2- 172.21.15.105:0/655066862 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x55e59898c7e0 0x55e59898ebd0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=293b96a0694e597d server_cookie=1e4bfa8ea32dfb11 in_seq=0 out_seq=0 2024-09-16T02:04:19.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.623+0000 7f49baffd640 1 -- 172.21.15.105:0/655066862 <== mds.2 v2:172.21.15.137:6836/2788541030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e598873d70 con 0x55e59898c7e0 2024-09-16T02:04:19.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.623+0000 7f49baffd640 10 client.15570 handle_command_reply: tid=0 2024-09-16T02:04:19.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.623+0000 7f49e427c740 1 client.15570 shutdown 2024-09-16T02:04:19.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.623+0000 7f49e427c740 1 -- 172.21.15.105:0/655066862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e598911370 msgr2=0x55e598916480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:19.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.623+0000 7f49e427c740 1 --2- 172.21.15.105:0/655066862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e598911370 0x55e598916480 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f49d4009ca0 tx=0x7f49d40095e0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.623+0000 7f49e427c740 1 -- 172.21.15.105:0/655066862 shutdown_connections 2024-09-16T02:04:19.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.623+0000 7f49e427c740 1 --2- 172.21.15.105:0/655066862 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x55e59898c7e0 0x55e59898ebd0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.623+0000 7f49e427c740 1 --2- 172.21.15.105:0/655066862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e598907b40 0x55e5989199f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.623+0000 7f49e427c740 1 --2- 172.21.15.105:0/655066862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e598911370 0x55e598916480 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.623+0000 7f49e427c740 1 --2- 172.21.15.105:0/655066862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e5989065d0 0x55e5989194b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.623+0000 7f49e427c740 1 -- 172.21.15.105:0/655066862 >> 172.21.15.105:0/655066862 conn(0x55e5988cf310 msgr2=0x55e598910040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:19.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.623+0000 7f49baffd640 0 client.15570 ms_handle_reset on v2:172.21.15.137:6836/2788541030 2024-09-16T02:04:19.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.623+0000 7f49e427c740 1 -- 172.21.15.105:0/655066862 shutdown_connections 2024-09-16T02:04:19.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.623+0000 7f49e427c740 1 -- 172.21.15.105:0/655066862 wait complete. 2024-09-16T02:04:19.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.623+0000 7f49e427c740 20 client.15570 trim_cache size 0 max 16384 2024-09-16T02:04:19.640 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-09-16T02:04:19.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.623+0000 7f49e1ea3640 1 -- 172.21.15.105:0/2670952158 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f49c40a32c0 msgr2=0x7f49c40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:19.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.623+0000 7f49e1ea3640 1 --2- 172.21.15.105:0/2670952158 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f49c40a32c0 0x7f49c40a5780 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f49d4005e30 tx=0x7f49d4005d80 comp rx=0 tx=0).stop 2024-09-16T02:04:19.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.623+0000 7f49e1ea3640 1 -- 172.21.15.105:0/2670952158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49dc102790 msgr2=0x7f49dc074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:19.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.623+0000 7f49e1ea3640 1 --2- 172.21.15.105:0/2670952158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49dc102790 0x7f49dc074a30 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f49d000d950 tx=0x7f49d000dd50 comp rx=0 tx=0).stop 2024-09-16T02:04:19.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.624+0000 7f49e1ea3640 1 -- 172.21.15.105:0/2670952158 shutdown_connections 2024-09-16T02:04:19.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.624+0000 7f49e1ea3640 1 --2- 172.21.15.105:0/2670952158 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f49c40a32c0 0x7f49c40a5780 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.624+0000 7f49e1ea3640 1 --2- 172.21.15.105:0/2670952158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49dc106f10 0x7f49dc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.624+0000 7f49e1ea3640 1 --2- 172.21.15.105:0/2670952158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49dc102790 0x7f49dc074a30 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.624+0000 7f49e1ea3640 1 --2- 172.21.15.105:0/2670952158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49dc101de0 0x7f49dc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.624+0000 7f49e1ea3640 1 -- 172.21.15.105:0/2670952158 >> 172.21.15.105:0/2670952158 conn(0x7f49dc0fd8c0 msgr2=0x7f49dc0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:19.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.624+0000 7f49e1ea3640 1 -- 172.21.15.105:0/2670952158 shutdown_connections 2024-09-16T02:04:19.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:19.624+0000 7f49e1ea3640 1 -- 172.21.15.105:0/2670952158 wait complete. 2024-09-16T02:04:20.346 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T02:04:20.591 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:20.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:20 smithi190 ceph-mon[34762]: pgmap v393: 97 pgs: 97 active+clean; 1.0 GiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 77 MiB/s wr, 30 op/s 2024-09-16T02:04:20.854 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:20 smithi105 ceph-mon[31197]: pgmap v393: 97 pgs: 97 active+clean; 1.0 GiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 77 MiB/s wr, 30 op/s 2024-09-16T02:04:20.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:20 smithi137 ceph-mon[34099]: pgmap v393: 97 pgs: 97 active+clean; 1.0 GiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 77 MiB/s wr, 30 op/s 2024-09-16T02:04:21.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.078+0000 7fb08f57d640 1 -- 172.21.15.105:0/4039457384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb088106a80 msgr2=0x7fb088106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:21.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.078+0000 7fb08f57d640 1 --2- 172.21.15.105:0/4039457384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb088106a80 0x7fb088106e60 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb07c009930 tx=0x7fb07c02a420 comp rx=0 tx=0).stop 2024-09-16T02:04:21.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.078+0000 7fb08f57d640 1 -- 172.21.15.105:0/4039457384 shutdown_connections 2024-09-16T02:04:21.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.078+0000 7fb08f57d640 1 --2- 172.21.15.105:0/4039457384 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb088101420 0x7fb08810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.078+0000 7fb08f57d640 1 --2- 172.21.15.105:0/4039457384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb088100a80 0x7fb088100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.078+0000 7fb08f57d640 1 --2- 172.21.15.105:0/4039457384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb088106a80 0x7fb088106e60 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.078+0000 7fb08f57d640 1 -- 172.21.15.105:0/4039457384 >> 172.21.15.105:0/4039457384 conn(0x7fb0880fc7a0 msgr2=0x7fb0880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:21.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.078+0000 7fb08f57d640 1 -- 172.21.15.105:0/4039457384 shutdown_connections 2024-09-16T02:04:21.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.078+0000 7fb08f57d640 1 -- 172.21.15.105:0/4039457384 wait complete. 2024-09-16T02:04:21.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.078+0000 7fb08f57d640 1 Processor -- start 2024-09-16T02:04:21.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.078+0000 7fb08f57d640 1 -- start start 2024-09-16T02:04:21.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.079+0000 7fb08f57d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb088100a80 0x7fb0881972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:21.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.079+0000 7fb08f57d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb088101420 0x7fb088197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:21.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.079+0000 7fb08f57d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb088106a80 0x7fb08819bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:21.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.079+0000 7fb08f57d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb088110d50 con 0x7fb088101420 2024-09-16T02:04:21.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.079+0000 7fb08f57d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb088110bd0 con 0x7fb088106a80 2024-09-16T02:04:21.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.079+0000 7fb08f57d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb088110ed0 con 0x7fb088100a80 2024-09-16T02:04:21.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.079+0000 7fb08cb18640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb088101420 0x7fb088197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:21.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.079+0000 7fb08cb18640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb088101420 0x7fb088197830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57146/0 (socket says 172.21.15.105:57146) 2024-09-16T02:04:21.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.079+0000 7fb08cb18640 1 -- 172.21.15.105:0/2379422136 learned_addr learned my addr 172.21.15.105:0/2379422136 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:21.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.079+0000 7fb08d319640 1 --2- 172.21.15.105:0/2379422136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb088100a80 0x7fb0881972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:21.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.079+0000 7fb08db1a640 1 --2- 172.21.15.105:0/2379422136 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb088106a80 0x7fb08819bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:21.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.079+0000 7fb08cb18640 1 -- 172.21.15.105:0/2379422136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb088100a80 msgr2=0x7fb0881972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:21.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.079+0000 7fb08cb18640 1 --2- 172.21.15.105:0/2379422136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb088100a80 0x7fb0881972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.079+0000 7fb08cb18640 1 -- 172.21.15.105:0/2379422136 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb088106a80 msgr2=0x7fb08819bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:21.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.079+0000 7fb08cb18640 1 --2- 172.21.15.105:0/2379422136 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb088106a80 0x7fb08819bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.079+0000 7fb08cb18640 1 -- 172.21.15.105:0/2379422136 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb08819c2a0 con 0x7fb088101420 2024-09-16T02:04:21.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.079+0000 7fb08db1a640 1 --2- 172.21.15.105:0/2379422136 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb088106a80 0x7fb08819bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:04:21.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.079+0000 7fb08cb18640 1 --2- 172.21.15.105:0/2379422136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb088101420 0x7fb088197830 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb08400b4a0 tx=0x7fb08400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:21.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.079+0000 7fb07a7fc640 1 -- 172.21.15.105:0/2379422136 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb084014070 con 0x7fb088101420 2024-09-16T02:04:21.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.079+0000 7fb08f57d640 1 -- 172.21.15.105:0/2379422136 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb08819c590 con 0x7fb088101420 2024-09-16T02:04:21.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.079+0000 7fb07a7fc640 1 -- 172.21.15.105:0/2379422136 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb084010ac0 con 0x7fb088101420 2024-09-16T02:04:21.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.079+0000 7fb07a7fc640 1 -- 172.21.15.105:0/2379422136 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb084017c30 con 0x7fb088101420 2024-09-16T02:04:21.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.080+0000 7fb08f57d640 1 -- 172.21.15.105:0/2379422136 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0881a3e70 con 0x7fb088101420 2024-09-16T02:04:21.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.082+0000 7fb07a7fc640 1 -- 172.21.15.105:0/2379422136 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fb084017070 con 0x7fb088101420 2024-09-16T02:04:21.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.082+0000 7fb091956740 1 Processor -- start 2024-09-16T02:04:21.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.082+0000 7fb091956740 1 -- start start 2024-09-16T02:04:21.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.082+0000 7fb091956740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbbed2ab40 0x55cbbed29090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:21.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.082+0000 7fb091956740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbbed295d0 0x55cbbed29a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:21.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.082+0000 7fb091956740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbbed36580 0x55cbbed38970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:21.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.082+0000 7fb091956740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55cbbec8f2f0 con 0x55cbbed2ab40 2024-09-16T02:04:21.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.082+0000 7fb091956740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55cbbec36cf0 con 0x55cbbed36580 2024-09-16T02:04:21.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.082+0000 7fb091956740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55cbbec8b2b0 con 0x55cbbed295d0 2024-09-16T02:04:21.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.083+0000 7fb08db1a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbbed2ab40 0x55cbbed29090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:21.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.083+0000 7fb08db1a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbbed2ab40 0x55cbbed29090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57162/0 (socket says 172.21.15.105:57162) 2024-09-16T02:04:21.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.083+0000 7fb08db1a640 1 -- 172.21.15.105:0/1629568261 learned_addr learned my addr 172.21.15.105:0/1629568261 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:21.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.083+0000 7fb07a7fc640 1 --2- 172.21.15.105:0/2379422136 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb0740a3240 0x7fb0740a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:21.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.083+0000 7fb08db1a640 1 -- 172.21.15.105:0/1629568261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbbed295d0 msgr2=0x55cbbed29a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:21.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.083+0000 7fb08db1a640 1 --2- 172.21.15.105:0/1629568261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbbed295d0 0x55cbbed29a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.083+0000 7fb08db1a640 1 -- 172.21.15.105:0/1629568261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbbed36580 msgr2=0x55cbbed38970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:21.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.083+0000 7fb08db1a640 1 --2- 172.21.15.105:0/1629568261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbbed36580 0x55cbbed38970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.083+0000 7fb08db1a640 1 -- 172.21.15.105:0/1629568261 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbbec1b3a0 con 0x55cbbed2ab40 2024-09-16T02:04:21.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.083+0000 7fb08db1a640 1 --2- 172.21.15.105:0/2379422136 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb0740a3240 0x7fb0740a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:21.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.083+0000 7fb07a7fc640 1 -- 172.21.15.105:0/2379422136 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fb0840cf030 con 0x7fb088101420 2024-09-16T02:04:21.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.083+0000 7fb08db1a640 1 --2- 172.21.15.105:0/1629568261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbbed2ab40 0x55cbbed29090 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb08000f390 tx=0x7fb08000f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5984e8dce3bd473 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:21.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.083+0000 7fb0737fe640 1 -- 172.21.15.105:0/1629568261 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08001a6d0 con 0x55cbbed2ab40 2024-09-16T02:04:21.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.083+0000 7fb0737fe640 1 -- 172.21.15.105:0/1629568261 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb08001ad20 con 0x55cbbed2ab40 2024-09-16T02:04:21.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.083+0000 7fb0737fe640 1 -- 172.21.15.105:0/1629568261 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb080010ad0 con 0x55cbbed2ab40 2024-09-16T02:04:21.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.083+0000 7fb08db1a640 1 --2- 172.21.15.105:0/2379422136 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb0740a3240 0x7fb0740a5700 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb080020000 tx=0x7fb08001d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:21.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.084+0000 7fb091956740 1 -- 172.21.15.105:0/1629568261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbbed2ab40 msgr2=0x55cbbed29090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:21.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.084+0000 7fb091956740 1 --2- 172.21.15.105:0/1629568261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbbed2ab40 0x55cbbed29090 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb08000f390 tx=0x7fb08000f850 comp rx=0 tx=0).stop 2024-09-16T02:04:21.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.084+0000 7fb091956740 1 -- 172.21.15.105:0/1629568261 shutdown_connections 2024-09-16T02:04:21.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.084+0000 7fb091956740 1 --2- 172.21.15.105:0/1629568261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbbed295d0 0x55cbbed29a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.084+0000 7fb091956740 1 --2- 172.21.15.105:0/1629568261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbbed36580 0x55cbbed38970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.084+0000 7fb091956740 1 --2- 172.21.15.105:0/1629568261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbbed2ab40 0x55cbbed29090 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.084+0000 7fb091956740 1 -- 172.21.15.105:0/1629568261 >> 172.21.15.105:0/1629568261 conn(0x55cbbecf2310 msgr2=0x55cbbed25c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:21.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.084+0000 7fb091956740 1 -- 172.21.15.105:0/1629568261 shutdown_connections 2024-09-16T02:04:21.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.084+0000 7fb091956740 1 -- 172.21.15.105:0/1629568261 wait complete. 2024-09-16T02:04:21.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.084+0000 7fb091956740 1 -- start start 2024-09-16T02:04:21.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.085+0000 7fb091956740 1 Processor -- start 2024-09-16T02:04:21.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.085+0000 7fb091956740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbbed295d0 0x55cbbed35fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:21.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.085+0000 7fb091956740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbbed2ab40 0x55cbbed33290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:21.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.085+0000 7fb091956740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbbed36580 0x55cbbed33a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:21.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.085+0000 7fb091956740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55cbbed39280 con 0x55cbbed2ab40 2024-09-16T02:04:21.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.085+0000 7fb091956740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55cbbec4a060 con 0x55cbbed295d0 2024-09-16T02:04:21.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.085+0000 7fb091956740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55cbbed39400 con 0x55cbbed36580 2024-09-16T02:04:21.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.085+0000 7fb091956740 10 client.0 _ll_register_callbacks cb 0x55cbbe99d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:04:21.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.085+0000 7fb08cb18640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbbed2ab40 0x55cbbed33290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:21.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.085+0000 7fb08cb18640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbbed2ab40 0x55cbbed33290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57170/0 (socket says 172.21.15.105:57170) 2024-09-16T02:04:21.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.085+0000 7fb08cb18640 1 -- 172.21.15.105:0/1245640286 learned_addr learned my addr 172.21.15.105:0/1245640286 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:21.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.085+0000 7fb08d319640 1 --2- 172.21.15.105:0/1245640286 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbbed295d0 0x55cbbed35fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:21.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.085+0000 7fb08db1a640 1 --2- 172.21.15.105:0/1245640286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbbed36580 0x55cbbed33a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:21.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.085+0000 7fb08cb18640 1 -- 172.21.15.105:0/1245640286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbbed36580 msgr2=0x55cbbed33a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:21.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.085+0000 7fb08cb18640 1 --2- 172.21.15.105:0/1245640286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbbed36580 0x55cbbed33a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.085+0000 7fb08cb18640 1 -- 172.21.15.105:0/1245640286 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbbed295d0 msgr2=0x55cbbed35fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:21.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.085+0000 7fb08cb18640 1 --2- 172.21.15.105:0/1245640286 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbbed295d0 0x55cbbed35fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.085+0000 7fb08cb18640 1 -- 172.21.15.105:0/1245640286 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbbea003c0 con 0x55cbbed2ab40 2024-09-16T02:04:21.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.085+0000 7fb08db1a640 1 --2- 172.21.15.105:0/1245640286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbbed36580 0x55cbbed33a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:04:21.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.086+0000 7fb08cb18640 1 --2- 172.21.15.105:0/1245640286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbbed2ab40 0x55cbbed33290 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb084097a50 tx=0x7fb08409b1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf1ed4338582c444 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:21.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.086+0000 7fb070ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:04:21.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.086+0000 7fb070ff9640 1 -- 172.21.15.105:0/1245640286 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08401e050 con 0x55cbbed2ab40 2024-09-16T02:04:21.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.086+0000 7fb070ff9640 1 -- 172.21.15.105:0/1245640286 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fb084097c60 con 0x55cbbed2ab40 2024-09-16T02:04:21.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.086+0000 7fb091956740 1 -- 172.21.15.105:0/1245640286 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb08001ad20 con 0x55cbbed2ab40 2024-09-16T02:04:21.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.086+0000 7fb070ff9640 1 -- 172.21.15.105:0/1245640286 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb0840a4260 con 0x55cbbed2ab40 2024-09-16T02:04:21.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.086+0000 7fb091956740 10 client.15582 fetch_fsmap learned FSMap version 28 2024-09-16T02:04:21.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.086+0000 7fb091956740 1 -- 172.21.15.105:0/1245640286 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cbbed44e00 con 0x55cbbed2ab40 2024-09-16T02:04:21.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.086+0000 7fb070ff9640 1 -- 172.21.15.105:0/1245640286 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb08001ad20 con 0x55cbbed2ab40 2024-09-16T02:04:21.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.086+0000 7fb070ff9640 1 -- 172.21.15.105:0/1245640286 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb0840a3e60 con 0x55cbbed2ab40 2024-09-16T02:04:21.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.086+0000 7fb091956740 10 client.15582 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:04:21.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.086+0000 7fb091956740 10 client.15582 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T02:04:21.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.086+0000 7fb091956740 20 client.15582 populate_metadata read hostname 'smithi105' 2024-09-16T02:04:21.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.086+0000 7fb091956740 1 --2- 172.21.15.105:0/1245640286 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55cbbed462c0 0x55cbbed486b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:21.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.086+0000 7fb091956740 4 client.15582 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:04:21.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.086+0000 7fb091956740 1 -- 172.21.15.105:0/1245640286 --> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbbec9c860 con 0x55cbbed462c0 2024-09-16T02:04:21.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.087+0000 7fb08d319640 1 --2- 172.21.15.105:0/1245640286 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55cbbed462c0 0x55cbbed486b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:21.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.087+0000 7fb070ff9640 10 client.15582 ms_handle_connect on v2:172.21.15.190:6838/1222573537 2024-09-16T02:04:21.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.087+0000 7fb08d319640 1 --2- 172.21.15.105:0/1245640286 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55cbbed462c0 0x55cbbed486b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d8ff45fdda9ea47f server_cookie=858ef32ec1c87f5 in_seq=0 out_seq=0 2024-09-16T02:04:21.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.088+0000 7fb070ff9640 1 -- 172.21.15.105:0/1245640286 <== mds.1 v2:172.21.15.190:6838/1222573537 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cbbec9c860 con 0x55cbbed462c0 2024-09-16T02:04:21.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.088+0000 7fb070ff9640 10 client.15582 handle_command_reply: tid=0 2024-09-16T02:04:21.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.088+0000 7fb091956740 1 client.15582 shutdown 2024-09-16T02:04:21.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.088+0000 7fb091956740 1 -- 172.21.15.105:0/1245640286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbbed2ab40 msgr2=0x55cbbed33290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:21.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.088+0000 7fb091956740 1 --2- 172.21.15.105:0/1245640286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbbed2ab40 0x55cbbed33290 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb084097a50 tx=0x7fb08409b1b0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.088+0000 7fb091956740 1 -- 172.21.15.105:0/1245640286 shutdown_connections 2024-09-16T02:04:21.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.088+0000 7fb091956740 1 --2- 172.21.15.105:0/1245640286 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55cbbed462c0 0x55cbbed486b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.088+0000 7fb091956740 1 --2- 172.21.15.105:0/1245640286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbbed36580 0x55cbbed33a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.088+0000 7fb091956740 1 --2- 172.21.15.105:0/1245640286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbbed2ab40 0x55cbbed33290 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.088+0000 7fb070ff9640 0 client.15582 ms_handle_reset on v2:172.21.15.190:6838/1222573537 2024-09-16T02:04:21.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.088+0000 7fb091956740 1 --2- 172.21.15.105:0/1245640286 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbbed295d0 0x55cbbed35fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.088+0000 7fb091956740 1 -- 172.21.15.105:0/1245640286 >> 172.21.15.105:0/1245640286 conn(0x55cbbecf2310 msgr2=0x55cbbed2b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:21.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.088+0000 7fb091956740 1 -- 172.21.15.105:0/1245640286 shutdown_connections 2024-09-16T02:04:21.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.088+0000 7fb091956740 1 -- 172.21.15.105:0/1245640286 wait complete. 2024-09-16T02:04:21.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.088+0000 7fb091956740 20 client.15582 trim_cache size 0 max 16384 2024-09-16T02:04:21.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.101+0000 7fb091956740 1 Processor -- start 2024-09-16T02:04:21.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.101+0000 7fb091956740 1 -- start start 2024-09-16T02:04:21.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.101+0000 7fb091956740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbbed295d0 0x55cbbed32270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:21.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.101+0000 7fb091956740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbbed2ab40 0x55cbbed4df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:21.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.101+0000 7fb091956740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbbed36580 0x55cbbed4e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:21.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.101+0000 7fb091956740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb0881ab4c0 con 0x55cbbed36580 2024-09-16T02:04:21.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.101+0000 7fb091956740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb0881ad000 con 0x55cbbed2ab40 2024-09-16T02:04:21.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.101+0000 7fb091956740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb0881abb90 con 0x55cbbed295d0 2024-09-16T02:04:21.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.101+0000 7fb08db1a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbbed36580 0x55cbbed4e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.101+0000 7fb08db1a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbbed36580 0x55cbbed4e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57180/0 (socket says 172.21.15.105:57180) 2024-09-16T02:04:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.101+0000 7fb08db1a640 1 -- 172.21.15.105:0/184877119 learned_addr learned my addr 172.21.15.105:0/184877119 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.101+0000 7fb08d319640 1 --2- 172.21.15.105:0/184877119 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbbed295d0 0x55cbbed32270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.101+0000 7fb08cb18640 1 --2- 172.21.15.105:0/184877119 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbbed2ab40 0x55cbbed4df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.101+0000 7fb08db1a640 1 -- 172.21.15.105:0/184877119 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbbed295d0 msgr2=0x55cbbed32270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.101+0000 7fb08db1a640 1 --2- 172.21.15.105:0/184877119 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbbed295d0 0x55cbbed32270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.101+0000 7fb08db1a640 1 -- 172.21.15.105:0/184877119 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbbed2ab40 msgr2=0x55cbbed4df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.101+0000 7fb08db1a640 1 --2- 172.21.15.105:0/184877119 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbbed2ab40 0x55cbbed4df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.101+0000 7fb08db1a640 1 -- 172.21.15.105:0/184877119 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbbed26920 con 0x55cbbed36580 2024-09-16T02:04:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.101+0000 7fb08cb18640 1 --2- 172.21.15.105:0/184877119 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbbed2ab40 0x55cbbed4df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:04:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.102+0000 7fb08db1a640 1 --2- 172.21.15.105:0/184877119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbbed36580 0x55cbbed4e470 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb0800093c0 tx=0x7fb080004a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c173360b5ff89719 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.102+0000 7fb072ffd640 1 -- 172.21.15.105:0/184877119 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0800094f0 con 0x55cbbed36580 2024-09-16T02:04:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.102+0000 7fb072ffd640 1 -- 172.21.15.105:0/184877119 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb080010d30 con 0x55cbbed36580 2024-09-16T02:04:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.102+0000 7fb072ffd640 1 -- 172.21.15.105:0/184877119 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08001a850 con 0x55cbbed36580 2024-09-16T02:04:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.102+0000 7fb091956740 1 -- 172.21.15.105:0/184877119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbbed36580 msgr2=0x55cbbed4e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.102+0000 7fb091956740 1 --2- 172.21.15.105:0/184877119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbbed36580 0x55cbbed4e470 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb0800093c0 tx=0x7fb080004a70 comp rx=0 tx=0).stop 2024-09-16T02:04:21.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.102+0000 7fb091956740 1 -- 172.21.15.105:0/184877119 shutdown_connections 2024-09-16T02:04:21.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.102+0000 7fb091956740 1 --2- 172.21.15.105:0/184877119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbbed36580 0x55cbbed4e470 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.102+0000 7fb091956740 1 --2- 172.21.15.105:0/184877119 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbbed2ab40 0x55cbbed4df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.102+0000 7fb091956740 1 --2- 172.21.15.105:0/184877119 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbbed295d0 0x55cbbed32270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.102+0000 7fb091956740 1 -- 172.21.15.105:0/184877119 >> 172.21.15.105:0/184877119 conn(0x55cbbecf2310 msgr2=0x55cbbed31b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:21.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.103+0000 7fb091956740 1 -- 172.21.15.105:0/184877119 shutdown_connections 2024-09-16T02:04:21.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.103+0000 7fb091956740 1 -- 172.21.15.105:0/184877119 wait complete. 2024-09-16T02:04:21.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.103+0000 7fb091956740 1 -- start start 2024-09-16T02:04:21.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.103+0000 7fb091956740 1 Processor -- start 2024-09-16T02:04:21.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.103+0000 7fb091956740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbbed295d0 0x55cbbed2e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:21.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.103+0000 7fb091956740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbbed2ab40 0x55cbbed2eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:21.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.103+0000 7fb091956740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbbed36580 0x55cbbed3e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:21.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.104+0000 7fb091956740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb0881ac930 con 0x55cbbed295d0 2024-09-16T02:04:21.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.104+0000 7fb091956740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb0881ac260 con 0x55cbbed2ab40 2024-09-16T02:04:21.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.104+0000 7fb091956740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb0881a94a0 con 0x55cbbed36580 2024-09-16T02:04:21.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.104+0000 7fb08d319640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbbed295d0 0x55cbbed2e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:21.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.104+0000 7fb091956740 10 client.0 _ll_register_callbacks cb 0x7fb0840175f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:04:21.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.104+0000 7fb08d319640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbbed295d0 0x55cbbed2e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57192/0 (socket says 172.21.15.105:57192) 2024-09-16T02:04:21.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.104+0000 7fb08d319640 1 -- 172.21.15.105:0/2771237226 learned_addr learned my addr 172.21.15.105:0/2771237226 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:21.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.104+0000 7fb08db1a640 1 --2- 172.21.15.105:0/2771237226 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbbed36580 0x55cbbed3e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:21.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.104+0000 7fb08cb18640 1 --2- 172.21.15.105:0/2771237226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbbed2ab40 0x55cbbed2eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:21.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.104+0000 7fb08d319640 1 -- 172.21.15.105:0/2771237226 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbbed36580 msgr2=0x55cbbed3e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:21.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.104+0000 7fb08d319640 1 --2- 172.21.15.105:0/2771237226 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbbed36580 0x55cbbed3e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.104+0000 7fb08d319640 1 -- 172.21.15.105:0/2771237226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbbed2ab40 msgr2=0x55cbbed2eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:21.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.104+0000 7fb08d319640 1 --2- 172.21.15.105:0/2771237226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbbed2ab40 0x55cbbed2eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.104+0000 7fb08d319640 1 -- 172.21.15.105:0/2771237226 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbbed3f0e0 con 0x55cbbed295d0 2024-09-16T02:04:21.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.104+0000 7fb08d319640 1 --2- 172.21.15.105:0/2771237226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbbed295d0 0x55cbbed2e5c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb07c02abd0 tx=0x7fb07c0042b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=139d6125b434dcc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:21.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.104+0000 7fb070ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:04:21.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.104+0000 7fb070ff9640 1 -- 172.21.15.105:0/2771237226 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07c048070 con 0x55cbbed295d0 2024-09-16T02:04:21.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.104+0000 7fb070ff9640 1 -- 172.21.15.105:0/2771237226 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fb07c0314a0 con 0x55cbbed295d0 2024-09-16T02:04:21.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.104+0000 7fb091956740 1 -- 172.21.15.105:0/2771237226 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb080010d30 con 0x55cbbed295d0 2024-09-16T02:04:21.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.105+0000 7fb070ff9640 1 -- 172.21.15.105:0/2771237226 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb07c035040 con 0x55cbbed295d0 2024-09-16T02:04:21.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.105+0000 7fb091956740 10 client.15588 fetch_fsmap learned FSMap version 28 2024-09-16T02:04:21.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.105+0000 7fb091956740 1 -- 172.21.15.105:0/2771237226 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cbbedb3300 con 0x55cbbed295d0 2024-09-16T02:04:21.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.105+0000 7fb070ff9640 1 -- 172.21.15.105:0/2771237226 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb080010d30 con 0x55cbbed295d0 2024-09-16T02:04:21.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.105+0000 7fb070ff9640 1 -- 172.21.15.105:0/2771237226 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb07c03d450 con 0x55cbbed295d0 2024-09-16T02:04:21.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.105+0000 7fb091956740 10 client.15588 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:04:21.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.105+0000 7fb091956740 10 client.15588 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T02:04:21.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.105+0000 7fb091956740 20 client.15588 populate_metadata read hostname 'smithi105' 2024-09-16T02:04:21.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.105+0000 7fb091956740 1 --2- 172.21.15.105:0/2771237226 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55cbbedb38b0 0x55cbbedb5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:21.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.105+0000 7fb091956740 4 client.15588 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T02:04:21.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.105+0000 7fb091956740 1 -- 172.21.15.105:0/2771237226 --> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cbbec96d70 con 0x55cbbedb38b0 2024-09-16T02:04:21.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.105+0000 7fb08cb18640 1 --2- 172.21.15.105:0/2771237226 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55cbbedb38b0 0x55cbbedb5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:21.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.106+0000 7fb08cb18640 1 --2- 172.21.15.105:0/2771237226 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55cbbedb38b0 0x55cbbedb5ca0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc28e2c44fdfef17 server_cookie=e8388a728f345cef in_seq=0 out_seq=0 2024-09-16T02:04:21.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.106+0000 7fb070ff9640 10 client.15588 ms_handle_connect on v2:172.21.15.190:6838/1222573537 2024-09-16T02:04:21.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.106+0000 7fb070ff9640 1 -- 172.21.15.105:0/2771237226 <== mds.1 v2:172.21.15.190:6838/1222573537 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cbbec96d70 con 0x55cbbedb38b0 2024-09-16T02:04:21.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.106+0000 7fb070ff9640 10 client.15588 handle_command_reply: tid=0 2024-09-16T02:04:21.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.106+0000 7fb091956740 1 client.15588 shutdown 2024-09-16T02:04:21.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.106+0000 7fb091956740 1 -- 172.21.15.105:0/2771237226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbbed295d0 msgr2=0x55cbbed2e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:21.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.106+0000 7fb091956740 1 --2- 172.21.15.105:0/2771237226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbbed295d0 0x55cbbed2e5c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb07c02abd0 tx=0x7fb07c0042b0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.106+0000 7fb091956740 1 -- 172.21.15.105:0/2771237226 shutdown_connections 2024-09-16T02:04:21.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.106+0000 7fb091956740 1 --2- 172.21.15.105:0/2771237226 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55cbbedb38b0 0x55cbbedb5ca0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.106+0000 7fb091956740 1 --2- 172.21.15.105:0/2771237226 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbbed36580 0x55cbbed3e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.106+0000 7fb091956740 1 --2- 172.21.15.105:0/2771237226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbbed2ab40 0x55cbbed2eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.106+0000 7fb091956740 1 --2- 172.21.15.105:0/2771237226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbbed295d0 0x55cbbed2e5c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.106+0000 7fb091956740 1 -- 172.21.15.105:0/2771237226 >> 172.21.15.105:0/2771237226 conn(0x55cbbecf2310 msgr2=0x55cbbed244e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:21.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.106+0000 7fb070ff9640 0 client.15588 ms_handle_reset on v2:172.21.15.190:6838/1222573537 2024-09-16T02:04:21.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.107+0000 7fb091956740 1 -- 172.21.15.105:0/2771237226 shutdown_connections 2024-09-16T02:04:21.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.107+0000 7fb091956740 1 -- 172.21.15.105:0/2771237226 wait complete. 2024-09-16T02:04:21.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.107+0000 7fb091956740 20 client.15588 trim_cache size 0 max 16384 2024-09-16T02:04:21.124 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-09-16T02:04:21.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.107+0000 7fb08f57d640 1 -- 172.21.15.105:0/2379422136 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb0740a3240 msgr2=0x7fb0740a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:21.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.107+0000 7fb08f57d640 1 --2- 172.21.15.105:0/2379422136 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb0740a3240 0x7fb0740a5700 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb080020000 tx=0x7fb08001d000 comp rx=0 tx=0).stop 2024-09-16T02:04:21.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.107+0000 7fb08f57d640 1 -- 172.21.15.105:0/2379422136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb088101420 msgr2=0x7fb088197830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:21.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.107+0000 7fb08f57d640 1 --2- 172.21.15.105:0/2379422136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb088101420 0x7fb088197830 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb08400b4a0 tx=0x7fb08400b960 comp rx=0 tx=0).stop 2024-09-16T02:04:21.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.107+0000 7fb08f57d640 1 -- 172.21.15.105:0/2379422136 shutdown_connections 2024-09-16T02:04:21.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.107+0000 7fb08f57d640 1 --2- 172.21.15.105:0/2379422136 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb0740a3240 0x7fb0740a5700 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.107+0000 7fb08f57d640 1 --2- 172.21.15.105:0/2379422136 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb088106a80 0x7fb08819bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.107+0000 7fb08f57d640 1 --2- 172.21.15.105:0/2379422136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb088101420 0x7fb088197830 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.107+0000 7fb08f57d640 1 --2- 172.21.15.105:0/2379422136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb088100a80 0x7fb0881972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.107+0000 7fb08f57d640 1 -- 172.21.15.105:0/2379422136 >> 172.21.15.105:0/2379422136 conn(0x7fb0880fc7a0 msgr2=0x7fb0880fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:21.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.107+0000 7fb08f57d640 1 -- 172.21.15.105:0/2379422136 shutdown_connections 2024-09-16T02:04:21.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:21.108+0000 7fb08f57d640 1 -- 172.21.15.105:0/2379422136 wait complete. 2024-09-16T02:04:21.834 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T02:04:22.080 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:22.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.521+0000 7f85f288b640 1 -- 172.21.15.105:0/2852603523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85ec103780 msgr2=0x7f85ec107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:22.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.521+0000 7f85f288b640 1 --2- 172.21.15.105:0/2852603523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85ec103780 0x7f85ec107bc0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f85d4009930 tx=0x7f85d402a420 comp rx=0 tx=0).stop 2024-09-16T02:04:22.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.521+0000 7f85f288b640 1 -- 172.21.15.105:0/2852603523 shutdown_connections 2024-09-16T02:04:22.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.521+0000 7f85f288b640 1 --2- 172.21.15.105:0/2852603523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85ec108300 0x7f85ec10ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.521+0000 7f85f288b640 1 --2- 172.21.15.105:0/2852603523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85ec103780 0x7f85ec107bc0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.521+0000 7f85f288b640 1 --2- 172.21.15.105:0/2852603523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f85ec102dd0 0x7f85ec1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.521+0000 7f85f288b640 1 -- 172.21.15.105:0/2852603523 >> 172.21.15.105:0/2852603523 conn(0x7f85ec0fc7d0 msgr2=0x7f85ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:22.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.521+0000 7f85f288b640 1 -- 172.21.15.105:0/2852603523 shutdown_connections 2024-09-16T02:04:22.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.521+0000 7f85f288b640 1 -- 172.21.15.105:0/2852603523 wait complete. 2024-09-16T02:04:22.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.522+0000 7f85f288b640 1 Processor -- start 2024-09-16T02:04:22.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.522+0000 7f85f288b640 1 -- start start 2024-09-16T02:04:22.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.522+0000 7f85f288b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85ec102dd0 0x7f85ec1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:22.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.522+0000 7f85f288b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85ec103780 0x7f85ec198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:22.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.522+0000 7f85f288b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f85ec108300 0x7f85ec19cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:22.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.522+0000 7f85f288b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec111f80 con 0x7f85ec102dd0 2024-09-16T02:04:22.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.522+0000 7f85f288b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec111e00 con 0x7f85ec108300 2024-09-16T02:04:22.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.522+0000 7f85f288b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec112100 con 0x7f85ec103780 2024-09-16T02:04:22.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.522+0000 7f85ebfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85ec102dd0 0x7f85ec1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:22.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.522+0000 7f85ebfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85ec102dd0 0x7f85ec1984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57224/0 (socket says 172.21.15.105:57224) 2024-09-16T02:04:22.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.522+0000 7f85ebfff640 1 -- 172.21.15.105:0/1721991176 learned_addr learned my addr 172.21.15.105:0/1721991176 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:22.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.522+0000 7f85e3fff640 1 --2- 172.21.15.105:0/1721991176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85ec103780 0x7f85ec198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:22.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.522+0000 7f85f0e28640 1 --2- 172.21.15.105:0/1721991176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f85ec108300 0x7f85ec19cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:22.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.523+0000 7f85ebfff640 1 -- 172.21.15.105:0/1721991176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85ec103780 msgr2=0x7f85ec198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:22.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.523+0000 7f85ebfff640 1 --2- 172.21.15.105:0/1721991176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85ec103780 0x7f85ec198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.523+0000 7f85ebfff640 1 -- 172.21.15.105:0/1721991176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f85ec108300 msgr2=0x7f85ec19cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:22.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.523+0000 7f85ebfff640 1 --2- 172.21.15.105:0/1721991176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f85ec108300 0x7f85ec19cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.523+0000 7f85ebfff640 1 -- 172.21.15.105:0/1721991176 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85ec19d490 con 0x7f85ec102dd0 2024-09-16T02:04:22.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.523+0000 7f85e3fff640 1 --2- 172.21.15.105:0/1721991176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85ec103780 0x7f85ec198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:04:22.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.523+0000 7f85f0e28640 1 --2- 172.21.15.105:0/1721991176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f85ec108300 0x7f85ec19cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:04:22.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.523+0000 7f85ebfff640 1 --2- 172.21.15.105:0/1721991176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85ec102dd0 0x7f85ec1984e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f85dc00b4a0 tx=0x7f85dc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:22.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.523+0000 7f85e9ffb640 1 -- 172.21.15.105:0/1721991176 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc014070 con 0x7f85ec102dd0 2024-09-16T02:04:22.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.523+0000 7f85f288b640 1 -- 172.21.15.105:0/1721991176 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85ec19d780 con 0x7f85ec102dd0 2024-09-16T02:04:22.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.523+0000 7f85e9ffb640 1 -- 172.21.15.105:0/1721991176 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85dc010ac0 con 0x7f85ec102dd0 2024-09-16T02:04:22.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.523+0000 7f85e9ffb640 1 -- 172.21.15.105:0/1721991176 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc017c30 con 0x7f85ec102dd0 2024-09-16T02:04:22.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.523+0000 7f85f288b640 1 -- 172.21.15.105:0/1721991176 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85ec1a5060 con 0x7f85ec102dd0 2024-09-16T02:04:22.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.525+0000 7f85e9ffb640 1 -- 172.21.15.105:0/1721991176 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f85dc017070 con 0x7f85ec102dd0 2024-09-16T02:04:22.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.526+0000 7f85f4c64740 1 Processor -- start 2024-09-16T02:04:22.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.526+0000 7f85f4c64740 1 -- start start 2024-09-16T02:04:22.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.526+0000 7f85f4c64740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a433b40 0x564c1a432090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:22.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.526+0000 7f85f4c64740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c1a4325d0 0x564c1a432a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:22.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.526+0000 7f85f4c64740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c1a43d370 0x564c1a43f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:22.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.526+0000 7f85f4c64740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564c1a3982f0 con 0x564c1a43d370 2024-09-16T02:04:22.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.526+0000 7f85f4c64740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564c1a33fcf0 con 0x564c1a433b40 2024-09-16T02:04:22.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.526+0000 7f85f4c64740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564c1a3942b0 con 0x564c1a4325d0 2024-09-16T02:04:22.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.526+0000 7f85e9ffb640 1 --2- 172.21.15.105:0/1721991176 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f85d80a3240 0x7f85d80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:22.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.526+0000 7f85e3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c1a43d370 0x564c1a43f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:22.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.526+0000 7f85e3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c1a43d370 0x564c1a43f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57228/0 (socket says 172.21.15.105:57228) 2024-09-16T02:04:22.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.526+0000 7f85e3fff640 1 -- 172.21.15.105:0/46091608 learned_addr learned my addr 172.21.15.105:0/46091608 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:22.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.526+0000 7f85ebfff640 1 --2- 172.21.15.105:0/46091608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c1a4325d0 0x564c1a432a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:22.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.526+0000 7f85f0e28640 1 --2- 172.21.15.105:0/1721991176 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f85d80a3240 0x7f85d80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:22.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.526+0000 7f85f0e28640 1 --2- 172.21.15.105:0/46091608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a433b40 0x564c1a432090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:22.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.526+0000 7f85e9ffb640 1 -- 172.21.15.105:0/1721991176 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f85dc0cf100 con 0x7f85ec102dd0 2024-09-16T02:04:22.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.526+0000 7f85e3fff640 1 -- 172.21.15.105:0/46091608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c1a4325d0 msgr2=0x564c1a432a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:22.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.526+0000 7f85e3fff640 1 --2- 172.21.15.105:0/46091608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c1a4325d0 0x564c1a432a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.526+0000 7f85e3fff640 1 -- 172.21.15.105:0/46091608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a433b40 msgr2=0x564c1a432090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:22.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.526+0000 7f85e3fff640 1 --2- 172.21.15.105:0/46091608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a433b40 0x564c1a432090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.526+0000 7f85e3fff640 1 -- 172.21.15.105:0/46091608 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c1a3243a0 con 0x564c1a43d370 2024-09-16T02:04:22.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.526+0000 7f85ebfff640 1 --2- 172.21.15.105:0/46091608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c1a4325d0 0x564c1a432a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:04:22.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.526+0000 7f85f0e28640 1 --2- 172.21.15.105:0/46091608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a433b40 0x564c1a432090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:04:22.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.527+0000 7f85f0e28640 1 --2- 172.21.15.105:0/1721991176 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f85d80a3240 0x7f85d80a5700 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f85e4007fd0 tx=0x7f85e40074b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:22.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.527+0000 7f85e3fff640 1 --2- 172.21.15.105:0/46091608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c1a43d370 0x564c1a43f760 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f85ec0697d0 tx=0x7f85d4031cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b33030ed750e312 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:22.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.527+0000 7f85e27fc640 1 -- 172.21.15.105:0/46091608 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d403f070 con 0x564c1a43d370 2024-09-16T02:04:22.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.527+0000 7f85e27fc640 1 -- 172.21.15.105:0/46091608 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85d40316a0 con 0x564c1a43d370 2024-09-16T02:04:22.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.527+0000 7f85e27fc640 1 -- 172.21.15.105:0/46091608 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4035070 con 0x564c1a43d370 2024-09-16T02:04:22.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.527+0000 7f85f4c64740 1 -- 172.21.15.105:0/46091608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c1a43d370 msgr2=0x564c1a43f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:22.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.527+0000 7f85f4c64740 1 --2- 172.21.15.105:0/46091608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c1a43d370 0x564c1a43f760 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f85ec0697d0 tx=0x7f85d4031cb0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.527+0000 7f85f4c64740 1 -- 172.21.15.105:0/46091608 shutdown_connections 2024-09-16T02:04:22.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.527+0000 7f85f4c64740 1 --2- 172.21.15.105:0/46091608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c1a4325d0 0x564c1a432a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.528+0000 7f85f4c64740 1 --2- 172.21.15.105:0/46091608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c1a43d370 0x564c1a43f760 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.528+0000 7f85f4c64740 1 --2- 172.21.15.105:0/46091608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a433b40 0x564c1a432090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.528+0000 7f85f4c64740 1 -- 172.21.15.105:0/46091608 >> 172.21.15.105:0/46091608 conn(0x564c1a3fb310 msgr2=0x564c1a42ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:22.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.528+0000 7f85f4c64740 1 -- 172.21.15.105:0/46091608 shutdown_connections 2024-09-16T02:04:22.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.528+0000 7f85f4c64740 1 -- 172.21.15.105:0/46091608 wait complete. 2024-09-16T02:04:22.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.528+0000 7f85f4c64740 1 -- start start 2024-09-16T02:04:22.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.528+0000 7f85f4c64740 1 Processor -- start 2024-09-16T02:04:22.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.528+0000 7f85f4c64740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a4325d0 0x564c1a43cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:22.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.528+0000 7f85f4c64740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c1a433b40 0x564c1a43a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:22.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.528+0000 7f85f4c64740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c1a43d370 0x564c1a43a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:22.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.528+0000 7f85f4c64740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564c1a440010 con 0x564c1a43d370 2024-09-16T02:04:22.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.528+0000 7f85f4c64740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564c1a353060 con 0x564c1a4325d0 2024-09-16T02:04:22.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.528+0000 7f85f4c64740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564c1a440190 con 0x564c1a433b40 2024-09-16T02:04:22.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.528+0000 7f85f4c64740 10 client.0 _ll_register_callbacks cb 0x564c1a0a68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:04:22.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.528+0000 7f85f0e28640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c1a43d370 0x564c1a43a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:22.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.528+0000 7f85f0e28640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c1a43d370 0x564c1a43a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57240/0 (socket says 172.21.15.105:57240) 2024-09-16T02:04:22.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.529+0000 7f85f0e28640 1 -- 172.21.15.105:0/1053250507 learned_addr learned my addr 172.21.15.105:0/1053250507 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:22.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.529+0000 7f85f0e28640 1 -- 172.21.15.105:0/1053250507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c1a433b40 msgr2=0x564c1a43a030 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T02:04:22.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.529+0000 7f85f0e28640 1 --2- 172.21.15.105:0/1053250507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c1a433b40 0x564c1a43a030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.529+0000 7f85f0e28640 1 -- 172.21.15.105:0/1053250507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a4325d0 msgr2=0x564c1a43cd80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T02:04:22.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.529+0000 7f85f0e28640 1 --2- 172.21.15.105:0/1053250507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a4325d0 0x564c1a43cd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.529+0000 7f85f0e28640 1 -- 172.21.15.105:0/1053250507 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c1a1093c0 con 0x564c1a43d370 2024-09-16T02:04:22.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.529+0000 7f85f0e28640 1 --2- 172.21.15.105:0/1053250507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c1a43d370 0x564c1a43a640 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f85e400ee20 tx=0x7f85e4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7820712fd34e0b57 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:22.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.529+0000 7f85cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:04:22.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.529+0000 7f85cb7fe640 1 -- 172.21.15.105:0/1053250507 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e4023030 con 0x564c1a43d370 2024-09-16T02:04:22.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.529+0000 7f85cb7fe640 1 -- 172.21.15.105:0/1053250507 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f85e4007660 con 0x564c1a43d370 2024-09-16T02:04:22.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.529+0000 7f85f4c64740 1 -- 172.21.15.105:0/1053250507 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85d40316a0 con 0x564c1a43d370 2024-09-16T02:04:22.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.529+0000 7f85cb7fe640 1 -- 172.21.15.105:0/1053250507 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f85e401f480 con 0x564c1a43d370 2024-09-16T02:04:22.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.529+0000 7f85f4c64740 10 client.15600 fetch_fsmap learned FSMap version 28 2024-09-16T02:04:22.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.529+0000 7f85f4c64740 1 -- 172.21.15.105:0/1053250507 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564c1a44bab0 con 0x564c1a43d370 2024-09-16T02:04:22.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.529+0000 7f85cb7fe640 1 -- 172.21.15.105:0/1053250507 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85d40316a0 con 0x564c1a43d370 2024-09-16T02:04:22.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.530+0000 7f85cb7fe640 1 -- 172.21.15.105:0/1053250507 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f85e401c050 con 0x564c1a43d370 2024-09-16T02:04:22.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.530+0000 7f85f4c64740 10 client.15600 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:04:22.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.530+0000 7f85f4c64740 10 client.15600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:04:22.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.530+0000 7f85f4c64740 20 client.15600 populate_metadata read hostname 'smithi105' 2024-09-16T02:04:22.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.530+0000 7f85f4c64740 1 --2- 172.21.15.105:0/1053250507 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x564c1a44d030 0x564c1a44f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:22.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.530+0000 7f85f4c64740 4 client.15600 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:04:22.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.530+0000 7f85f4c64740 1 -- 172.21.15.105:0/1053250507 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c1a3a5860 con 0x564c1a44d030 2024-09-16T02:04:22.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.530+0000 7f85ebfff640 1 --2- 172.21.15.105:0/1053250507 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x564c1a44d030 0x564c1a44f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:22.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.530+0000 7f85ebfff640 1 --2- 172.21.15.105:0/1053250507 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x564c1a44d030 0x564c1a44f4f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42b2aad835133573 server_cookie=f2ee65f5b71a266f in_seq=0 out_seq=0 2024-09-16T02:04:22.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.530+0000 7f85cb7fe640 10 client.15600 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:04:22.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.531+0000 7f85cb7fe640 1 -- 172.21.15.105:0/1053250507 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564c1a3a5860 con 0x564c1a44d030 2024-09-16T02:04:22.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.531+0000 7f85cb7fe640 10 client.15600 handle_command_reply: tid=0 2024-09-16T02:04:22.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.531+0000 7f85f4c64740 1 client.15600 shutdown 2024-09-16T02:04:22.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.531+0000 7f85f4c64740 1 -- 172.21.15.105:0/1053250507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c1a43d370 msgr2=0x564c1a43a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:22.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.531+0000 7f85f4c64740 1 --2- 172.21.15.105:0/1053250507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c1a43d370 0x564c1a43a640 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f85e400ee20 tx=0x7f85e4004ab0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.531+0000 7f85f4c64740 1 -- 172.21.15.105:0/1053250507 shutdown_connections 2024-09-16T02:04:22.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.531+0000 7f85f4c64740 1 --2- 172.21.15.105:0/1053250507 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x564c1a44d030 0x564c1a44f4f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.531+0000 7f85f4c64740 1 --2- 172.21.15.105:0/1053250507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c1a433b40 0x564c1a43a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.531+0000 7f85f4c64740 1 --2- 172.21.15.105:0/1053250507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c1a43d370 0x564c1a43a640 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.531+0000 7f85cb7fe640 0 client.15600 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:04:22.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.531+0000 7f85f4c64740 1 --2- 172.21.15.105:0/1053250507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a4325d0 0x564c1a43cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.531+0000 7f85f4c64740 1 -- 172.21.15.105:0/1053250507 >> 172.21.15.105:0/1053250507 conn(0x564c1a3fb310 msgr2=0x564c1a434960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:22.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.531+0000 7f85f4c64740 1 -- 172.21.15.105:0/1053250507 shutdown_connections 2024-09-16T02:04:22.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.531+0000 7f85f4c64740 1 -- 172.21.15.105:0/1053250507 wait complete. 2024-09-16T02:04:22.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.531+0000 7f85f4c64740 20 client.15600 trim_cache size 0 max 16384 2024-09-16T02:04:22.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.544+0000 7f85f4c64740 1 Processor -- start 2024-09-16T02:04:22.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.544+0000 7f85f4c64740 1 -- start start 2024-09-16T02:04:22.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.544+0000 7f85f4c64740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c1a4325d0 0x564c1a43bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:22.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.544+0000 7f85f4c64740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a433b40 0x564c1a435120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:22.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.544+0000 7f85f4c64740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c1a43d370 0x564c1a435660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:22.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.544+0000 7f85f4c64740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec1ad640 con 0x564c1a4325d0 2024-09-16T02:04:22.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.544+0000 7f85f4c64740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec1af180 con 0x564c1a433b40 2024-09-16T02:04:22.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.544+0000 7f85f4c64740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec1add10 con 0x564c1a43d370 2024-09-16T02:04:22.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.544+0000 7f85ebfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c1a4325d0 0x564c1a43bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:22.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.545+0000 7f85ebfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c1a4325d0 0x564c1a43bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57250/0 (socket says 172.21.15.105:57250) 2024-09-16T02:04:22.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.545+0000 7f85ebfff640 1 -- 172.21.15.105:0/2443329327 learned_addr learned my addr 172.21.15.105:0/2443329327 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:22.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.545+0000 7f85f0e28640 1 --2- 172.21.15.105:0/2443329327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c1a43d370 0x564c1a435660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:22.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.545+0000 7f85ebfff640 1 -- 172.21.15.105:0/2443329327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c1a43d370 msgr2=0x564c1a435660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:22.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.545+0000 7f85ebfff640 1 --2- 172.21.15.105:0/2443329327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c1a43d370 0x564c1a435660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.545+0000 7f85ebfff640 1 -- 172.21.15.105:0/2443329327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a433b40 msgr2=0x564c1a435120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:22.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.545+0000 7f85ebfff640 1 --2- 172.21.15.105:0/2443329327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a433b40 0x564c1a435120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.545+0000 7f85ebfff640 1 -- 172.21.15.105:0/2443329327 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c1a44fb70 con 0x564c1a4325d0 2024-09-16T02:04:22.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.545+0000 7f85f0e28640 1 --2- 172.21.15.105:0/2443329327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c1a43d370 0x564c1a435660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:04:22.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.545+0000 7f85ebfff640 1 --2- 172.21.15.105:0/2443329327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c1a4325d0 0x564c1a43bfa0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f85dc0ccba0 tx=0x7f85dc090ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=739091ede278b795 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:22.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.545+0000 7f85e1ffb640 1 -- 172.21.15.105:0/2443329327 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc09b6c0 con 0x564c1a4325d0 2024-09-16T02:04:22.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.545+0000 7f85e1ffb640 1 -- 172.21.15.105:0/2443329327 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85dc09fa70 con 0x564c1a4325d0 2024-09-16T02:04:22.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.545+0000 7f85e1ffb640 1 -- 172.21.15.105:0/2443329327 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc0997f0 con 0x564c1a4325d0 2024-09-16T02:04:22.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.546+0000 7f85f4c64740 1 -- 172.21.15.105:0/2443329327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c1a4325d0 msgr2=0x564c1a43bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:22.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.546+0000 7f85f4c64740 1 --2- 172.21.15.105:0/2443329327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c1a4325d0 0x564c1a43bfa0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f85dc0ccba0 tx=0x7f85dc090ba0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.546+0000 7f85f4c64740 1 -- 172.21.15.105:0/2443329327 shutdown_connections 2024-09-16T02:04:22.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.546+0000 7f85f4c64740 1 --2- 172.21.15.105:0/2443329327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c1a43d370 0x564c1a435660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.546+0000 7f85f4c64740 1 --2- 172.21.15.105:0/2443329327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a433b40 0x564c1a435120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.546+0000 7f85f4c64740 1 --2- 172.21.15.105:0/2443329327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c1a4325d0 0x564c1a43bfa0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.546+0000 7f85f4c64740 1 -- 172.21.15.105:0/2443329327 >> 172.21.15.105:0/2443329327 conn(0x564c1a3fb310 msgr2=0x564c1a43b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:22.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.546+0000 7f85f4c64740 1 -- 172.21.15.105:0/2443329327 shutdown_connections 2024-09-16T02:04:22.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.546+0000 7f85f4c64740 1 -- 172.21.15.105:0/2443329327 wait complete. 2024-09-16T02:04:22.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.546+0000 7f85f4c64740 1 -- start start 2024-09-16T02:04:22.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.546+0000 7f85f4c64740 1 Processor -- start 2024-09-16T02:04:22.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.546+0000 7f85f4c64740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c1a4325d0 0x564c1a436fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:22.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.546+0000 7f85f4c64740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c1a433b40 0x564c1a4374f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:22.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.546+0000 7f85f4c64740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a43d370 0x564c1a437a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:22.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.546+0000 7f85f4c64740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec1aeab0 con 0x564c1a4325d0 2024-09-16T02:04:22.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.547+0000 7f85f4c64740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec1ae3e0 con 0x564c1a43d370 2024-09-16T02:04:22.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.547+0000 7f85f4c64740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec1ab620 con 0x564c1a433b40 2024-09-16T02:04:22.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.547+0000 7f85f4c64740 10 client.0 _ll_register_callbacks cb 0x7f85e4007af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:04:22.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.547+0000 7f85ebfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c1a4325d0 0x564c1a436fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:22.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.547+0000 7f85ebfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c1a4325d0 0x564c1a436fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57260/0 (socket says 172.21.15.105:57260) 2024-09-16T02:04:22.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.547+0000 7f85ebfff640 1 -- 172.21.15.105:0/3086623829 learned_addr learned my addr 172.21.15.105:0/3086623829 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:22.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.547+0000 7f85e3fff640 1 --2- 172.21.15.105:0/3086623829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c1a433b40 0x564c1a4374f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:22.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.547+0000 7f85f0e28640 1 --2- 172.21.15.105:0/3086623829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a43d370 0x564c1a437a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:22.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.547+0000 7f85ebfff640 1 -- 172.21.15.105:0/3086623829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c1a433b40 msgr2=0x564c1a4374f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:22.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.547+0000 7f85ebfff640 1 --2- 172.21.15.105:0/3086623829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c1a433b40 0x564c1a4374f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.547+0000 7f85ebfff640 1 -- 172.21.15.105:0/3086623829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a43d370 msgr2=0x564c1a437a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:22.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.547+0000 7f85ebfff640 1 --2- 172.21.15.105:0/3086623829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a43d370 0x564c1a437a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.547+0000 7f85ebfff640 1 -- 172.21.15.105:0/3086623829 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c1a4b6d90 con 0x564c1a4325d0 2024-09-16T02:04:22.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.547+0000 7f85e3fff640 1 --2- 172.21.15.105:0/3086623829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c1a433b40 0x564c1a4374f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:04:22.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.547+0000 7f85ebfff640 1 --2- 172.21.15.105:0/3086623829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c1a4325d0 0x564c1a436fb0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f85dc009420 tx=0x7f85dc09b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7829a11b097b264e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:22.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.547+0000 7f85cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:04:22.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.547+0000 7f85cb7fe640 1 -- 172.21.15.105:0/3086623829 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc094070 con 0x564c1a4325d0 2024-09-16T02:04:22.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.547+0000 7f85cb7fe640 1 -- 172.21.15.105:0/3086623829 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f85dc0999c0 con 0x564c1a4325d0 2024-09-16T02:04:22.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.547+0000 7f85f4c64740 1 -- 172.21.15.105:0/3086623829 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85dc09fa70 con 0x564c1a4325d0 2024-09-16T02:04:22.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.547+0000 7f85cb7fe640 1 -- 172.21.15.105:0/3086623829 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f85dc0b3070 con 0x564c1a4325d0 2024-09-16T02:04:22.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.547+0000 7f85f4c64740 10 client.15606 fetch_fsmap learned FSMap version 28 2024-09-16T02:04:22.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.547+0000 7f85f4c64740 1 -- 172.21.15.105:0/3086623829 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564c1a4b7ee0 con 0x564c1a4325d0 2024-09-16T02:04:22.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.548+0000 7f85cb7fe640 1 -- 172.21.15.105:0/3086623829 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85dc09fa70 con 0x564c1a4325d0 2024-09-16T02:04:22.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.548+0000 7f85cb7fe640 1 -- 172.21.15.105:0/3086623829 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f85dc080e80 con 0x564c1a4325d0 2024-09-16T02:04:22.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.548+0000 7f85f4c64740 10 client.15606 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:04:22.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.548+0000 7f85f4c64740 10 client.15606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:04:22.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.548+0000 7f85f4c64740 20 client.15606 populate_metadata read hostname 'smithi105' 2024-09-16T02:04:22.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.548+0000 7f85f4c64740 1 --2- 172.21.15.105:0/3086623829 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x564c1a4b85a0 0x564c1a4ba990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:22.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.548+0000 7f85f4c64740 4 client.15606 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T02:04:22.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.548+0000 7f85f4c64740 1 -- 172.21.15.105:0/3086623829 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564c1a39fd70 con 0x564c1a4b85a0 2024-09-16T02:04:22.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.548+0000 7f85e3fff640 1 --2- 172.21.15.105:0/3086623829 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x564c1a4b85a0 0x564c1a4ba990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:22.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.548+0000 7f85e3fff640 1 --2- 172.21.15.105:0/3086623829 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x564c1a4b85a0 0x564c1a4ba990 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4630300af2ce2b server_cookie=46f1518cfe7e0920 in_seq=0 out_seq=0 2024-09-16T02:04:22.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.548+0000 7f85cb7fe640 10 client.15606 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:04:22.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.549+0000 7f85cb7fe640 1 -- 172.21.15.105:0/3086623829 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564c1a39fd70 con 0x564c1a4b85a0 2024-09-16T02:04:22.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.549+0000 7f85cb7fe640 10 client.15606 handle_command_reply: tid=0 2024-09-16T02:04:22.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.549+0000 7f85f4c64740 1 client.15606 shutdown 2024-09-16T02:04:22.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.549+0000 7f85f4c64740 1 -- 172.21.15.105:0/3086623829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c1a4325d0 msgr2=0x564c1a436fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:22.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.549+0000 7f85f4c64740 1 --2- 172.21.15.105:0/3086623829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c1a4325d0 0x564c1a436fb0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f85dc009420 tx=0x7f85dc09b6c0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.549+0000 7f85f4c64740 1 -- 172.21.15.105:0/3086623829 shutdown_connections 2024-09-16T02:04:22.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.549+0000 7f85f4c64740 1 --2- 172.21.15.105:0/3086623829 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x564c1a4b85a0 0x564c1a4ba990 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.549+0000 7f85f4c64740 1 --2- 172.21.15.105:0/3086623829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c1a433b40 0x564c1a4374f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.549+0000 7f85f4c64740 1 --2- 172.21.15.105:0/3086623829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a43d370 0x564c1a437a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.549+0000 7f85f4c64740 1 --2- 172.21.15.105:0/3086623829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c1a4325d0 0x564c1a436fb0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.549+0000 7f85cb7fe640 0 client.15606 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:04:22.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.549+0000 7f85f4c64740 1 -- 172.21.15.105:0/3086623829 >> 172.21.15.105:0/3086623829 conn(0x564c1a3fb310 msgr2=0x564c1a43c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:22.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.549+0000 7f85f4c64740 1 -- 172.21.15.105:0/3086623829 shutdown_connections 2024-09-16T02:04:22.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.549+0000 7f85f4c64740 1 -- 172.21.15.105:0/3086623829 wait complete. 2024-09-16T02:04:22.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.549+0000 7f85f4c64740 20 client.15606 trim_cache size 0 max 16384 2024-09-16T02:04:22.569 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-09-16T02:04:22.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.550+0000 7f85f288b640 1 -- 172.21.15.105:0/1721991176 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f85d80a3240 msgr2=0x7f85d80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:22.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.550+0000 7f85f288b640 1 --2- 172.21.15.105:0/1721991176 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f85d80a3240 0x7f85d80a5700 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f85e4007fd0 tx=0x7f85e40074b0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.550+0000 7f85f288b640 1 -- 172.21.15.105:0/1721991176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85ec102dd0 msgr2=0x7f85ec1984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:22.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.550+0000 7f85f288b640 1 --2- 172.21.15.105:0/1721991176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85ec102dd0 0x7f85ec1984e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f85dc00b4a0 tx=0x7f85dc00b960 comp rx=0 tx=0).stop 2024-09-16T02:04:22.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.550+0000 7f85f288b640 1 -- 172.21.15.105:0/1721991176 shutdown_connections 2024-09-16T02:04:22.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.550+0000 7f85f288b640 1 --2- 172.21.15.105:0/1721991176 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f85d80a3240 0x7f85d80a5700 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.550+0000 7f85f288b640 1 --2- 172.21.15.105:0/1721991176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85ec103780 0x7f85ec198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.550+0000 7f85f288b640 1 --2- 172.21.15.105:0/1721991176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f85ec108300 0x7f85ec19cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.550+0000 7f85f288b640 1 --2- 172.21.15.105:0/1721991176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85ec102dd0 0x7f85ec1984e0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.550+0000 7f85f288b640 1 -- 172.21.15.105:0/1721991176 >> 172.21.15.105:0/1721991176 conn(0x7f85ec0fc7d0 msgr2=0x7f85ec105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:22.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.550+0000 7f85f288b640 1 -- 172.21.15.105:0/1721991176 shutdown_connections 2024-09-16T02:04:22.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:22.550+0000 7f85f288b640 1 -- 172.21.15.105:0/1721991176 wait complete. 2024-09-16T02:04:22.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:22 smithi105 ceph-mon[31197]: pgmap v394: 97 pgs: 97 active+clean; 1.0 GiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 61 MiB/s wr, 25 op/s 2024-09-16T02:04:22.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:22 smithi190 ceph-mon[34762]: pgmap v394: 97 pgs: 97 active+clean; 1.0 GiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 61 MiB/s wr, 25 op/s 2024-09-16T02:04:22.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:22 smithi137 ceph-mon[34099]: pgmap v394: 97 pgs: 97 active+clean; 1.0 GiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 61 MiB/s wr, 25 op/s 2024-09-16T02:04:24.559 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T02:04:24.559 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T02:04:24.819 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:24.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:24 smithi190 ceph-mon[34762]: pgmap v395: 97 pgs: 97 active+clean; 1.0 GiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 24 MiB/s wr, 16 op/s 2024-09-16T02:04:24.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:24 smithi137 ceph-mon[34099]: pgmap v395: 97 pgs: 97 active+clean; 1.0 GiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 24 MiB/s wr, 16 op/s 2024-09-16T02:04:24.939 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:24 smithi105 ceph-mon[31197]: pgmap v395: 97 pgs: 97 active+clean; 1.0 GiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 24 MiB/s wr, 16 op/s 2024-09-16T02:04:25.255 INFO:tasks.workunit.client.0.smithi105.stdout: 2024-09-16T02:04:25.255 INFO:tasks.workunit.client.0.smithi105.stdout: 2024-09-16T02:04:25.255 INFO:tasks.workunit.client.0.smithi105.stdout: 2024-09-16T02:04:25.256 INFO:tasks.workunit.client.0.smithi105.stdout:iozone test complete. 2024-09-16T02:04:25.256 INFO:tasks.workunit.client.0.smithi105.stderr:+ iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-09-16T02:04:25.320 INFO:tasks.workunit.client.0.smithi105.stdout: Iozone: Performance Test of File I/O 2024-09-16T02:04:25.321 INFO:tasks.workunit.client.0.smithi105.stdout: Version $Revision: 3.494 $ 2024-09-16T02:04:25.321 INFO:tasks.workunit.client.0.smithi105.stdout: Compiled for 64 bit mode. 2024-09-16T02:04:25.321 INFO:tasks.workunit.client.0.smithi105.stdout: Build: linux-AMD64 2024-09-16T02:04:25.321 INFO:tasks.workunit.client.0.smithi105.stdout: 2024-09-16T02:04:25.321 INFO:tasks.workunit.client.0.smithi105.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-09-16T02:04:25.321 INFO:tasks.workunit.client.0.smithi105.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-09-16T02:04:25.321 INFO:tasks.workunit.client.0.smithi105.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-09-16T02:04:25.321 INFO:tasks.workunit.client.0.smithi105.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-09-16T02:04:25.321 INFO:tasks.workunit.client.0.smithi105.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-09-16T02:04:25.321 INFO:tasks.workunit.client.0.smithi105.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-09-16T02:04:25.321 INFO:tasks.workunit.client.0.smithi105.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-09-16T02:04:25.322 INFO:tasks.workunit.client.0.smithi105.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-09-16T02:04:25.322 INFO:tasks.workunit.client.0.smithi105.stdout: Alexey Skidanov, Sudhir Kumar. 2024-09-16T02:04:25.322 INFO:tasks.workunit.client.0.smithi105.stdout: 2024-09-16T02:04:25.322 INFO:tasks.workunit.client.0.smithi105.stdout: Run began: Mon Sep 16 02:04:25 2024 2024-09-16T02:04:25.322 INFO:tasks.workunit.client.0.smithi105.stdout: 2024-09-16T02:04:25.322 INFO:tasks.workunit.client.0.smithi105.stdout: Include close in write timing 2024-09-16T02:04:25.322 INFO:tasks.workunit.client.0.smithi105.stdout: Include fsync in write timing 2024-09-16T02:04:25.322 INFO:tasks.workunit.client.0.smithi105.stdout: File size set to 1048576 kB 2024-09-16T02:04:25.322 INFO:tasks.workunit.client.0.smithi105.stdout: Record Size 1024 kB 2024-09-16T02:04:25.322 INFO:tasks.workunit.client.0.smithi105.stdout: Command line used: iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-09-16T02:04:25.322 INFO:tasks.workunit.client.0.smithi105.stdout: Output is in kBytes/sec 2024-09-16T02:04:25.322 INFO:tasks.workunit.client.0.smithi105.stdout: Time Resolution = 0.000001 seconds. 2024-09-16T02:04:25.323 INFO:tasks.workunit.client.0.smithi105.stdout: Processor cache size set to 1024 kBytes. 2024-09-16T02:04:25.323 INFO:tasks.workunit.client.0.smithi105.stdout: Processor cache line size set to 32 bytes. 2024-09-16T02:04:25.323 INFO:tasks.workunit.client.0.smithi105.stdout: File stride size set to 17 * record size. 2024-09-16T02:04:25.323 INFO:tasks.workunit.client.0.smithi105.stdout: Throughput test with 1 process 2024-09-16T02:04:25.323 INFO:tasks.workunit.client.0.smithi105.stdout: Each process writes a 1048576 kByte file in 1024 kByte records 2024-09-16T02:04:25.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.620+0000 7ff4aa75b640 1 -- 172.21.15.105:0/1289958111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4a4100aa0 msgr2=0x7ff4a410ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:25.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.620+0000 7ff4aa75b640 1 --2- 172.21.15.105:0/1289958111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4a4100aa0 0x7ff4a410ea20 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff49c009ca0 tx=0x7ff49c02a8c0 comp rx=0 tx=0).stop 2024-09-16T02:04:25.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.620+0000 7ff4aa75b640 1 -- 172.21.15.105:0/1289958111 shutdown_connections 2024-09-16T02:04:25.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.620+0000 7ff4aa75b640 1 --2- 172.21.15.105:0/1289958111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4a4100aa0 0x7ff4a410ea20 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:25.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.620+0000 7ff4aa75b640 1 --2- 172.21.15.105:0/1289958111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4a4100100 0x7ff4a4100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:25.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.620+0000 7ff4aa75b640 1 --2- 172.21.15.105:0/1289958111 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4a4106b80 0x7ff4a4106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:25.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.620+0000 7ff4aa75b640 1 -- 172.21.15.105:0/1289958111 >> 172.21.15.105:0/1289958111 conn(0x7ff4a4077230 msgr2=0x7ff4a40fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:25.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.620+0000 7ff4aa75b640 1 -- 172.21.15.105:0/1289958111 shutdown_connections 2024-09-16T02:04:25.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.620+0000 7ff4aa75b640 1 -- 172.21.15.105:0/1289958111 wait complete. 2024-09-16T02:04:25.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.621+0000 7ff4aa75b640 1 Processor -- start 2024-09-16T02:04:25.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.621+0000 7ff4aa75b640 1 -- start start 2024-09-16T02:04:25.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.622+0000 7ff4aa75b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4a4100100 0x7ff4a4198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:25.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.622+0000 7ff4aa75b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4a4100aa0 0x7ff4a4198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:25.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.622+0000 7ff4aa75b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4a4106b80 0x7ff4a419cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:25.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.622+0000 7ff4aa75b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a4111fa0 con 0x7ff4a4100aa0 2024-09-16T02:04:25.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.622+0000 7ff4aa75b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a4111e20 con 0x7ff4a4100100 2024-09-16T02:04:25.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.622+0000 7ff4aa75b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a4112120 con 0x7ff4a4106b80 2024-09-16T02:04:25.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.623+0000 7ff4a37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4a4100aa0 0x7ff4a4198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:25.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.623+0000 7ff4a37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4a4100aa0 0x7ff4a4198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57288/0 (socket says 172.21.15.105:57288) 2024-09-16T02:04:25.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.623+0000 7ff4a37fe640 1 -- 172.21.15.105:0/3749874968 learned_addr learned my addr 172.21.15.105:0/3749874968 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:25.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.623+0000 7ff4a8cf8640 1 --2- 172.21.15.105:0/3749874968 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4a4106b80 0x7ff4a419cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:25.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.623+0000 7ff4a3fff640 1 --2- 172.21.15.105:0/3749874968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4a4100100 0x7ff4a4198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:25.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.623+0000 7ff4a37fe640 1 -- 172.21.15.105:0/3749874968 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4a4106b80 msgr2=0x7ff4a419cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:25.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.623+0000 7ff4a37fe640 1 --2- 172.21.15.105:0/3749874968 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4a4106b80 0x7ff4a419cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:25.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.623+0000 7ff4a37fe640 1 -- 172.21.15.105:0/3749874968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4a4100100 msgr2=0x7ff4a4198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:25.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.623+0000 7ff4a37fe640 1 --2- 172.21.15.105:0/3749874968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4a4100100 0x7ff4a4198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:25.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.623+0000 7ff4a37fe640 1 -- 172.21.15.105:0/3749874968 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a419d4d0 con 0x7ff4a4100aa0 2024-09-16T02:04:25.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.624+0000 7ff4a37fe640 1 --2- 172.21.15.105:0/3749874968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4a4100aa0 0x7ff4a4198a60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff49800b4a0 tx=0x7ff49800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:25.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.624+0000 7ff4a17fa640 1 -- 172.21.15.105:0/3749874968 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff498014070 con 0x7ff4a4100aa0 2024-09-16T02:04:25.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.624+0000 7ff4a17fa640 1 -- 172.21.15.105:0/3749874968 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff498010ac0 con 0x7ff4a4100aa0 2024-09-16T02:04:25.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.624+0000 7ff4a17fa640 1 -- 172.21.15.105:0/3749874968 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff498017c30 con 0x7ff4a4100aa0 2024-09-16T02:04:25.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.624+0000 7ff4aa75b640 1 -- 172.21.15.105:0/3749874968 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4a419d7c0 con 0x7ff4a4100aa0 2024-09-16T02:04:25.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.625+0000 7ff4aa75b640 1 -- 172.21.15.105:0/3749874968 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4a4101210 con 0x7ff4a4100aa0 2024-09-16T02:04:25.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.627+0000 7ff4a17fa640 1 -- 172.21.15.105:0/3749874968 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7ff498017070 con 0x7ff4a4100aa0 2024-09-16T02:04:25.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.628+0000 7ff4a17fa640 1 --2- 172.21.15.105:0/3749874968 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff48c0a32c0 0x7ff48c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:25.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.628+0000 7ff4a17fa640 1 -- 172.21.15.105:0/3749874968 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7ff4980cefb0 con 0x7ff4a4100aa0 2024-09-16T02:04:25.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.628+0000 7ff4a3fff640 1 --2- 172.21.15.105:0/3749874968 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff48c0a32c0 0x7ff48c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:25.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.629+0000 7ff4acb34740 1 Processor -- start 2024-09-16T02:04:25.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.629+0000 7ff4acb34740 1 -- start start 2024-09-16T02:04:25.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.629+0000 7ff4a3fff640 1 --2- 172.21.15.105:0/3749874968 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff48c0a32c0 0x7ff48c0a5780 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff49400a810 tx=0x7ff494008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:25.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.629+0000 7ff4acb34740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1a3dd1b60 0x55a1a3dd1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:25.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.629+0000 7ff4acb34740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1a3dd2480 0x55a1a3ddc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:25.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.629+0000 7ff4acb34740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1a3ddcd40 0x55a1a3ddf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:25.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.629+0000 7ff4acb34740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a1a3d37280 con 0x55a1a3dd2480 2024-09-16T02:04:25.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.629+0000 7ff4acb34740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a1a3cdeb10 con 0x55a1a3dd1b60 2024-09-16T02:04:25.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.629+0000 7ff4acb34740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a1a3d33240 con 0x55a1a3ddcd40 2024-09-16T02:04:25.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.629+0000 7ff4a37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1a3dd2480 0x55a1a3ddc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:25.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.629+0000 7ff4a37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1a3dd2480 0x55a1a3ddc800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57300/0 (socket says 172.21.15.105:57300) 2024-09-16T02:04:25.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.629+0000 7ff4a8cf8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1a3ddcd40 0x55a1a3ddf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:25.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.629+0000 7ff4a37fe640 1 -- 172.21.15.105:0/3451349206 learned_addr learned my addr 172.21.15.105:0/3451349206 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:25.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.629+0000 7ff4a3fff640 1 --2- 172.21.15.105:0/3451349206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1a3dd1b60 0x55a1a3dd1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:25.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.630+0000 7ff4a37fe640 1 -- 172.21.15.105:0/3451349206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1a3ddcd40 msgr2=0x55a1a3ddf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:25.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.630+0000 7ff4a37fe640 1 --2- 172.21.15.105:0/3451349206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1a3ddcd40 0x55a1a3ddf130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:25.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.630+0000 7ff4a37fe640 1 -- 172.21.15.105:0/3451349206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1a3dd1b60 msgr2=0x55a1a3dd1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:25.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.630+0000 7ff4a37fe640 1 --2- 172.21.15.105:0/3451349206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1a3dd1b60 0x55a1a3dd1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:25.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.630+0000 7ff4a37fe640 1 -- 172.21.15.105:0/3451349206 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1a3ceb1f0 con 0x55a1a3dd2480 2024-09-16T02:04:25.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.630+0000 7ff4a8cf8640 1 --2- 172.21.15.105:0/3451349206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1a3ddcd40 0x55a1a3ddf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:04:25.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.630+0000 7ff4a37fe640 1 --2- 172.21.15.105:0/3451349206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1a3dd2480 0x55a1a3ddc800 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff49809b270 tx=0x7ff49809af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ef82ebb33ba6db server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:25.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.630+0000 7ff48a7fc640 1 -- 172.21.15.105:0/3451349206 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4980b4070 con 0x55a1a3dd2480 2024-09-16T02:04:25.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.630+0000 7ff48a7fc640 1 -- 172.21.15.105:0/3451349206 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff49809f920 con 0x55a1a3dd2480 2024-09-16T02:04:25.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.630+0000 7ff48a7fc640 1 -- 172.21.15.105:0/3451349206 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49801e070 con 0x55a1a3dd2480 2024-09-16T02:04:25.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.631+0000 7ff4acb34740 1 -- 172.21.15.105:0/3451349206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1a3dd2480 msgr2=0x55a1a3ddc800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:25.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.631+0000 7ff4acb34740 1 --2- 172.21.15.105:0/3451349206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1a3dd2480 0x55a1a3ddc800 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff49809b270 tx=0x7ff49809af90 comp rx=0 tx=0).stop 2024-09-16T02:04:25.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.631+0000 7ff4acb34740 1 -- 172.21.15.105:0/3451349206 shutdown_connections 2024-09-16T02:04:25.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.631+0000 7ff4acb34740 1 --2- 172.21.15.105:0/3451349206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1a3ddcd40 0x55a1a3ddf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:25.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.631+0000 7ff4acb34740 1 --2- 172.21.15.105:0/3451349206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1a3dd2480 0x55a1a3ddc800 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:25.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.631+0000 7ff4acb34740 1 --2- 172.21.15.105:0/3451349206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1a3dd1b60 0x55a1a3dd1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:25.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.631+0000 7ff4acb34740 1 -- 172.21.15.105:0/3451349206 >> 172.21.15.105:0/3451349206 conn(0x55a1a3dce0f0 msgr2=0x55a1a3dce4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:25.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.631+0000 7ff4acb34740 1 -- 172.21.15.105:0/3451349206 shutdown_connections 2024-09-16T02:04:25.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.631+0000 7ff4acb34740 1 -- 172.21.15.105:0/3451349206 wait complete. 2024-09-16T02:04:25.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.632+0000 7ff4acb34740 1 -- start start 2024-09-16T02:04:25.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.632+0000 7ff4acb34740 1 Processor -- start 2024-09-16T02:04:25.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.633+0000 7ff4acb34740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1a3dd1b60 0x55a1a3dd9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:25.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.633+0000 7ff4a3fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1a3dd1b60 0x55a1a3dd9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:25.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.633+0000 7ff4a3fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1a3dd1b60 0x55a1a3dd9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.105:40876/0 (socket says 172.21.15.105:40876) 2024-09-16T02:04:25.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.633+0000 7ff4acb34740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1a3ddcd40 0x55a1a3dda2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:25.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.634+0000 7ff4a3fff640 1 -- 172.21.15.105:0/3163506007 learned_addr learned my addr 172.21.15.105:0/3163506007 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:25.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.634+0000 7ff4acb34740 1 --2- 172.21.15.105:0/3163506007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1a3dda830 0x55a1a3de8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:25.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.634+0000 7ff4a37fe640 1 --2- 172.21.15.105:0/3163506007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1a3ddcd40 0x55a1a3dda2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:25.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.634+0000 7ff4acb34740 1 -- 172.21.15.105:0/3163506007 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a1a3ddf910 con 0x55a1a3ddcd40 2024-09-16T02:04:25.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.634+0000 7ff4acb34740 1 -- 172.21.15.105:0/3163506007 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a1a3cf1f10 con 0x55a1a3dda830 2024-09-16T02:04:25.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.634+0000 7ff4acb34740 1 -- 172.21.15.105:0/3163506007 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a1a3ddfa90 con 0x55a1a3dd1b60 2024-09-16T02:04:25.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.634+0000 7ff4acb34740 10 client.0 _ll_register_callbacks cb 0x55a1a3cbaa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:04:25.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.635+0000 7ff4a3fff640 1 -- 172.21.15.105:0/3163506007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1a3dda830 msgr2=0x55a1a3de8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:25.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.635+0000 7ff4a3fff640 1 --2- 172.21.15.105:0/3163506007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1a3dda830 0x55a1a3de8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:25.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.635+0000 7ff4a3fff640 1 -- 172.21.15.105:0/3163506007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1a3ddcd40 msgr2=0x55a1a3dda2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:25.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.635+0000 7ff4a3fff640 1 --2- 172.21.15.105:0/3163506007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1a3ddcd40 0x55a1a3dda2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:25.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.635+0000 7ff4a3fff640 1 -- 172.21.15.105:0/3163506007 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1a3aa86a0 con 0x55a1a3dd1b60 2024-09-16T02:04:25.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.635+0000 7ff4a3fff640 1 --2- 172.21.15.105:0/3163506007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1a3dd1b60 0x55a1a3dd9db0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff494006e50 tx=0x7ff4940046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2599a3fc7d54465b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:25.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.635+0000 7ff4837fe640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-16T02:04:25.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.635+0000 7ff4837fe640 1 -- 172.21.15.105:0/3163506007 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff494013e30 con 0x55a1a3dd1b60 2024-09-16T02:04:25.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.635+0000 7ff4837fe640 1 -- 172.21.15.105:0/3163506007 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7ff4940115e0 con 0x55a1a3dd1b60 2024-09-16T02:04:25.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.635+0000 7ff4acb34740 1 -- 172.21.15.105:0/3163506007 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff49809f920 con 0x55a1a3dd1b60 2024-09-16T02:04:25.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.636+0000 7ff4837fe640 1 -- 172.21.15.105:0/3163506007 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff494011800 con 0x55a1a3dd1b60 2024-09-16T02:04:25.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.636+0000 7ff4acb34740 10 client.24970 fetch_fsmap learned FSMap version 28 2024-09-16T02:04:25.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.636+0000 7ff4acb34740 1 -- 172.21.15.105:0/3163506007 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a1a3debb50 con 0x55a1a3dd1b60 2024-09-16T02:04:25.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.636+0000 7ff4837fe640 1 -- 172.21.15.105:0/3163506007 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff49809f920 con 0x55a1a3dd1b60 2024-09-16T02:04:25.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.636+0000 7ff4837fe640 1 -- 172.21.15.105:0/3163506007 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff49401fa20 con 0x55a1a3dd1b60 2024-09-16T02:04:25.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.637+0000 7ff4acb34740 10 client.24970 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:04:25.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.637+0000 7ff4acb34740 10 client.24970 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:04:25.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.637+0000 7ff4acb34740 20 client.24970 populate_metadata read hostname 'smithi105' 2024-09-16T02:04:25.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.637+0000 7ff4acb34740 1 --2- 172.21.15.105:0/3163506007 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55a1a3ded390 0x55a1a3def850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:25.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.638+0000 7ff4acb34740 4 client.24970 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:04:25.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.638+0000 7ff4acb34740 1 -- 172.21.15.105:0/3163506007 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1a3d447f0 con 0x55a1a3ded390 2024-09-16T02:04:25.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.638+0000 7ff4a37fe640 1 --2- 172.21.15.105:0/3163506007 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55a1a3ded390 0x55a1a3def850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:25.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.639+0000 7ff4a37fe640 1 --2- 172.21.15.105:0/3163506007 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55a1a3ded390 0x55a1a3def850 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68d03ce7db5dc52c server_cookie=a82e002ca7658c10 in_seq=0 out_seq=0 2024-09-16T02:04:25.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.639+0000 7ff4837fe640 10 client.24970 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:04:25.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.640+0000 7ff4837fe640 1 -- 172.21.15.105:0/3163506007 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a1a3d447f0 con 0x55a1a3ded390 2024-09-16T02:04:25.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.640+0000 7ff4837fe640 10 client.24970 handle_command_reply: tid=0 2024-09-16T02:04:25.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.640+0000 7ff4acb34740 1 client.24970 shutdown 2024-09-16T02:04:25.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.641+0000 7ff4acb34740 1 -- 172.21.15.105:0/3163506007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1a3dd1b60 msgr2=0x55a1a3dd9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:25.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.641+0000 7ff4acb34740 1 --2- 172.21.15.105:0/3163506007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1a3dd1b60 0x55a1a3dd9db0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff494006e50 tx=0x7ff4940046c0 comp rx=0 tx=0).stop 2024-09-16T02:04:25.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.641+0000 7ff4acb34740 1 -- 172.21.15.105:0/3163506007 shutdown_connections 2024-09-16T02:04:25.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.641+0000 7ff4acb34740 1 --2- 172.21.15.105:0/3163506007 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55a1a3ded390 0x55a1a3def850 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:25.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.641+0000 7ff4acb34740 1 --2- 172.21.15.105:0/3163506007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1a3dda830 0x55a1a3de8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:25.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.641+0000 7ff4acb34740 1 --2- 172.21.15.105:0/3163506007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1a3ddcd40 0x55a1a3dda2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:25.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.641+0000 7ff4acb34740 1 --2- 172.21.15.105:0/3163506007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1a3dd1b60 0x55a1a3dd9db0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:25.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.641+0000 7ff4acb34740 1 -- 172.21.15.105:0/3163506007 >> 172.21.15.105:0/3163506007 conn(0x55a1a3dce0f0 msgr2=0x55a1a3dd3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:25.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.642+0000 7ff4837fe640 0 client.24970 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:04:25.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.642+0000 7ff4acb34740 1 -- 172.21.15.105:0/3163506007 shutdown_connections 2024-09-16T02:04:25.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.642+0000 7ff4acb34740 1 -- 172.21.15.105:0/3163506007 wait complete. 2024-09-16T02:04:25.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.642+0000 7ff4acb34740 20 client.24970 trim_cache size 0 max 16384 2024-09-16T02:04:25.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.670+0000 7ff4acb34740 1 Processor -- start 2024-09-16T02:04:25.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.670+0000 7ff4acb34740 1 -- start start 2024-09-16T02:04:25.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.670+0000 7ff4acb34740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1a3dd1b60 0x55a1a3de8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:25.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.670+0000 7ff4acb34740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1a3dda830 0x55a1a3ddbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:25.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.670+0000 7ff4acb34740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1a3ddcd40 0x55a1a3df5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:25.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.670+0000 7ff4acb34740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a41ab510 con 0x55a1a3dd1b60 2024-09-16T02:04:25.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.670+0000 7ff4acb34740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a41ad050 con 0x55a1a3ddcd40 2024-09-16T02:04:25.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.670+0000 7ff4acb34740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a41abbe0 con 0x55a1a3dda830 2024-09-16T02:04:25.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.670+0000 7ff4a3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1a3dd1b60 0x55a1a3de8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:25.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.670+0000 7ff4a3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1a3dd1b60 0x55a1a3de8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57316/0 (socket says 172.21.15.105:57316) 2024-09-16T02:04:25.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.670+0000 7ff4a3fff640 1 -- 172.21.15.105:0/1278768291 learned_addr learned my addr 172.21.15.105:0/1278768291 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:25.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.671+0000 7ff4a3fff640 1 -- 172.21.15.105:0/1278768291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1a3dda830 msgr2=0x55a1a3ddbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:25.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.671+0000 7ff4a3fff640 1 --2- 172.21.15.105:0/1278768291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1a3dda830 0x55a1a3ddbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:25.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.671+0000 7ff4a3fff640 1 -- 172.21.15.105:0/1278768291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1a3ddcd40 msgr2=0x55a1a3df5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:25.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.671+0000 7ff4a3fff640 1 --2- 172.21.15.105:0/1278768291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1a3ddcd40 0x55a1a3df5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:25.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.671+0000 7ff4a3fff640 1 -- 172.21.15.105:0/1278768291 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1a3dd8d80 con 0x55a1a3dd1b60 2024-09-16T02:04:25.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.671+0000 7ff4a3fff640 1 --2- 172.21.15.105:0/1278768291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1a3dd1b60 0x55a1a3de8b70 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff49400b010 tx=0x7ff494000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=697ea0ff4afebb2a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:25.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.671+0000 7ff489ffb640 1 -- 172.21.15.105:0/1278768291 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff494028650 con 0x55a1a3dd1b60 2024-09-16T02:04:25.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.671+0000 7ff489ffb640 1 -- 172.21.15.105:0/1278768291 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff494024070 con 0x55a1a3dd1b60 2024-09-16T02:04:25.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.671+0000 7ff489ffb640 1 -- 172.21.15.105:0/1278768291 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49401f680 con 0x55a1a3dd1b60 2024-09-16T02:04:25.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.673+0000 7ff4acb34740 1 -- 172.21.15.105:0/1278768291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1a3dd1b60 msgr2=0x55a1a3de8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:25.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.673+0000 7ff4acb34740 1 --2- 172.21.15.105:0/1278768291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1a3dd1b60 0x55a1a3de8b70 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff49400b010 tx=0x7ff494000bc0 comp rx=0 tx=0).stop 2024-09-16T02:04:25.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.673+0000 7ff4acb34740 1 -- 172.21.15.105:0/1278768291 shutdown_connections 2024-09-16T02:04:25.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.673+0000 7ff4acb34740 1 --2- 172.21.15.105:0/1278768291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1a3dda830 0x55a1a3ddbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:25.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.673+0000 7ff4acb34740 1 --2- 172.21.15.105:0/1278768291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1a3ddcd40 0x55a1a3df5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:25.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.673+0000 7ff4acb34740 1 --2- 172.21.15.105:0/1278768291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1a3dd1b60 0x55a1a3de8b70 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:25.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.673+0000 7ff4acb34740 1 -- 172.21.15.105:0/1278768291 >> 172.21.15.105:0/1278768291 conn(0x55a1a3dce0f0 msgr2=0x55a1a3ddb610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:25.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.673+0000 7ff4acb34740 1 -- 172.21.15.105:0/1278768291 shutdown_connections 2024-09-16T02:04:25.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.673+0000 7ff4acb34740 1 -- 172.21.15.105:0/1278768291 wait complete. 2024-09-16T02:04:25.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.674+0000 7ff4acb34740 1 -- start start 2024-09-16T02:04:25.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.675+0000 7ff4acb34740 1 Processor -- start 2024-09-16T02:04:25.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.675+0000 7ff4acb34740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1a3dd1b60 0x55a1a3dd6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:25.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.675+0000 7ff4acb34740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1a3dda830 0x55a1a3dd74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:25.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.676+0000 7ff4acb34740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1a3ddcd40 0x55a1a3dd79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:25.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.676+0000 7ff4a3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1a3dd1b60 0x55a1a3dd6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:25.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.676+0000 7ff4a3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1a3dd1b60 0x55a1a3dd6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57326/0 (socket says 172.21.15.105:57326) 2024-09-16T02:04:25.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.676+0000 7ff4acb34740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a41ac980 con 0x55a1a3dd1b60 2024-09-16T02:04:25.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.676+0000 7ff4a3fff640 1 -- 172.21.15.105:0/1574962599 learned_addr learned my addr 172.21.15.105:0/1574962599 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:25.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.676+0000 7ff4acb34740 1 -- 172.21.15.105:0/1574962599 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a41ac2b0 con 0x55a1a3dda830 2024-09-16T02:04:25.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.676+0000 7ff4acb34740 1 -- 172.21.15.105:0/1574962599 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a41a94f0 con 0x55a1a3ddcd40 2024-09-16T02:04:25.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.676+0000 7ff4acb34740 10 client.0 _ll_register_callbacks cb 0x7ff49400a4e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:04:25.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.676+0000 7ff4a3fff640 1 -- 172.21.15.105:0/1574962599 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1a3ddcd40 msgr2=0x55a1a3dd79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:25.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.676+0000 7ff4a3fff640 1 --2- 172.21.15.105:0/1574962599 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1a3ddcd40 0x55a1a3dd79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:25.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.676+0000 7ff4a3fff640 1 -- 172.21.15.105:0/1574962599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1a3dda830 msgr2=0x55a1a3dd74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:25.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.676+0000 7ff4a3fff640 1 --2- 172.21.15.105:0/1574962599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1a3dda830 0x55a1a3dd74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:25.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.676+0000 7ff4a3fff640 1 -- 172.21.15.105:0/1574962599 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1a3e5fac0 con 0x55a1a3dd1b60 2024-09-16T02:04:25.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.676+0000 7ff4a3fff640 1 --2- 172.21.15.105:0/1574962599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1a3dd1b60 0x55a1a3dd6f70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff49400e010 tx=0x7ff4940289e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee4225ebb88a3f1a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:25.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.676+0000 7ff4837fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:04:25.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.677+0000 7ff4837fe640 1 -- 172.21.15.105:0/1574962599 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff494028450 con 0x55a1a3dd1b60 2024-09-16T02:04:25.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.677+0000 7ff4837fe640 1 -- 172.21.15.105:0/1574962599 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7ff4940285f0 con 0x55a1a3dd1b60 2024-09-16T02:04:25.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.677+0000 7ff4acb34740 1 -- 172.21.15.105:0/1574962599 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff494024070 con 0x55a1a3dd1b60 2024-09-16T02:04:25.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.677+0000 7ff4837fe640 1 -- 172.21.15.105:0/1574962599 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff49401f420 con 0x55a1a3dd1b60 2024-09-16T02:04:25.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.677+0000 7ff4acb34740 10 client.15624 fetch_fsmap learned FSMap version 28 2024-09-16T02:04:25.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.677+0000 7ff4acb34740 1 -- 172.21.15.105:0/1574962599 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a1a3e60de0 con 0x55a1a3dd1b60 2024-09-16T02:04:25.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.677+0000 7ff4837fe640 1 -- 172.21.15.105:0/1574962599 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff494024070 con 0x55a1a3dd1b60 2024-09-16T02:04:25.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.678+0000 7ff4837fe640 1 -- 172.21.15.105:0/1574962599 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff49403e6f0 con 0x55a1a3dd1b60 2024-09-16T02:04:25.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.678+0000 7ff4acb34740 10 client.15624 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:04:25.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.678+0000 7ff4acb34740 10 client.15624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:04:25.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.678+0000 7ff4acb34740 20 client.15624 populate_metadata read hostname 'smithi105' 2024-09-16T02:04:25.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.678+0000 7ff4acb34740 1 --2- 172.21.15.105:0/1574962599 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55a1a3e61460 0x55a1a3e638a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:25.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.678+0000 7ff4acb34740 4 client.15624 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T02:04:25.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.678+0000 7ff4acb34740 1 -- 172.21.15.105:0/1574962599 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a1a3d3ed00 con 0x55a1a3e61460 2024-09-16T02:04:25.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.679+0000 7ff4a37fe640 1 --2- 172.21.15.105:0/1574962599 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55a1a3e61460 0x55a1a3e638a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:25.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.679+0000 7ff4a37fe640 1 --2- 172.21.15.105:0/1574962599 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55a1a3e61460 0x55a1a3e638a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62fd8162567e3a1c server_cookie=8de1420b47ebda4 in_seq=0 out_seq=0 2024-09-16T02:04:25.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.679+0000 7ff4837fe640 10 client.15624 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:04:25.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.680+0000 7ff4837fe640 1 -- 172.21.15.105:0/1574962599 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a1a3d3ed00 con 0x55a1a3e61460 2024-09-16T02:04:25.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.680+0000 7ff4837fe640 10 client.15624 handle_command_reply: tid=0 2024-09-16T02:04:25.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.681+0000 7ff4acb34740 1 client.15624 shutdown 2024-09-16T02:04:25.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.681+0000 7ff4acb34740 1 -- 172.21.15.105:0/1574962599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1a3dd1b60 msgr2=0x55a1a3dd6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:25.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.681+0000 7ff4acb34740 1 --2- 172.21.15.105:0/1574962599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1a3dd1b60 0x55a1a3dd6f70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff49400e010 tx=0x7ff4940289e0 comp rx=0 tx=0).stop 2024-09-16T02:04:25.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.681+0000 7ff4acb34740 1 -- 172.21.15.105:0/1574962599 shutdown_connections 2024-09-16T02:04:25.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.681+0000 7ff4acb34740 1 --2- 172.21.15.105:0/1574962599 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55a1a3e61460 0x55a1a3e638a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:25.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.681+0000 7ff4acb34740 1 --2- 172.21.15.105:0/1574962599 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1a3ddcd40 0x55a1a3dd79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:25.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.681+0000 7ff4acb34740 1 --2- 172.21.15.105:0/1574962599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1a3dda830 0x55a1a3dd74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:25.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.682+0000 7ff4acb34740 1 --2- 172.21.15.105:0/1574962599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1a3dd1b60 0x55a1a3dd6f70 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:25.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.682+0000 7ff4acb34740 1 -- 172.21.15.105:0/1574962599 >> 172.21.15.105:0/1574962599 conn(0x55a1a3dce0f0 msgr2=0x55a1a3ddbe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:25.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.682+0000 7ff4837fe640 0 client.15624 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:04:25.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.682+0000 7ff4acb34740 1 -- 172.21.15.105:0/1574962599 shutdown_connections 2024-09-16T02:04:25.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.682+0000 7ff4acb34740 1 -- 172.21.15.105:0/1574962599 wait complete. 2024-09-16T02:04:25.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.682+0000 7ff4acb34740 20 client.15624 trim_cache size 0 max 16384 2024-09-16T02:04:25.684 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-16T02:04:25.684 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-09-16T02:04:25.684 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "b7aa2a6e-a7fc-4553-8d71-c33dc2a1e7d4", 2024-09-16T02:04:25.685 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-09-16T02:04:25.685 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-16T02:04:25.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.683+0000 7ff4aa75b640 1 -- 172.21.15.105:0/3749874968 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff48c0a32c0 msgr2=0x7ff48c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:25.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.683+0000 7ff4aa75b640 1 --2- 172.21.15.105:0/3749874968 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff48c0a32c0 0x7ff48c0a5780 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff49400a810 tx=0x7ff494008040 comp rx=0 tx=0).stop 2024-09-16T02:04:25.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.683+0000 7ff4aa75b640 1 -- 172.21.15.105:0/3749874968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4a4100aa0 msgr2=0x7ff4a4198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:25.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.683+0000 7ff4aa75b640 1 --2- 172.21.15.105:0/3749874968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4a4100aa0 0x7ff4a4198a60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff49800b4a0 tx=0x7ff49800b960 comp rx=0 tx=0).stop 2024-09-16T02:04:25.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.683+0000 7ff4aa75b640 1 -- 172.21.15.105:0/3749874968 shutdown_connections 2024-09-16T02:04:25.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.683+0000 7ff4aa75b640 1 --2- 172.21.15.105:0/3749874968 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff48c0a32c0 0x7ff48c0a5780 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:25.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.683+0000 7ff4aa75b640 1 --2- 172.21.15.105:0/3749874968 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4a4106b80 0x7ff4a419cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:25.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.683+0000 7ff4aa75b640 1 --2- 172.21.15.105:0/3749874968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4a4100aa0 0x7ff4a4198a60 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:25.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.683+0000 7ff4aa75b640 1 --2- 172.21.15.105:0/3749874968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4a4100100 0x7ff4a4198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:25.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.683+0000 7ff4aa75b640 1 -- 172.21.15.105:0/3749874968 >> 172.21.15.105:0/3749874968 conn(0x7ff4a4077230 msgr2=0x7ff4a4104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:25.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.683+0000 7ff4aa75b640 1 -- 172.21.15.105:0/3749874968 shutdown_connections 2024-09-16T02:04:25.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:25.683+0000 7ff4aa75b640 1 -- 172.21.15.105:0/3749874968 wait complete. 2024-09-16T02:04:26.535 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T02:04:26.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:26 smithi105 ceph-mon[31197]: pgmap v396: 97 pgs: 97 active+clean; 1.0 GiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-09-16T02:04:26.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:26 smithi137 ceph-mon[34099]: pgmap v396: 97 pgs: 97 active+clean; 1.0 GiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-09-16T02:04:26.945 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:26.992 INFO:tasks.workunit.client.0.smithi105.stdout: 2024-09-16T02:04:26.992 INFO:tasks.workunit.client.0.smithi105.stdout: Children see throughput for 1 initial writers = 639711.38 kB/sec 2024-09-16T02:04:26.992 INFO:tasks.workunit.client.0.smithi105.stdout: Parent sees throughput for 1 initial writers = 639619.77 kB/sec 2024-09-16T02:04:26.992 INFO:tasks.workunit.client.0.smithi105.stdout: Min throughput per process = 639711.38 kB/sec 2024-09-16T02:04:26.992 INFO:tasks.workunit.client.0.smithi105.stdout: Max throughput per process = 639711.38 kB/sec 2024-09-16T02:04:26.992 INFO:tasks.workunit.client.0.smithi105.stdout: Avg throughput per process = 639711.38 kB/sec 2024-09-16T02:04:26.992 INFO:tasks.workunit.client.0.smithi105.stdout: Min xfer = 1048576.00 kB 2024-09-16T02:04:27.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:26 smithi190 ceph-mon[34762]: pgmap v396: 97 pgs: 97 active+clean; 1.0 GiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-09-16T02:04:27.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:27 smithi137 ceph-mon[34099]: scrub queued for path: / 2024-09-16T02:04:27.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:27 smithi137 ceph-mon[34099]: scrub summary: idle+waiting paths [/] 2024-09-16T02:04:27.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:27 smithi137 ceph-mon[34099]: scrub summary: active paths [/] 2024-09-16T02:04:27.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:27 smithi137 ceph-mon[34099]: scrub summary: idle+waiting paths [/] 2024-09-16T02:04:27.995 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:27 smithi105 ceph-mon[31197]: scrub queued for path: / 2024-09-16T02:04:27.995 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:27 smithi105 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-16T02:04:27.995 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:27 smithi105 ceph-mon[31197]: scrub summary: active paths [/] 2024-09-16T02:04:27.995 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:27 smithi105 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-16T02:04:28.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:27 smithi190 ceph-mon[34762]: scrub queued for path: / 2024-09-16T02:04:28.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:27 smithi190 ceph-mon[34762]: scrub summary: idle+waiting paths [/] 2024-09-16T02:04:28.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:27 smithi190 ceph-mon[34762]: scrub summary: active paths [/] 2024-09-16T02:04:28.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:27 smithi190 ceph-mon[34762]: scrub summary: idle+waiting paths [/] 2024-09-16T02:04:28.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.342+0000 7fb929b18640 1 -- 172.21.15.105:0/2073428712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb924100ca0 msgr2=0x7fb924106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:28.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.342+0000 7fb929b18640 1 --2- 172.21.15.105:0/2073428712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb924100ca0 0x7fb924106840 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb918009930 tx=0x7fb91802a420 comp rx=0 tx=0).stop 2024-09-16T02:04:28.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.342+0000 7fb929b18640 1 -- 172.21.15.105:0/2073428712 shutdown_connections 2024-09-16T02:04:28.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.342+0000 7fb929b18640 1 --2- 172.21.15.105:0/2073428712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb924106f50 0x7fb92410eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:28.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.342+0000 7fb929b18640 1 --2- 172.21.15.105:0/2073428712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb924100ca0 0x7fb924106840 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:28.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.342+0000 7fb929b18640 1 --2- 172.21.15.105:0/2073428712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9241002f0 0x7fb9241006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:28.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.342+0000 7fb929b18640 1 -- 172.21.15.105:0/2073428712 >> 172.21.15.105:0/2073428712 conn(0x7fb924077210 msgr2=0x7fb9240fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:28.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.342+0000 7fb929b18640 1 -- 172.21.15.105:0/2073428712 shutdown_connections 2024-09-16T02:04:28.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.342+0000 7fb929b18640 1 -- 172.21.15.105:0/2073428712 wait complete. 2024-09-16T02:04:28.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.343+0000 7fb929b18640 1 Processor -- start 2024-09-16T02:04:28.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.343+0000 7fb929b18640 1 -- start start 2024-09-16T02:04:28.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.343+0000 7fb929b18640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9241002f0 0x7fb924198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:28.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.343+0000 7fb929b18640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb924100ca0 0x7fb9241989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:28.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.343+0000 7fb929b18640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb924106f50 0x7fb92419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:28.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.343+0000 7fb929b18640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb924111e50 con 0x7fb9241002f0 2024-09-16T02:04:28.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.343+0000 7fb929b18640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb924111cd0 con 0x7fb924106f50 2024-09-16T02:04:28.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.343+0000 7fb929b18640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb924111fd0 con 0x7fb924100ca0 2024-09-16T02:04:28.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.344+0000 7fb9237fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9241002f0 0x7fb924198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:28.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.344+0000 7fb9237fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9241002f0 0x7fb924198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52908/0 (socket says 172.21.15.105:52908) 2024-09-16T02:04:28.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.344+0000 7fb9237fe640 1 -- 172.21.15.105:0/956834586 learned_addr learned my addr 172.21.15.105:0/956834586 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:28.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.344+0000 7fb923fff640 1 --2- 172.21.15.105:0/956834586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb924106f50 0x7fb92419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:28.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.344+0000 7fb922ffd640 1 --2- 172.21.15.105:0/956834586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb924100ca0 0x7fb9241989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:28.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.344+0000 7fb9237fe640 1 -- 172.21.15.105:0/956834586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb924100ca0 msgr2=0x7fb9241989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:28.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.344+0000 7fb9237fe640 1 --2- 172.21.15.105:0/956834586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb924100ca0 0x7fb9241989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:28.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.344+0000 7fb9237fe640 1 -- 172.21.15.105:0/956834586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb924106f50 msgr2=0x7fb92419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:28.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.344+0000 7fb9237fe640 1 --2- 172.21.15.105:0/956834586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb924106f50 0x7fb92419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:28.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.344+0000 7fb9237fe640 1 -- 172.21.15.105:0/956834586 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92419d440 con 0x7fb9241002f0 2024-09-16T02:04:28.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.344+0000 7fb923fff640 1 --2- 172.21.15.105:0/956834586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb924106f50 0x7fb92419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:04:28.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.344+0000 7fb9237fe640 1 --2- 172.21.15.105:0/956834586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9241002f0 0x7fb924198490 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb91400cce0 tx=0x7fb91400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:28.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.344+0000 7fb920ff9640 1 -- 172.21.15.105:0/956834586 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb914004b20 con 0x7fb9241002f0 2024-09-16T02:04:28.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.344+0000 7fb920ff9640 1 -- 172.21.15.105:0/956834586 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb914004cc0 con 0x7fb9241002f0 2024-09-16T02:04:28.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.344+0000 7fb929b18640 1 -- 172.21.15.105:0/956834586 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb92419d730 con 0x7fb9241002f0 2024-09-16T02:04:28.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.344+0000 7fb920ff9640 1 -- 172.21.15.105:0/956834586 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91400f850 con 0x7fb9241002f0 2024-09-16T02:04:28.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.344+0000 7fb929b18640 1 -- 172.21.15.105:0/956834586 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9241a5010 con 0x7fb9241002f0 2024-09-16T02:04:28.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.346+0000 7fb920ff9640 1 -- 172.21.15.105:0/956834586 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fb914005020 con 0x7fb9241002f0 2024-09-16T02:04:28.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.347+0000 7fb92bef1740 1 Processor -- start 2024-09-16T02:04:28.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.347+0000 7fb92bef1740 1 -- start start 2024-09-16T02:04:28.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.347+0000 7fb92bef1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650f7f2b4a0 0x5650f7f2b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:28.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.347+0000 7fb92bef1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650f7f2bdc0 0x5650f7f36140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:28.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.347+0000 7fb92bef1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650f7f36680 0x5650f7f38a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:28.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.347+0000 7fb92bef1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5650f7e912e0 con 0x5650f7f2b4a0 2024-09-16T02:04:28.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.347+0000 7fb92bef1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5650f7e38ce0 con 0x5650f7f36680 2024-09-16T02:04:28.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.347+0000 7fb92bef1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5650f7e8d2a0 con 0x5650f7f2bdc0 2024-09-16T02:04:28.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.347+0000 7fb923fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650f7f2b4a0 0x5650f7f2b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:28.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.347+0000 7fb923fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650f7f2b4a0 0x5650f7f2b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52920/0 (socket says 172.21.15.105:52920) 2024-09-16T02:04:28.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.347+0000 7fb923fff640 1 -- 172.21.15.105:0/3618905872 learned_addr learned my addr 172.21.15.105:0/3618905872 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:28.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.347+0000 7fb920ff9640 1 --2- 172.21.15.105:0/956834586 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb90c0a3240 0x7fb90c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:28.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.347+0000 7fb922ffd640 1 --2- 172.21.15.105:0/3618905872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650f7f36680 0x5650f7f38a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:28.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.347+0000 7fb9237fe640 1 --2- 172.21.15.105:0/3618905872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650f7f2bdc0 0x5650f7f36140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:28.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.347+0000 7fb923fff640 1 --2- 172.21.15.105:0/956834586 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb90c0a3240 0x7fb90c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:28.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.347+0000 7fb923fff640 1 -- 172.21.15.105:0/3618905872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650f7f2bdc0 msgr2=0x5650f7f36140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:28.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.347+0000 7fb920ff9640 1 -- 172.21.15.105:0/956834586 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fb9140cfae0 con 0x7fb9241002f0 2024-09-16T02:04:28.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.347+0000 7fb923fff640 1 --2- 172.21.15.105:0/3618905872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650f7f2bdc0 0x5650f7f36140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:28.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.347+0000 7fb923fff640 1 -- 172.21.15.105:0/3618905872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650f7f36680 msgr2=0x5650f7f38a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:28.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.347+0000 7fb923fff640 1 --2- 172.21.15.105:0/3618905872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650f7f36680 0x5650f7f38a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:28.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.347+0000 7fb923fff640 1 -- 172.21.15.105:0/3618905872 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650f7e1d440 con 0x5650f7f2b4a0 2024-09-16T02:04:28.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.348+0000 7fb9237fe640 1 --2- 172.21.15.105:0/3618905872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650f7f2bdc0 0x5650f7f36140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:04:28.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.348+0000 7fb923fff640 1 --2- 172.21.15.105:0/3618905872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650f7f2b4a0 0x5650f7f2b880 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb91c013790 tx=0x7fb91c013c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b8956ec7d57291 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:28.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.348+0000 7fb909ffb640 1 -- 172.21.15.105:0/3618905872 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c00bd60 con 0x5650f7f2b4a0 2024-09-16T02:04:28.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.348+0000 7fb909ffb640 1 -- 172.21.15.105:0/3618905872 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb91c004c50 con 0x5650f7f2b4a0 2024-09-16T02:04:28.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.348+0000 7fb909ffb640 1 -- 172.21.15.105:0/3618905872 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c01d5f0 con 0x5650f7f2b4a0 2024-09-16T02:04:28.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.348+0000 7fb923fff640 1 --2- 172.21.15.105:0/956834586 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb90c0a3240 0x7fb90c0a5700 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb91c020000 tx=0x7fb91c01f000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:28.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.349+0000 7fb92bef1740 1 -- 172.21.15.105:0/3618905872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650f7f2b4a0 msgr2=0x5650f7f2b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:28.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.349+0000 7fb92bef1740 1 --2- 172.21.15.105:0/3618905872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650f7f2b4a0 0x5650f7f2b880 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb91c013790 tx=0x7fb91c013c50 comp rx=0 tx=0).stop 2024-09-16T02:04:28.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.349+0000 7fb92bef1740 1 -- 172.21.15.105:0/3618905872 shutdown_connections 2024-09-16T02:04:28.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.349+0000 7fb92bef1740 1 --2- 172.21.15.105:0/3618905872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650f7f2bdc0 0x5650f7f36140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:28.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.349+0000 7fb92bef1740 1 --2- 172.21.15.105:0/3618905872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650f7f36680 0x5650f7f38a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:28.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.349+0000 7fb92bef1740 1 --2- 172.21.15.105:0/3618905872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650f7f2b4a0 0x5650f7f2b880 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:28.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.349+0000 7fb92bef1740 1 -- 172.21.15.105:0/3618905872 >> 172.21.15.105:0/3618905872 conn(0x5650f7f27a30 msgr2=0x5650f7f27e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:28.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.349+0000 7fb92bef1740 1 -- 172.21.15.105:0/3618905872 shutdown_connections 2024-09-16T02:04:28.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.349+0000 7fb92bef1740 1 -- 172.21.15.105:0/3618905872 wait complete. 2024-09-16T02:04:28.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.349+0000 7fb92bef1740 1 -- start start 2024-09-16T02:04:28.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.349+0000 7fb92bef1740 1 Processor -- start 2024-09-16T02:04:28.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.349+0000 7fb92bef1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650f7f2b4a0 0x5650f7f35f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:28.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.349+0000 7fb92bef1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650f7f2bdc0 0x5650f7f33220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:28.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.349+0000 7fb92bef1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650f7f36680 0x5650f7f33760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:28.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.349+0000 7fb92bef1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5650f7f39250 con 0x5650f7f36680 2024-09-16T02:04:28.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.349+0000 7fb92bef1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5650f7e4c050 con 0x5650f7f2bdc0 2024-09-16T02:04:28.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.349+0000 7fb92bef1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5650f7f393d0 con 0x5650f7f2b4a0 2024-09-16T02:04:28.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.350+0000 7fb92bef1740 10 client.0 _ll_register_callbacks cb 0x5650f7e8c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:04:28.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.350+0000 7fb923fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650f7f36680 0x5650f7f33760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:28.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.350+0000 7fb923fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650f7f36680 0x5650f7f33760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52926/0 (socket says 172.21.15.105:52926) 2024-09-16T02:04:28.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.350+0000 7fb923fff640 1 -- 172.21.15.105:0/250869954 learned_addr learned my addr 172.21.15.105:0/250869954 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:28.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.350+0000 7fb922ffd640 1 --2- 172.21.15.105:0/250869954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650f7f2bdc0 0x5650f7f33220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:28.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.350+0000 7fb923fff640 1 -- 172.21.15.105:0/250869954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650f7f2b4a0 msgr2=0x5650f7f35f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:28.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.350+0000 7fb923fff640 1 --2- 172.21.15.105:0/250869954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650f7f2b4a0 0x5650f7f35f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:28.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.350+0000 7fb923fff640 1 -- 172.21.15.105:0/250869954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650f7f2bdc0 msgr2=0x5650f7f33220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:28.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.350+0000 7fb923fff640 1 --2- 172.21.15.105:0/250869954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650f7f2bdc0 0x5650f7f33220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:28.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.350+0000 7fb923fff640 1 -- 172.21.15.105:0/250869954 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650f7c023c0 con 0x5650f7f36680 2024-09-16T02:04:28.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.350+0000 7fb923fff640 1 --2- 172.21.15.105:0/250869954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650f7f36680 0x5650f7f33760 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb91c00b470 tx=0x7fb91c011eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c200679c49261e94 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:28.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.350+0000 7fb902ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:04:28.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.350+0000 7fb902ffd640 1 -- 172.21.15.105:0/250869954 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c00b620 con 0x5650f7f36680 2024-09-16T02:04:28.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.350+0000 7fb902ffd640 1 -- 172.21.15.105:0/250869954 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fb91c00bd40 con 0x5650f7f36680 2024-09-16T02:04:28.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.350+0000 7fb92bef1740 1 -- 172.21.15.105:0/250869954 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb91c004c50 con 0x5650f7f36680 2024-09-16T02:04:28.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.350+0000 7fb902ffd640 1 -- 172.21.15.105:0/250869954 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb91c01d3f0 con 0x5650f7f36680 2024-09-16T02:04:28.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.351+0000 7fb92bef1740 10 client.15636 fetch_fsmap learned FSMap version 28 2024-09-16T02:04:28.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.351+0000 7fb92bef1740 1 -- 172.21.15.105:0/250869954 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5650f7f44bc0 con 0x5650f7f36680 2024-09-16T02:04:28.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.351+0000 7fb902ffd640 1 -- 172.21.15.105:0/250869954 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb91c004c50 con 0x5650f7f36680 2024-09-16T02:04:28.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.351+0000 7fb902ffd640 1 -- 172.21.15.105:0/250869954 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb91c026470 con 0x5650f7f36680 2024-09-16T02:04:28.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.351+0000 7fb92bef1740 10 client.15636 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:04:28.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.351+0000 7fb92bef1740 10 client.15636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:04:28.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.351+0000 7fb92bef1740 20 client.15636 populate_metadata read hostname 'smithi105' 2024-09-16T02:04:28.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.351+0000 7fb92bef1740 1 --2- 172.21.15.105:0/250869954 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x5650f7f45f10 0x5650f7f48300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:28.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.351+0000 7fb92bef1740 4 client.15636 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:04:28.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.351+0000 7fb92bef1740 1 -- 172.21.15.105:0/250869954 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650f7e9e850 con 0x5650f7f45f10 2024-09-16T02:04:28.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.352+0000 7fb9237fe640 1 --2- 172.21.15.105:0/250869954 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x5650f7f45f10 0x5650f7f48300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:28.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.352+0000 7fb9237fe640 1 --2- 172.21.15.105:0/250869954 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x5650f7f45f10 0x5650f7f48300 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e089dbf7737ee7f server_cookie=440f84899d4ff8a8 in_seq=0 out_seq=0 2024-09-16T02:04:28.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.352+0000 7fb902ffd640 10 client.15636 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:04:28.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.353+0000 7fb902ffd640 1 -- 172.21.15.105:0/250869954 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5650f7e9e850 con 0x5650f7f45f10 2024-09-16T02:04:28.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.353+0000 7fb902ffd640 10 client.15636 handle_command_reply: tid=0 2024-09-16T02:04:28.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.353+0000 7fb92bef1740 1 client.15636 shutdown 2024-09-16T02:04:28.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.353+0000 7fb92bef1740 1 -- 172.21.15.105:0/250869954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650f7f36680 msgr2=0x5650f7f33760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:28.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.353+0000 7fb92bef1740 1 --2- 172.21.15.105:0/250869954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650f7f36680 0x5650f7f33760 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb91c00b470 tx=0x7fb91c011eb0 comp rx=0 tx=0).stop 2024-09-16T02:04:28.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.353+0000 7fb92bef1740 1 -- 172.21.15.105:0/250869954 shutdown_connections 2024-09-16T02:04:28.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.353+0000 7fb92bef1740 1 --2- 172.21.15.105:0/250869954 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x5650f7f45f10 0x5650f7f48300 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:28.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.353+0000 7fb92bef1740 1 --2- 172.21.15.105:0/250869954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650f7f36680 0x5650f7f33760 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:28.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.353+0000 7fb92bef1740 1 --2- 172.21.15.105:0/250869954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650f7f2bdc0 0x5650f7f33220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:28.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.353+0000 7fb92bef1740 1 --2- 172.21.15.105:0/250869954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650f7f2b4a0 0x5650f7f35f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:28.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.353+0000 7fb902ffd640 0 client.15636 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:04:28.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.353+0000 7fb92bef1740 1 -- 172.21.15.105:0/250869954 >> 172.21.15.105:0/250869954 conn(0x5650f7f27a30 msgr2=0x5650f7f26b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:28.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.353+0000 7fb92bef1740 1 -- 172.21.15.105:0/250869954 shutdown_connections 2024-09-16T02:04:28.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.353+0000 7fb92bef1740 1 -- 172.21.15.105:0/250869954 wait complete. 2024-09-16T02:04:28.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.353+0000 7fb92bef1740 20 client.15636 trim_cache size 0 max 16384 2024-09-16T02:04:28.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.370+0000 7fb92bef1740 1 Processor -- start 2024-09-16T02:04:28.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.370+0000 7fb92bef1740 1 -- start start 2024-09-16T02:04:28.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.370+0000 7fb92bef1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650f7f2b4a0 0x5650f7f424b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:28.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.370+0000 7fb92bef1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650f7f2bdc0 0x5650f7f2e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:28.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.370+0000 7fb92bef1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650f7f36680 0x5650f7f32200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:28.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.370+0000 7fb92bef1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb9241ad710 con 0x5650f7f2b4a0 2024-09-16T02:04:28.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.370+0000 7fb92bef1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb9241af250 con 0x5650f7f36680 2024-09-16T02:04:28.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.370+0000 7fb92bef1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb9241adde0 con 0x5650f7f2bdc0 2024-09-16T02:04:28.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.370+0000 7fb9237fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650f7f2b4a0 0x5650f7f424b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:28.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.371+0000 7fb9237fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650f7f2b4a0 0x5650f7f424b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52930/0 (socket says 172.21.15.105:52930) 2024-09-16T02:04:28.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.371+0000 7fb9237fe640 1 -- 172.21.15.105:0/3432439041 learned_addr learned my addr 172.21.15.105:0/3432439041 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:28.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.371+0000 7fb9237fe640 1 -- 172.21.15.105:0/3432439041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650f7f2bdc0 msgr2=0x5650f7f2e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:28.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.371+0000 7fb9237fe640 1 --2- 172.21.15.105:0/3432439041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650f7f2bdc0 0x5650f7f2e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:28.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.371+0000 7fb9237fe640 1 -- 172.21.15.105:0/3432439041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650f7f36680 msgr2=0x5650f7f32200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:28.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.371+0000 7fb923fff640 1 --2- 172.21.15.105:0/3432439041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650f7f36680 0x5650f7f32200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:28.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.371+0000 7fb9237fe640 1 --2- 172.21.15.105:0/3432439041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650f7f36680 0x5650f7f32200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:28.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.371+0000 7fb9237fe640 1 -- 172.21.15.105:0/3432439041 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650f7f2bbf0 con 0x5650f7f2b4a0 2024-09-16T02:04:28.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.371+0000 7fb9237fe640 1 --2- 172.21.15.105:0/3432439041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650f7f2b4a0 0x5650f7f424b0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb914098a70 tx=0x7fb91407f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab1167495af1c53 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:28.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.371+0000 7fb9097fa640 1 -- 172.21.15.105:0/3432439041 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91401d040 con 0x5650f7f2b4a0 2024-09-16T02:04:28.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.371+0000 7fb9097fa640 1 -- 172.21.15.105:0/3432439041 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb914066f40 con 0x5650f7f2b4a0 2024-09-16T02:04:28.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.371+0000 7fb9097fa640 1 -- 172.21.15.105:0/3432439041 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91409a140 con 0x5650f7f2b4a0 2024-09-16T02:04:28.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.372+0000 7fb92bef1740 1 -- 172.21.15.105:0/3432439041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650f7f2b4a0 msgr2=0x5650f7f424b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:28.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.372+0000 7fb92bef1740 1 --2- 172.21.15.105:0/3432439041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650f7f2b4a0 0x5650f7f424b0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb914098a70 tx=0x7fb91407f040 comp rx=0 tx=0).stop 2024-09-16T02:04:28.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.372+0000 7fb92bef1740 1 -- 172.21.15.105:0/3432439041 shutdown_connections 2024-09-16T02:04:28.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.372+0000 7fb92bef1740 1 --2- 172.21.15.105:0/3432439041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650f7f2bdc0 0x5650f7f2e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:28.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.372+0000 7fb92bef1740 1 --2- 172.21.15.105:0/3432439041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650f7f36680 0x5650f7f32200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:28.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.372+0000 7fb92bef1740 1 --2- 172.21.15.105:0/3432439041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650f7f2b4a0 0x5650f7f424b0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:28.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.372+0000 7fb92bef1740 1 -- 172.21.15.105:0/3432439041 >> 172.21.15.105:0/3432439041 conn(0x5650f7f27a30 msgr2=0x5650f7f31af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:28.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.372+0000 7fb92bef1740 1 -- 172.21.15.105:0/3432439041 shutdown_connections 2024-09-16T02:04:28.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.372+0000 7fb92bef1740 1 -- 172.21.15.105:0/3432439041 wait complete. 2024-09-16T02:04:28.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.372+0000 7fb92bef1740 1 -- start start 2024-09-16T02:04:28.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.372+0000 7fb92bef1740 1 Processor -- start 2024-09-16T02:04:28.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.373+0000 7fb92bef1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650f7f2b4a0 0x5650f7f3e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:28.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.373+0000 7fb92bef1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650f7f2bdc0 0x5650f7f3ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:28.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.373+0000 7fb92bef1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650f7f36680 0x5650f7fa9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:28.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.373+0000 7fb92bef1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb9241aeb80 con 0x5650f7f36680 2024-09-16T02:04:28.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.373+0000 7fb92bef1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb9241ae4b0 con 0x5650f7f2bdc0 2024-09-16T02:04:28.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.373+0000 7fb92bef1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb9241ab6f0 con 0x5650f7f2b4a0 2024-09-16T02:04:28.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.373+0000 7fb92bef1740 10 client.0 _ll_register_callbacks cb 0x7fb918009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:04:28.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.373+0000 7fb923fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650f7f36680 0x5650f7fa9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:28.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.373+0000 7fb923fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650f7f36680 0x5650f7fa9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52934/0 (socket says 172.21.15.105:52934) 2024-09-16T02:04:28.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.373+0000 7fb923fff640 1 -- 172.21.15.105:0/334011096 learned_addr learned my addr 172.21.15.105:0/334011096 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:04:28.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.373+0000 7fb922ffd640 1 --2- 172.21.15.105:0/334011096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650f7f2bdc0 0x5650f7f3ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:28.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.373+0000 7fb9237fe640 1 --2- 172.21.15.105:0/334011096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650f7f2b4a0 0x5650f7f3e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:28.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.373+0000 7fb923fff640 1 -- 172.21.15.105:0/334011096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650f7f2b4a0 msgr2=0x5650f7f3e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:28.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.373+0000 7fb923fff640 1 --2- 172.21.15.105:0/334011096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650f7f2b4a0 0x5650f7f3e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:28.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.373+0000 7fb923fff640 1 -- 172.21.15.105:0/334011096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650f7f2bdc0 msgr2=0x5650f7f3ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:28.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.373+0000 7fb923fff640 1 --2- 172.21.15.105:0/334011096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650f7f2bdc0 0x5650f7f3ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:28.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.373+0000 7fb923fff640 1 -- 172.21.15.105:0/334011096 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650f7fa9980 con 0x5650f7f36680 2024-09-16T02:04:28.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.373+0000 7fb923fff640 1 --2- 172.21.15.105:0/334011096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650f7f36680 0x5650f7fa9180 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb91c026840 tx=0x7fb91c011880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cd08a0d8aee3634 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:28.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.373+0000 7fb902ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:04:28.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.373+0000 7fb902ffd640 1 -- 172.21.15.105:0/334011096 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c01d590 con 0x5650f7f36680 2024-09-16T02:04:28.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.373+0000 7fb902ffd640 1 -- 172.21.15.105:0/334011096 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fb91c026470 con 0x5650f7f36680 2024-09-16T02:04:28.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.373+0000 7fb92bef1740 1 -- 172.21.15.105:0/334011096 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb914066f40 con 0x5650f7f36680 2024-09-16T02:04:28.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.374+0000 7fb902ffd640 1 -- 172.21.15.105:0/334011096 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb91c004890 con 0x5650f7f36680 2024-09-16T02:04:28.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.374+0000 7fb92bef1740 10 client.15642 fetch_fsmap learned FSMap version 28 2024-09-16T02:04:28.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.374+0000 7fb92bef1740 1 -- 172.21.15.105:0/334011096 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5650f7f3c290 con 0x5650f7f36680 2024-09-16T02:04:28.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.374+0000 7fb902ffd640 1 -- 172.21.15.105:0/334011096 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb914066f40 con 0x5650f7f36680 2024-09-16T02:04:28.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.374+0000 7fb902ffd640 1 -- 172.21.15.105:0/334011096 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb91c01d950 con 0x5650f7f36680 2024-09-16T02:04:28.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.374+0000 7fb92bef1740 10 client.15642 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:04:28.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.374+0000 7fb92bef1740 10 client.15642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:04:28.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.374+0000 7fb92bef1740 20 client.15642 populate_metadata read hostname 'smithi105' 2024-09-16T02:04:28.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.374+0000 7fb92bef1740 1 --2- 172.21.15.105:0/334011096 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x5650f7fb50b0 0x5650f7fb74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:28.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.374+0000 7fb92bef1740 4 client.15642 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T02:04:28.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.374+0000 7fb92bef1740 1 -- 172.21.15.105:0/334011096 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5650f7e98d60 con 0x5650f7fb50b0 2024-09-16T02:04:28.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.374+0000 7fb9237fe640 1 --2- 172.21.15.105:0/334011096 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x5650f7fb50b0 0x5650f7fb74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:28.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.375+0000 7fb902ffd640 10 client.15642 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:04:28.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.375+0000 7fb9237fe640 1 --2- 172.21.15.105:0/334011096 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x5650f7fb50b0 0x5650f7fb74a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75e65b4af54f85dc server_cookie=a15ae1bb25caa3c1 in_seq=0 out_seq=0 2024-09-16T02:04:28.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.375+0000 7fb902ffd640 1 -- 172.21.15.105:0/334011096 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5650f7e98d60 con 0x5650f7fb50b0 2024-09-16T02:04:28.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.375+0000 7fb902ffd640 10 client.15642 handle_command_reply: tid=0 2024-09-16T02:04:28.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.375+0000 7fb92bef1740 1 client.15642 shutdown 2024-09-16T02:04:28.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.375+0000 7fb92bef1740 1 -- 172.21.15.105:0/334011096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650f7f36680 msgr2=0x5650f7fa9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:28.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.375+0000 7fb92bef1740 1 --2- 172.21.15.105:0/334011096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650f7f36680 0x5650f7fa9180 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb91c026840 tx=0x7fb91c011880 comp rx=0 tx=0).stop 2024-09-16T02:04:28.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.375+0000 7fb92bef1740 1 -- 172.21.15.105:0/334011096 shutdown_connections 2024-09-16T02:04:28.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.375+0000 7fb92bef1740 1 --2- 172.21.15.105:0/334011096 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x5650f7fb50b0 0x5650f7fb74a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:28.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.375+0000 7fb92bef1740 1 --2- 172.21.15.105:0/334011096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650f7f36680 0x5650f7fa9180 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:28.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.375+0000 7fb92bef1740 1 --2- 172.21.15.105:0/334011096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650f7f2bdc0 0x5650f7f3ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:28.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.375+0000 7fb902ffd640 0 client.15642 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:04:28.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.375+0000 7fb92bef1740 1 --2- 172.21.15.105:0/334011096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650f7f2b4a0 0x5650f7f3e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:28.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.375+0000 7fb92bef1740 1 -- 172.21.15.105:0/334011096 >> 172.21.15.105:0/334011096 conn(0x5650f7f27a30 msgr2=0x5650f7f34a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:28.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.375+0000 7fb92bef1740 1 -- 172.21.15.105:0/334011096 shutdown_connections 2024-09-16T02:04:28.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.375+0000 7fb92bef1740 1 -- 172.21.15.105:0/334011096 wait complete. 2024-09-16T02:04:28.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.375+0000 7fb92bef1740 20 client.15642 trim_cache size 0 max 16384 2024-09-16T02:04:28.389 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-16T02:04:28.389 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T02:04:28.389 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-09-16T02:04:28.390 INFO:teuthology.orchestra.run.smithi105.stdout: "b7aa2a6e-a7fc-4553-8d71-c33dc2a1e7d4": { 2024-09-16T02:04:28.390 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-09-16T02:04:28.390 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "b7aa2a6e-a7fc-4553-8d71-c33dc2a1e7d4", 2024-09-16T02:04:28.390 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-09-16T02:04:28.390 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:04:28.390 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:04:28.390 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-16T02:04:28.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.376+0000 7fb929b18640 1 -- 172.21.15.105:0/956834586 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb90c0a3240 msgr2=0x7fb90c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:28.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.376+0000 7fb929b18640 1 --2- 172.21.15.105:0/956834586 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb90c0a3240 0x7fb90c0a5700 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb91c020000 tx=0x7fb91c01f000 comp rx=0 tx=0).stop 2024-09-16T02:04:28.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.376+0000 7fb929b18640 1 -- 172.21.15.105:0/956834586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9241002f0 msgr2=0x7fb924198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:28.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.376+0000 7fb929b18640 1 --2- 172.21.15.105:0/956834586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9241002f0 0x7fb924198490 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb91400cce0 tx=0x7fb91400a620 comp rx=0 tx=0).stop 2024-09-16T02:04:28.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.376+0000 7fb929b18640 1 -- 172.21.15.105:0/956834586 shutdown_connections 2024-09-16T02:04:28.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.376+0000 7fb929b18640 1 --2- 172.21.15.105:0/956834586 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb90c0a3240 0x7fb90c0a5700 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:28.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.376+0000 7fb929b18640 1 --2- 172.21.15.105:0/956834586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb924100ca0 0x7fb9241989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:28.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.376+0000 7fb929b18640 1 --2- 172.21.15.105:0/956834586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb924106f50 0x7fb92419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:28.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.376+0000 7fb929b18640 1 --2- 172.21.15.105:0/956834586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9241002f0 0x7fb924198490 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:28.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.376+0000 7fb929b18640 1 -- 172.21.15.105:0/956834586 >> 172.21.15.105:0/956834586 conn(0x7fb924077210 msgr2=0x7fb9241052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:28.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.376+0000 7fb929b18640 1 -- 172.21.15.105:0/956834586 shutdown_connections 2024-09-16T02:04:28.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:04:28.376+0000 7fb929b18640 1 -- 172.21.15.105:0/956834586 wait complete. 2024-09-16T02:04:28.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:28 smithi137 ceph-mon[34099]: pgmap v397: 97 pgs: 97 active+clean; 1.0 GiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 6.0 MiB/s wr, 11 op/s 2024-09-16T02:04:29.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:28 smithi105 ceph-mon[31197]: pgmap v397: 97 pgs: 97 active+clean; 1.0 GiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 6.0 MiB/s wr, 11 op/s 2024-09-16T02:04:29.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:28 smithi190 ceph-mon[34762]: pgmap v397: 97 pgs: 97 active+clean; 1.0 GiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 6.0 MiB/s wr, 11 op/s 2024-09-16T02:04:29.099 INFO:tasks.cephfs.filesystem:scrub status for tag:b7aa2a6e-a7fc-4553-8d71-c33dc2a1e7d4 - {'path': '/', 'tag': 'b7aa2a6e-a7fc-4553-8d71-c33dc2a1e7d4', 'options': 'recursive,force'} 2024-09-16T02:04:30.920 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:30 smithi137 ceph-mon[34099]: pgmap v398: 97 pgs: 97 active+clean; 1013 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 25 MiB/s wr, 26 op/s 2024-09-16T02:04:30.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:30 smithi137 ceph-mon[34099]: scrub summary: idle 2024-09-16T02:04:31.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:30 smithi105 ceph-mon[31197]: pgmap v398: 97 pgs: 97 active+clean; 1013 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 25 MiB/s wr, 26 op/s 2024-09-16T02:04:31.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:30 smithi105 ceph-mon[31197]: scrub summary: idle 2024-09-16T02:04:31.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:30 smithi190 ceph-mon[34762]: pgmap v398: 97 pgs: 97 active+clean; 1013 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 25 MiB/s wr, 26 op/s 2024-09-16T02:04:31.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:30 smithi190 ceph-mon[34762]: scrub summary: idle 2024-09-16T02:04:31.175 INFO:tasks.workunit.client.0.smithi105.stdout: 2024-09-16T02:04:31.175 INFO:tasks.workunit.client.0.smithi105.stdout: Children see throughput for 1 rewriters = 782433.56 kB/sec 2024-09-16T02:04:31.175 INFO:tasks.workunit.client.0.smithi105.stdout: Parent sees throughput for 1 rewriters = 782310.97 kB/sec 2024-09-16T02:04:31.175 INFO:tasks.workunit.client.0.smithi105.stdout: Min throughput per process = 782433.56 kB/sec 2024-09-16T02:04:31.175 INFO:tasks.workunit.client.0.smithi105.stdout: Max throughput per process = 782433.56 kB/sec 2024-09-16T02:04:31.175 INFO:tasks.workunit.client.0.smithi105.stdout: Avg throughput per process = 782433.56 kB/sec 2024-09-16T02:04:31.175 INFO:tasks.workunit.client.0.smithi105.stdout: Min xfer = 1048576.00 kB 2024-09-16T02:04:32.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:32 smithi137 ceph-mon[34099]: pgmap v399: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 101 MiB/s wr, 59 op/s 2024-09-16T02:04:33.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:32 smithi105 ceph-mon[31197]: pgmap v399: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 101 MiB/s wr, 59 op/s 2024-09-16T02:04:33.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:32 smithi190 ceph-mon[34762]: pgmap v399: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 101 MiB/s wr, 59 op/s 2024-09-16T02:04:34.010 INFO:tasks.workunit.client.0.smithi105.stdout: 2024-09-16T02:04:34.010 INFO:tasks.workunit.client.0.smithi105.stdout: Children see throughput for 1 readers = 7179853.50 kB/sec 2024-09-16T02:04:34.010 INFO:tasks.workunit.client.0.smithi105.stdout: Parent sees throughput for 1 readers = 7168677.08 kB/sec 2024-09-16T02:04:34.010 INFO:tasks.workunit.client.0.smithi105.stdout: Min throughput per process = 7179853.50 kB/sec 2024-09-16T02:04:34.010 INFO:tasks.workunit.client.0.smithi105.stdout: Max throughput per process = 7179853.50 kB/sec 2024-09-16T02:04:34.010 INFO:tasks.workunit.client.0.smithi105.stdout: Avg throughput per process = 7179853.50 kB/sec 2024-09-16T02:04:34.011 INFO:tasks.workunit.client.0.smithi105.stdout: Min xfer = 1048576.00 kB 2024-09-16T02:04:34.865 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:34 smithi105 ceph-mon[31197]: pgmap v400: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 114 MiB/s wr, 62 op/s 2024-09-16T02:04:34.924 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:34 smithi137 ceph-mon[34099]: pgmap v400: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 114 MiB/s wr, 62 op/s 2024-09-16T02:04:35.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:34 smithi190 ceph-mon[34762]: pgmap v400: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 114 MiB/s wr, 62 op/s 2024-09-16T02:04:35.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:35 smithi137 ceph-mon[34099]: pgmap v401: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 159 MiB/s wr, 74 op/s 2024-09-16T02:04:36.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:35 smithi105 ceph-mon[31197]: pgmap v401: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 159 MiB/s wr, 74 op/s 2024-09-16T02:04:36.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:35 smithi190 ceph-mon[34762]: pgmap v401: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 159 MiB/s wr, 74 op/s 2024-09-16T02:04:36.244 INFO:tasks.workunit.client.0.smithi105.stdout: 2024-09-16T02:04:36.244 INFO:tasks.workunit.client.0.smithi105.stdout: Children see throughput for 1 re-readers = 7880592.00 kB/sec 2024-09-16T02:04:36.244 INFO:tasks.workunit.client.0.smithi105.stdout: Parent sees throughput for 1 re-readers = 7868762.43 kB/sec 2024-09-16T02:04:36.244 INFO:tasks.workunit.client.0.smithi105.stdout: Min throughput per process = 7880592.00 kB/sec 2024-09-16T02:04:36.244 INFO:tasks.workunit.client.0.smithi105.stdout: Max throughput per process = 7880592.00 kB/sec 2024-09-16T02:04:36.244 INFO:tasks.workunit.client.0.smithi105.stdout: Avg throughput per process = 7880592.00 kB/sec 2024-09-16T02:04:36.244 INFO:tasks.workunit.client.0.smithi105.stdout: Min xfer = 1048576.00 kB 2024-09-16T02:04:38.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:38 smithi190 ceph-mon[34762]: pgmap v402: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 171 MiB/s wr, 76 op/s 2024-09-16T02:04:38.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:38 smithi137 ceph-mon[34099]: pgmap v402: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 171 MiB/s wr, 76 op/s 2024-09-16T02:04:39.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:38 smithi105 ceph-mon[31197]: pgmap v402: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 171 MiB/s wr, 76 op/s 2024-09-16T02:04:40.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:40 smithi190 ceph-mon[34762]: pgmap v403: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 169 MiB/s wr, 77 op/s 2024-09-16T02:04:40.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:40 smithi137 ceph-mon[34099]: pgmap v403: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 169 MiB/s wr, 77 op/s 2024-09-16T02:04:41.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:40 smithi105 ceph-mon[31197]: pgmap v403: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 169 MiB/s wr, 77 op/s 2024-09-16T02:04:42.594 INFO:tasks.workunit.client.0.smithi105.stdout: 2024-09-16T02:04:42.594 INFO:tasks.workunit.client.0.smithi105.stdout: 2024-09-16T02:04:42.594 INFO:tasks.workunit.client.0.smithi105.stdout: 2024-09-16T02:04:42.594 INFO:tasks.workunit.client.0.smithi105.stdout:iozone test complete. 2024-09-16T02:04:42.594 INFO:tasks.workunit.client.0.smithi105.stderr:+ iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-09-16T02:04:42.660 INFO:tasks.workunit.client.0.smithi105.stdout: Iozone: Performance Test of File I/O 2024-09-16T02:04:42.660 INFO:tasks.workunit.client.0.smithi105.stdout: Version $Revision: 3.494 $ 2024-09-16T02:04:42.660 INFO:tasks.workunit.client.0.smithi105.stdout: Compiled for 64 bit mode. 2024-09-16T02:04:42.660 INFO:tasks.workunit.client.0.smithi105.stdout: Build: linux-AMD64 2024-09-16T02:04:42.660 INFO:tasks.workunit.client.0.smithi105.stdout: 2024-09-16T02:04:42.660 INFO:tasks.workunit.client.0.smithi105.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-09-16T02:04:42.660 INFO:tasks.workunit.client.0.smithi105.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-09-16T02:04:42.660 INFO:tasks.workunit.client.0.smithi105.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-09-16T02:04:42.660 INFO:tasks.workunit.client.0.smithi105.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-09-16T02:04:42.660 INFO:tasks.workunit.client.0.smithi105.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-09-16T02:04:42.661 INFO:tasks.workunit.client.0.smithi105.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-09-16T02:04:42.661 INFO:tasks.workunit.client.0.smithi105.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-09-16T02:04:42.661 INFO:tasks.workunit.client.0.smithi105.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-09-16T02:04:42.661 INFO:tasks.workunit.client.0.smithi105.stdout: Alexey Skidanov, Sudhir Kumar. 2024-09-16T02:04:42.661 INFO:tasks.workunit.client.0.smithi105.stdout: 2024-09-16T02:04:42.661 INFO:tasks.workunit.client.0.smithi105.stdout: Run began: Mon Sep 16 02:04:42 2024 2024-09-16T02:04:42.661 INFO:tasks.workunit.client.0.smithi105.stdout: 2024-09-16T02:04:42.661 INFO:tasks.workunit.client.0.smithi105.stdout: Include close in write timing 2024-09-16T02:04:42.661 INFO:tasks.workunit.client.0.smithi105.stdout: Include fsync in write timing 2024-09-16T02:04:42.661 INFO:tasks.workunit.client.0.smithi105.stdout: File size set to 10485760 kB 2024-09-16T02:04:42.661 INFO:tasks.workunit.client.0.smithi105.stdout: Record Size 1024 kB 2024-09-16T02:04:42.661 INFO:tasks.workunit.client.0.smithi105.stdout: Command line used: iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-09-16T02:04:42.661 INFO:tasks.workunit.client.0.smithi105.stdout: Output is in kBytes/sec 2024-09-16T02:04:42.662 INFO:tasks.workunit.client.0.smithi105.stdout: Time Resolution = 0.000001 seconds. 2024-09-16T02:04:42.662 INFO:tasks.workunit.client.0.smithi105.stdout: Processor cache size set to 1024 kBytes. 2024-09-16T02:04:42.662 INFO:tasks.workunit.client.0.smithi105.stdout: Processor cache line size set to 32 bytes. 2024-09-16T02:04:42.662 INFO:tasks.workunit.client.0.smithi105.stdout: File stride size set to 17 * record size. 2024-09-16T02:04:42.662 INFO:tasks.workunit.client.0.smithi105.stdout: Throughput test with 1 process 2024-09-16T02:04:42.662 INFO:tasks.workunit.client.0.smithi105.stdout: Each process writes a 10485760 kByte file in 1024 kByte records 2024-09-16T02:04:42.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:42 smithi190 ceph-mon[34762]: pgmap v404: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 145 MiB/s wr, 62 op/s 2024-09-16T02:04:42.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:42 smithi137 ceph-mon[34099]: pgmap v404: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 145 MiB/s wr, 62 op/s 2024-09-16T02:04:43.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:42 smithi105 ceph-mon[31197]: pgmap v404: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 145 MiB/s wr, 62 op/s 2024-09-16T02:04:44.845 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:44 smithi190 ceph-mon[34762]: pgmap v405: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 73 MiB/s wr, 28 op/s 2024-09-16T02:04:44.920 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:44 smithi137 ceph-mon[34099]: pgmap v405: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 73 MiB/s wr, 28 op/s 2024-09-16T02:04:45.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:44 smithi105 ceph-mon[31197]: pgmap v405: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 73 MiB/s wr, 28 op/s 2024-09-16T02:04:46.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:46 smithi190 ceph-mon[34762]: pgmap v406: 97 pgs: 97 active+clean; 1.5 GiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 131 MiB/s wr, 50 op/s 2024-09-16T02:04:46.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:46 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:04:46.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:46 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:04:46.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:46 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:04:46.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:46 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:04:46.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:46 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:04:46.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:46 smithi137 ceph-mon[34099]: pgmap v406: 97 pgs: 97 active+clean; 1.5 GiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 131 MiB/s wr, 50 op/s 2024-09-16T02:04:46.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:46 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:04:46.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:46 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:04:46.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:46 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:04:46.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:46 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:04:46.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:46 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:04:47.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:46 smithi105 ceph-mon[31197]: pgmap v406: 97 pgs: 97 active+clean; 1.5 GiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 131 MiB/s wr, 50 op/s 2024-09-16T02:04:47.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:46 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:04:47.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:46 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:04:47.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:46 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:04:47.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:46 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:04:47.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:46 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:04:47.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:47 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:04:47.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:47 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:04:47.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:47 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:04:47.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:47 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:04:47.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:47 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:04:47.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:47 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:04:48.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:47 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:04:48.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:47 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:04:48.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:47 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:04:48.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:48 smithi190 ceph-mon[34762]: pgmap v407: 97 pgs: 97 active+clean; 1.7 GiB data, 706 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 115 MiB/s wr, 48 op/s 2024-09-16T02:04:48.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:48 smithi137 ceph-mon[34099]: pgmap v407: 97 pgs: 97 active+clean; 1.7 GiB data, 706 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 115 MiB/s wr, 48 op/s 2024-09-16T02:04:49.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:48 smithi105 ceph-mon[31197]: pgmap v407: 97 pgs: 97 active+clean; 1.7 GiB data, 706 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 115 MiB/s wr, 48 op/s 2024-09-16T02:04:50.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:50 smithi190 ceph-mon[34762]: pgmap v408: 97 pgs: 97 active+clean; 2.5 GiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 191 MiB/s wr, 77 op/s 2024-09-16T02:04:50.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:50 smithi137 ceph-mon[34099]: pgmap v408: 97 pgs: 97 active+clean; 2.5 GiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 191 MiB/s wr, 77 op/s 2024-09-16T02:04:51.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:50 smithi105 ceph-mon[31197]: pgmap v408: 97 pgs: 97 active+clean; 2.5 GiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 191 MiB/s wr, 77 op/s 2024-09-16T02:04:52.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:52 smithi190 ceph-mon[34762]: pgmap v409: 97 pgs: 97 active+clean; 4.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 386 MiB/s wr, 130 op/s 2024-09-16T02:04:52.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:52 smithi137 ceph-mon[34099]: pgmap v409: 97 pgs: 97 active+clean; 4.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 386 MiB/s wr, 130 op/s 2024-09-16T02:04:53.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:52 smithi105 ceph-mon[31197]: pgmap v409: 97 pgs: 97 active+clean; 4.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 386 MiB/s wr, 130 op/s 2024-09-16T02:04:54.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:54 smithi137 ceph-mon[34099]: pgmap v410: 97 pgs: 97 active+clean; 5.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 431 MiB/s wr, 142 op/s 2024-09-16T02:04:55.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:54 smithi105 ceph-mon[31197]: pgmap v410: 97 pgs: 97 active+clean; 5.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 431 MiB/s wr, 142 op/s 2024-09-16T02:04:55.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:54 smithi190 ceph-mon[34762]: pgmap v410: 97 pgs: 97 active+clean; 5.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 431 MiB/s wr, 142 op/s 2024-09-16T02:04:56.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:56 smithi137 ceph-mon[34099]: pgmap v411: 97 pgs: 97 active+clean; 7.8 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 665 MiB/s wr, 201 op/s 2024-09-16T02:04:57.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:56 smithi105 ceph-mon[31197]: pgmap v411: 97 pgs: 97 active+clean; 7.8 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 665 MiB/s wr, 201 op/s 2024-09-16T02:04:57.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:56 smithi190 ceph-mon[34762]: pgmap v411: 97 pgs: 97 active+clean; 7.8 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 665 MiB/s wr, 201 op/s 2024-09-16T02:04:57.665 INFO:tasks.workunit.client.0.smithi105.stdout: 2024-09-16T02:04:57.665 INFO:tasks.workunit.client.0.smithi105.stdout: Children see throughput for 1 initial writers = 700306.69 kB/sec 2024-09-16T02:04:57.665 INFO:tasks.workunit.client.0.smithi105.stdout: Parent sees throughput for 1 initial writers = 700297.20 kB/sec 2024-09-16T02:04:57.665 INFO:tasks.workunit.client.0.smithi105.stdout: Min throughput per process = 700306.69 kB/sec 2024-09-16T02:04:57.665 INFO:tasks.workunit.client.0.smithi105.stdout: Max throughput per process = 700306.69 kB/sec 2024-09-16T02:04:57.666 INFO:tasks.workunit.client.0.smithi105.stdout: Avg throughput per process = 700306.69 kB/sec 2024-09-16T02:04:57.666 INFO:tasks.workunit.client.0.smithi105.stdout: Min xfer = 10485760.00 kB 2024-09-16T02:04:58.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:04:58 smithi137 ceph-mon[34099]: pgmap v412: 97 pgs: 97 active+clean; 7.9 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 602 MiB/s wr, 176 op/s 2024-09-16T02:04:59.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:04:58 smithi105 ceph-mon[31197]: pgmap v412: 97 pgs: 97 active+clean; 7.9 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 602 MiB/s wr, 176 op/s 2024-09-16T02:04:59.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:04:58 smithi190 ceph-mon[34762]: pgmap v412: 97 pgs: 97 active+clean; 7.9 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 602 MiB/s wr, 176 op/s 2024-09-16T02:04:59.100 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T02:04:59.356 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:00.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:00 smithi137 ceph-mon[34099]: pgmap v413: 97 pgs: 97 active+clean; 8.8 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 644 MiB/s wr, 185 op/s 2024-09-16T02:05:01.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:00 smithi105 ceph-mon[31197]: pgmap v413: 97 pgs: 97 active+clean; 8.8 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 644 MiB/s wr, 185 op/s 2024-09-16T02:05:01.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:00 smithi190 ceph-mon[34762]: pgmap v413: 97 pgs: 97 active+clean; 8.8 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 644 MiB/s wr, 185 op/s 2024-09-16T02:05:02.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:02 smithi137 ceph-mon[34099]: pgmap v414: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 662 MiB/s wr, 181 op/s 2024-09-16T02:05:02.921 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:02 smithi105 ceph-mon[31197]: pgmap v414: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 662 MiB/s wr, 181 op/s 2024-09-16T02:05:02.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.918+0000 7f162a0e0640 1 -- 172.21.15.105:0/643385324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1624106f10 msgr2=0x7f162410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:02.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.918+0000 7f162a0e0640 1 --2- 172.21.15.105:0/643385324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1624106f10 0x7f162410ea90 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f161c009ca0 tx=0x7f161c02a8c0 comp rx=0 tx=0).stop 2024-09-16T02:05:02.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.918+0000 7f162a0e0640 1 -- 172.21.15.105:0/643385324 shutdown_connections 2024-09-16T02:05:02.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.918+0000 7f162a0e0640 1 --2- 172.21.15.105:0/643385324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1624102790 0x7f16241067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:02.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.918+0000 7f162a0e0640 1 --2- 172.21.15.105:0/643385324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1624106f10 0x7f162410ea90 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:02.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.918+0000 7f162a0e0640 1 --2- 172.21.15.105:0/643385324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1624101de0 0x7f16241021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:02.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.918+0000 7f162a0e0640 1 -- 172.21.15.105:0/643385324 >> 172.21.15.105:0/643385324 conn(0x7f16240fd8c0 msgr2=0x7f16240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:02.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.918+0000 7f162a0e0640 1 -- 172.21.15.105:0/643385324 shutdown_connections 2024-09-16T02:05:02.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.918+0000 7f162a0e0640 1 -- 172.21.15.105:0/643385324 wait complete. 2024-09-16T02:05:02.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.919+0000 7f162a0e0640 1 Processor -- start 2024-09-16T02:05:02.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.919+0000 7f162a0e0640 1 -- start start 2024-09-16T02:05:02.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.919+0000 7f162a0e0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1624101de0 0x7f16240744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:02.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.919+0000 7f162a0e0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1624102790 0x7f1624074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:02.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.919+0000 7f162a0e0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1624106f10 0x7f1624074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:02.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.919+0000 7f162a0e0640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1624111e20 con 0x7f1624102790 2024-09-16T02:05:02.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.919+0000 7f162a0e0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1624111ca0 con 0x7f1624106f10 2024-09-16T02:05:02.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.919+0000 7f162a0e0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1624111fa0 con 0x7f1624101de0 2024-09-16T02:05:02.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.919+0000 7f1622ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1624102790 0x7f1624074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:02.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.919+0000 7f1622ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1624102790 0x7f1624074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54966/0 (socket says 172.21.15.105:54966) 2024-09-16T02:05:02.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.919+0000 7f1622ffd640 1 -- 172.21.15.105:0/507080905 learned_addr learned my addr 172.21.15.105:0/507080905 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:02.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.919+0000 7f1623fff640 1 --2- 172.21.15.105:0/507080905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1624106f10 0x7f1624074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:02.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.919+0000 7f16237fe640 1 --2- 172.21.15.105:0/507080905 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1624101de0 0x7f16240744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:02.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.920+0000 7f1622ffd640 1 -- 172.21.15.105:0/507080905 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1624101de0 msgr2=0x7f16240744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:02.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.920+0000 7f1622ffd640 1 --2- 172.21.15.105:0/507080905 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1624101de0 0x7f16240744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:02.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.920+0000 7f1622ffd640 1 -- 172.21.15.105:0/507080905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1624106f10 msgr2=0x7f1624074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:02.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.920+0000 7f1622ffd640 1 --2- 172.21.15.105:0/507080905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1624106f10 0x7f1624074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:02.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.920+0000 7f1622ffd640 1 -- 172.21.15.105:0/507080905 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16241a6760 con 0x7f1624102790 2024-09-16T02:05:02.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.920+0000 7f1622ffd640 1 --2- 172.21.15.105:0/507080905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1624102790 0x7f1624074a30 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f161800d770 tx=0x7f161800db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:02.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.920+0000 7f1620ff9640 1 -- 172.21.15.105:0/507080905 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1618017070 con 0x7f1624102790 2024-09-16T02:05:02.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.920+0000 7f1620ff9640 1 -- 172.21.15.105:0/507080905 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1618009820 con 0x7f1624102790 2024-09-16T02:05:02.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.920+0000 7f162a0e0640 1 -- 172.21.15.105:0/507080905 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16241a6a50 con 0x7f1624102790 2024-09-16T02:05:02.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.920+0000 7f1620ff9640 1 -- 172.21.15.105:0/507080905 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1618004ba0 con 0x7f1624102790 2024-09-16T02:05:02.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.920+0000 7f162a0e0640 1 -- 172.21.15.105:0/507080905 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16241a6f40 con 0x7f1624102790 2024-09-16T02:05:02.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.923+0000 7f1620ff9640 1 -- 172.21.15.105:0/507080905 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f1618010460 con 0x7f1624102790 2024-09-16T02:05:02.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.923+0000 7f162c4b9740 1 Processor -- start 2024-09-16T02:05:02.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.923+0000 7f162c4b9740 1 -- start start 2024-09-16T02:05:02.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.923+0000 7f162c4b9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647623154a0 0x564762315880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:02.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.923+0000 7f162c4b9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564762315dc0 0x564762320140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:02.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.923+0000 7f162c4b9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564762320680 0x564762322a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:02.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.923+0000 7f162c4b9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56476227b2e0 con 0x564762320680 2024-09-16T02:05:02.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.923+0000 7f162c4b9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564762222ce0 con 0x564762315dc0 2024-09-16T02:05:02.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.923+0000 7f162c4b9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5647622772a0 con 0x5647623154a0 2024-09-16T02:05:02.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.923+0000 7f1622ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564762320680 0x564762322a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:02.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.923+0000 7f1620ff9640 1 --2- 172.21.15.105:0/507080905 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f160c0a3240 0x7f160c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:02.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.924+0000 7f1622ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564762320680 0x564762322a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54976/0 (socket says 172.21.15.105:54976) 2024-09-16T02:05:02.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.924+0000 7f1622ffd640 1 -- 172.21.15.105:0/905466801 learned_addr learned my addr 172.21.15.105:0/905466801 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:02.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.924+0000 7f1623fff640 1 --2- 172.21.15.105:0/905466801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647623154a0 0x564762315880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:02.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.924+0000 7f1623fff640 1 --2- 172.21.15.105:0/507080905 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f160c0a3240 0x7f160c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:02.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.924+0000 7f16237fe640 1 --2- 172.21.15.105:0/905466801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564762315dc0 0x564762320140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:02.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.924+0000 7f1620ff9640 1 -- 172.21.15.105:0/507080905 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f16180cf350 con 0x7f1624102790 2024-09-16T02:05:02.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.924+0000 7f1623fff640 1 -- 172.21.15.105:0/905466801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564762315dc0 msgr2=0x564762320140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:02.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.924+0000 7f1623fff640 1 --2- 172.21.15.105:0/905466801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564762315dc0 0x564762320140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:02.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.924+0000 7f1623fff640 1 -- 172.21.15.105:0/905466801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564762320680 msgr2=0x564762322a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:02.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.924+0000 7f1623fff640 1 --2- 172.21.15.105:0/905466801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564762320680 0x564762322a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:02.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.924+0000 7f1623fff640 1 -- 172.21.15.105:0/905466801 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564762207440 con 0x5647623154a0 2024-09-16T02:05:02.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.924+0000 7f1622ffd640 1 --2- 172.21.15.105:0/905466801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564762320680 0x564762322a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:05:02.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.924+0000 7f1623fff640 1 --2- 172.21.15.105:0/507080905 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f160c0a3240 0x7f160c0a5700 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f161c004af0 tx=0x7f161c03b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:02.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.924+0000 7f1623fff640 1 --2- 172.21.15.105:0/905466801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647623154a0 0x564762315880 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f161c031040 tx=0x7f161c032d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=671a36cab79f2809 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:02.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.925+0000 7f1609ffb640 1 -- 172.21.15.105:0/905466801 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c04e030 con 0x5647623154a0 2024-09-16T02:05:02.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.925+0000 7f1609ffb640 1 -- 172.21.15.105:0/905466801 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f161c036040 con 0x5647623154a0 2024-09-16T02:05:02.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.925+0000 7f1609ffb640 1 -- 172.21.15.105:0/905466801 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c0042a0 con 0x5647623154a0 2024-09-16T02:05:02.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.925+0000 7f162c4b9740 1 -- 172.21.15.105:0/905466801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647623154a0 msgr2=0x564762315880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:02.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.925+0000 7f162c4b9740 1 --2- 172.21.15.105:0/905466801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647623154a0 0x564762315880 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f161c031040 tx=0x7f161c032d70 comp rx=0 tx=0).stop 2024-09-16T02:05:02.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.926+0000 7f162c4b9740 1 -- 172.21.15.105:0/905466801 shutdown_connections 2024-09-16T02:05:02.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.926+0000 7f162c4b9740 1 --2- 172.21.15.105:0/905466801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564762320680 0x564762322a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:02.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.926+0000 7f162c4b9740 1 --2- 172.21.15.105:0/905466801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564762315dc0 0x564762320140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:02.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.926+0000 7f162c4b9740 1 --2- 172.21.15.105:0/905466801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647623154a0 0x564762315880 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:02.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.926+0000 7f162c4b9740 1 -- 172.21.15.105:0/905466801 >> 172.21.15.105:0/905466801 conn(0x564762311a30 msgr2=0x564762311e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:02.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.926+0000 7f162c4b9740 1 -- 172.21.15.105:0/905466801 shutdown_connections 2024-09-16T02:05:02.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.926+0000 7f162c4b9740 1 -- 172.21.15.105:0/905466801 wait complete. 2024-09-16T02:05:02.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.926+0000 7f162c4b9740 1 -- start start 2024-09-16T02:05:02.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.926+0000 7f162c4b9740 1 Processor -- start 2024-09-16T02:05:02.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.926+0000 7f162c4b9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647623154a0 0x56476231ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:02.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.926+0000 7f162c4b9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564762315dc0 0x56476231d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:02.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.927+0000 7f162c4b9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564762320680 0x56476231d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:02.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.927+0000 7f162c4b9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5647623230d0 con 0x564762315dc0 2024-09-16T02:05:02.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.927+0000 7f162c4b9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564762236050 con 0x5647623154a0 2024-09-16T02:05:02.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.927+0000 7f162c4b9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564762323250 con 0x564762320680 2024-09-16T02:05:02.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.927+0000 7f162c4b9740 10 client.0 _ll_register_callbacks cb 0x564762276160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:02.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.927+0000 7f1622ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564762315dc0 0x56476231d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:02.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.927+0000 7f1622ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564762315dc0 0x56476231d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54992/0 (socket says 172.21.15.105:54992) 2024-09-16T02:05:02.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.927+0000 7f1622ffd640 1 -- 172.21.15.105:0/848901137 learned_addr learned my addr 172.21.15.105:0/848901137 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:02.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.927+0000 7f1622ffd640 1 -- 172.21.15.105:0/848901137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564762320680 msgr2=0x56476231d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:02.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.927+0000 7f1622ffd640 1 --2- 172.21.15.105:0/848901137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564762320680 0x56476231d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:02.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.927+0000 7f1622ffd640 1 -- 172.21.15.105:0/848901137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647623154a0 msgr2=0x56476231ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:02.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.927+0000 7f1622ffd640 1 --2- 172.21.15.105:0/848901137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647623154a0 0x56476231ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:02.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.927+0000 7f1622ffd640 1 -- 172.21.15.105:0/848901137 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564761fec3c0 con 0x564762315dc0 2024-09-16T02:05:02.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.927+0000 7f1622ffd640 1 --2- 172.21.15.105:0/848901137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564762315dc0 0x56476231d290 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1618098280 tx=0x7f1618015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b98bba1a25ed2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:02.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.927+0000 7f1602ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:05:02.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.927+0000 7f1602ffd640 1 -- 172.21.15.105:0/848901137 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1618020070 con 0x564762315dc0 2024-09-16T02:05:02.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.927+0000 7f1602ffd640 1 -- 172.21.15.105:0/848901137 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f1618004d40 con 0x564762315dc0 2024-09-16T02:05:02.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.927+0000 7f162c4b9740 1 -- 172.21.15.105:0/848901137 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f161c036040 con 0x564762315dc0 2024-09-16T02:05:02.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.927+0000 7f1602ffd640 1 -- 172.21.15.105:0/848901137 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f16180a4520 con 0x564762315dc0 2024-09-16T02:05:02.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.927+0000 7f162c4b9740 10 client.15654 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:02.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.927+0000 7f162c4b9740 1 -- 172.21.15.105:0/848901137 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56476232eb00 con 0x564762315dc0 2024-09-16T02:05:02.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.928+0000 7f1602ffd640 1 -- 172.21.15.105:0/848901137 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f161c036040 con 0x564762315dc0 2024-09-16T02:05:02.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.928+0000 7f1602ffd640 1 -- 172.21.15.105:0/848901137 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f161809cbe0 con 0x564762315dc0 2024-09-16T02:05:02.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.928+0000 7f162c4b9740 10 client.15654 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:02.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.928+0000 7f162c4b9740 10 client.15654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:05:02.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.928+0000 7f162c4b9740 20 client.15654 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:02.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.928+0000 7f162c4b9740 1 --2- 172.21.15.105:0/848901137 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x56476232ffc0 0x564762332430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:02.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.928+0000 7f162c4b9740 4 client.15654 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:05:02.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.928+0000 7f162c4b9740 1 -- 172.21.15.105:0/848901137 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564762288850 con 0x56476232ffc0 2024-09-16T02:05:02.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.928+0000 7f16237fe640 1 --2- 172.21.15.105:0/848901137 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x56476232ffc0 0x564762332430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:02.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.928+0000 7f1602ffd640 10 client.15654 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:05:02.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.928+0000 7f16237fe640 1 --2- 172.21.15.105:0/848901137 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x56476232ffc0 0x564762332430 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e402099ef386e0fa server_cookie=ef0c2002cde69135 in_seq=0 out_seq=0 2024-09-16T02:05:02.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.929+0000 7f1602ffd640 1 -- 172.21.15.105:0/848901137 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564762288850 con 0x56476232ffc0 2024-09-16T02:05:02.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.929+0000 7f1602ffd640 10 client.15654 handle_command_reply: tid=0 2024-09-16T02:05:02.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.929+0000 7f162c4b9740 1 client.15654 shutdown 2024-09-16T02:05:02.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.929+0000 7f162c4b9740 1 -- 172.21.15.105:0/848901137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564762315dc0 msgr2=0x56476231d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:02.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.929+0000 7f162c4b9740 1 --2- 172.21.15.105:0/848901137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564762315dc0 0x56476231d290 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1618098280 tx=0x7f1618015000 comp rx=0 tx=0).stop 2024-09-16T02:05:02.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.929+0000 7f162c4b9740 1 -- 172.21.15.105:0/848901137 shutdown_connections 2024-09-16T02:05:02.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.929+0000 7f162c4b9740 1 --2- 172.21.15.105:0/848901137 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x56476232ffc0 0x564762332430 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:02.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.929+0000 7f162c4b9740 1 --2- 172.21.15.105:0/848901137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564762320680 0x56476231d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:02.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.929+0000 7f162c4b9740 1 --2- 172.21.15.105:0/848901137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564762315dc0 0x56476231d290 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:02.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.929+0000 7f162c4b9740 1 --2- 172.21.15.105:0/848901137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647623154a0 0x56476231ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:02.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.929+0000 7f1602ffd640 0 client.15654 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:05:02.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.929+0000 7f162c4b9740 1 -- 172.21.15.105:0/848901137 >> 172.21.15.105:0/848901137 conn(0x564762311a30 msgr2=0x564762317760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:02.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.929+0000 7f162c4b9740 1 -- 172.21.15.105:0/848901137 shutdown_connections 2024-09-16T02:05:02.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.929+0000 7f162c4b9740 1 -- 172.21.15.105:0/848901137 wait complete. 2024-09-16T02:05:02.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.930+0000 7f162c4b9740 20 client.15654 trim_cache size 0 max 16384 2024-09-16T02:05:02.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.946+0000 7f162c4b9740 1 Processor -- start 2024-09-16T02:05:02.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.946+0000 7f162c4b9740 1 -- start start 2024-09-16T02:05:02.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.947+0000 7f162c4b9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647623154a0 0x564762337cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:02.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.947+0000 7f162c4b9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564762315dc0 0x5647623381f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:02.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.947+0000 7f162c4b9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564762320680 0x56476231a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:02.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.947+0000 7f162c4b9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f16241af760 con 0x564762315dc0 2024-09-16T02:05:02.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.947+0000 7f162c4b9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f16241b12a0 con 0x5647623154a0 2024-09-16T02:05:02.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.947+0000 7f162c4b9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f16241afe30 con 0x564762320680 2024-09-16T02:05:02.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.947+0000 7f1622ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564762315dc0 0x5647623381f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:02.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.947+0000 7f1622ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564762315dc0 0x5647623381f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55000/0 (socket says 172.21.15.105:55000) 2024-09-16T02:05:02.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.947+0000 7f1622ffd640 1 -- 172.21.15.105:0/3297635693 learned_addr learned my addr 172.21.15.105:0/3297635693 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:02.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.947+0000 7f1623fff640 1 --2- 172.21.15.105:0/3297635693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564762320680 0x56476231a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:02.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.947+0000 7f16237fe640 1 --2- 172.21.15.105:0/3297635693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647623154a0 0x564762337cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:02.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.947+0000 7f1622ffd640 1 -- 172.21.15.105:0/3297635693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564762320680 msgr2=0x56476231a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:02.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.947+0000 7f1622ffd640 1 --2- 172.21.15.105:0/3297635693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564762320680 0x56476231a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:02.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.947+0000 7f1622ffd640 1 -- 172.21.15.105:0/3297635693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647623154a0 msgr2=0x564762337cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:02.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.947+0000 7f1622ffd640 1 --2- 172.21.15.105:0/3297635693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647623154a0 0x564762337cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:02.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.947+0000 7f1622ffd640 1 -- 172.21.15.105:0/3297635693 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56476231db20 con 0x564762315dc0 2024-09-16T02:05:02.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.947+0000 7f16237fe640 1 --2- 172.21.15.105:0/3297635693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647623154a0 0x564762337cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:05:02.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.947+0000 7f1622ffd640 1 --2- 172.21.15.105:0/3297635693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564762315dc0 0x5647623381f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f1618005220 tx=0x7f1618097c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cfbb0028bd48461 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:02.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.947+0000 7f16097fa640 1 -- 172.21.15.105:0/3297635693 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1618020030 con 0x564762315dc0 2024-09-16T02:05:02.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.947+0000 7f16097fa640 1 -- 172.21.15.105:0/3297635693 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16180ba1c0 con 0x564762315dc0 2024-09-16T02:05:02.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.947+0000 7f16097fa640 1 -- 172.21.15.105:0/3297635693 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16180052a0 con 0x564762315dc0 2024-09-16T02:05:02.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.948+0000 7f162c4b9740 1 -- 172.21.15.105:0/3297635693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564762315dc0 msgr2=0x5647623381f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:02.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.948+0000 7f162c4b9740 1 --2- 172.21.15.105:0/3297635693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564762315dc0 0x5647623381f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f1618005220 tx=0x7f1618097c50 comp rx=0 tx=0).stop 2024-09-16T02:05:02.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.949+0000 7f162c4b9740 1 -- 172.21.15.105:0/3297635693 shutdown_connections 2024-09-16T02:05:02.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.949+0000 7f162c4b9740 1 --2- 172.21.15.105:0/3297635693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564762320680 0x56476231a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:02.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.949+0000 7f162c4b9740 1 --2- 172.21.15.105:0/3297635693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564762315dc0 0x5647623381f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:02.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.949+0000 7f162c4b9740 1 --2- 172.21.15.105:0/3297635693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647623154a0 0x564762337cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:02.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.949+0000 7f162c4b9740 1 -- 172.21.15.105:0/3297635693 >> 172.21.15.105:0/3297635693 conn(0x564762311a30 msgr2=0x56476231f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:02.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.949+0000 7f162c4b9740 1 -- 172.21.15.105:0/3297635693 shutdown_connections 2024-09-16T02:05:02.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.949+0000 7f162c4b9740 1 -- 172.21.15.105:0/3297635693 wait complete. 2024-09-16T02:05:02.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.949+0000 7f162c4b9740 1 -- start start 2024-09-16T02:05:02.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.949+0000 7f162c4b9740 1 Processor -- start 2024-09-16T02:05:02.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.949+0000 7f162c4b9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647623154a0 0x5647623265a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:02.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.949+0000 7f162c4b9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564762315dc0 0x564762326ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:02.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.949+0000 7f162c4b9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564762320680 0x564762399c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:02.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.949+0000 7f162c4b9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f16241b0bd0 con 0x564762320680 2024-09-16T02:05:02.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.949+0000 7f162c4b9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f16241b0500 con 0x5647623154a0 2024-09-16T02:05:02.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.949+0000 7f162c4b9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f16241ad740 con 0x564762315dc0 2024-09-16T02:05:02.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.950+0000 7f162c4b9740 10 client.0 _ll_register_callbacks cb 0x7f161809fc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:02.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.950+0000 7f1623fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564762320680 0x564762399c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:02.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.950+0000 7f1623fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564762320680 0x564762399c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55008/0 (socket says 172.21.15.105:55008) 2024-09-16T02:05:02.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.950+0000 7f1623fff640 1 -- 172.21.15.105:0/2199647565 learned_addr learned my addr 172.21.15.105:0/2199647565 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:02.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.950+0000 7f1622ffd640 1 --2- 172.21.15.105:0/2199647565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564762315dc0 0x564762326ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:02.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.950+0000 7f16237fe640 1 --2- 172.21.15.105:0/2199647565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647623154a0 0x5647623265a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:02.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.950+0000 7f1623fff640 1 -- 172.21.15.105:0/2199647565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564762315dc0 msgr2=0x564762326ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:02.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.950+0000 7f1623fff640 1 --2- 172.21.15.105:0/2199647565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564762315dc0 0x564762326ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:02.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.950+0000 7f1623fff640 1 -- 172.21.15.105:0/2199647565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647623154a0 msgr2=0x5647623265a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:02.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.950+0000 7f1623fff640 1 --2- 172.21.15.105:0/2199647565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647623154a0 0x5647623265a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:02.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.950+0000 7f1623fff640 1 -- 172.21.15.105:0/2199647565 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564762327370 con 0x564762320680 2024-09-16T02:05:02.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.950+0000 7f1622ffd640 1 --2- 172.21.15.105:0/2199647565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564762315dc0 0x564762326ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:05:02.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.950+0000 7f1623fff640 1 --2- 172.21.15.105:0/2199647565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564762320680 0x564762399c90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x56476230d5a0 tx=0x7f161c02f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a86a39830a2c0db server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:02.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.950+0000 7f1602ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:05:02.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.950+0000 7f1602ffd640 1 -- 172.21.15.105:0/2199647565 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c04e040 con 0x564762320680 2024-09-16T02:05:02.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.950+0000 7f1602ffd640 1 -- 172.21.15.105:0/2199647565 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f161c004510 con 0x564762320680 2024-09-16T02:05:02.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.950+0000 7f162c4b9740 1 -- 172.21.15.105:0/2199647565 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16180ba1c0 con 0x564762320680 2024-09-16T02:05:02.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.951+0000 7f1602ffd640 1 -- 172.21.15.105:0/2199647565 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f161c004710 con 0x564762320680 2024-09-16T02:05:02.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.951+0000 7f162c4b9740 10 client.15660 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:02.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.951+0000 7f162c4b9740 1 -- 172.21.15.105:0/2199647565 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56476239b220 con 0x564762320680 2024-09-16T02:05:02.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.951+0000 7f1602ffd640 1 -- 172.21.15.105:0/2199647565 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16180ba1c0 con 0x564762320680 2024-09-16T02:05:02.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.951+0000 7f1602ffd640 1 -- 172.21.15.105:0/2199647565 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f161c0308f0 con 0x564762320680 2024-09-16T02:05:02.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.951+0000 7f162c4b9740 10 client.15660 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:02.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.951+0000 7f162c4b9740 10 client.15660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:05:02.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.951+0000 7f162c4b9740 20 client.15660 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:02.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.951+0000 7f162c4b9740 1 --2- 172.21.15.105:0/2199647565 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x56476239b660 0x56476239da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:02.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.951+0000 7f162c4b9740 4 client.15660 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T02:05:02.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.951+0000 7f162c4b9740 1 -- 172.21.15.105:0/2199647565 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564762282d60 con 0x56476239b660 2024-09-16T02:05:02.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.951+0000 7f16237fe640 1 --2- 172.21.15.105:0/2199647565 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x56476239b660 0x56476239da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:02.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.952+0000 7f1602ffd640 10 client.15660 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:05:02.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.952+0000 7f16237fe640 1 --2- 172.21.15.105:0/2199647565 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x56476239b660 0x56476239da50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=795457013aeabce server_cookie=320a222fa66ce6db in_seq=0 out_seq=0 2024-09-16T02:05:02.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.952+0000 7f1602ffd640 1 -- 172.21.15.105:0/2199647565 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564762282d60 con 0x56476239b660 2024-09-16T02:05:02.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.952+0000 7f1602ffd640 10 client.15660 handle_command_reply: tid=0 2024-09-16T02:05:02.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.952+0000 7f162c4b9740 1 client.15660 shutdown 2024-09-16T02:05:02.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.952+0000 7f162c4b9740 1 -- 172.21.15.105:0/2199647565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564762320680 msgr2=0x564762399c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:02.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.952+0000 7f162c4b9740 1 --2- 172.21.15.105:0/2199647565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564762320680 0x564762399c90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x56476230d5a0 tx=0x7f161c02f310 comp rx=0 tx=0).stop 2024-09-16T02:05:02.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.952+0000 7f162c4b9740 1 -- 172.21.15.105:0/2199647565 shutdown_connections 2024-09-16T02:05:02.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.952+0000 7f162c4b9740 1 --2- 172.21.15.105:0/2199647565 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x56476239b660 0x56476239da50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:02.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.952+0000 7f162c4b9740 1 --2- 172.21.15.105:0/2199647565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564762315dc0 0x564762326ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:02.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.952+0000 7f162c4b9740 1 --2- 172.21.15.105:0/2199647565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564762320680 0x564762399c90 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:02.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.952+0000 7f162c4b9740 1 --2- 172.21.15.105:0/2199647565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647623154a0 0x5647623265a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:02.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.952+0000 7f162c4b9740 1 -- 172.21.15.105:0/2199647565 >> 172.21.15.105:0/2199647565 conn(0x564762311a30 msgr2=0x564762310350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:02.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.952+0000 7f1602ffd640 0 client.15660 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:05:02.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.952+0000 7f162c4b9740 1 -- 172.21.15.105:0/2199647565 shutdown_connections 2024-09-16T02:05:02.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.952+0000 7f162c4b9740 1 -- 172.21.15.105:0/2199647565 wait complete. 2024-09-16T02:05:02.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.952+0000 7f162c4b9740 20 client.15660 trim_cache size 0 max 16384 2024-09-16T02:05:02.975 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-16T02:05:02.976 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-09-16T02:05:02.976 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-09-16T02:05:02.976 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-16T02:05:02.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.953+0000 7f162a0e0640 1 -- 172.21.15.105:0/507080905 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f160c0a3240 msgr2=0x7f160c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:02.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.953+0000 7f162a0e0640 1 --2- 172.21.15.105:0/507080905 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f160c0a3240 0x7f160c0a5700 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f161c004af0 tx=0x7f161c03b000 comp rx=0 tx=0).stop 2024-09-16T02:05:02.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.953+0000 7f162a0e0640 1 -- 172.21.15.105:0/507080905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1624102790 msgr2=0x7f1624074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:02.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.953+0000 7f162a0e0640 1 --2- 172.21.15.105:0/507080905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1624102790 0x7f1624074a30 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f161800d770 tx=0x7f161800db70 comp rx=0 tx=0).stop 2024-09-16T02:05:02.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.953+0000 7f162a0e0640 1 -- 172.21.15.105:0/507080905 shutdown_connections 2024-09-16T02:05:02.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.953+0000 7f162a0e0640 1 --2- 172.21.15.105:0/507080905 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f160c0a3240 0x7f160c0a5700 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:02.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.953+0000 7f162a0e0640 1 --2- 172.21.15.105:0/507080905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1624106f10 0x7f1624074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:02.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.953+0000 7f162a0e0640 1 --2- 172.21.15.105:0/507080905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1624102790 0x7f1624074a30 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:02.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.953+0000 7f162a0e0640 1 --2- 172.21.15.105:0/507080905 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1624101de0 0x7f16240744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:02.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.953+0000 7f162a0e0640 1 -- 172.21.15.105:0/507080905 >> 172.21.15.105:0/507080905 conn(0x7f16240fd8c0 msgr2=0x7f16240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:02.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.953+0000 7f162a0e0640 1 -- 172.21.15.105:0/507080905 shutdown_connections 2024-09-16T02:05:02.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:02.953+0000 7f162a0e0640 1 -- 172.21.15.105:0/507080905 wait complete. 2024-09-16T02:05:03.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:02 smithi190 ceph-mon[34762]: pgmap v414: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 662 MiB/s wr, 181 op/s 2024-09-16T02:05:03.853 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T02:05:03.854 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:05:04.105 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:04.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:04.657+0000 7f0df9991640 1 -- 172.21.15.105:0/1850738526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0df4102ef0 msgr2=0x7f0df410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:04.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:04.657+0000 7f0df9991640 1 --2- 172.21.15.105:0/1850738526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0df4102ef0 0x7f0df410da40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0dec009ca0 tx=0x7f0dec02a950 comp rx=0 tx=0).stop 2024-09-16T02:05:04.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:04.658+0000 7f0df9991640 1 -- 172.21.15.105:0/1850738526 shutdown_connections 2024-09-16T02:05:04.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:04.658+0000 7f0df9991640 1 --2- 172.21.15.105:0/1850738526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0df4102570 0x7f0df41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:04.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:04.658+0000 7f0df9991640 1 --2- 172.21.15.105:0/1850738526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0df4102ef0 0x7f0df410da40 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:04.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:04.658+0000 7f0df9991640 1 --2- 172.21.15.105:0/1850738526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0df4100e20 0x7f0df4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:04.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:04.658+0000 7f0df9991640 1 -- 172.21.15.105:0/1850738526 >> 172.21.15.105:0/1850738526 conn(0x7f0df40fc7d0 msgr2=0x7f0df40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:04.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:04.658+0000 7f0df9991640 1 -- 172.21.15.105:0/1850738526 shutdown_connections 2024-09-16T02:05:04.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:04.659+0000 7f0df9991640 1 -- 172.21.15.105:0/1850738526 wait complete. 2024-09-16T02:05:04.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:04.660+0000 7f0df9991640 1 Processor -- start 2024-09-16T02:05:04.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:04.661+0000 7f0df9991640 1 -- start start 2024-09-16T02:05:04.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:04.661+0000 7f0df9991640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0df4100e20 0x7f0df4196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:04.661+0000 7f0df9991640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0df4102570 0x7f0df4196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:04.661+0000 7f0df9991640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0df4102ef0 0x7f0df419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:04.662+0000 7f0df9991640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0df410fd20 con 0x7f0df4102ef0 2024-09-16T02:05:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:04.662+0000 7f0df9991640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0df410fba0 con 0x7f0df4100e20 2024-09-16T02:05:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:04.662+0000 7f0df9991640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0df410fea0 con 0x7f0df4102570 2024-09-16T02:05:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:04.662+0000 7f0df37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0df4102ef0 0x7f0df419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:04.662+0000 7f0df37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0df4102ef0 0x7f0df419aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55036/0 (socket says 172.21.15.105:55036) 2024-09-16T02:05:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:04.662+0000 7f0df37fe640 1 -- 172.21.15.105:0/3603008467 learned_addr learned my addr 172.21.15.105:0/3603008467 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:04.663+0000 7f0df27fc640 1 --2- 172.21.15.105:0/3603008467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0df4102570 0x7f0df4196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:04.663+0000 7f0df37fe640 1 -- 172.21.15.105:0/3603008467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0df4102570 msgr2=0x7f0df4196740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:04.663+0000 7f0df37fe640 1 --2- 172.21.15.105:0/3603008467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0df4102570 0x7f0df4196740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:04.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:04.663+0000 7f0df37fe640 1 -- 172.21.15.105:0/3603008467 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0df4100e20 msgr2=0x7f0df4196200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:04.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:04.663+0000 7f0df37fe640 1 --2- 172.21.15.105:0/3603008467 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0df4100e20 0x7f0df4196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:04.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:04.663+0000 7f0df37fe640 1 -- 172.21.15.105:0/3603008467 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df419b1b0 con 0x7f0df4102ef0 2024-09-16T02:05:04.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:04.663+0000 7f0df37fe640 1 --2- 172.21.15.105:0/3603008467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0df4102ef0 0x7f0df419aad0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0dec02ae70 tx=0x7f0dec02f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:04.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:04.664+0000 7f0df898f640 1 -- 172.21.15.105:0/3603008467 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dec034040 con 0x7f0df4102ef0 2024-09-16T02:05:04.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:04.664+0000 7f0df898f640 1 -- 172.21.15.105:0/3603008467 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0dec0308f0 con 0x7f0df4102ef0 2024-09-16T02:05:04.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:04.664+0000 7f0df898f640 1 -- 172.21.15.105:0/3603008467 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dec004780 con 0x7f0df4102ef0 2024-09-16T02:05:04.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:04.664+0000 7f0df9991640 1 -- 172.21.15.105:0/3603008467 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0df419b440 con 0x7f0df4102ef0 2024-09-16T02:05:04.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:04.664+0000 7f0df9991640 1 -- 172.21.15.105:0/3603008467 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0df41a2d80 con 0x7f0df4102ef0 2024-09-16T02:05:04.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:04.665+0000 7f0df9991640 1 -- 172.21.15.105:0/3603008467 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0df4101200 con 0x7f0df4102ef0 2024-09-16T02:05:04.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:04.669+0000 7f0df898f640 1 -- 172.21.15.105:0/3603008467 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f0dec037020 con 0x7f0df4102ef0 2024-09-16T02:05:04.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:04.670+0000 7f0df898f640 1 --2- 172.21.15.105:0/3603008467 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f0ddc0a76d0 0x7f0ddc0a9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:04.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:04.671+0000 7f0df898f640 1 -- 172.21.15.105:0/3603008467 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f0dec0f0a60 con 0x7f0df4102ef0 2024-09-16T02:05:04.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:04.671+0000 7f0df2ffd640 1 --2- 172.21.15.105:0/3603008467 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f0ddc0a76d0 0x7f0ddc0a9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:04.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:04.672+0000 7f0df2ffd640 1 --2- 172.21.15.105:0/3603008467 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f0ddc0a76d0 0x7f0ddc0a9b90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0de4005fd0 tx=0x7f0de4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:04.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:04.677+0000 7f0df898f640 1 -- 172.21.15.105:0/3603008467 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0dec0bd7b0 con 0x7f0df4102ef0 2024-09-16T02:05:04.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:04 smithi105 ceph-mon[31197]: pgmap v415: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 467 MiB/s wr, 127 op/s 2024-09-16T02:05:04.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:04 smithi137 ceph-mon[34099]: pgmap v415: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 467 MiB/s wr, 127 op/s 2024-09-16T02:05:05.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:05.027+0000 7f0df9991640 1 -- 172.21.15.105:0/3603008467 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0df4109090 con 0x7f0df4102ef0 2024-09-16T02:05:05.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:05.028+0000 7f0df898f640 1 -- 172.21.15.105:0/3603008467 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11991 (secure 0 0 0) 0x7f0dec0dd0e0 con 0x7f0df4102ef0 2024-09-16T02:05:05.030 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:05:05.030 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":28,"btime":"2024-09-16T02:02:41:554178+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2057699920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2057699920},{"type":"v1","addr":"172.21.15.105:6835","nonce":2057699920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1764886948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1764886948},{"type":"v1","addr":"172.21.15.105:6837","nonce":1764886948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T02:00:35.112266+0000","modified":"2024-09-16T02:02:40.885614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24469,"mds_2":24461,"mds_3":24457,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/4112062361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":4112062361},{"type":"v1","addr":"172.21.15.105:6839","nonce":4112062361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6841/163156514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":163156514},{"type":"v1","addr":"172.21.15.105:6841","nonce":163156514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/683519097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":683519097},{"type":"v1","addr":"172.21.15.190:6833","nonce":683519097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3025442862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3025442862},{"type":"v1","addr":"172.21.15.137:6833","nonce":3025442862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/763783602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":763783602},{"type":"v1","addr":"172.21.15.137:6835","nonce":763783602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.190:6835/1851042281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1851042281},{"type":"v1","addr":"172.21.15.190:6835","nonce":1851042281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/3181257549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3181257549},{"type":"v1","addr":"172.21.15.190:6837","nonce":3181257549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6837/2788541030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2788541030},{"type":"v1","addr":"172.21.15.137:6837","nonce":2788541030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.190:6839/1222573537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":1222573537},{"type":"v1","addr":"172.21.15.190:6839","nonce":1222573537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/3454924009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3454924009},{"type":"v1","addr":"172.21.15.137:6839","nonce":3454924009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24445,24457,24461,24469,24482]},"id":1}]} 2024-09-16T02:05:05.030 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 28 2024-09-16T02:05:05.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:05.030+0000 7f0df9991640 1 -- 172.21.15.105:0/3603008467 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f0ddc0a76d0 msgr2=0x7f0ddc0a9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:05.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:05.030+0000 7f0df9991640 1 --2- 172.21.15.105:0/3603008467 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f0ddc0a76d0 0x7f0ddc0a9b90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0de4005fd0 tx=0x7f0de4005850 comp rx=0 tx=0).stop 2024-09-16T02:05:05.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:05.031+0000 7f0df9991640 1 -- 172.21.15.105:0/3603008467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0df4102ef0 msgr2=0x7f0df419aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:05.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:05.031+0000 7f0df9991640 1 --2- 172.21.15.105:0/3603008467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0df4102ef0 0x7f0df419aad0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0dec02ae70 tx=0x7f0dec02f5d0 comp rx=0 tx=0).stop 2024-09-16T02:05:05.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:05.031+0000 7f0df9991640 1 -- 172.21.15.105:0/3603008467 shutdown_connections 2024-09-16T02:05:05.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:05.031+0000 7f0df9991640 1 --2- 172.21.15.105:0/3603008467 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f0ddc0a76d0 0x7f0ddc0a9b90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:05.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:05.031+0000 7f0df9991640 1 --2- 172.21.15.105:0/3603008467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0df4102570 0x7f0df4196740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:05.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:05.031+0000 7f0df9991640 1 --2- 172.21.15.105:0/3603008467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0df4102ef0 0x7f0df419aad0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:05.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:05.031+0000 7f0df9991640 1 --2- 172.21.15.105:0/3603008467 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0df4100e20 0x7f0df4196200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:05.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:05.031+0000 7f0df9991640 1 -- 172.21.15.105:0/3603008467 >> 172.21.15.105:0/3603008467 conn(0x7f0df40fc7d0 msgr2=0x7f0df40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:05.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:05.031+0000 7f0df9991640 1 -- 172.21.15.105:0/3603008467 shutdown_connections 2024-09-16T02:05:05.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:05.031+0000 7f0df9991640 1 -- 172.21.15.105:0/3603008467 wait complete. 2024-09-16T02:05:05.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:04 smithi190 ceph-mon[34762]: pgmap v415: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 467 MiB/s wr, 127 op/s 2024-09-16T02:05:05.877 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:05 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/3603008467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:05:05.879 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-16T02:05:05.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:05 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/3603008467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:05:06.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:05 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/3603008467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:05:06.320 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:06.768 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:06 smithi105 ceph-mon[31197]: pgmap v416: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 441 MiB/s wr, 121 op/s 2024-09-16T02:05:06.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.914+0000 7fbc60abc640 1 -- 172.21.15.105:0/200579236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc5c102780 msgr2=0x7fbc5c1067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:06.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.914+0000 7fbc60abc640 1 --2- 172.21.15.105:0/200579236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc5c102780 0x7fbc5c1067d0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fbc50009900 tx=0x7fbc5002a3f0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.914+0000 7fbc60abc640 1 -- 172.21.15.105:0/200579236 shutdown_connections 2024-09-16T02:05:06.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.914+0000 7fbc60abc640 1 --2- 172.21.15.105:0/200579236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc5c106f00 0x7fbc5c10ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.914+0000 7fbc60abc640 1 --2- 172.21.15.105:0/200579236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc5c102780 0x7fbc5c1067d0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.914+0000 7fbc60abc640 1 --2- 172.21.15.105:0/200579236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc5c101dd0 0x7fbc5c1021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.914+0000 7fbc60abc640 1 -- 172.21.15.105:0/200579236 >> 172.21.15.105:0/200579236 conn(0x7fbc5c0fd870 msgr2=0x7fbc5c0ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:06.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.914+0000 7fbc60abc640 1 -- 172.21.15.105:0/200579236 shutdown_connections 2024-09-16T02:05:06.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.914+0000 7fbc60abc640 1 -- 172.21.15.105:0/200579236 wait complete. 2024-09-16T02:05:06.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.915+0000 7fbc60abc640 1 Processor -- start 2024-09-16T02:05:06.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.915+0000 7fbc60abc640 1 -- start start 2024-09-16T02:05:06.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.916+0000 7fbc60abc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc5c101dd0 0x7fbc5c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:06.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.916+0000 7fbc60abc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc5c102780 0x7fbc5c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:06.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.916+0000 7fbc60abc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc5c106f00 0x7fbc5c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:06.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.916+0000 7fbc60abc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5c111e30 con 0x7fbc5c102780 2024-09-16T02:05:06.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.916+0000 7fbc60abc640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5c111cb0 con 0x7fbc5c101dd0 2024-09-16T02:05:06.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.916+0000 7fbc60abc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5c111fb0 con 0x7fbc5c106f00 2024-09-16T02:05:06.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.916+0000 7fbc59d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc5c102780 0x7fbc5c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:06.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.916+0000 7fbc59d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc5c102780 0x7fbc5c198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55050/0 (socket says 172.21.15.105:55050) 2024-09-16T02:05:06.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.916+0000 7fbc59d9b640 1 -- 172.21.15.105:0/3623578850 learned_addr learned my addr 172.21.15.105:0/3623578850 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:06.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.916+0000 7fbc5ad9d640 1 --2- 172.21.15.105:0/3623578850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc5c106f00 0x7fbc5c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:06.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.916+0000 7fbc5a59c640 1 --2- 172.21.15.105:0/3623578850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc5c101dd0 0x7fbc5c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:06.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.917+0000 7fbc59d9b640 1 -- 172.21.15.105:0/3623578850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc5c106f00 msgr2=0x7fbc5c19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:06.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.917+0000 7fbc59d9b640 1 --2- 172.21.15.105:0/3623578850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc5c106f00 0x7fbc5c19cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.917+0000 7fbc59d9b640 1 -- 172.21.15.105:0/3623578850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc5c101dd0 msgr2=0x7fbc5c1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:06.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.917+0000 7fbc59d9b640 1 --2- 172.21.15.105:0/3623578850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc5c101dd0 0x7fbc5c1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.917+0000 7fbc59d9b640 1 -- 172.21.15.105:0/3623578850 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc5c19d380 con 0x7fbc5c102780 2024-09-16T02:05:06.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.917+0000 7fbc5a59c640 1 --2- 172.21.15.105:0/3623578850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc5c101dd0 0x7fbc5c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:05:06.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.917+0000 7fbc59d9b640 1 --2- 172.21.15.105:0/3623578850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc5c102780 0x7fbc5c198910 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fbc50006010 tx=0x7fbc50004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:06.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.918+0000 7fbc4b7fe640 1 -- 172.21.15.105:0/3623578850 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5003f030 con 0x7fbc5c102780 2024-09-16T02:05:06.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.918+0000 7fbc4b7fe640 1 -- 172.21.15.105:0/3623578850 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc50003800 con 0x7fbc5c102780 2024-09-16T02:05:06.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.918+0000 7fbc60abc640 1 -- 172.21.15.105:0/3623578850 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc5c19d610 con 0x7fbc5c102780 2024-09-16T02:05:06.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:06 smithi137 ceph-mon[34099]: pgmap v416: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 441 MiB/s wr, 121 op/s 2024-09-16T02:05:06.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.919+0000 7fbc60abc640 1 -- 172.21.15.105:0/3623578850 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc5c1a4f50 con 0x7fbc5c102780 2024-09-16T02:05:06.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.919+0000 7fbc4b7fe640 1 -- 172.21.15.105:0/3623578850 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc50035070 con 0x7fbc5c102780 2024-09-16T02:05:06.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.922+0000 7fbc4b7fe640 1 -- 172.21.15.105:0/3623578850 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fbc500039b0 con 0x7fbc5c102780 2024-09-16T02:05:06.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.923+0000 7fbc62e95740 1 Processor -- start 2024-09-16T02:05:06.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.923+0000 7fbc4b7fe640 1 --2- 172.21.15.105:0/3623578850 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fbc440a76d0 0x7fbc440a9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:06.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.924+0000 7fbc5ad9d640 1 --2- 172.21.15.105:0/3623578850 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fbc440a76d0 0x7fbc440a9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:06.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.924+0000 7fbc4b7fe640 1 -- 172.21.15.105:0/3623578850 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fbc500f0050 con 0x7fbc5c102780 2024-09-16T02:05:06.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.924+0000 7fbc62e95740 1 -- start start 2024-09-16T02:05:06.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.924+0000 7fbc62e95740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a5700cb40 0x562a5700b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:06.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.924+0000 7fbc62e95740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a5700b5d0 0x562a5700ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:06.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.924+0000 7fbc62e95740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a57016370 0x562a57018760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:06.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.924+0000 7fbc5a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a5700cb40 0x562a5700b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:06.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.924+0000 7fbc5a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a5700cb40 0x562a5700b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55052/0 (socket says 172.21.15.105:55052) 2024-09-16T02:05:06.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.924+0000 7fbc5a59c640 1 -- 172.21.15.105:0/2765321028 learned_addr learned my addr 172.21.15.105:0/2765321028 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:06.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.924+0000 7fbc62e95740 1 -- 172.21.15.105:0/2765321028 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562a56f712f0 con 0x562a5700cb40 2024-09-16T02:05:06.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.924+0000 7fbc62e95740 1 -- 172.21.15.105:0/2765321028 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562a56f18cf0 con 0x562a5700b5d0 2024-09-16T02:05:06.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.924+0000 7fbc62e95740 1 -- 172.21.15.105:0/2765321028 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x562a56f6d2b0 con 0x562a57016370 2024-09-16T02:05:06.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.924+0000 7fbc5ad9d640 1 --2- 172.21.15.105:0/3623578850 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fbc440a76d0 0x7fbc440a9b90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbc54009ea0 tx=0x7fbc54009410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:06.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.925+0000 7fbc5a59c640 1 -- 172.21.15.105:0/2765321028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a57016370 msgr2=0x562a57018760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T02:05:06.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.925+0000 7fbc5a59c640 1 --2- 172.21.15.105:0/2765321028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a57016370 0x562a57018760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.925+0000 7fbc5a59c640 1 -- 172.21.15.105:0/2765321028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a5700b5d0 msgr2=0x562a5700ba10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T02:05:06.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.925+0000 7fbc5a59c640 1 --2- 172.21.15.105:0/2765321028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a5700b5d0 0x562a5700ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.925+0000 7fbc5a59c640 1 -- 172.21.15.105:0/2765321028 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a56efd3a0 con 0x562a5700cb40 2024-09-16T02:05:06.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.925+0000 7fbc5a59c640 1 --2- 172.21.15.105:0/2765321028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a5700cb40 0x562a5700b090 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c00bc20 tx=0x7fbc4c00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ce02bb1af53fa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:06.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.925+0000 7fbc48ff9640 1 -- 172.21.15.105:0/2765321028 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c017620 con 0x562a5700cb40 2024-09-16T02:05:06.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.925+0000 7fbc48ff9640 1 -- 172.21.15.105:0/2765321028 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc4c017c70 con 0x562a5700cb40 2024-09-16T02:05:06.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.925+0000 7fbc48ff9640 1 -- 172.21.15.105:0/2765321028 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c00fc20 con 0x562a5700cb40 2024-09-16T02:05:06.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.926+0000 7fbc62e95740 1 -- 172.21.15.105:0/2765321028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a5700cb40 msgr2=0x562a5700b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:06.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.926+0000 7fbc62e95740 1 --2- 172.21.15.105:0/2765321028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a5700cb40 0x562a5700b090 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c00bc20 tx=0x7fbc4c00d710 comp rx=0 tx=0).stop 2024-09-16T02:05:06.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.926+0000 7fbc62e95740 1 -- 172.21.15.105:0/2765321028 shutdown_connections 2024-09-16T02:05:06.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.926+0000 7fbc62e95740 1 --2- 172.21.15.105:0/2765321028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a57016370 0x562a57018760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.926+0000 7fbc62e95740 1 --2- 172.21.15.105:0/2765321028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a5700b5d0 0x562a5700ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.926+0000 7fbc62e95740 1 --2- 172.21.15.105:0/2765321028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a5700cb40 0x562a5700b090 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.926+0000 7fbc62e95740 1 -- 172.21.15.105:0/2765321028 >> 172.21.15.105:0/2765321028 conn(0x562a56fd4310 msgr2=0x562a57007c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:06.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.926+0000 7fbc62e95740 1 -- 172.21.15.105:0/2765321028 shutdown_connections 2024-09-16T02:05:06.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.926+0000 7fbc62e95740 1 -- 172.21.15.105:0/2765321028 wait complete. 2024-09-16T02:05:06.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.927+0000 7fbc62e95740 1 -- start start 2024-09-16T02:05:06.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.927+0000 7fbc62e95740 1 Processor -- start 2024-09-16T02:05:06.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.927+0000 7fbc62e95740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a5700b5d0 0x562a57015db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:06.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.927+0000 7fbc62e95740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a5700cb40 0x562a57013060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:06.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.927+0000 7fbc62e95740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a57016370 0x562a570137f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:06.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.927+0000 7fbc62e95740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562a57019070 con 0x562a5700b5d0 2024-09-16T02:05:06.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.927+0000 7fbc62e95740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562a56f2c060 con 0x562a57016370 2024-09-16T02:05:06.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.927+0000 7fbc62e95740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x562a570191f0 con 0x562a5700cb40 2024-09-16T02:05:06.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.927+0000 7fbc62e95740 10 client.0 _ll_register_callbacks cb 0x562a56c7f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:06.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.927+0000 7fbc5a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a5700b5d0 0x562a57015db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:06.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.928+0000 7fbc5a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a5700b5d0 0x562a57015db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55062/0 (socket says 172.21.15.105:55062) 2024-09-16T02:05:06.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.928+0000 7fbc5a59c640 1 -- 172.21.15.105:0/2148687685 learned_addr learned my addr 172.21.15.105:0/2148687685 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:06.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.928+0000 7fbc5a59c640 1 -- 172.21.15.105:0/2148687685 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a5700cb40 msgr2=0x562a57013060 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T02:05:06.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.928+0000 7fbc5a59c640 1 --2- 172.21.15.105:0/2148687685 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a5700cb40 0x562a57013060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.928+0000 7fbc5a59c640 1 -- 172.21.15.105:0/2148687685 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a57016370 msgr2=0x562a570137f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:06.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.928+0000 7fbc5a59c640 1 --2- 172.21.15.105:0/2148687685 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a57016370 0x562a570137f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.928+0000 7fbc5a59c640 1 -- 172.21.15.105:0/2148687685 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a56ce23c0 con 0x562a5700b5d0 2024-09-16T02:05:06.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.928+0000 7fbc5a59c640 1 --2- 172.21.15.105:0/2148687685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a5700b5d0 0x562a57015db0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c0058c0 tx=0x7fbc4c00fda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fae96f10fb53a16c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:06.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.928+0000 7fbc41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:05:06.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.929+0000 7fbc41ffb640 1 -- 172.21.15.105:0/2148687685 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c026030 con 0x562a5700b5d0 2024-09-16T02:05:06.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.929+0000 7fbc62e95740 1 -- 172.21.15.105:0/2148687685 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc4c017c70 con 0x562a5700b5d0 2024-09-16T02:05:06.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.929+0000 7fbc41ffb640 1 -- 172.21.15.105:0/2148687685 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fbc4c01d040 con 0x562a5700b5d0 2024-09-16T02:05:06.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.929+0000 7fbc41ffb640 1 -- 172.21.15.105:0/2148687685 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbc4c030050 con 0x562a5700b5d0 2024-09-16T02:05:06.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.929+0000 7fbc62e95740 10 client.15678 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:06.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.929+0000 7fbc62e95740 1 -- 172.21.15.105:0/2148687685 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562a57024b60 con 0x562a5700b5d0 2024-09-16T02:05:06.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.929+0000 7fbc41ffb640 1 -- 172.21.15.105:0/2148687685 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc4c017c70 con 0x562a5700b5d0 2024-09-16T02:05:06.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.930+0000 7fbc41ffb640 1 -- 172.21.15.105:0/2148687685 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbc4c0053f0 con 0x562a5700b5d0 2024-09-16T02:05:06.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.930+0000 7fbc62e95740 10 client.15678 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:06.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.930+0000 7fbc62e95740 10 client.15678 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T02:05:06.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.930+0000 7fbc62e95740 20 client.15678 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:06.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.930+0000 7fbc62e95740 1 --2- 172.21.15.105:0/2148687685 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x562a57025f10 0x562a57028300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:06.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.930+0000 7fbc62e95740 4 client.15678 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:05:06.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.930+0000 7fbc62e95740 1 -- 172.21.15.105:0/2148687685 --> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a56f7e860 con 0x562a57025f10 2024-09-16T02:05:06.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.931+0000 7fbc59d9b640 1 --2- 172.21.15.105:0/2148687685 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x562a57025f10 0x562a57028300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:06.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.931+0000 7fbc59d9b640 1 --2- 172.21.15.105:0/2148687685 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x562a57025f10 0x562a57028300 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d70afffaca2edca0 server_cookie=c3cfd0c3f1e97ac in_seq=0 out_seq=0 2024-09-16T02:05:06.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.931+0000 7fbc41ffb640 10 client.15678 ms_handle_connect on v2:172.21.15.190:6834/1851042281 2024-09-16T02:05:06.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.932+0000 7fbc41ffb640 1 -- 172.21.15.105:0/2148687685 <== mds.4 v2:172.21.15.190:6834/1851042281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562a56f7e860 con 0x562a57025f10 2024-09-16T02:05:06.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.932+0000 7fbc41ffb640 10 client.15678 handle_command_reply: tid=0 2024-09-16T02:05:06.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.932+0000 7fbc62e95740 1 client.15678 shutdown 2024-09-16T02:05:06.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.932+0000 7fbc62e95740 1 -- 172.21.15.105:0/2148687685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a5700b5d0 msgr2=0x562a57015db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:06.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.932+0000 7fbc62e95740 1 --2- 172.21.15.105:0/2148687685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a5700b5d0 0x562a57015db0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c0058c0 tx=0x7fbc4c00fda0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.933+0000 7fbc62e95740 1 -- 172.21.15.105:0/2148687685 shutdown_connections 2024-09-16T02:05:06.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.933+0000 7fbc62e95740 1 --2- 172.21.15.105:0/2148687685 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x562a57025f10 0x562a57028300 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.933+0000 7fbc62e95740 1 --2- 172.21.15.105:0/2148687685 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a5700cb40 0x562a57013060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.933+0000 7fbc62e95740 1 --2- 172.21.15.105:0/2148687685 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a57016370 0x562a570137f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.933+0000 7fbc41ffb640 0 client.15678 ms_handle_reset on v2:172.21.15.190:6834/1851042281 2024-09-16T02:05:06.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.933+0000 7fbc62e95740 1 --2- 172.21.15.105:0/2148687685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a5700b5d0 0x562a57015db0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.933+0000 7fbc62e95740 1 -- 172.21.15.105:0/2148687685 >> 172.21.15.105:0/2148687685 conn(0x562a56fd4310 msgr2=0x562a5700d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:06.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.933+0000 7fbc62e95740 1 -- 172.21.15.105:0/2148687685 shutdown_connections 2024-09-16T02:05:06.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.933+0000 7fbc62e95740 1 -- 172.21.15.105:0/2148687685 wait complete. 2024-09-16T02:05:06.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.933+0000 7fbc62e95740 20 client.15678 trim_cache size 0 max 16384 2024-09-16T02:05:06.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.956+0000 7fbc62e95740 1 Processor -- start 2024-09-16T02:05:06.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.956+0000 7fbc62e95740 1 -- start start 2024-09-16T02:05:06.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.957+0000 7fbc62e95740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a5700b5d0 0x562a57012920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:06.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.957+0000 7fbc62e95740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a5700cb40 0x562a5700e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:06.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.957+0000 7fbc62e95740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a57016370 0x562a5700e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:06.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.957+0000 7fbc62e95740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5c1ab410 con 0x562a5700cb40 2024-09-16T02:05:06.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.957+0000 7fbc62e95740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5c1acf50 con 0x562a5700b5d0 2024-09-16T02:05:06.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.957+0000 7fbc62e95740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5c1abae0 con 0x562a57016370 2024-09-16T02:05:06.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.958+0000 7fbc59d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a5700cb40 0x562a5700e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:06.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.958+0000 7fbc59d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a5700cb40 0x562a5700e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55066/0 (socket says 172.21.15.105:55066) 2024-09-16T02:05:06.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.958+0000 7fbc59d9b640 1 -- 172.21.15.105:0/3966387337 learned_addr learned my addr 172.21.15.105:0/3966387337 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:06.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.959+0000 7fbc5a59c640 1 --2- 172.21.15.105:0/3966387337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a5700b5d0 0x562a57012920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:06.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.959+0000 7fbc5ad9d640 1 --2- 172.21.15.105:0/3966387337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a57016370 0x562a5700e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:06.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.959+0000 7fbc59d9b640 1 -- 172.21.15.105:0/3966387337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a57016370 msgr2=0x562a5700e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:06.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.959+0000 7fbc59d9b640 1 --2- 172.21.15.105:0/3966387337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a57016370 0x562a5700e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.959+0000 7fbc59d9b640 1 -- 172.21.15.105:0/3966387337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a5700b5d0 msgr2=0x562a57012920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:06.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.959+0000 7fbc59d9b640 1 --2- 172.21.15.105:0/3966387337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a5700b5d0 0x562a57012920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.959+0000 7fbc59d9b640 1 -- 172.21.15.105:0/3966387337 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a570115b0 con 0x562a5700cb40 2024-09-16T02:05:06.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.959+0000 7fbc59d9b640 1 --2- 172.21.15.105:0/3966387337 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a5700cb40 0x562a5700e120 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fbc50005ec0 tx=0x7fbc500ed9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dadec1274140488 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:06.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.959+0000 7fbc43fff640 1 -- 172.21.15.105:0/3966387337 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc50049040 con 0x562a5700cb40 2024-09-16T02:05:06.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.959+0000 7fbc43fff640 1 -- 172.21.15.105:0/3966387337 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc500ba6b0 con 0x562a5700cb40 2024-09-16T02:05:06.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.960+0000 7fbc43fff640 1 -- 172.21.15.105:0/3966387337 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc500c5050 con 0x562a5700cb40 2024-09-16T02:05:06.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.960+0000 7fbc62e95740 1 -- 172.21.15.105:0/3966387337 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a5700cb40 msgr2=0x562a5700e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:06.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.961+0000 7fbc62e95740 1 --2- 172.21.15.105:0/3966387337 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a5700cb40 0x562a5700e120 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fbc50005ec0 tx=0x7fbc500ed9c0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.961+0000 7fbc62e95740 1 -- 172.21.15.105:0/3966387337 shutdown_connections 2024-09-16T02:05:06.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.961+0000 7fbc62e95740 1 --2- 172.21.15.105:0/3966387337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a57016370 0x562a5700e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.961+0000 7fbc62e95740 1 --2- 172.21.15.105:0/3966387337 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a5700cb40 0x562a5700e120 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.961+0000 7fbc62e95740 1 --2- 172.21.15.105:0/3966387337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a5700b5d0 0x562a57012920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.961+0000 7fbc62e95740 1 -- 172.21.15.105:0/3966387337 >> 172.21.15.105:0/3966387337 conn(0x562a56fd4310 msgr2=0x562a57012150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:06.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.961+0000 7fbc62e95740 1 -- 172.21.15.105:0/3966387337 shutdown_connections 2024-09-16T02:05:06.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.961+0000 7fbc62e95740 1 -- 172.21.15.105:0/3966387337 wait complete. 2024-09-16T02:05:06.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.961+0000 7fbc62e95740 1 -- start start 2024-09-16T02:05:06.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.961+0000 7fbc62e95740 1 Processor -- start 2024-09-16T02:05:06.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.962+0000 7fbc62e95740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a5700b5d0 0x562a5701e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:06.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.962+0000 7fbc62e95740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a5700cb40 0x562a5701eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:06.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.962+0000 7fbc62e95740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a57016370 0x562a570199a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:06.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.962+0000 7fbc62e95740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5c1ac880 con 0x562a5700b5d0 2024-09-16T02:05:06.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.962+0000 7fbc62e95740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5c1ac1b0 con 0x562a57016370 2024-09-16T02:05:06.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.962+0000 7fbc62e95740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5c1a93f0 con 0x562a5700cb40 2024-09-16T02:05:06.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.962+0000 7fbc62e95740 10 client.0 _ll_register_callbacks cb 0x7fbc4c00ba70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:06.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.962+0000 7fbc5a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a5700b5d0 0x562a5701e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:06.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.962+0000 7fbc5a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a5700b5d0 0x562a5701e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55080/0 (socket says 172.21.15.105:55080) 2024-09-16T02:05:06.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.962+0000 7fbc5a59c640 1 -- 172.21.15.105:0/1458876406 learned_addr learned my addr 172.21.15.105:0/1458876406 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:06.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.962+0000 7fbc5a59c640 1 -- 172.21.15.105:0/1458876406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a5700cb40 msgr2=0x562a5701eb20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T02:05:06.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.962+0000 7fbc5a59c640 1 --2- 172.21.15.105:0/1458876406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a5700cb40 0x562a5701eb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.962+0000 7fbc5a59c640 1 -- 172.21.15.105:0/1458876406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a57016370 msgr2=0x562a570199a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:06.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.962+0000 7fbc5a59c640 1 --2- 172.21.15.105:0/1458876406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a57016370 0x562a570199a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.963+0000 7fbc5a59c640 1 -- 172.21.15.105:0/1458876406 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a5701a1d0 con 0x562a5700b5d0 2024-09-16T02:05:06.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.963+0000 7fbc5a59c640 1 --2- 172.21.15.105:0/1458876406 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a5700b5d0 0x562a5701e5e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c00f7f0 tx=0x7fbc4c00fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0fa52a860ab32ff server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:06.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.963+0000 7fbc41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:05:06.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.963+0000 7fbc41ffb640 1 -- 172.21.15.105:0/1458876406 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c038020 con 0x562a5700b5d0 2024-09-16T02:05:06.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.963+0000 7fbc41ffb640 1 -- 172.21.15.105:0/1458876406 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fbc4c005050 con 0x562a5700b5d0 2024-09-16T02:05:06.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.963+0000 7fbc62e95740 1 -- 172.21.15.105:0/1458876406 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc500ba6b0 con 0x562a5700b5d0 2024-09-16T02:05:06.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.964+0000 7fbc41ffb640 1 -- 172.21.15.105:0/1458876406 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbc4c026070 con 0x562a5700b5d0 2024-09-16T02:05:06.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.964+0000 7fbc62e95740 10 client.15684 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:06.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.964+0000 7fbc62e95740 1 -- 172.21.15.105:0/1458876406 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562a57090450 con 0x562a5700b5d0 2024-09-16T02:05:06.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.965+0000 7fbc41ffb640 1 -- 172.21.15.105:0/1458876406 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc500ba6b0 con 0x562a5700b5d0 2024-09-16T02:05:06.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.965+0000 7fbc41ffb640 1 -- 172.21.15.105:0/1458876406 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbc4c017ba0 con 0x562a5700b5d0 2024-09-16T02:05:06.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.965+0000 7fbc62e95740 10 client.15684 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:06.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.965+0000 7fbc62e95740 10 client.15684 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T02:05:06.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.965+0000 7fbc62e95740 20 client.15684 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:06.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.965+0000 7fbc62e95740 1 --2- 172.21.15.105:0/1458876406 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x562a57090b30 0x562a57092ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:06.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.965+0000 7fbc62e95740 4 client.15684 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T02:05:06.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.965+0000 7fbc62e95740 1 -- 172.21.15.105:0/1458876406 --> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562a56f78d70 con 0x562a57090b30 2024-09-16T02:05:06.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.966+0000 7fbc59d9b640 1 --2- 172.21.15.105:0/1458876406 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x562a57090b30 0x562a57092ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:06.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.968+0000 7fbc59d9b640 1 --2- 172.21.15.105:0/1458876406 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x562a57090b30 0x562a57092ff0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5ef4f246cac9c586 server_cookie=f6e5b312e90c4be2 in_seq=0 out_seq=0 2024-09-16T02:05:06.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.968+0000 7fbc41ffb640 10 client.15684 ms_handle_connect on v2:172.21.15.190:6834/1851042281 2024-09-16T02:05:06.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.968+0000 7fbc41ffb640 1 -- 172.21.15.105:0/1458876406 <== mds.4 v2:172.21.15.190:6834/1851042281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562a56f78d70 con 0x562a57090b30 2024-09-16T02:05:06.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.969+0000 7fbc41ffb640 10 client.15684 handle_command_reply: tid=0 2024-09-16T02:05:06.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.969+0000 7fbc62e95740 1 client.15684 shutdown 2024-09-16T02:05:06.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.969+0000 7fbc62e95740 1 -- 172.21.15.105:0/1458876406 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a5700b5d0 msgr2=0x562a5701e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:06.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.969+0000 7fbc62e95740 1 --2- 172.21.15.105:0/1458876406 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a5700b5d0 0x562a5701e5e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c00f7f0 tx=0x7fbc4c00fa30 comp rx=0 tx=0).stop 2024-09-16T02:05:06.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.969+0000 7fbc62e95740 1 -- 172.21.15.105:0/1458876406 shutdown_connections 2024-09-16T02:05:06.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.969+0000 7fbc62e95740 1 --2- 172.21.15.105:0/1458876406 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x562a57090b30 0x562a57092ff0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.969+0000 7fbc62e95740 1 --2- 172.21.15.105:0/1458876406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a5700cb40 0x562a5701eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.969+0000 7fbc62e95740 1 --2- 172.21.15.105:0/1458876406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a57016370 0x562a570199a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.969+0000 7fbc62e95740 1 --2- 172.21.15.105:0/1458876406 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a5700b5d0 0x562a5701e5e0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.969+0000 7fbc62e95740 1 -- 172.21.15.105:0/1458876406 >> 172.21.15.105:0/1458876406 conn(0x562a56fd4310 msgr2=0x562a57012150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:06.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.969+0000 7fbc41ffb640 0 client.15684 ms_handle_reset on v2:172.21.15.190:6834/1851042281 2024-09-16T02:05:06.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.970+0000 7fbc62e95740 1 -- 172.21.15.105:0/1458876406 shutdown_connections 2024-09-16T02:05:06.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.970+0000 7fbc62e95740 1 -- 172.21.15.105:0/1458876406 wait complete. 2024-09-16T02:05:06.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.970+0000 7fbc62e95740 20 client.15684 trim_cache size 0 max 16384 2024-09-16T02:05:06.973 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-09-16T02:05:06.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.971+0000 7fbc60abc640 1 -- 172.21.15.105:0/3623578850 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fbc440a76d0 msgr2=0x7fbc440a9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:06.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.971+0000 7fbc60abc640 1 --2- 172.21.15.105:0/3623578850 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fbc440a76d0 0x7fbc440a9b90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbc54009ea0 tx=0x7fbc54009410 comp rx=0 tx=0).stop 2024-09-16T02:05:06.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.971+0000 7fbc60abc640 1 -- 172.21.15.105:0/3623578850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc5c102780 msgr2=0x7fbc5c198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:06.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.971+0000 7fbc60abc640 1 --2- 172.21.15.105:0/3623578850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc5c102780 0x7fbc5c198910 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fbc50006010 tx=0x7fbc50004270 comp rx=0 tx=0).stop 2024-09-16T02:05:06.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.971+0000 7fbc60abc640 1 -- 172.21.15.105:0/3623578850 shutdown_connections 2024-09-16T02:05:06.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.971+0000 7fbc60abc640 1 --2- 172.21.15.105:0/3623578850 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fbc440a76d0 0x7fbc440a9b90 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.971+0000 7fbc60abc640 1 --2- 172.21.15.105:0/3623578850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc5c106f00 0x7fbc5c19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.971+0000 7fbc60abc640 1 --2- 172.21.15.105:0/3623578850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc5c102780 0x7fbc5c198910 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.971+0000 7fbc60abc640 1 --2- 172.21.15.105:0/3623578850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc5c101dd0 0x7fbc5c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.971+0000 7fbc60abc640 1 -- 172.21.15.105:0/3623578850 >> 172.21.15.105:0/3623578850 conn(0x7fbc5c0fd870 msgr2=0x7fbc5c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:06.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.971+0000 7fbc60abc640 1 -- 172.21.15.105:0/3623578850 shutdown_connections 2024-09-16T02:05:06.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:06.971+0000 7fbc60abc640 1 -- 172.21.15.105:0/3623578850 wait complete. 2024-09-16T02:05:07.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:06 smithi190 ceph-mon[34762]: pgmap v416: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 441 MiB/s wr, 121 op/s 2024-09-16T02:05:07.784 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:07 smithi105 ceph-mon[31197]: pgmap v417: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 207 MiB/s wr, 61 op/s 2024-09-16T02:05:07.851 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-16T02:05:07.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:07 smithi137 ceph-mon[34099]: pgmap v417: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 207 MiB/s wr, 61 op/s 2024-09-16T02:05:08.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:07 smithi190 ceph-mon[34762]: pgmap v417: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 207 MiB/s wr, 61 op/s 2024-09-16T02:05:08.272 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:08.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.854+0000 7f6831bd6640 1 -- 172.21.15.105:0/320592263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f682c074590 msgr2=0x7f682c074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:08.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.854+0000 7f6831bd6640 1 --2- 172.21.15.105:0/320592263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f682c074590 0x7f682c074970 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f681c009900 tx=0x7f681c02a4a0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.855+0000 7f6831bd6640 1 -- 172.21.15.105:0/320592263 shutdown_connections 2024-09-16T02:05:08.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.855+0000 7f6831bd6640 1 --2- 172.21.15.105:0/320592263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f682c112ba0 0x7f682c114f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.855+0000 7f6831bd6640 1 --2- 172.21.15.105:0/320592263 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f682c074eb0 0x7f682c112660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.855+0000 7f6831bd6640 1 --2- 172.21.15.105:0/320592263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f682c074590 0x7f682c074970 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.855+0000 7f6831bd6640 1 -- 172.21.15.105:0/320592263 >> 172.21.15.105:0/320592263 conn(0x7f682c0fd8f0 msgr2=0x7f682c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:08.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.855+0000 7f6831bd6640 1 -- 172.21.15.105:0/320592263 shutdown_connections 2024-09-16T02:05:08.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.855+0000 7f6831bd6640 1 -- 172.21.15.105:0/320592263 wait complete. 2024-09-16T02:05:08.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.857+0000 7f6831bd6640 1 Processor -- start 2024-09-16T02:05:08.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.857+0000 7f6831bd6640 1 -- start start 2024-09-16T02:05:08.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.858+0000 7f6831bd6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f682c074590 0x7f682c112340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:08.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.858+0000 7f6831bd6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f682c074eb0 0x7f682c10c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:08.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.858+0000 7f6831bd6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f682c112ba0 0x7f682c10c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:08.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.858+0000 7f682b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f682c074590 0x7f682c112340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:08.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.858+0000 7f6831bd6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f682c106730 con 0x7f682c074590 2024-09-16T02:05:08.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.858+0000 7f6831bd6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f682c1065b0 con 0x7f682c112ba0 2024-09-16T02:05:08.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.858+0000 7f6831bd6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f682c1068b0 con 0x7f682c074eb0 2024-09-16T02:05:08.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.858+0000 7f682b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f682c074590 0x7f682c112340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55208/0 (socket says 172.21.15.105:55208) 2024-09-16T02:05:08.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.858+0000 7f682b7fe640 1 -- 172.21.15.105:0/1382373663 learned_addr learned my addr 172.21.15.105:0/1382373663 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:08.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.858+0000 7f682b7fe640 1 -- 172.21.15.105:0/1382373663 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f682c074eb0 msgr2=0x7f682c10c3c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T02:05:08.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.858+0000 7f682b7fe640 1 --2- 172.21.15.105:0/1382373663 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f682c074eb0 0x7f682c10c3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.858+0000 7f682b7fe640 1 -- 172.21.15.105:0/1382373663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f682c112ba0 msgr2=0x7f682c10c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:08.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.858+0000 7f682b7fe640 1 --2- 172.21.15.105:0/1382373663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f682c112ba0 0x7f682c10c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.858+0000 7f682b7fe640 1 -- 172.21.15.105:0/1382373663 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f682c10d080 con 0x7f682c074590 2024-09-16T02:05:08.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.859+0000 7f682b7fe640 1 --2- 172.21.15.105:0/1382373663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f682c074590 0x7f682c112340 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f681c02aa90 tx=0x7f681c003de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:08.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.860+0000 7f6828ff9640 1 -- 172.21.15.105:0/1382373663 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681c03f070 con 0x7f682c074590 2024-09-16T02:05:08.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.860+0000 7f6828ff9640 1 -- 172.21.15.105:0/1382373663 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f681c031d60 con 0x7f682c074590 2024-09-16T02:05:08.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.860+0000 7f6828ff9640 1 -- 172.21.15.105:0/1382373663 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681c035030 con 0x7f682c074590 2024-09-16T02:05:08.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.860+0000 7f6831bd6640 1 -- 172.21.15.105:0/1382373663 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f682c1b39b0 con 0x7f682c074590 2024-09-16T02:05:08.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.860+0000 7f6831bd6640 1 -- 172.21.15.105:0/1382373663 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f682c1b3db0 con 0x7f682c074590 2024-09-16T02:05:08.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.863+0000 7f6833faf740 1 Processor -- start 2024-09-16T02:05:08.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.863+0000 7f6833faf740 1 -- start start 2024-09-16T02:05:08.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.863+0000 7f6833faf740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560850388b40 0x560850387090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:08.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.863+0000 7f6833faf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608503875d0 0x560850387a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:08.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.864+0000 7f6833faf740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560850392370 0x560850394760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:08.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.864+0000 7f6833faf740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5608502ed2f0 con 0x560850388b40 2024-09-16T02:05:08.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.864+0000 7f6833faf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560850294cf0 con 0x5608503875d0 2024-09-16T02:05:08.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.864+0000 7f6833faf740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5608502e92b0 con 0x560850392370 2024-09-16T02:05:08.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.864+0000 7f682bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560850388b40 0x560850387090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:08.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.864+0000 7f682bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560850388b40 0x560850387090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55220/0 (socket says 172.21.15.105:55220) 2024-09-16T02:05:08.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.864+0000 7f682bfff640 1 -- 172.21.15.105:0/500695682 learned_addr learned my addr 172.21.15.105:0/500695682 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:08.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.864+0000 7f682affd640 1 --2- 172.21.15.105:0/500695682 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560850392370 0x560850394760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:08.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.864+0000 7f682bfff640 1 -- 172.21.15.105:0/500695682 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560850392370 msgr2=0x560850394760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:08.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.864+0000 7f682bfff640 1 --2- 172.21.15.105:0/500695682 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560850392370 0x560850394760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.864+0000 7f682bfff640 1 -- 172.21.15.105:0/500695682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608503875d0 msgr2=0x560850387a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T02:05:08.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.864+0000 7f682bfff640 1 --2- 172.21.15.105:0/500695682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608503875d0 0x560850387a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.864+0000 7f682bfff640 1 -- 172.21.15.105:0/500695682 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608502793a0 con 0x560850388b40 2024-09-16T02:05:08.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.865+0000 7f6828ff9640 1 -- 172.21.15.105:0/1382373663 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f681c042070 con 0x7f682c074590 2024-09-16T02:05:08.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.865+0000 7f682bfff640 1 --2- 172.21.15.105:0/500695682 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560850388b40 0x560850387090 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6824009cf0 tx=0x7f682400b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc4236e14d3c0ffb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:08.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.865+0000 7f6811ffb640 1 -- 172.21.15.105:0/500695682 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682400bd40 con 0x560850388b40 2024-09-16T02:05:08.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.865+0000 7f6811ffb640 1 -- 172.21.15.105:0/500695682 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6824017070 con 0x560850388b40 2024-09-16T02:05:08.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.865+0000 7f6811ffb640 1 -- 172.21.15.105:0/500695682 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68240115f0 con 0x560850388b40 2024-09-16T02:05:08.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.866+0000 7f6833faf740 1 -- 172.21.15.105:0/500695682 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560850388b40 msgr2=0x560850387090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:08.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.866+0000 7f6833faf740 1 --2- 172.21.15.105:0/500695682 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560850388b40 0x560850387090 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6824009cf0 tx=0x7f682400b420 comp rx=0 tx=0).stop 2024-09-16T02:05:08.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.866+0000 7f6833faf740 1 -- 172.21.15.105:0/500695682 shutdown_connections 2024-09-16T02:05:08.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.866+0000 7f6833faf740 1 --2- 172.21.15.105:0/500695682 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560850392370 0x560850394760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.866+0000 7f6833faf740 1 --2- 172.21.15.105:0/500695682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608503875d0 0x560850387a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.866+0000 7f6833faf740 1 --2- 172.21.15.105:0/500695682 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560850388b40 0x560850387090 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.866+0000 7f6833faf740 1 -- 172.21.15.105:0/500695682 >> 172.21.15.105:0/500695682 conn(0x560850350310 msgr2=0x560850383c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:08.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.866+0000 7f6833faf740 1 -- 172.21.15.105:0/500695682 shutdown_connections 2024-09-16T02:05:08.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.866+0000 7f6833faf740 1 -- 172.21.15.105:0/500695682 wait complete. 2024-09-16T02:05:08.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.867+0000 7f6828ff9640 1 --2- 172.21.15.105:0/1382373663 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f68140a31d0 0x7f68140a5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:08.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.867+0000 7f6833faf740 1 -- start start 2024-09-16T02:05:08.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.867+0000 7f682affd640 1 --2- 172.21.15.105:0/1382373663 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f68140a31d0 0x7f68140a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:08.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.867+0000 7f6833faf740 1 Processor -- start 2024-09-16T02:05:08.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.867+0000 7f6828ff9640 1 -- 172.21.15.105:0/1382373663 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f681c0f0310 con 0x7f682c074590 2024-09-16T02:05:08.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.868+0000 7f6833faf740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5608503875d0 0x560850391d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:08.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.868+0000 7f6833faf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560850388b40 0x56085038f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:08.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.868+0000 7f6833faf740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560850392370 0x56085038f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:08.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.868+0000 7f6833faf740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560850395070 con 0x5608503875d0 2024-09-16T02:05:08.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.868+0000 7f6833faf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5608502a8060 con 0x560850388b40 2024-09-16T02:05:08.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.868+0000 7f6833faf740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5608503951f0 con 0x560850392370 2024-09-16T02:05:08.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.868+0000 7f6833faf740 10 client.0 _ll_register_callbacks cb 0x56084fffb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:08.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.868+0000 7f682b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5608503875d0 0x560850391d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:08.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.868+0000 7f682affd640 1 --2- 172.21.15.105:0/1382373663 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f68140a31d0 0x7f68140a5690 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x560850388070 tx=0x7f6820008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:08.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.868+0000 7f682b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5608503875d0 0x560850391d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55234/0 (socket says 172.21.15.105:55234) 2024-09-16T02:05:08.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.868+0000 7f682b7fe640 1 -- 172.21.15.105:0/1305801609 learned_addr learned my addr 172.21.15.105:0/1305801609 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:08.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.868+0000 7f682b7fe640 1 -- 172.21.15.105:0/1305801609 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560850392370 msgr2=0x56085038f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:08.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.868+0000 7f682b7fe640 1 --2- 172.21.15.105:0/1305801609 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560850392370 0x56085038f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.868+0000 7f682b7fe640 1 -- 172.21.15.105:0/1305801609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560850388b40 msgr2=0x56085038f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:08.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.868+0000 7f682b7fe640 1 --2- 172.21.15.105:0/1305801609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560850388b40 0x56085038f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.868+0000 7f682b7fe640 1 -- 172.21.15.105:0/1305801609 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56085005e3c0 con 0x5608503875d0 2024-09-16T02:05:08.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.868+0000 7f682b7fe640 1 --2- 172.21.15.105:0/1305801609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5608503875d0 0x560850391d80 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f681c0bc860 tx=0x7f681c0bc410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3adc07241b427dba server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:08.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.868+0000 7f680affd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:05:08.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.869+0000 7f680affd640 1 -- 172.21.15.105:0/1305801609 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681c0bc8d0 con 0x5608503875d0 2024-09-16T02:05:08.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.869+0000 7f680affd640 1 -- 172.21.15.105:0/1305801609 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f681c03b460 con 0x5608503875d0 2024-09-16T02:05:08.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.869+0000 7f6833faf740 1 -- 172.21.15.105:0/1305801609 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6824017070 con 0x5608503875d0 2024-09-16T02:05:08.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.869+0000 7f680affd640 1 -- 172.21.15.105:0/1305801609 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f681c0c0c80 con 0x5608503875d0 2024-09-16T02:05:08.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.869+0000 7f6833faf740 10 client.15696 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:08.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.869+0000 7f6833faf740 1 -- 172.21.15.105:0/1305801609 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5608503a0b70 con 0x5608503875d0 2024-09-16T02:05:08.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.869+0000 7f680affd640 1 -- 172.21.15.105:0/1305801609 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f681c0c5540 con 0x5608503875d0 2024-09-16T02:05:08.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.869+0000 7f680affd640 1 -- 172.21.15.105:0/1305801609 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f681c0b8f40 con 0x5608503875d0 2024-09-16T02:05:08.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.869+0000 7f6833faf740 10 client.15696 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:08.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.869+0000 7f6833faf740 10 client.15696 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T02:05:08.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.869+0000 7f6833faf740 20 client.15696 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:08.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.869+0000 7f6833faf740 1 --2- 172.21.15.105:0/1305801609 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x5608503a20a0 0x5608503a4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:08.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.869+0000 7f6833faf740 4 client.15696 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:05:08.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.869+0000 7f6833faf740 1 -- 172.21.15.105:0/1305801609 --> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6824017070 con 0x5608503a20a0 2024-09-16T02:05:08.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.870+0000 7f682affd640 1 --2- 172.21.15.105:0/1305801609 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x5608503a20a0 0x5608503a4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:08.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.871+0000 7f682affd640 1 --2- 172.21.15.105:0/1305801609 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x5608503a20a0 0x5608503a4490 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a92859658b6b828b server_cookie=98d0747357db2c2e in_seq=0 out_seq=0 2024-09-16T02:05:08.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.871+0000 7f680affd640 10 client.15696 ms_handle_connect on v2:172.21.15.190:6836/3181257549 2024-09-16T02:05:08.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.872+0000 7f680affd640 1 -- 172.21.15.105:0/1305801609 <== mds.3 v2:172.21.15.190:6836/3181257549 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f6824017070 con 0x5608503a20a0 2024-09-16T02:05:08.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.872+0000 7f680affd640 10 client.15696 handle_command_reply: tid=0 2024-09-16T02:05:08.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.872+0000 7f6833faf740 1 client.15696 shutdown 2024-09-16T02:05:08.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.872+0000 7f6833faf740 1 -- 172.21.15.105:0/1305801609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5608503875d0 msgr2=0x560850391d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:08.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.872+0000 7f6833faf740 1 --2- 172.21.15.105:0/1305801609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5608503875d0 0x560850391d80 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f681c0bc860 tx=0x7f681c0bc410 comp rx=0 tx=0).stop 2024-09-16T02:05:08.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.873+0000 7f6833faf740 1 -- 172.21.15.105:0/1305801609 shutdown_connections 2024-09-16T02:05:08.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.873+0000 7f6833faf740 1 --2- 172.21.15.105:0/1305801609 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560850392370 0x56085038f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.873+0000 7f6833faf740 1 --2- 172.21.15.105:0/1305801609 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x5608503a20a0 0x5608503a4490 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.873+0000 7f6833faf740 1 --2- 172.21.15.105:0/1305801609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560850388b40 0x56085038f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.873+0000 7f6833faf740 1 --2- 172.21.15.105:0/1305801609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5608503875d0 0x560850391d80 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.873+0000 7f680affd640 0 client.15696 ms_handle_reset on v2:172.21.15.190:6836/3181257549 2024-09-16T02:05:08.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.873+0000 7f6833faf740 1 -- 172.21.15.105:0/1305801609 >> 172.21.15.105:0/1305801609 conn(0x560850350310 msgr2=0x560850389930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:08.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.873+0000 7f6833faf740 1 -- 172.21.15.105:0/1305801609 shutdown_connections 2024-09-16T02:05:08.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.873+0000 7f6833faf740 1 -- 172.21.15.105:0/1305801609 wait complete. 2024-09-16T02:05:08.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.873+0000 7f6833faf740 20 client.15696 trim_cache size 0 max 16384 2024-09-16T02:05:08.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.897+0000 7f6833faf740 1 Processor -- start 2024-09-16T02:05:08.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.897+0000 7f6833faf740 1 -- start start 2024-09-16T02:05:08.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.898+0000 7f6833faf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608503875d0 0x5608503a4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:08.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.898+0000 7f6833faf740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560850388b40 0x560850382bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:08.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.898+0000 7f6833faf740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560850392370 0x560850383130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:08.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.898+0000 7f6833faf740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f682c1bc4c0 con 0x560850392370 2024-09-16T02:05:08.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.898+0000 7f6833faf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f682c1be000 con 0x5608503875d0 2024-09-16T02:05:08.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.898+0000 7f6833faf740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f682c1bcb90 con 0x560850388b40 2024-09-16T02:05:08.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.898+0000 7f682bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560850392370 0x560850383130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:08.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.898+0000 7f682bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560850392370 0x560850383130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55250/0 (socket says 172.21.15.105:55250) 2024-09-16T02:05:08.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.898+0000 7f682bfff640 1 -- 172.21.15.105:0/3630943676 learned_addr learned my addr 172.21.15.105:0/3630943676 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:08.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.899+0000 7f682bfff640 1 -- 172.21.15.105:0/3630943676 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560850388b40 msgr2=0x560850382bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T02:05:08.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.899+0000 7f682bfff640 1 --2- 172.21.15.105:0/3630943676 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560850388b40 0x560850382bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.899+0000 7f682bfff640 1 -- 172.21.15.105:0/3630943676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608503875d0 msgr2=0x5608503a4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:08.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.899+0000 7f682bfff640 1 --2- 172.21.15.105:0/3630943676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608503875d0 0x5608503a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.899+0000 7f682bfff640 1 -- 172.21.15.105:0/3630943676 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608503918a0 con 0x560850392370 2024-09-16T02:05:08.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.899+0000 7f682bfff640 1 --2- 172.21.15.105:0/3630943676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560850392370 0x560850383130 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6824009a90 tx=0x7f6824009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=313e32f91013f0a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:08.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.900+0000 7f68117fa640 1 -- 172.21.15.105:0/3630943676 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6824021070 con 0x560850392370 2024-09-16T02:05:08.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.900+0000 7f68117fa640 1 -- 172.21.15.105:0/3630943676 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6824017040 con 0x560850392370 2024-09-16T02:05:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.901+0000 7f68117fa640 1 -- 172.21.15.105:0/3630943676 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682401dad0 con 0x560850392370 2024-09-16T02:05:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.901+0000 7f6833faf740 1 -- 172.21.15.105:0/3630943676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560850392370 msgr2=0x560850383130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.901+0000 7f6833faf740 1 --2- 172.21.15.105:0/3630943676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560850392370 0x560850383130 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6824009a90 tx=0x7f6824009ac0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.901+0000 7f6833faf740 1 -- 172.21.15.105:0/3630943676 shutdown_connections 2024-09-16T02:05:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.901+0000 7f6833faf740 1 --2- 172.21.15.105:0/3630943676 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560850388b40 0x560850382bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.901+0000 7f6833faf740 1 --2- 172.21.15.105:0/3630943676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560850392370 0x560850383130 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.901+0000 7f6833faf740 1 --2- 172.21.15.105:0/3630943676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608503875d0 0x5608503a4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.901+0000 7f6833faf740 1 -- 172.21.15.105:0/3630943676 >> 172.21.15.105:0/3630943676 conn(0x560850350310 msgr2=0x5608503824e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.901+0000 7f6833faf740 1 -- 172.21.15.105:0/3630943676 shutdown_connections 2024-09-16T02:05:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.902+0000 7f6833faf740 1 -- 172.21.15.105:0/3630943676 wait complete. 2024-09-16T02:05:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.902+0000 7f6833faf740 1 -- start start 2024-09-16T02:05:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.902+0000 7f6833faf740 1 Processor -- start 2024-09-16T02:05:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.903+0000 7f6833faf740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608503875d0 0x56085039ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.903+0000 7f6833faf740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560850388b40 0x56085038bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.903+0000 7f6833faf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560850392370 0x56085038c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.903+0000 7f6833faf740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f682c1bd930 con 0x560850388b40 2024-09-16T02:05:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.903+0000 7f6833faf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f682c1bd260 con 0x560850392370 2024-09-16T02:05:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.903+0000 7f6833faf740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f682c1ba4a0 con 0x5608503875d0 2024-09-16T02:05:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.903+0000 7f6833faf740 10 client.0 _ll_register_callbacks cb 0x7f681c005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.903+0000 7f682bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560850392370 0x56085038c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.903+0000 7f682affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560850388b40 0x56085038bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.903+0000 7f682affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560850388b40 0x56085038bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55258/0 (socket says 172.21.15.105:55258) 2024-09-16T02:05:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.903+0000 7f682bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560850392370 0x56085038c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.105:44626/0 (socket says 172.21.15.105:44626) 2024-09-16T02:05:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.903+0000 7f682affd640 1 -- 172.21.15.105:0/1808006225 learned_addr learned my addr 172.21.15.105:0/1808006225 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.904+0000 7f682b7fe640 1 --2- 172.21.15.105:0/1808006225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608503875d0 0x56085039ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.904+0000 7f682affd640 1 -- 172.21.15.105:0/1808006225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608503875d0 msgr2=0x56085039ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.904+0000 7f682affd640 1 --2- 172.21.15.105:0/1808006225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608503875d0 0x56085039ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.904+0000 7f682affd640 1 -- 172.21.15.105:0/1808006225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560850392370 msgr2=0x56085038c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.904+0000 7f682affd640 1 --2- 172.21.15.105:0/1808006225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560850392370 0x56085038c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.904+0000 7f682affd640 1 -- 172.21.15.105:0/1808006225 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56085040bc80 con 0x560850388b40 2024-09-16T02:05:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.904+0000 7f682bfff640 1 --2- 172.21.15.105:0/1808006225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560850392370 0x56085038c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:05:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.904+0000 7f682affd640 1 --2- 172.21.15.105:0/1808006225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560850388b40 0x56085038bea0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x56085038e8c0 tx=0x7f682000ddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a062b7a06082a5f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.904+0000 7f680affd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:05:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.905+0000 7f680affd640 1 -- 172.21.15.105:0/1808006225 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6820005bd0 con 0x560850388b40 2024-09-16T02:05:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.905+0000 7f680affd640 1 -- 172.21.15.105:0/1808006225 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f6820005d70 con 0x560850388b40 2024-09-16T02:05:08.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.905+0000 7f6833faf740 1 -- 172.21.15.105:0/1808006225 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6824017040 con 0x560850388b40 2024-09-16T02:05:08.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.905+0000 7f680affd640 1 -- 172.21.15.105:0/1808006225 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f68200148e0 con 0x560850388b40 2024-09-16T02:05:08.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.905+0000 7f6833faf740 10 client.15702 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:08.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.905+0000 7f6833faf740 1 -- 172.21.15.105:0/1808006225 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56085040ce60 con 0x560850388b40 2024-09-16T02:05:08.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.906+0000 7f680affd640 1 -- 172.21.15.105:0/1808006225 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6824017040 con 0x560850388b40 2024-09-16T02:05:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.906+0000 7f680affd640 1 -- 172.21.15.105:0/1808006225 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6820013050 con 0x560850388b40 2024-09-16T02:05:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.906+0000 7f6833faf740 10 client.15702 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.906+0000 7f6833faf740 10 client.15702 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T02:05:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.906+0000 7f6833faf740 20 client.15702 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.906+0000 7f6833faf740 1 --2- 172.21.15.105:0/1808006225 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x56085040d610 0x56085040fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.906+0000 7f6833faf740 4 client.15702 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T02:05:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.906+0000 7f6833faf740 1 -- 172.21.15.105:0/1808006225 --> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5608502fa860 con 0x56085040d610 2024-09-16T02:05:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.908+0000 7f682b7fe640 1 --2- 172.21.15.105:0/1808006225 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x56085040d610 0x56085040fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:08.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.911+0000 7f680affd640 10 client.15702 ms_handle_connect on v2:172.21.15.190:6836/3181257549 2024-09-16T02:05:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.911+0000 7f682b7fe640 1 --2- 172.21.15.105:0/1808006225 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x56085040d610 0x56085040fa30 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=91973c2ef985d378 server_cookie=d54d4e28b75afe23 in_seq=0 out_seq=0 2024-09-16T02:05:08.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.912+0000 7f680affd640 1 -- 172.21.15.105:0/1808006225 <== mds.3 v2:172.21.15.190:6836/3181257549 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5608502fa860 con 0x56085040d610 2024-09-16T02:05:08.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.912+0000 7f680affd640 10 client.15702 handle_command_reply: tid=0 2024-09-16T02:05:08.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.912+0000 7f6833faf740 1 client.15702 shutdown 2024-09-16T02:05:08.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.913+0000 7f6833faf740 1 -- 172.21.15.105:0/1808006225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560850388b40 msgr2=0x56085038bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:08.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.913+0000 7f6833faf740 1 --2- 172.21.15.105:0/1808006225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560850388b40 0x56085038bea0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x56085038e8c0 tx=0x7f682000ddd0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.913+0000 7f6833faf740 1 -- 172.21.15.105:0/1808006225 shutdown_connections 2024-09-16T02:05:08.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.913+0000 7f6833faf740 1 --2- 172.21.15.105:0/1808006225 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x56085040d610 0x56085040fa30 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.913+0000 7f6833faf740 1 --2- 172.21.15.105:0/1808006225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560850392370 0x56085038c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.913+0000 7f6833faf740 1 --2- 172.21.15.105:0/1808006225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560850388b40 0x56085038bea0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.913+0000 7f6833faf740 1 --2- 172.21.15.105:0/1808006225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608503875d0 0x56085039ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.913+0000 7f6833faf740 1 -- 172.21.15.105:0/1808006225 >> 172.21.15.105:0/1808006225 conn(0x560850350310 msgr2=0x5608503910b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:08.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.913+0000 7f680affd640 0 client.15702 ms_handle_reset on v2:172.21.15.190:6836/3181257549 2024-09-16T02:05:08.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.913+0000 7f6833faf740 1 -- 172.21.15.105:0/1808006225 shutdown_connections 2024-09-16T02:05:08.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.913+0000 7f6833faf740 1 -- 172.21.15.105:0/1808006225 wait complete. 2024-09-16T02:05:08.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.913+0000 7f6833faf740 20 client.15702 trim_cache size 0 max 16384 2024-09-16T02:05:08.917 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-09-16T02:05:08.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.915+0000 7f6831bd6640 1 -- 172.21.15.105:0/1382373663 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f68140a31d0 msgr2=0x7f68140a5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:08.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.915+0000 7f6831bd6640 1 --2- 172.21.15.105:0/1382373663 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f68140a31d0 0x7f68140a5690 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x560850388070 tx=0x7f6820008040 comp rx=0 tx=0).stop 2024-09-16T02:05:08.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.915+0000 7f6831bd6640 1 -- 172.21.15.105:0/1382373663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f682c074590 msgr2=0x7f682c112340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:08.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.915+0000 7f6831bd6640 1 --2- 172.21.15.105:0/1382373663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f682c074590 0x7f682c112340 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f681c02aa90 tx=0x7f681c003de0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.915+0000 7f6831bd6640 1 -- 172.21.15.105:0/1382373663 shutdown_connections 2024-09-16T02:05:08.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.915+0000 7f6831bd6640 1 --2- 172.21.15.105:0/1382373663 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f68140a31d0 0x7f68140a5690 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.915+0000 7f6831bd6640 1 --2- 172.21.15.105:0/1382373663 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f682c074eb0 0x7f682c10c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.915+0000 7f6831bd6640 1 --2- 172.21.15.105:0/1382373663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f682c112ba0 0x7f682c10c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.915+0000 7f6831bd6640 1 --2- 172.21.15.105:0/1382373663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f682c074590 0x7f682c112340 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.915+0000 7f6831bd6640 1 -- 172.21.15.105:0/1382373663 >> 172.21.15.105:0/1382373663 conn(0x7f682c0fd8f0 msgr2=0x7f682c0ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:08.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.916+0000 7f6831bd6640 1 -- 172.21.15.105:0/1382373663 shutdown_connections 2024-09-16T02:05:08.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:08.916+0000 7f6831bd6640 1 -- 172.21.15.105:0/1382373663 wait complete. 2024-09-16T02:05:09.834 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T02:05:10.270 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:10.726 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:10 smithi105 ceph-mon[31197]: pgmap v418: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 277 MiB/s wr, 81 op/s 2024-09-16T02:05:10.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:10 smithi190 ceph-mon[34762]: pgmap v418: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 277 MiB/s wr, 81 op/s 2024-09-16T02:05:10.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.863+0000 7f33fd690640 1 -- 172.21.15.105:0/1230458752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33f8105ec0 msgr2=0x7f33f8106340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:10.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.863+0000 7f33fd690640 1 --2- 172.21.15.105:0/1230458752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33f8105ec0 0x7f33f8106340 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f33f0009900 tx=0x7f33f002a3f0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.863+0000 7f33fd690640 1 -- 172.21.15.105:0/1230458752 shutdown_connections 2024-09-16T02:05:10.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.863+0000 7f33fd690640 1 --2- 172.21.15.105:0/1230458752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33f8105ec0 0x7f33f8106340 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.863+0000 7f33fd690640 1 --2- 172.21.15.105:0/1230458752 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33f81011e0 0x7f33f8105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.863+0000 7f33fd690640 1 --2- 172.21.15.105:0/1230458752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33f81008c0 0x7f33f8100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.863+0000 7f33fd690640 1 -- 172.21.15.105:0/1230458752 >> 172.21.15.105:0/1230458752 conn(0x7f33f80fc7d0 msgr2=0x7f33f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:10.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.863+0000 7f33fd690640 1 -- 172.21.15.105:0/1230458752 shutdown_connections 2024-09-16T02:05:10.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.863+0000 7f33fd690640 1 -- 172.21.15.105:0/1230458752 wait complete. 2024-09-16T02:05:10.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.864+0000 7f33fd690640 1 Processor -- start 2024-09-16T02:05:10.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.864+0000 7f33fd690640 1 -- start start 2024-09-16T02:05:10.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.864+0000 7f33fd690640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33f81008c0 0x7f33f81a4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:10.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.864+0000 7f33fd690640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33f81011e0 0x7f33f81a4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:10.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.864+0000 7f33fd690640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33f8105ec0 0x7f33f819e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:10.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.864+0000 7f33fd690640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f33f81185c0 con 0x7f33f8105ec0 2024-09-16T02:05:10.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.864+0000 7f33fd690640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f33f8118440 con 0x7f33f81011e0 2024-09-16T02:05:10.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.864+0000 7f33fd690640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f33f8118740 con 0x7f33f81008c0 2024-09-16T02:05:10.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.865+0000 7f33f77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33f8105ec0 0x7f33f819e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:10.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.865+0000 7f33f77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33f8105ec0 0x7f33f819e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55270/0 (socket says 172.21.15.105:55270) 2024-09-16T02:05:10.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.865+0000 7f33f77fe640 1 -- 172.21.15.105:0/947040164 learned_addr learned my addr 172.21.15.105:0/947040164 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:10.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.865+0000 7f33f6ffd640 1 --2- 172.21.15.105:0/947040164 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33f81008c0 0x7f33f81a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:10.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.865+0000 7f33e7fff640 1 --2- 172.21.15.105:0/947040164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33f81011e0 0x7f33f81a4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:10.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.865+0000 7f33f77fe640 1 -- 172.21.15.105:0/947040164 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33f81008c0 msgr2=0x7f33f81a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:10.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.865+0000 7f33f77fe640 1 --2- 172.21.15.105:0/947040164 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33f81008c0 0x7f33f81a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.865+0000 7f33f77fe640 1 -- 172.21.15.105:0/947040164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33f81011e0 msgr2=0x7f33f81a4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:10.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.865+0000 7f33f77fe640 1 --2- 172.21.15.105:0/947040164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33f81011e0 0x7f33f81a4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.865+0000 7f33f77fe640 1 -- 172.21.15.105:0/947040164 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33f819f030 con 0x7f33f8105ec0 2024-09-16T02:05:10.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.865+0000 7f33e7fff640 1 --2- 172.21.15.105:0/947040164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33f81011e0 0x7f33f81a4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:05:10.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.866+0000 7f33f77fe640 1 --2- 172.21.15.105:0/947040164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33f8105ec0 0x7f33f819e900 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f33f0009230 tx=0x7f33f00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:10.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.866+0000 7f33f4ff9640 1 -- 172.21.15.105:0/947040164 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f003f030 con 0x7f33f8105ec0 2024-09-16T02:05:10.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.866+0000 7f33f4ff9640 1 -- 172.21.15.105:0/947040164 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33f0003800 con 0x7f33f8105ec0 2024-09-16T02:05:10.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.866+0000 7f33f4ff9640 1 -- 172.21.15.105:0/947040164 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f0035070 con 0x7f33f8105ec0 2024-09-16T02:05:10.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.866+0000 7f33fd690640 1 -- 172.21.15.105:0/947040164 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33f819f2c0 con 0x7f33f8105ec0 2024-09-16T02:05:10.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.866+0000 7f33fd690640 1 -- 172.21.15.105:0/947040164 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33f81ab610 con 0x7f33f8105ec0 2024-09-16T02:05:10.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.869+0000 7f33f4ff9640 1 -- 172.21.15.105:0/947040164 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f33f00039b0 con 0x7f33f8105ec0 2024-09-16T02:05:10.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.870+0000 7f33ffa69740 1 Processor -- start 2024-09-16T02:05:10.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.870+0000 7f33ffa69740 1 -- start start 2024-09-16T02:05:10.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.870+0000 7f33f4ff9640 1 --2- 172.21.15.105:0/947040164 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f33e00a3240 0x7f33e00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:10.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.870+0000 7f33e7fff640 1 --2- 172.21.15.105:0/947040164 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f33e00a3240 0x7f33e00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:10.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.871+0000 7f33ffa69740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1e4057b40 0x55e1e4056090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:10.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.871+0000 7f33f4ff9640 1 -- 172.21.15.105:0/947040164 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f33f00f03a0 con 0x7f33f8105ec0 2024-09-16T02:05:10.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.871+0000 7f33ffa69740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1e40565d0 0x55e1e4056a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:10.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.871+0000 7f33ffa69740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1e4061370 0x55e1e4063760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:10.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.871+0000 7f33ffa69740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e1e3fbc2f0 con 0x55e1e40565d0 2024-09-16T02:05:10.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.871+0000 7f33ffa69740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e1e3f63cf0 con 0x55e1e4061370 2024-09-16T02:05:10.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.871+0000 7f33ffa69740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e1e3fb82b0 con 0x55e1e4057b40 2024-09-16T02:05:10.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.871+0000 7f33e7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1e40565d0 0x55e1e4056a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:10.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.871+0000 7f33e7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1e40565d0 0x55e1e4056a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55284/0 (socket says 172.21.15.105:55284) 2024-09-16T02:05:10.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.871+0000 7f33e7fff640 1 -- 172.21.15.105:0/3951804616 learned_addr learned my addr 172.21.15.105:0/3951804616 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:10.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.871+0000 7f33e7fff640 1 --2- 172.21.15.105:0/947040164 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f33e00a3240 0x7f33e00a5700 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f33ec006fd0 tx=0x7f33ec00af00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:10.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.872+0000 7f33f77fe640 1 --2- 172.21.15.105:0/3951804616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1e4061370 0x55e1e4063760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:10.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.872+0000 7f33e7fff640 1 -- 172.21.15.105:0/3951804616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1e4057b40 msgr2=0x55e1e4056090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T02:05:10.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.872+0000 7f33e7fff640 1 --2- 172.21.15.105:0/3951804616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1e4057b40 0x55e1e4056090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.872+0000 7f33e7fff640 1 -- 172.21.15.105:0/3951804616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1e4061370 msgr2=0x55e1e4063760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:10.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.872+0000 7f33e7fff640 1 --2- 172.21.15.105:0/3951804616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1e4061370 0x55e1e4063760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.872+0000 7f33e7fff640 1 -- 172.21.15.105:0/3951804616 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1e3f483a0 con 0x55e1e40565d0 2024-09-16T02:05:10.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.872+0000 7f33e7fff640 1 --2- 172.21.15.105:0/3951804616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1e40565d0 0x55e1e4056a10 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f33ec010e20 tx=0x7f33ec017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b168304a18d6968e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:10.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.873+0000 7f33e57fa640 1 -- 172.21.15.105:0/3951804616 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec01e650 con 0x55e1e40565d0 2024-09-16T02:05:10.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.873+0000 7f33e57fa640 1 -- 172.21.15.105:0/3951804616 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33ec023070 con 0x55e1e40565d0 2024-09-16T02:05:10.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.873+0000 7f33e57fa640 1 -- 172.21.15.105:0/3951804616 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec016ad0 con 0x55e1e40565d0 2024-09-16T02:05:10.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.874+0000 7f33ffa69740 1 -- 172.21.15.105:0/3951804616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1e40565d0 msgr2=0x55e1e4056a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:10.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.874+0000 7f33ffa69740 1 --2- 172.21.15.105:0/3951804616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1e40565d0 0x55e1e4056a10 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f33ec010e20 tx=0x7f33ec017000 comp rx=0 tx=0).stop 2024-09-16T02:05:10.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.874+0000 7f33ffa69740 1 -- 172.21.15.105:0/3951804616 shutdown_connections 2024-09-16T02:05:10.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.874+0000 7f33ffa69740 1 --2- 172.21.15.105:0/3951804616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1e4061370 0x55e1e4063760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.874+0000 7f33ffa69740 1 --2- 172.21.15.105:0/3951804616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1e40565d0 0x55e1e4056a10 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.874+0000 7f33ffa69740 1 --2- 172.21.15.105:0/3951804616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1e4057b40 0x55e1e4056090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.874+0000 7f33ffa69740 1 -- 172.21.15.105:0/3951804616 >> 172.21.15.105:0/3951804616 conn(0x55e1e401f310 msgr2=0x55e1e4052c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:10.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.874+0000 7f33ffa69740 1 -- 172.21.15.105:0/3951804616 shutdown_connections 2024-09-16T02:05:10.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.874+0000 7f33ffa69740 1 -- 172.21.15.105:0/3951804616 wait complete. 2024-09-16T02:05:10.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.874+0000 7f33ffa69740 1 -- start start 2024-09-16T02:05:10.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.875+0000 7f33ffa69740 1 Processor -- start 2024-09-16T02:05:10.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.875+0000 7f33ffa69740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1e40565d0 0x55e1e4060de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:10.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.875+0000 7f33ffa69740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1e4057b40 0x55e1e405e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:10.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.875+0000 7f33ffa69740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1e4061370 0x55e1e405e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:10.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.875+0000 7f33ffa69740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e1e4064010 con 0x55e1e40565d0 2024-09-16T02:05:10.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.875+0000 7f33ffa69740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e1e3f77060 con 0x55e1e4061370 2024-09-16T02:05:10.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.875+0000 7f33ffa69740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e1e4064190 con 0x55e1e4057b40 2024-09-16T02:05:10.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.875+0000 7f33ffa69740 10 client.0 _ll_register_callbacks cb 0x55e1e3cca8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:10.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.876+0000 7f33f6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1e40565d0 0x55e1e4060de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:10.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.876+0000 7f33f6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1e40565d0 0x55e1e4060de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55292/0 (socket says 172.21.15.105:55292) 2024-09-16T02:05:10.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.876+0000 7f33f6ffd640 1 -- 172.21.15.105:0/2292373643 learned_addr learned my addr 172.21.15.105:0/2292373643 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:10.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.876+0000 7f33e7fff640 1 --2- 172.21.15.105:0/2292373643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1e4057b40 0x55e1e405e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:10.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.876+0000 7f33f77fe640 1 --2- 172.21.15.105:0/2292373643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1e4061370 0x55e1e405e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:10.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.876+0000 7f33f6ffd640 1 -- 172.21.15.105:0/2292373643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1e4057b40 msgr2=0x55e1e405e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:10.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.876+0000 7f33f6ffd640 1 --2- 172.21.15.105:0/2292373643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1e4057b40 0x55e1e405e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.876+0000 7f33f6ffd640 1 -- 172.21.15.105:0/2292373643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1e4061370 msgr2=0x55e1e405e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:10.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.876+0000 7f33f6ffd640 1 --2- 172.21.15.105:0/2292373643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1e4061370 0x55e1e405e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.876+0000 7f33f6ffd640 1 -- 172.21.15.105:0/2292373643 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1e3d2d3c0 con 0x55e1e40565d0 2024-09-16T02:05:10.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.876+0000 7f33e7fff640 1 --2- 172.21.15.105:0/2292373643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1e4057b40 0x55e1e405e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:05:10.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.877+0000 7f33f6ffd640 1 --2- 172.21.15.105:0/2292373643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1e40565d0 0x55e1e4060de0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f33e800ca10 tx=0x7f33e800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e75abf60509614 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:10.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.877+0000 7f33de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:05:10.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.877+0000 7f33de7fc640 1 -- 172.21.15.105:0/2292373643 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e801a060 con 0x55e1e40565d0 2024-09-16T02:05:10.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.877+0000 7f33de7fc640 1 -- 172.21.15.105:0/2292373643 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f33e80079b0 con 0x55e1e40565d0 2024-09-16T02:05:10.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.877+0000 7f33ffa69740 1 -- 172.21.15.105:0/2292373643 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33ec023070 con 0x55e1e40565d0 2024-09-16T02:05:10.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.878+0000 7f33de7fc640 1 -- 172.21.15.105:0/2292373643 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f33e8007bb0 con 0x55e1e40565d0 2024-09-16T02:05:10.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.878+0000 7f33ffa69740 10 client.15714 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:10.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.878+0000 7f33ffa69740 1 -- 172.21.15.105:0/2292373643 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e1e406fb10 con 0x55e1e40565d0 2024-09-16T02:05:10.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.878+0000 7f33de7fc640 1 -- 172.21.15.105:0/2292373643 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33ec023070 con 0x55e1e40565d0 2024-09-16T02:05:10.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.879+0000 7f33de7fc640 1 -- 172.21.15.105:0/2292373643 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f33e801e030 con 0x55e1e40565d0 2024-09-16T02:05:10.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.879+0000 7f33ffa69740 10 client.15714 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:10.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.879+0000 7f33ffa69740 10 client.15714 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T02:05:10.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.879+0000 7f33ffa69740 20 client.15714 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:10.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.879+0000 7f33ffa69740 1 --2- 172.21.15.105:0/2292373643 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x55e1e4070fd0 0x55e1e4073420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:10.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.879+0000 7f33ffa69740 4 client.15714 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:05:10.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.879+0000 7f33ffa69740 1 -- 172.21.15.105:0/2292373643 --> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1e3fc9860 con 0x55e1e4070fd0 2024-09-16T02:05:10.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.882+0000 7f33e7fff640 1 --2- 172.21.15.105:0/2292373643 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x55e1e4070fd0 0x55e1e4073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:10.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.885+0000 7f33e7fff640 1 --2- 172.21.15.105:0/2292373643 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x55e1e4070fd0 0x55e1e4073420 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a6e28b7aaadeb948 server_cookie=fe38c311083e2e77 in_seq=0 out_seq=0 2024-09-16T02:05:10.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.885+0000 7f33de7fc640 10 client.15714 ms_handle_connect on v2:172.21.15.137:6836/2788541030 2024-09-16T02:05:10.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.887+0000 7f33de7fc640 1 -- 172.21.15.105:0/2292373643 <== mds.2 v2:172.21.15.137:6836/2788541030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e1e3fc9860 con 0x55e1e4070fd0 2024-09-16T02:05:10.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.887+0000 7f33de7fc640 10 client.15714 handle_command_reply: tid=0 2024-09-16T02:05:10.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.887+0000 7f33ffa69740 1 client.15714 shutdown 2024-09-16T02:05:10.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.888+0000 7f33ffa69740 1 -- 172.21.15.105:0/2292373643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1e40565d0 msgr2=0x55e1e4060de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:10.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.888+0000 7f33ffa69740 1 --2- 172.21.15.105:0/2292373643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1e40565d0 0x55e1e4060de0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f33e800ca10 tx=0x7f33e800ced0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.888+0000 7f33ffa69740 1 -- 172.21.15.105:0/2292373643 shutdown_connections 2024-09-16T02:05:10.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.888+0000 7f33ffa69740 1 --2- 172.21.15.105:0/2292373643 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x55e1e4070fd0 0x55e1e4073420 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.888+0000 7f33ffa69740 1 --2- 172.21.15.105:0/2292373643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1e4057b40 0x55e1e405e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.888+0000 7f33ffa69740 1 --2- 172.21.15.105:0/2292373643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1e4061370 0x55e1e405e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.888+0000 7f33ffa69740 1 --2- 172.21.15.105:0/2292373643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1e40565d0 0x55e1e4060de0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.888+0000 7f33ffa69740 1 -- 172.21.15.105:0/2292373643 >> 172.21.15.105:0/2292373643 conn(0x55e1e401f310 msgr2=0x55e1e4058960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:10.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.888+0000 7f33de7fc640 0 client.15714 ms_handle_reset on v2:172.21.15.137:6836/2788541030 2024-09-16T02:05:10.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.888+0000 7f33ffa69740 1 -- 172.21.15.105:0/2292373643 shutdown_connections 2024-09-16T02:05:10.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.888+0000 7f33ffa69740 1 -- 172.21.15.105:0/2292373643 wait complete. 2024-09-16T02:05:10.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.888+0000 7f33ffa69740 20 client.15714 trim_cache size 0 max 16384 2024-09-16T02:05:10.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.911+0000 7f33ffa69740 1 Processor -- start 2024-09-16T02:05:10.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.911+0000 7f33ffa69740 1 -- start start 2024-09-16T02:05:10.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.912+0000 7f33ffa69740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1e40565d0 0x55e1e406d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:10.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.912+0000 7f33ffa69740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1e4057b40 0x55e1e405d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:10.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.912+0000 7f33ffa69740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1e4061370 0x55e1e4078cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:10.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.912+0000 7f33ffa69740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f33f81b3dc0 con 0x55e1e4057b40 2024-09-16T02:05:10.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.912+0000 7f33ffa69740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f33f81b5900 con 0x55e1e4061370 2024-09-16T02:05:10.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.912+0000 7f33ffa69740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f33f81b4490 con 0x55e1e40565d0 2024-09-16T02:05:10.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.912+0000 7f33e7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1e4057b40 0x55e1e405d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:10.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.912+0000 7f33e7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1e4057b40 0x55e1e405d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55294/0 (socket says 172.21.15.105:55294) 2024-09-16T02:05:10.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.912+0000 7f33e7fff640 1 -- 172.21.15.105:0/1505336278 learned_addr learned my addr 172.21.15.105:0/1505336278 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:10.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.913+0000 7f33e7fff640 1 -- 172.21.15.105:0/1505336278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1e40565d0 msgr2=0x55e1e406d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:10.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.913+0000 7f33e7fff640 1 --2- 172.21.15.105:0/1505336278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1e40565d0 0x55e1e406d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.913+0000 7f33e7fff640 1 -- 172.21.15.105:0/1505336278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1e4061370 msgr2=0x55e1e4078cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:10.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.913+0000 7f33e7fff640 1 --2- 172.21.15.105:0/1505336278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1e4061370 0x55e1e4078cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.913+0000 7f33e7fff640 1 -- 172.21.15.105:0/1505336278 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1e4052de0 con 0x55e1e4057b40 2024-09-16T02:05:10.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.913+0000 7f33e7fff640 1 --2- 172.21.15.105:0/1505336278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1e4057b40 0x55e1e405d090 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f33ec015ca0 tx=0x7f33ec0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b483232b48fadea server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:10.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.913+0000 7f33e4ff9640 1 -- 172.21.15.105:0/1505336278 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec01e7d0 con 0x55e1e4057b40 2024-09-16T02:05:10.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.914+0000 7f33e4ff9640 1 -- 172.21.15.105:0/1505336278 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33ec023070 con 0x55e1e4057b40 2024-09-16T02:05:10.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.914+0000 7f33e4ff9640 1 -- 172.21.15.105:0/1505336278 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec032b70 con 0x55e1e4057b40 2024-09-16T02:05:10.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.914+0000 7f33ffa69740 1 -- 172.21.15.105:0/1505336278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1e4057b40 msgr2=0x55e1e405d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:10.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.914+0000 7f33ffa69740 1 --2- 172.21.15.105:0/1505336278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1e4057b40 0x55e1e405d090 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f33ec015ca0 tx=0x7f33ec0047d0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.915+0000 7f33ffa69740 1 -- 172.21.15.105:0/1505336278 shutdown_connections 2024-09-16T02:05:10.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.915+0000 7f33ffa69740 1 --2- 172.21.15.105:0/1505336278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1e4061370 0x55e1e4078cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.915+0000 7f33ffa69740 1 --2- 172.21.15.105:0/1505336278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1e4057b40 0x55e1e405d090 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.915+0000 7f33ffa69740 1 --2- 172.21.15.105:0/1505336278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1e40565d0 0x55e1e406d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.915+0000 7f33ffa69740 1 -- 172.21.15.105:0/1505336278 >> 172.21.15.105:0/1505336278 conn(0x55e1e401f310 msgr2=0x55e1e405c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:10.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.915+0000 7f33ffa69740 1 -- 172.21.15.105:0/1505336278 shutdown_connections 2024-09-16T02:05:10.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.915+0000 7f33ffa69740 1 -- 172.21.15.105:0/1505336278 wait complete. 2024-09-16T02:05:10.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.915+0000 7f33ffa69740 1 -- start start 2024-09-16T02:05:10.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.916+0000 7f33ffa69740 1 Processor -- start 2024-09-16T02:05:10.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.916+0000 7f33ffa69740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1e40565d0 0x55e1e4069720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:10.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.916+0000 7f33ffa69740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1e4057b40 0x55e1e4069c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:10.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.916+0000 7f33ffa69740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1e4061370 0x55e1e405aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:10.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.916+0000 7f33ffa69740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f33f81b5230 con 0x55e1e4057b40 2024-09-16T02:05:10.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.916+0000 7f33ffa69740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f33f81b4b60 con 0x55e1e4061370 2024-09-16T02:05:10.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.916+0000 7f33ffa69740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f33f81b1da0 con 0x55e1e40565d0 2024-09-16T02:05:10.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.916+0000 7f33e7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1e4057b40 0x55e1e4069c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:10.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.916+0000 7f33e7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1e4057b40 0x55e1e4069c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55298/0 (socket says 172.21.15.105:55298) 2024-09-16T02:05:10.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.916+0000 7f33e7fff640 1 -- 172.21.15.105:0/82471032 learned_addr learned my addr 172.21.15.105:0/82471032 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:10.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.916+0000 7f33ffa69740 10 client.0 _ll_register_callbacks cb 0x7f33e80096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:10.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.916+0000 7f33e7fff640 1 -- 172.21.15.105:0/82471032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1e40565d0 msgr2=0x55e1e4069720 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T02:05:10.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.916+0000 7f33e7fff640 1 --2- 172.21.15.105:0/82471032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1e40565d0 0x55e1e4069720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.916+0000 7f33e7fff640 1 -- 172.21.15.105:0/82471032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1e4061370 msgr2=0x55e1e405aa90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T02:05:10.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.916+0000 7f33e7fff640 1 --2- 172.21.15.105:0/82471032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1e4061370 0x55e1e405aa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.916+0000 7f33e7fff640 1 -- 172.21.15.105:0/82471032 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1e405b260 con 0x55e1e4057b40 2024-09-16T02:05:10.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.917+0000 7f33e7fff640 1 --2- 172.21.15.105:0/82471032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1e4057b40 0x55e1e4069c60 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f33ec010a30 tx=0x7f33ec032610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dec10df92e40bf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:10.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.917+0000 7f33de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:05:10.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.917+0000 7f33de7fc640 1 -- 172.21.15.105:0/82471032 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec040060 con 0x55e1e4057b40 2024-09-16T02:05:10.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.917+0000 7f33de7fc640 1 -- 172.21.15.105:0/82471032 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f33ec032d00 con 0x55e1e4057b40 2024-09-16T02:05:10.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.917+0000 7f33ffa69740 1 -- 172.21.15.105:0/82471032 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33ec023070 con 0x55e1e4057b40 2024-09-16T02:05:10.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.917+0000 7f33de7fc640 1 -- 172.21.15.105:0/82471032 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f33ec0059c0 con 0x55e1e4057b40 2024-09-16T02:05:10.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.917+0000 7f33ffa69740 10 client.15720 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:10.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.917+0000 7f33ffa69740 1 -- 172.21.15.105:0/82471032 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e1e40db860 con 0x55e1e4057b40 2024-09-16T02:05:10.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.918+0000 7f33de7fc640 1 -- 172.21.15.105:0/82471032 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33ec023070 con 0x55e1e4057b40 2024-09-16T02:05:10.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.918+0000 7f33de7fc640 1 -- 172.21.15.105:0/82471032 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f33ec031070 con 0x55e1e4057b40 2024-09-16T02:05:10.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.918+0000 7f33ffa69740 10 client.15720 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:10.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.918+0000 7f33ffa69740 10 client.15720 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T02:05:10.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.918+0000 7f33ffa69740 20 client.15720 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:10.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.918+0000 7f33ffa69740 1 --2- 172.21.15.105:0/82471032 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x55e1e40dbbf0 0x55e1e40ddfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:10.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.918+0000 7f33ffa69740 4 client.15720 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T02:05:10.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.918+0000 7f33ffa69740 1 -- 172.21.15.105:0/82471032 --> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e1e3fc3d70 con 0x55e1e40dbbf0 2024-09-16T02:05:10.924 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:10 smithi137 ceph-mon[34099]: pgmap v418: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 277 MiB/s wr, 81 op/s 2024-09-16T02:05:10.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.920+0000 7f33f6ffd640 1 --2- 172.21.15.105:0/82471032 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x55e1e40dbbf0 0x55e1e40ddfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:10.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.923+0000 7f33de7fc640 10 client.15720 ms_handle_connect on v2:172.21.15.137:6836/2788541030 2024-09-16T02:05:10.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.923+0000 7f33f6ffd640 1 --2- 172.21.15.105:0/82471032 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x55e1e40dbbf0 0x55e1e40ddfe0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d8e79c87b810464b server_cookie=76b9314095be002b in_seq=0 out_seq=0 2024-09-16T02:05:10.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.923+0000 7f33de7fc640 1 -- 172.21.15.105:0/82471032 <== mds.2 v2:172.21.15.137:6836/2788541030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e1e3fc3d70 con 0x55e1e40dbbf0 2024-09-16T02:05:10.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.924+0000 7f33de7fc640 10 client.15720 handle_command_reply: tid=0 2024-09-16T02:05:10.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.924+0000 7f33ffa69740 1 client.15720 shutdown 2024-09-16T02:05:10.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.924+0000 7f33ffa69740 1 -- 172.21.15.105:0/82471032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1e4057b40 msgr2=0x55e1e4069c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:10.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.924+0000 7f33ffa69740 1 --2- 172.21.15.105:0/82471032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1e4057b40 0x55e1e4069c60 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f33ec010a30 tx=0x7f33ec032610 comp rx=0 tx=0).stop 2024-09-16T02:05:10.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.924+0000 7f33ffa69740 1 -- 172.21.15.105:0/82471032 shutdown_connections 2024-09-16T02:05:10.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.924+0000 7f33ffa69740 1 --2- 172.21.15.105:0/82471032 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x55e1e40dbbf0 0x55e1e40ddfe0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.924+0000 7f33ffa69740 1 --2- 172.21.15.105:0/82471032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1e4061370 0x55e1e405aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.924+0000 7f33ffa69740 1 --2- 172.21.15.105:0/82471032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1e4057b40 0x55e1e4069c60 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.924+0000 7f33ffa69740 1 --2- 172.21.15.105:0/82471032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1e40565d0 0x55e1e4069720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.924+0000 7f33ffa69740 1 -- 172.21.15.105:0/82471032 >> 172.21.15.105:0/82471032 conn(0x55e1e401f310 msgr2=0x55e1e405d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:10.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.924+0000 7f33de7fc640 0 client.15720 ms_handle_reset on v2:172.21.15.137:6836/2788541030 2024-09-16T02:05:10.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.924+0000 7f33ffa69740 1 -- 172.21.15.105:0/82471032 shutdown_connections 2024-09-16T02:05:10.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.924+0000 7f33ffa69740 1 -- 172.21.15.105:0/82471032 wait complete. 2024-09-16T02:05:10.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.924+0000 7f33ffa69740 20 client.15720 trim_cache size 0 max 16384 2024-09-16T02:05:10.928 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-09-16T02:05:10.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.925+0000 7f33fd690640 1 -- 172.21.15.105:0/947040164 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f33e00a3240 msgr2=0x7f33e00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:10.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.925+0000 7f33fd690640 1 --2- 172.21.15.105:0/947040164 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f33e00a3240 0x7f33e00a5700 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f33ec006fd0 tx=0x7f33ec00af00 comp rx=0 tx=0).stop 2024-09-16T02:05:10.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.925+0000 7f33fd690640 1 -- 172.21.15.105:0/947040164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33f8105ec0 msgr2=0x7f33f819e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:10.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.925+0000 7f33fd690640 1 --2- 172.21.15.105:0/947040164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33f8105ec0 0x7f33f819e900 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f33f0009230 tx=0x7f33f00042f0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.926+0000 7f33fd690640 1 -- 172.21.15.105:0/947040164 shutdown_connections 2024-09-16T02:05:10.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.926+0000 7f33fd690640 1 --2- 172.21.15.105:0/947040164 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f33e00a3240 0x7f33e00a5700 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.926+0000 7f33fd690640 1 --2- 172.21.15.105:0/947040164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33f8105ec0 0x7f33f819e900 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.926+0000 7f33fd690640 1 --2- 172.21.15.105:0/947040164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33f81011e0 0x7f33f81a4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.926+0000 7f33fd690640 1 --2- 172.21.15.105:0/947040164 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33f81008c0 0x7f33f81a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.926+0000 7f33fd690640 1 -- 172.21.15.105:0/947040164 >> 172.21.15.105:0/947040164 conn(0x7f33f80fc7d0 msgr2=0x7f33f810f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:10.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.926+0000 7f33fd690640 1 -- 172.21.15.105:0/947040164 shutdown_connections 2024-09-16T02:05:10.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:10.926+0000 7f33fd690640 1 -- 172.21.15.105:0/947040164 wait complete. 2024-09-16T02:05:11.928 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T02:05:12.356 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:12.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:12 smithi105 ceph-mon[31197]: pgmap v419: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 423 MiB/s wr, 116 op/s 2024-09-16T02:05:12.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:12 smithi190 ceph-mon[34762]: pgmap v419: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 423 MiB/s wr, 116 op/s 2024-09-16T02:05:12.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:12 smithi137 ceph-mon[34099]: pgmap v419: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 423 MiB/s wr, 116 op/s 2024-09-16T02:05:12.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.967+0000 7f0010f31640 1 -- 172.21.15.105:0/2261665227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f000c106f10 msgr2=0x7f000c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:12.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.967+0000 7f0010f31640 1 --2- 172.21.15.105:0/2261665227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f000c106f10 0x7f000c10ea90 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0004009ca0 tx=0x7f000402a8e0 comp rx=0 tx=0).stop 2024-09-16T02:05:12.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.967+0000 7f0010f31640 1 -- 172.21.15.105:0/2261665227 shutdown_connections 2024-09-16T02:05:12.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.967+0000 7f0010f31640 1 --2- 172.21.15.105:0/2261665227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f000c102790 0x7f000c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:12.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.967+0000 7f0010f31640 1 --2- 172.21.15.105:0/2261665227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f000c106f10 0x7f000c10ea90 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:12.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.967+0000 7f0010f31640 1 --2- 172.21.15.105:0/2261665227 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f000c101de0 0x7f000c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:12.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.967+0000 7f0010f31640 1 -- 172.21.15.105:0/2261665227 >> 172.21.15.105:0/2261665227 conn(0x7f000c0fd8c0 msgr2=0x7f000c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:12.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.968+0000 7f0010f31640 1 -- 172.21.15.105:0/2261665227 shutdown_connections 2024-09-16T02:05:12.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.968+0000 7f0010f31640 1 -- 172.21.15.105:0/2261665227 wait complete. 2024-09-16T02:05:12.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.969+0000 7f0010f31640 1 Processor -- start 2024-09-16T02:05:12.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.970+0000 7f0010f31640 1 -- start start 2024-09-16T02:05:12.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.970+0000 7f0010f31640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f000c101de0 0x7f000c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:12.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.970+0000 7f0010f31640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f000c102790 0x7f000c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:12.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.970+0000 7f0010f31640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f000c106f10 0x7f000c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:12.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.970+0000 7f0010f31640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f000c111f30 con 0x7f000c102790 2024-09-16T02:05:12.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.970+0000 7f0010f31640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f000c111db0 con 0x7f000c106f10 2024-09-16T02:05:12.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.970+0000 7f0010f31640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f000c1120b0 con 0x7f000c101de0 2024-09-16T02:05:12.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.970+0000 7f0009d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f000c102790 0x7f000c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:12.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.970+0000 7f0009d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f000c102790 0x7f000c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55314/0 (socket says 172.21.15.105:55314) 2024-09-16T02:05:12.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.971+0000 7f0009d9b640 1 -- 172.21.15.105:0/2171118917 learned_addr learned my addr 172.21.15.105:0/2171118917 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:12.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.971+0000 7f000a59c640 1 --2- 172.21.15.105:0/2171118917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f000c101de0 0x7f000c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:12.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.971+0000 7f000ad9d640 1 --2- 172.21.15.105:0/2171118917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f000c106f10 0x7f000c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:12.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.971+0000 7f0009d9b640 1 -- 172.21.15.105:0/2171118917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f000c101de0 msgr2=0x7f000c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:12.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.971+0000 7f0009d9b640 1 --2- 172.21.15.105:0/2171118917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f000c101de0 0x7f000c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:12.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.971+0000 7f0009d9b640 1 -- 172.21.15.105:0/2171118917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f000c106f10 msgr2=0x7f000c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:12.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.971+0000 7f0009d9b640 1 --2- 172.21.15.105:0/2171118917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f000c106f10 0x7f000c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:12.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.971+0000 7f0009d9b640 1 -- 172.21.15.105:0/2171118917 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f000c1a69b0 con 0x7f000c102790 2024-09-16T02:05:12.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.971+0000 7f000ad9d640 1 --2- 172.21.15.105:0/2171118917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f000c106f10 0x7f000c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:05:12.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.972+0000 7f0009d9b640 1 --2- 172.21.15.105:0/2171118917 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f000c102790 0x7f000c074a30 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f000000b4a0 tx=0x7f000000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:12.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.972+0000 7efffb7fe640 1 -- 172.21.15.105:0/2171118917 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0000014070 con 0x7f000c102790 2024-09-16T02:05:12.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.972+0000 7f0010f31640 1 -- 172.21.15.105:0/2171118917 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f000c1a6ca0 con 0x7f000c102790 2024-09-16T02:05:12.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.972+0000 7f0010f31640 1 -- 172.21.15.105:0/2171118917 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f000c1a7190 con 0x7f000c102790 2024-09-16T02:05:12.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.972+0000 7efffb7fe640 1 -- 172.21.15.105:0/2171118917 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0000010ac0 con 0x7f000c102790 2024-09-16T02:05:12.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.972+0000 7efffb7fe640 1 -- 172.21.15.105:0/2171118917 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0000017c30 con 0x7f000c102790 2024-09-16T02:05:12.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.976+0000 7f001330a740 1 Processor -- start 2024-09-16T02:05:12.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.977+0000 7f001330a740 1 -- start start 2024-09-16T02:05:12.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.977+0000 7efffb7fe640 1 -- 172.21.15.105:0/2171118917 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f0000017070 con 0x7f000c102790 2024-09-16T02:05:12.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.977+0000 7f001330a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc82f85350 0x55dc82f838a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:12.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.978+0000 7f001330a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc82f83de0 0x55dc82f84220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:12.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.978+0000 7f001330a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc82f8eb80 0x55dc82f90f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:12.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.978+0000 7f001330a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55dc82ee12f0 con 0x55dc82f83de0 2024-09-16T02:05:12.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.978+0000 7f001330a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dc82e88cf0 con 0x55dc82f8eb80 2024-09-16T02:05:12.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.978+0000 7f001330a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55dc82edd2b0 con 0x55dc82f85350 2024-09-16T02:05:12.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.978+0000 7f000a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc82f83de0 0x55dc82f84220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:12.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.978+0000 7f000a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc82f83de0 0x55dc82f84220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55328/0 (socket says 172.21.15.105:55328) 2024-09-16T02:05:12.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.978+0000 7f000a59c640 1 -- 172.21.15.105:0/2527682274 learned_addr learned my addr 172.21.15.105:0/2527682274 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:12.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.978+0000 7f000ad9d640 1 --2- 172.21.15.105:0/2527682274 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc82f85350 0x55dc82f838a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:12.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.978+0000 7f0009d9b640 1 --2- 172.21.15.105:0/2527682274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc82f8eb80 0x55dc82f90f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:12.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.978+0000 7f000a59c640 1 -- 172.21.15.105:0/2527682274 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc82f85350 msgr2=0x55dc82f838a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:12.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.978+0000 7f000a59c640 1 --2- 172.21.15.105:0/2527682274 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc82f85350 0x55dc82f838a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:12.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.978+0000 7f000a59c640 1 -- 172.21.15.105:0/2527682274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc82f8eb80 msgr2=0x55dc82f90f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:12.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.978+0000 7f000a59c640 1 --2- 172.21.15.105:0/2527682274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc82f8eb80 0x55dc82f90f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:12.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.978+0000 7f000a59c640 1 -- 172.21.15.105:0/2527682274 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc82e6d3a0 con 0x55dc82f83de0 2024-09-16T02:05:12.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.979+0000 7f000ad9d640 1 --2- 172.21.15.105:0/2527682274 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc82f85350 0x55dc82f838a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:05:12.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.979+0000 7f000a59c640 1 --2- 172.21.15.105:0/2527682274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc82f83de0 0x55dc82f84220 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7efffc009a40 tx=0x7efffc009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ca406f2d712b13 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:12.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.979+0000 7efff8ff9640 1 -- 172.21.15.105:0/2527682274 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efffc017070 con 0x55dc82f83de0 2024-09-16T02:05:12.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.979+0000 7efff8ff9640 1 -- 172.21.15.105:0/2527682274 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efffc013470 con 0x55dc82f83de0 2024-09-16T02:05:12.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.979+0000 7efff8ff9640 1 -- 172.21.15.105:0/2527682274 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efffc012630 con 0x55dc82f83de0 2024-09-16T02:05:12.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.979+0000 7efffb7fe640 1 --2- 172.21.15.105:0/2171118917 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7efff40a32c0 0x7efff40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:12.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.980+0000 7efffb7fe640 1 -- 172.21.15.105:0/2171118917 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f00000cebf0 con 0x7f000c102790 2024-09-16T02:05:12.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.980+0000 7f000ad9d640 1 --2- 172.21.15.105:0/2171118917 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7efff40a32c0 0x7efff40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:12.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.981+0000 7f001330a740 1 -- 172.21.15.105:0/2527682274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc82f83de0 msgr2=0x55dc82f84220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:12.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.981+0000 7f000ad9d640 1 --2- 172.21.15.105:0/2171118917 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7efff40a32c0 0x7efff40a5780 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0004004af0 tx=0x7f00040098f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:12.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.981+0000 7f001330a740 1 --2- 172.21.15.105:0/2527682274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc82f83de0 0x55dc82f84220 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7efffc009a40 tx=0x7efffc009f00 comp rx=0 tx=0).stop 2024-09-16T02:05:12.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.981+0000 7f001330a740 1 -- 172.21.15.105:0/2527682274 shutdown_connections 2024-09-16T02:05:12.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.981+0000 7f001330a740 1 --2- 172.21.15.105:0/2527682274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc82f8eb80 0x55dc82f90f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:12.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.981+0000 7f001330a740 1 --2- 172.21.15.105:0/2527682274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc82f83de0 0x55dc82f84220 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:12.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.981+0000 7f001330a740 1 --2- 172.21.15.105:0/2527682274 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc82f85350 0x55dc82f838a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:12.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.981+0000 7f001330a740 1 -- 172.21.15.105:0/2527682274 >> 172.21.15.105:0/2527682274 conn(0x55dc82f44310 msgr2=0x55dc82f80470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:12.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.981+0000 7f001330a740 1 -- 172.21.15.105:0/2527682274 shutdown_connections 2024-09-16T02:05:12.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.981+0000 7f001330a740 1 -- 172.21.15.105:0/2527682274 wait complete. 2024-09-16T02:05:12.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.982+0000 7f001330a740 1 -- start start 2024-09-16T02:05:12.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.983+0000 7f001330a740 1 Processor -- start 2024-09-16T02:05:12.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.984+0000 7f001330a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc82f83de0 0x55dc82f8e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:12.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.984+0000 7f001330a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc82f85350 0x55dc82f8b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:12.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.985+0000 7f001330a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc82f8eb80 0x55dc82f8bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:12.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.985+0000 7f000a59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc82f83de0 0x55dc82f8e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:12.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.985+0000 7f000ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc82f8eb80 0x55dc82f8bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:12.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.985+0000 7f000a59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc82f83de0 0x55dc82f8e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.105:44720/0 (socket says 172.21.15.105:44720) 2024-09-16T02:05:12.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.985+0000 7f000a59c640 1 -- 172.21.15.105:0/2540862302 learned_addr learned my addr 172.21.15.105:0/2540862302 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:12.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.985+0000 7f001330a740 1 -- 172.21.15.105:0/2540862302 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55dc82f91880 con 0x55dc82f8eb80 2024-09-16T02:05:12.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.985+0000 7f0009d9b640 1 --2- 172.21.15.105:0/2540862302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc82f85350 0x55dc82f8b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:12.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.986+0000 7f001330a740 1 -- 172.21.15.105:0/2540862302 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dc82e9c060 con 0x55dc82f83de0 2024-09-16T02:05:12.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.986+0000 7f001330a740 1 -- 172.21.15.105:0/2540862302 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55dc82f91a00 con 0x55dc82f85350 2024-09-16T02:05:12.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.986+0000 7f001330a740 10 client.0 _ll_register_callbacks cb 0x55dc82bef8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:12.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.987+0000 7f000ad9d640 1 -- 172.21.15.105:0/2540862302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc82f85350 msgr2=0x55dc82f8b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:12.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.987+0000 7f000ad9d640 1 --2- 172.21.15.105:0/2540862302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc82f85350 0x55dc82f8b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:12.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.987+0000 7f000ad9d640 1 -- 172.21.15.105:0/2540862302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc82f83de0 msgr2=0x55dc82f8e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:12.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.987+0000 7f000ad9d640 1 --2- 172.21.15.105:0/2540862302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc82f83de0 0x55dc82f8e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:12.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.987+0000 7f000ad9d640 1 -- 172.21.15.105:0/2540862302 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc82c523c0 con 0x55dc82f8eb80 2024-09-16T02:05:12.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.987+0000 7f0009d9b640 1 --2- 172.21.15.105:0/2540862302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc82f85350 0x55dc82f8b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:05:12.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.987+0000 7f000ad9d640 1 --2- 172.21.15.105:0/2540862302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc82f8eb80 0x55dc82f8bc30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f0004005e30 tx=0x7f00040095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbd3728a7ebfcffb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:12.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.988+0000 7efff1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:05:12.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.988+0000 7efff1ffb640 1 -- 172.21.15.105:0/2540862302 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0004031470 con 0x55dc82f8eb80 2024-09-16T02:05:12.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.988+0000 7efff1ffb640 1 -- 172.21.15.105:0/2540862302 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f00040305e0 con 0x55dc82f8eb80 2024-09-16T02:05:12.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.988+0000 7f001330a740 1 -- 172.21.15.105:0/2540862302 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efffc013470 con 0x55dc82f8eb80 2024-09-16T02:05:12.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.988+0000 7efff1ffb640 1 -- 172.21.15.105:0/2540862302 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f00040307e0 con 0x55dc82f8eb80 2024-09-16T02:05:12.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.988+0000 7f001330a740 10 client.15732 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:12.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.988+0000 7f001330a740 1 -- 172.21.15.105:0/2540862302 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dc82f9d1d0 con 0x55dc82f8eb80 2024-09-16T02:05:12.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.989+0000 7efff1ffb640 1 -- 172.21.15.105:0/2540862302 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efffc013470 con 0x55dc82f8eb80 2024-09-16T02:05:12.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.989+0000 7efff1ffb640 1 -- 172.21.15.105:0/2540862302 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0004048910 con 0x55dc82f8eb80 2024-09-16T02:05:12.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.989+0000 7f001330a740 10 client.15732 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:12.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.989+0000 7f001330a740 10 client.15732 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T02:05:12.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.989+0000 7f001330a740 20 client.15732 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:12.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.989+0000 7f001330a740 1 --2- 172.21.15.105:0/2540862302 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55dc82f9e680 0x55dc82fa0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:12.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.989+0000 7f001330a740 4 client.15732 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:05:12.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.989+0000 7f001330a740 1 -- 172.21.15.105:0/2540862302 --> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc82eee860 con 0x55dc82f9e680 2024-09-16T02:05:12.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.990+0000 7f000a59c640 1 --2- 172.21.15.105:0/2540862302 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55dc82f9e680 0x55dc82fa0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:12.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.992+0000 7f000a59c640 1 --2- 172.21.15.105:0/2540862302 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55dc82f9e680 0x55dc82fa0b00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6eae6370bb4d09cb server_cookie=35a1fe816650ef in_seq=0 out_seq=0 2024-09-16T02:05:12.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.992+0000 7efff1ffb640 10 client.15732 ms_handle_connect on v2:172.21.15.190:6838/1222573537 2024-09-16T02:05:12.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.993+0000 7efff1ffb640 1 -- 172.21.15.105:0/2540862302 <== mds.1 v2:172.21.15.190:6838/1222573537 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dc82eee860 con 0x55dc82f9e680 2024-09-16T02:05:12.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.993+0000 7efff1ffb640 10 client.15732 handle_command_reply: tid=0 2024-09-16T02:05:12.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.993+0000 7f001330a740 1 client.15732 shutdown 2024-09-16T02:05:12.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.993+0000 7f001330a740 1 -- 172.21.15.105:0/2540862302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc82f8eb80 msgr2=0x55dc82f8bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:12.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.993+0000 7f001330a740 1 --2- 172.21.15.105:0/2540862302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc82f8eb80 0x55dc82f8bc30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f0004005e30 tx=0x7f00040095c0 comp rx=0 tx=0).stop 2024-09-16T02:05:13.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.994+0000 7f001330a740 1 -- 172.21.15.105:0/2540862302 shutdown_connections 2024-09-16T02:05:13.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.994+0000 7f001330a740 1 --2- 172.21.15.105:0/2540862302 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55dc82f9e680 0x55dc82fa0b00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:13.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.994+0000 7f001330a740 1 --2- 172.21.15.105:0/2540862302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc82f85350 0x55dc82f8b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:13.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.994+0000 7f001330a740 1 --2- 172.21.15.105:0/2540862302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc82f8eb80 0x55dc82f8bc30 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:13.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.994+0000 7f001330a740 1 --2- 172.21.15.105:0/2540862302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc82f83de0 0x55dc82f8e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:13.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.994+0000 7efff1ffb640 0 client.15732 ms_handle_reset on v2:172.21.15.190:6838/1222573537 2024-09-16T02:05:13.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.994+0000 7f001330a740 1 -- 172.21.15.105:0/2540862302 >> 172.21.15.105:0/2540862302 conn(0x55dc82f44310 msgr2=0x55dc82f86140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:13.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.994+0000 7f001330a740 1 -- 172.21.15.105:0/2540862302 shutdown_connections 2024-09-16T02:05:13.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.994+0000 7f001330a740 1 -- 172.21.15.105:0/2540862302 wait complete. 2024-09-16T02:05:13.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:12.994+0000 7f001330a740 20 client.15732 trim_cache size 0 max 16384 2024-09-16T02:05:13.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.010+0000 7f001330a740 1 Processor -- start 2024-09-16T02:05:13.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.010+0000 7f001330a740 1 -- start start 2024-09-16T02:05:13.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.011+0000 7f001330a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc82f83de0 0x55dc82f9abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:13.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.011+0000 7f001330a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc82f85350 0x55dc82f8ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:13.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.011+0000 7f001330a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc82f8eb80 0x55dc82f904b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:13.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.011+0000 7f001330a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7effec00e600 con 0x55dc82f85350 2024-09-16T02:05:13.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.011+0000 7f001330a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7effec010140 con 0x55dc82f8eb80 2024-09-16T02:05:13.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.011+0000 7f001330a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7effec00ecd0 con 0x55dc82f83de0 2024-09-16T02:05:13.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.012+0000 7f000ad9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc82f8eb80 0x55dc82f904b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:13.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.012+0000 7f000ad9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc82f8eb80 0x55dc82f904b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.105:44722/0 (socket says 172.21.15.105:44722) 2024-09-16T02:05:13.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.012+0000 7f000ad9d640 1 -- 172.21.15.105:0/191357668 learned_addr learned my addr 172.21.15.105:0/191357668 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:13.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.012+0000 7f0009d9b640 1 --2- 172.21.15.105:0/191357668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc82f85350 0x55dc82f8ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:13.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.013+0000 7f000a59c640 1 --2- 172.21.15.105:0/191357668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc82f83de0 0x55dc82f9abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:13.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.014+0000 7f0009d9b640 1 -- 172.21.15.105:0/191357668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc82f83de0 msgr2=0x55dc82f9abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:13.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.014+0000 7f0009d9b640 1 --2- 172.21.15.105:0/191357668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc82f83de0 0x55dc82f9abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:13.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.014+0000 7f0009d9b640 1 -- 172.21.15.105:0/191357668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc82f8eb80 msgr2=0x55dc82f904b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:13.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.014+0000 7f0009d9b640 1 --2- 172.21.15.105:0/191357668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc82f8eb80 0x55dc82f904b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:13.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.014+0000 7f0009d9b640 1 -- 172.21.15.105:0/191357668 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc82fa1180 con 0x55dc82f85350 2024-09-16T02:05:13.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.014+0000 7f000ad9d640 1 --2- 172.21.15.105:0/191357668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc82f8eb80 0x55dc82f904b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:05:13.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.014+0000 7f0009d9b640 1 --2- 172.21.15.105:0/191357668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc82f85350 0x55dc82f8ff70 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f00000cc6e0 tx=0x7f000000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=913cb6230cc2efa3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:13.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.015+0000 7efff3fff640 1 -- 172.21.15.105:0/191357668 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000001e040 con 0x55dc82f85350 2024-09-16T02:05:13.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.015+0000 7efff3fff640 1 -- 172.21.15.105:0/191357668 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00000a3e20 con 0x55dc82f85350 2024-09-16T02:05:13.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.015+0000 7efff3fff640 1 -- 172.21.15.105:0/191357668 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0000097820 con 0x55dc82f85350 2024-09-16T02:05:13.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.016+0000 7f001330a740 1 -- 172.21.15.105:0/191357668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc82f85350 msgr2=0x55dc82f8ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:13.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.016+0000 7f001330a740 1 --2- 172.21.15.105:0/191357668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc82f85350 0x55dc82f8ff70 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f00000cc6e0 tx=0x7f000000bf90 comp rx=0 tx=0).stop 2024-09-16T02:05:13.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.017+0000 7f001330a740 1 -- 172.21.15.105:0/191357668 shutdown_connections 2024-09-16T02:05:13.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.017+0000 7f001330a740 1 --2- 172.21.15.105:0/191357668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc82f8eb80 0x55dc82f904b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:13.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.017+0000 7f001330a740 1 --2- 172.21.15.105:0/191357668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc82f85350 0x55dc82f8ff70 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:13.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.017+0000 7f001330a740 1 --2- 172.21.15.105:0/191357668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc82f83de0 0x55dc82f9abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:13.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.017+0000 7f001330a740 1 -- 172.21.15.105:0/191357668 >> 172.21.15.105:0/191357668 conn(0x55dc82f44310 msgr2=0x55dc82f8b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:13.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.017+0000 7f001330a740 1 -- 172.21.15.105:0/191357668 shutdown_connections 2024-09-16T02:05:13.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.017+0000 7f001330a740 1 -- 172.21.15.105:0/191357668 wait complete. 2024-09-16T02:05:13.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.018+0000 7f001330a740 1 -- start start 2024-09-16T02:05:13.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.018+0000 7f001330a740 1 Processor -- start 2024-09-16T02:05:13.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.019+0000 7f001330a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc82f83de0 0x55dc82f96d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:13.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.019+0000 7f001330a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc82f85350 0x55dc82f97270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:13.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.019+0000 7f000a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc82f83de0 0x55dc82f96d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:13.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.019+0000 7f000a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc82f83de0 0x55dc82f96d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55356/0 (socket says 172.21.15.105:55356) 2024-09-16T02:05:13.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.019+0000 7f000a59c640 1 -- 172.21.15.105:0/3351708754 learned_addr learned my addr 172.21.15.105:0/3351708754 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:13.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.019+0000 7f001330a740 1 --2- 172.21.15.105:0/3351708754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc82f8eb80 0x55dc82f91ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:13.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.019+0000 7f001330a740 1 -- 172.21.15.105:0/3351708754 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7effec00fa70 con 0x55dc82f83de0 2024-09-16T02:05:13.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.019+0000 7f001330a740 1 -- 172.21.15.105:0/3351708754 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7effec00f3a0 con 0x55dc82f8eb80 2024-09-16T02:05:13.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.019+0000 7f001330a740 1 -- 172.21.15.105:0/3351708754 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7effec00c5e0 con 0x55dc82f85350 2024-09-16T02:05:13.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.020+0000 7f001330a740 10 client.0 _ll_register_callbacks cb 0x7f0004009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:13.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.020+0000 7f000ad9d640 1 --2- 172.21.15.105:0/3351708754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc82f8eb80 0x55dc82f91ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:13.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.020+0000 7f000a59c640 1 -- 172.21.15.105:0/3351708754 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc82f85350 msgr2=0x55dc82f97270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:13.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.020+0000 7f000a59c640 1 --2- 172.21.15.105:0/3351708754 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc82f85350 0x55dc82f97270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:13.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.020+0000 7f000a59c640 1 -- 172.21.15.105:0/3351708754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc82f8eb80 msgr2=0x55dc82f91ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:13.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.020+0000 7f000a59c640 1 --2- 172.21.15.105:0/3351708754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc82f8eb80 0x55dc82f91ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:13.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.020+0000 7f000a59c640 1 -- 172.21.15.105:0/3351708754 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc82f92220 con 0x55dc82f83de0 2024-09-16T02:05:13.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.020+0000 7f000ad9d640 1 --2- 172.21.15.105:0/3351708754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc82f8eb80 0x55dc82f91ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:05:13.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.021+0000 7f000a59c640 1 --2- 172.21.15.105:0/3351708754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc82f83de0 0x55dc82f96d30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7efffc00e660 tx=0x7efffc0046e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3e96d6ef48f708 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:13.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.021+0000 7efff1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:05:13.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.022+0000 7efff1ffb640 1 -- 172.21.15.105:0/3351708754 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efffc02d070 con 0x55dc82f83de0 2024-09-16T02:05:13.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.022+0000 7efff1ffb640 1 -- 172.21.15.105:0/3351708754 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7efffc004220 con 0x55dc82f83de0 2024-09-16T02:05:13.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.022+0000 7f001330a740 1 -- 172.21.15.105:0/3351708754 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00000a3e20 con 0x55dc82f83de0 2024-09-16T02:05:13.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.022+0000 7efff1ffb640 1 -- 172.21.15.105:0/3351708754 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efffc017040 con 0x55dc82f83de0 2024-09-16T02:05:13.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.022+0000 7f001330a740 10 client.15738 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:13.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.022+0000 7f001330a740 1 -- 172.21.15.105:0/3351708754 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dc83009a30 con 0x55dc82f83de0 2024-09-16T02:05:13.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.022+0000 7efff1ffb640 1 -- 172.21.15.105:0/3351708754 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00000a3e20 con 0x55dc82f83de0 2024-09-16T02:05:13.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.023+0000 7efff1ffb640 1 -- 172.21.15.105:0/3351708754 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7efffc026070 con 0x55dc82f83de0 2024-09-16T02:05:13.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.023+0000 7f001330a740 10 client.15738 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:13.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.023+0000 7f001330a740 10 client.15738 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T02:05:13.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.023+0000 7f001330a740 20 client.15738 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:13.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.023+0000 7f001330a740 1 --2- 172.21.15.105:0/3351708754 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55dc83009e20 0x55dc8300c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:13.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.023+0000 7f001330a740 4 client.15738 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T02:05:13.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.023+0000 7f001330a740 1 -- 172.21.15.105:0/3351708754 --> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dc82ee8d70 con 0x55dc83009e20 2024-09-16T02:05:13.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.025+0000 7f0009d9b640 1 --2- 172.21.15.105:0/3351708754 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55dc83009e20 0x55dc8300c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:13.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.030+0000 7f0009d9b640 1 --2- 172.21.15.105:0/3351708754 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55dc83009e20 0x55dc8300c210 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cbc92f415b598ef5 server_cookie=aff51f5ccbea6990 in_seq=0 out_seq=0 2024-09-16T02:05:13.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.030+0000 7efff1ffb640 10 client.15738 ms_handle_connect on v2:172.21.15.190:6838/1222573537 2024-09-16T02:05:13.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.031+0000 7efff1ffb640 1 -- 172.21.15.105:0/3351708754 <== mds.1 v2:172.21.15.190:6838/1222573537 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dc82ee8d70 con 0x55dc83009e20 2024-09-16T02:05:13.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.031+0000 7efff1ffb640 10 client.15738 handle_command_reply: tid=0 2024-09-16T02:05:13.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.031+0000 7f001330a740 1 client.15738 shutdown 2024-09-16T02:05:13.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.031+0000 7f001330a740 1 -- 172.21.15.105:0/3351708754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc82f83de0 msgr2=0x55dc82f96d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:13.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.031+0000 7f001330a740 1 --2- 172.21.15.105:0/3351708754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc82f83de0 0x55dc82f96d30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7efffc00e660 tx=0x7efffc0046e0 comp rx=0 tx=0).stop 2024-09-16T02:05:13.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.031+0000 7f001330a740 1 -- 172.21.15.105:0/3351708754 shutdown_connections 2024-09-16T02:05:13.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.031+0000 7f001330a740 1 --2- 172.21.15.105:0/3351708754 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55dc83009e20 0x55dc8300c210 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:13.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.031+0000 7f001330a740 1 --2- 172.21.15.105:0/3351708754 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc82f85350 0x55dc82f97270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:13.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.031+0000 7f001330a740 1 --2- 172.21.15.105:0/3351708754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc82f8eb80 0x55dc82f91ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:13.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.031+0000 7f001330a740 1 --2- 172.21.15.105:0/3351708754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc82f83de0 0x55dc82f96d30 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:13.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.031+0000 7f001330a740 1 -- 172.21.15.105:0/3351708754 >> 172.21.15.105:0/3351708754 conn(0x55dc82f44310 msgr2=0x55dc82f764e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:13.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.031+0000 7efff1ffb640 0 client.15738 ms_handle_reset on v2:172.21.15.190:6838/1222573537 2024-09-16T02:05:13.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.031+0000 7f001330a740 1 -- 172.21.15.105:0/3351708754 shutdown_connections 2024-09-16T02:05:13.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.031+0000 7f001330a740 1 -- 172.21.15.105:0/3351708754 wait complete. 2024-09-16T02:05:13.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.032+0000 7f001330a740 20 client.15738 trim_cache size 0 max 16384 2024-09-16T02:05:13.034 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-09-16T02:05:13.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.033+0000 7f0010f31640 1 -- 172.21.15.105:0/2171118917 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7efff40a32c0 msgr2=0x7efff40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:13.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.033+0000 7f0010f31640 1 --2- 172.21.15.105:0/2171118917 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7efff40a32c0 0x7efff40a5780 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0004004af0 tx=0x7f00040098f0 comp rx=0 tx=0).stop 2024-09-16T02:05:13.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.033+0000 7f0010f31640 1 -- 172.21.15.105:0/2171118917 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f000c102790 msgr2=0x7f000c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:13.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.033+0000 7f0010f31640 1 --2- 172.21.15.105:0/2171118917 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f000c102790 0x7f000c074a30 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f000000b4a0 tx=0x7f000000b960 comp rx=0 tx=0).stop 2024-09-16T02:05:13.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.034+0000 7f0010f31640 1 -- 172.21.15.105:0/2171118917 shutdown_connections 2024-09-16T02:05:13.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.034+0000 7f0010f31640 1 --2- 172.21.15.105:0/2171118917 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7efff40a32c0 0x7efff40a5780 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:13.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.034+0000 7f0010f31640 1 --2- 172.21.15.105:0/2171118917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f000c106f10 0x7f000c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:13.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.034+0000 7f0010f31640 1 --2- 172.21.15.105:0/2171118917 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f000c102790 0x7f000c074a30 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:13.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.034+0000 7f0010f31640 1 --2- 172.21.15.105:0/2171118917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f000c101de0 0x7f000c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:13.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.034+0000 7f0010f31640 1 -- 172.21.15.105:0/2171118917 >> 172.21.15.105:0/2171118917 conn(0x7f000c0fd8c0 msgr2=0x7f000c0fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:13.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.034+0000 7f0010f31640 1 -- 172.21.15.105:0/2171118917 shutdown_connections 2024-09-16T02:05:13.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:13.034+0000 7f0010f31640 1 -- 172.21.15.105:0/2171118917 wait complete. 2024-09-16T02:05:13.794 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T02:05:14.209 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:14.660 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:14 smithi105 ceph-mon[31197]: pgmap v420: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 366 MiB/s wr, 102 op/s 2024-09-16T02:05:14.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.818+0000 7f915ebbf640 1 -- 172.21.15.105:0/1370981702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9158100de0 msgr2=0x7f9158104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:14.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.818+0000 7f915ebbf640 1 --2- 172.21.15.105:0/1370981702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9158100de0 0x7f9158104e30 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f914400aa10 tx=0x7f914402b760 comp rx=0 tx=0).stop 2024-09-16T02:05:14.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.818+0000 7f915ebbf640 1 -- 172.21.15.105:0/1370981702 shutdown_connections 2024-09-16T02:05:14.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.818+0000 7f915ebbf640 1 --2- 172.21.15.105:0/1370981702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9158105560 0x7f915810d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.818+0000 7f915ebbf640 1 --2- 172.21.15.105:0/1370981702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9158100de0 0x7f9158104e30 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.818+0000 7f915ebbf640 1 --2- 172.21.15.105:0/1370981702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9158100430 0x7f9158100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.818+0000 7f915ebbf640 1 -- 172.21.15.105:0/1370981702 >> 172.21.15.105:0/1370981702 conn(0x7f9158077230 msgr2=0x7f91580fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:14.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.818+0000 7f915ebbf640 1 -- 172.21.15.105:0/1370981702 shutdown_connections 2024-09-16T02:05:14.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.819+0000 7f915ebbf640 1 -- 172.21.15.105:0/1370981702 wait complete. 2024-09-16T02:05:14.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.820+0000 7f915ebbf640 1 Processor -- start 2024-09-16T02:05:14.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.820+0000 7f915ebbf640 1 -- start start 2024-09-16T02:05:14.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.820+0000 7f915ebbf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9158100430 0x7f915819d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:14.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.820+0000 7f915ebbf640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9158100de0 0x7f91581961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:14.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.820+0000 7f915ebbf640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9158105560 0x7f91581966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:14.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.821+0000 7f915ebbf640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f915810fcd0 con 0x7f9158100430 2024-09-16T02:05:14.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.821+0000 7f915ebbf640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f915810fb50 con 0x7f9158100de0 2024-09-16T02:05:14.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.821+0000 7f915ebbf640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f915810fe50 con 0x7f9158105560 2024-09-16T02:05:14.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.821+0000 7f915d15c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9158105560 0x7f91581966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:14.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.821+0000 7f915d15c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9158105560 0x7f91581966e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.105:57614/0 (socket says 172.21.15.105:57614) 2024-09-16T02:05:14.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.821+0000 7f915d15c640 1 -- 172.21.15.105:0/230106756 learned_addr learned my addr 172.21.15.105:0/230106756 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:14.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.821+0000 7f915c95b640 1 --2- 172.21.15.105:0/230106756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9158100430 0x7f915819d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:14.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.822+0000 7f915d15c640 1 -- 172.21.15.105:0/230106756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9158100de0 msgr2=0x7f91581961a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T02:05:14.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.822+0000 7f915d15c640 1 --2- 172.21.15.105:0/230106756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9158100de0 0x7f91581961a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.822+0000 7f915d15c640 1 -- 172.21.15.105:0/230106756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9158100430 msgr2=0x7f915819d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:14.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.822+0000 7f915d15c640 1 --2- 172.21.15.105:0/230106756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9158100430 0x7f915819d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.822+0000 7f915d15c640 1 -- 172.21.15.105:0/230106756 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9158196e60 con 0x7f9158105560 2024-09-16T02:05:14.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.822+0000 7f915d15c640 1 --2- 172.21.15.105:0/230106756 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9158105560 0x7f91581966e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f915000b990 tx=0x7f915000be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:14.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.823+0000 7f9155ffb640 1 -- 172.21.15.105:0/230106756 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9150019070 con 0x7f9158105560 2024-09-16T02:05:14.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.823+0000 7f915ebbf640 1 -- 172.21.15.105:0/230106756 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f915819ac40 con 0x7f9158105560 2024-09-16T02:05:14.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.823+0000 7f915ebbf640 1 -- 172.21.15.105:0/230106756 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f915819b120 con 0x7f9158105560 2024-09-16T02:05:14.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.823+0000 7f9155ffb640 1 -- 172.21.15.105:0/230106756 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f915000cba0 con 0x7f9158105560 2024-09-16T02:05:14.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.824+0000 7f9155ffb640 1 -- 172.21.15.105:0/230106756 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91500075d0 con 0x7f9158105560 2024-09-16T02:05:14.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.828+0000 7f9155ffb640 1 -- 172.21.15.105:0/230106756 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f915001c070 con 0x7f9158105560 2024-09-16T02:05:14.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.829+0000 7f9160f98740 1 Processor -- start 2024-09-16T02:05:14.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.829+0000 7f9155ffb640 1 --2- 172.21.15.105:0/230106756 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f91480a3390 0x7f91480a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:14.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.829+0000 7f9160f98740 1 -- start start 2024-09-16T02:05:14.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.830+0000 7f9155ffb640 1 -- 172.21.15.105:0/230106756 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f91500ced70 con 0x7f9158105560 2024-09-16T02:05:14.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.830+0000 7f9160f98740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd24eceb40 0x55fd24ecd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:14.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.830+0000 7f9160f98740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd24ecd5d0 0x55fd24ecda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:14.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.830+0000 7f9160f98740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd24ed8370 0x55fd24eda760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:14.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.830+0000 7f9157fff640 1 --2- 172.21.15.105:0/230106756 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f91480a3390 0x7f91480a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:14.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.830+0000 7f9160f98740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55fd24e332f0 con 0x55fd24ecd5d0 2024-09-16T02:05:14.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.830+0000 7f9160f98740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fd24ddacf0 con 0x55fd24eceb40 2024-09-16T02:05:14.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.830+0000 7f9160f98740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55fd24e2f2b0 con 0x55fd24ed8370 2024-09-16T02:05:14.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.830+0000 7f915d15c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd24ed8370 0x55fd24eda760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:14.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.830+0000 7f9157fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd24ecd5d0 0x55fd24ecda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:14.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.830+0000 7f9157fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd24ecd5d0 0x55fd24ecda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55384/0 (socket says 172.21.15.105:55384) 2024-09-16T02:05:14.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.830+0000 7f9157fff640 1 -- 172.21.15.105:0/915950159 learned_addr learned my addr 172.21.15.105:0/915950159 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:14.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.830+0000 7f915c95b640 1 --2- 172.21.15.105:0/915950159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd24eceb40 0x55fd24ecd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:14.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.831+0000 7f915d15c640 1 -- 172.21.15.105:0/915950159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd24eceb40 msgr2=0x55fd24ecd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:14.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.831+0000 7f915d15c640 1 --2- 172.21.15.105:0/915950159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd24eceb40 0x55fd24ecd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.831+0000 7f915d15c640 1 -- 172.21.15.105:0/915950159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd24ecd5d0 msgr2=0x55fd24ecda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:14.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.831+0000 7f915d15c640 1 --2- 172.21.15.105:0/915950159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd24ecd5d0 0x55fd24ecda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.831+0000 7f915d15c640 1 -- 172.21.15.105:0/915950159 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd24dbf3a0 con 0x55fd24ed8370 2024-09-16T02:05:14.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.831+0000 7f9157fff640 1 --2- 172.21.15.105:0/230106756 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f91480a3390 0x7f91480a5850 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9144030e40 tx=0x7f9144030e70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:14.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.832+0000 7f915d15c640 1 --2- 172.21.15.105:0/915950159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd24ed8370 0x55fd24eda760 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f91500cc810 tx=0x7f915009ae00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3308bd16b94a254c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:14.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.832+0000 7f9142ffd640 1 -- 172.21.15.105:0/915950159 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91500a3fa0 con 0x55fd24ed8370 2024-09-16T02:05:14.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.833+0000 7f9142ffd640 1 -- 172.21.15.105:0/915950159 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f915000cd50 con 0x55fd24ed8370 2024-09-16T02:05:14.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.834+0000 7f9142ffd640 1 -- 172.21.15.105:0/915950159 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91500bb0b0 con 0x55fd24ed8370 2024-09-16T02:05:14.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.834+0000 7f9160f98740 1 -- 172.21.15.105:0/915950159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd24ed8370 msgr2=0x55fd24eda760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:14.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.834+0000 7f9160f98740 1 --2- 172.21.15.105:0/915950159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd24ed8370 0x55fd24eda760 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f91500cc810 tx=0x7f915009ae00 comp rx=0 tx=0).stop 2024-09-16T02:05:14.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.834+0000 7f9160f98740 1 -- 172.21.15.105:0/915950159 shutdown_connections 2024-09-16T02:05:14.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.834+0000 7f9160f98740 1 --2- 172.21.15.105:0/915950159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd24ed8370 0x55fd24eda760 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.834+0000 7f9160f98740 1 --2- 172.21.15.105:0/915950159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd24ecd5d0 0x55fd24ecda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.834+0000 7f9160f98740 1 --2- 172.21.15.105:0/915950159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd24eceb40 0x55fd24ecd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.834+0000 7f9160f98740 1 -- 172.21.15.105:0/915950159 >> 172.21.15.105:0/915950159 conn(0x55fd24e96310 msgr2=0x55fd24ec9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:14.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.835+0000 7f9160f98740 1 -- 172.21.15.105:0/915950159 shutdown_connections 2024-09-16T02:05:14.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.835+0000 7f9160f98740 1 -- 172.21.15.105:0/915950159 wait complete. 2024-09-16T02:05:14.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.836+0000 7f9160f98740 1 -- start start 2024-09-16T02:05:14.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.836+0000 7f9160f98740 1 Processor -- start 2024-09-16T02:05:14.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.837+0000 7f9160f98740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd24ecd5d0 0x55fd24ed7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:14.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.837+0000 7f9160f98740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd24eceb40 0x55fd24ed4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:14.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.837+0000 7f9160f98740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd24ed8370 0x55fd24ed54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:14.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.837+0000 7f9160f98740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55fd24edb0c0 con 0x55fd24ed8370 2024-09-16T02:05:14.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.837+0000 7f9160f98740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fd24dee060 con 0x55fd24ecd5d0 2024-09-16T02:05:14.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.837+0000 7f9160f98740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55fd24edb240 con 0x55fd24eceb40 2024-09-16T02:05:14.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.837+0000 7f9160f98740 10 client.0 _ll_register_callbacks cb 0x55fd24b418c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:14.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.838+0000 7f915d15c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd24ed8370 0x55fd24ed54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:14.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.838+0000 7f915d15c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd24ed8370 0x55fd24ed54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55400/0 (socket says 172.21.15.105:55400) 2024-09-16T02:05:14.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.838+0000 7f915d15c640 1 -- 172.21.15.105:0/1062094461 learned_addr learned my addr 172.21.15.105:0/1062094461 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:14.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.838+0000 7f9157fff640 1 --2- 172.21.15.105:0/1062094461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd24eceb40 0x55fd24ed4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:14.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.838+0000 7f915d15c640 1 -- 172.21.15.105:0/1062094461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd24eceb40 msgr2=0x55fd24ed4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:14.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.838+0000 7f915d15c640 1 --2- 172.21.15.105:0/1062094461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd24eceb40 0x55fd24ed4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.838+0000 7f915d15c640 1 -- 172.21.15.105:0/1062094461 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd24ecd5d0 msgr2=0x55fd24ed7ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T02:05:14.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.838+0000 7f915d15c640 1 --2- 172.21.15.105:0/1062094461 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd24ecd5d0 0x55fd24ed7ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.838+0000 7f915d15c640 1 -- 172.21.15.105:0/1062094461 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd24ba43c0 con 0x55fd24ed8370 2024-09-16T02:05:14.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.838+0000 7f915d15c640 1 --2- 172.21.15.105:0/1062094461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd24ed8370 0x55fd24ed54f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9150097840 tx=0x7f915000c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6996cfbdb1e7b9d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:14.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.840+0000 7f9137fff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:05:14.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.841+0000 7f9137fff640 1 -- 172.21.15.105:0/1062094461 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91500b4060 con 0x55fd24ed8370 2024-09-16T02:05:14.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.841+0000 7f9160f98740 1 -- 172.21.15.105:0/1062094461 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f915000cd50 con 0x55fd24ed8370 2024-09-16T02:05:14.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.841+0000 7f9137fff640 1 -- 172.21.15.105:0/1062094461 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f91500b4200 con 0x55fd24ed8370 2024-09-16T02:05:14.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:14 smithi190 ceph-mon[34762]: pgmap v420: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 366 MiB/s wr, 102 op/s 2024-09-16T02:05:14.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.841+0000 7f9137fff640 1 -- 172.21.15.105:0/1062094461 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f91500a3c60 con 0x55fd24ed8370 2024-09-16T02:05:14.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.841+0000 7f9160f98740 10 client.15750 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:14.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.841+0000 7f9160f98740 1 -- 172.21.15.105:0/1062094461 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fd24ee6b00 con 0x55fd24ed8370 2024-09-16T02:05:14.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.843+0000 7f9137fff640 1 -- 172.21.15.105:0/1062094461 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f915000cd50 con 0x55fd24ed8370 2024-09-16T02:05:14.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.843+0000 7f9137fff640 1 -- 172.21.15.105:0/1062094461 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f915009f160 con 0x55fd24ed8370 2024-09-16T02:05:14.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.843+0000 7f9160f98740 10 client.15750 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:14.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.843+0000 7f9160f98740 10 client.15750 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:05:14.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.843+0000 7f9160f98740 20 client.15750 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:14.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.843+0000 7f9160f98740 1 --2- 172.21.15.105:0/1062094461 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55fd24ee8030 0x55fd24eea4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:14.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.843+0000 7f9160f98740 4 client.15750 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:05:14.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.843+0000 7f9160f98740 1 -- 172.21.15.105:0/1062094461 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd24e40860 con 0x55fd24ee8030 2024-09-16T02:05:14.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.844+0000 7f915c95b640 1 --2- 172.21.15.105:0/1062094461 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55fd24ee8030 0x55fd24eea4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:14.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.845+0000 7f915c95b640 1 --2- 172.21.15.105:0/1062094461 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55fd24ee8030 0x55fd24eea4d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e164453d7677f44 server_cookie=a78d05ed99dd1 in_seq=0 out_seq=0 2024-09-16T02:05:14.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.845+0000 7f9137fff640 10 client.15750 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:05:14.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.846+0000 7f9137fff640 1 -- 172.21.15.105:0/1062094461 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fd24e40860 con 0x55fd24ee8030 2024-09-16T02:05:14.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.846+0000 7f9137fff640 10 client.15750 handle_command_reply: tid=0 2024-09-16T02:05:14.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.846+0000 7f9160f98740 1 client.15750 shutdown 2024-09-16T02:05:14.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.847+0000 7f9160f98740 1 -- 172.21.15.105:0/1062094461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd24ed8370 msgr2=0x55fd24ed54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:14.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.847+0000 7f9160f98740 1 --2- 172.21.15.105:0/1062094461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd24ed8370 0x55fd24ed54f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9150097840 tx=0x7f915000c610 comp rx=0 tx=0).stop 2024-09-16T02:05:14.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.847+0000 7f9160f98740 1 -- 172.21.15.105:0/1062094461 shutdown_connections 2024-09-16T02:05:14.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.847+0000 7f9160f98740 1 --2- 172.21.15.105:0/1062094461 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55fd24ee8030 0x55fd24eea4d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.847+0000 7f9160f98740 1 --2- 172.21.15.105:0/1062094461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd24eceb40 0x55fd24ed4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.847+0000 7f9160f98740 1 --2- 172.21.15.105:0/1062094461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd24ed8370 0x55fd24ed54f0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.847+0000 7f9160f98740 1 --2- 172.21.15.105:0/1062094461 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd24ecd5d0 0x55fd24ed7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.847+0000 7f9160f98740 1 -- 172.21.15.105:0/1062094461 >> 172.21.15.105:0/1062094461 conn(0x55fd24e96310 msgr2=0x55fd24ecf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:14.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.847+0000 7f9137fff640 0 client.15750 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:05:14.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.847+0000 7f9160f98740 1 -- 172.21.15.105:0/1062094461 shutdown_connections 2024-09-16T02:05:14.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.847+0000 7f9160f98740 1 -- 172.21.15.105:0/1062094461 wait complete. 2024-09-16T02:05:14.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.847+0000 7f9160f98740 20 client.15750 trim_cache size 0 max 16384 2024-09-16T02:05:14.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.865+0000 7f9160f98740 1 Processor -- start 2024-09-16T02:05:14.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.865+0000 7f9160f98740 1 -- start start 2024-09-16T02:05:14.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.866+0000 7f9160f98740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd24ecd5d0 0x55fd24ec9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:14.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.866+0000 7f9160f98740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd24eceb40 0x55fd24ee4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:14.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.866+0000 7f9160f98740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd24ed8370 0x55fd24ed0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:14.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.866+0000 7f9160f98740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f913c00e6c0 con 0x55fd24eceb40 2024-09-16T02:05:14.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.866+0000 7f9160f98740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f913c010200 con 0x55fd24ed8370 2024-09-16T02:05:14.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.866+0000 7f9160f98740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f913c00ed90 con 0x55fd24ecd5d0 2024-09-16T02:05:14.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.866+0000 7f9157fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd24eceb40 0x55fd24ee4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:14.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.866+0000 7f9157fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd24eceb40 0x55fd24ee4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55412/0 (socket says 172.21.15.105:55412) 2024-09-16T02:05:14.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.866+0000 7f9157fff640 1 -- 172.21.15.105:0/2348931815 learned_addr learned my addr 172.21.15.105:0/2348931815 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:14.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.866+0000 7f915d15c640 1 --2- 172.21.15.105:0/2348931815 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd24ed8370 0x55fd24ed0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:14.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.867+0000 7f9157fff640 1 -- 172.21.15.105:0/2348931815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd24ecd5d0 msgr2=0x55fd24ec9e10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T02:05:14.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.867+0000 7f9157fff640 1 --2- 172.21.15.105:0/2348931815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd24ecd5d0 0x55fd24ec9e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.867+0000 7f9157fff640 1 -- 172.21.15.105:0/2348931815 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd24ed8370 msgr2=0x55fd24ed0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:14.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.867+0000 7f9157fff640 1 --2- 172.21.15.105:0/2348931815 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd24ed8370 0x55fd24ed0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.867+0000 7f9157fff640 1 -- 172.21.15.105:0/2348931815 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd24ecadf0 con 0x55fd24eceb40 2024-09-16T02:05:14.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.867+0000 7f9157fff640 1 --2- 172.21.15.105:0/2348931815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd24eceb40 0x55fd24ee4390 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f914400aa10 tx=0x7f9144041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc07b96d8e2c14a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:14.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.867+0000 7f91427fc640 1 -- 172.21.15.105:0/2348931815 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f914404e070 con 0x55fd24eceb40 2024-09-16T02:05:14.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.867+0000 7f91427fc640 1 -- 172.21.15.105:0/2348931815 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9144003ca0 con 0x55fd24eceb40 2024-09-16T02:05:14.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.867+0000 7f91427fc640 1 -- 172.21.15.105:0/2348931815 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9144036040 con 0x55fd24eceb40 2024-09-16T02:05:14.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.869+0000 7f9160f98740 1 -- 172.21.15.105:0/2348931815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd24eceb40 msgr2=0x55fd24ee4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:14.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.869+0000 7f9160f98740 1 --2- 172.21.15.105:0/2348931815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd24eceb40 0x55fd24ee4390 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f914400aa10 tx=0x7f9144041000 comp rx=0 tx=0).stop 2024-09-16T02:05:14.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.869+0000 7f9160f98740 1 -- 172.21.15.105:0/2348931815 shutdown_connections 2024-09-16T02:05:14.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.869+0000 7f9160f98740 1 --2- 172.21.15.105:0/2348931815 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd24ed8370 0x55fd24ed0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.869+0000 7f9160f98740 1 --2- 172.21.15.105:0/2348931815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd24eceb40 0x55fd24ee4390 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.869+0000 7f9160f98740 1 --2- 172.21.15.105:0/2348931815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd24ecd5d0 0x55fd24ec9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.869+0000 7f9160f98740 1 -- 172.21.15.105:0/2348931815 >> 172.21.15.105:0/2348931815 conn(0x55fd24e96310 msgr2=0x55fd24ec9700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:14.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.869+0000 7f9160f98740 1 -- 172.21.15.105:0/2348931815 shutdown_connections 2024-09-16T02:05:14.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.869+0000 7f9160f98740 1 -- 172.21.15.105:0/2348931815 wait complete. 2024-09-16T02:05:14.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.870+0000 7f9160f98740 1 -- start start 2024-09-16T02:05:14.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.870+0000 7f9160f98740 1 Processor -- start 2024-09-16T02:05:14.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.871+0000 7f9160f98740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd24ecd5d0 0x55fd24ed2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:14.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.871+0000 7f9160f98740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd24eceb40 0x55fd24ee0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:14.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.871+0000 7f9160f98740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd24ed8370 0x55fd24ee0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:14.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.871+0000 7f9160f98740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f913c00fb30 con 0x55fd24ed8370 2024-09-16T02:05:14.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.871+0000 7f9160f98740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f913c00f460 con 0x55fd24eceb40 2024-09-16T02:05:14.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.871+0000 7f9160f98740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f913c00c6a0 con 0x55fd24ecd5d0 2024-09-16T02:05:14.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.871+0000 7f9160f98740 10 client.0 _ll_register_callbacks cb 0x7f914400a890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:14.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.871+0000 7f915d15c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd24ed8370 0x55fd24ee0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:14.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.871+0000 7f915d15c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd24ed8370 0x55fd24ee0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55414/0 (socket says 172.21.15.105:55414) 2024-09-16T02:05:14.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.871+0000 7f915d15c640 1 -- 172.21.15.105:0/3383533670 learned_addr learned my addr 172.21.15.105:0/3383533670 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:14.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.871+0000 7f915d15c640 1 -- 172.21.15.105:0/3383533670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd24ecd5d0 msgr2=0x55fd24ed2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:14.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.871+0000 7f915d15c640 1 --2- 172.21.15.105:0/3383533670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd24ecd5d0 0x55fd24ed2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.871+0000 7f915d15c640 1 -- 172.21.15.105:0/3383533670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd24eceb40 msgr2=0x55fd24ee0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:14.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.871+0000 7f915d15c640 1 --2- 172.21.15.105:0/3383533670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd24eceb40 0x55fd24ee0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.871+0000 7f915d15c640 1 -- 172.21.15.105:0/3383533670 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd24ed1200 con 0x55fd24ed8370 2024-09-16T02:05:14.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.872+0000 7f915d15c640 1 --2- 172.21.15.105:0/3383533670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd24ed8370 0x55fd24ee0a40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f91500a4160 tx=0x7f91500a3b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e05a72f6a2fd1705 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:14.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.872+0000 7f9137fff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:05:14.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.872+0000 7f9137fff640 1 -- 172.21.15.105:0/3383533670 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915009f160 con 0x55fd24ed8370 2024-09-16T02:05:14.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.872+0000 7f9137fff640 1 -- 172.21.15.105:0/3383533670 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f91500b4060 con 0x55fd24ed8370 2024-09-16T02:05:14.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.872+0000 7f9160f98740 1 -- 172.21.15.105:0/3383533670 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9144003ca0 con 0x55fd24ed8370 2024-09-16T02:05:14.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.872+0000 7f9137fff640 1 -- 172.21.15.105:0/3383533670 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f915009f300 con 0x55fd24ed8370 2024-09-16T02:05:14.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.872+0000 7f9160f98740 10 client.15756 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:14.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.872+0000 7f9160f98740 1 -- 172.21.15.105:0/3383533670 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fd24f51f70 con 0x55fd24ed8370 2024-09-16T02:05:14.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.872+0000 7f9137fff640 1 -- 172.21.15.105:0/3383533670 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9144003ca0 con 0x55fd24ed8370 2024-09-16T02:05:14.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.872+0000 7f9137fff640 1 -- 172.21.15.105:0/3383533670 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f91500bb140 con 0x55fd24ed8370 2024-09-16T02:05:14.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.872+0000 7f9160f98740 10 client.15756 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:14.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.872+0000 7f9160f98740 10 client.15756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:05:14.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.872+0000 7f9160f98740 20 client.15756 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:14.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.872+0000 7f9160f98740 1 --2- 172.21.15.105:0/3383533670 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55fd24f524d0 0x55fd24f54950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:14.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.873+0000 7f9160f98740 4 client.15756 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T02:05:14.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.873+0000 7f9160f98740 1 -- 172.21.15.105:0/3383533670 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fd24e3ad70 con 0x55fd24f524d0 2024-09-16T02:05:14.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.873+0000 7f915c95b640 1 --2- 172.21.15.105:0/3383533670 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55fd24f524d0 0x55fd24f54950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:14.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.875+0000 7f9137fff640 10 client.15756 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:05:14.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.875+0000 7f915c95b640 1 --2- 172.21.15.105:0/3383533670 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55fd24f524d0 0x55fd24f54950 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb8067f8e8564d17 server_cookie=df1fe858d3222a16 in_seq=0 out_seq=0 2024-09-16T02:05:14.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.876+0000 7f9137fff640 1 -- 172.21.15.105:0/3383533670 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fd24e3ad70 con 0x55fd24f524d0 2024-09-16T02:05:14.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.876+0000 7f9137fff640 10 client.15756 handle_command_reply: tid=0 2024-09-16T02:05:14.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.876+0000 7f9160f98740 1 client.15756 shutdown 2024-09-16T02:05:14.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.876+0000 7f9160f98740 1 -- 172.21.15.105:0/3383533670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd24ed8370 msgr2=0x55fd24ee0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:14.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.876+0000 7f9160f98740 1 --2- 172.21.15.105:0/3383533670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd24ed8370 0x55fd24ee0a40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f91500a4160 tx=0x7f91500a3b90 comp rx=0 tx=0).stop 2024-09-16T02:05:14.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.876+0000 7f9160f98740 1 -- 172.21.15.105:0/3383533670 shutdown_connections 2024-09-16T02:05:14.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.876+0000 7f9160f98740 1 --2- 172.21.15.105:0/3383533670 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55fd24f524d0 0x55fd24f54950 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.876+0000 7f9160f98740 1 --2- 172.21.15.105:0/3383533670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd24ed8370 0x55fd24ee0a40 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.876+0000 7f9160f98740 1 --2- 172.21.15.105:0/3383533670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd24eceb40 0x55fd24ee0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.876+0000 7f9137fff640 0 client.15756 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:05:14.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.876+0000 7f9160f98740 1 --2- 172.21.15.105:0/3383533670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd24ecd5d0 0x55fd24ed2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.876+0000 7f9160f98740 1 -- 172.21.15.105:0/3383533670 >> 172.21.15.105:0/3383533670 conn(0x55fd24e96310 msgr2=0x55fd24ed7010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:14.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.877+0000 7f9160f98740 1 -- 172.21.15.105:0/3383533670 shutdown_connections 2024-09-16T02:05:14.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.877+0000 7f9160f98740 1 -- 172.21.15.105:0/3383533670 wait complete. 2024-09-16T02:05:14.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.877+0000 7f9160f98740 20 client.15756 trim_cache size 0 max 16384 2024-09-16T02:05:14.880 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-09-16T02:05:14.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.879+0000 7f915ebbf640 1 -- 172.21.15.105:0/230106756 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f91480a3390 msgr2=0x7f91480a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:14.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.879+0000 7f915ebbf640 1 --2- 172.21.15.105:0/230106756 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f91480a3390 0x7f91480a5850 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9144030e40 tx=0x7f9144030e70 comp rx=0 tx=0).stop 2024-09-16T02:05:14.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.879+0000 7f915ebbf640 1 -- 172.21.15.105:0/230106756 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9158105560 msgr2=0x7f91581966e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:14.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.879+0000 7f915ebbf640 1 --2- 172.21.15.105:0/230106756 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9158105560 0x7f91581966e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f915000b990 tx=0x7f915000be50 comp rx=0 tx=0).stop 2024-09-16T02:05:14.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.879+0000 7f915ebbf640 1 -- 172.21.15.105:0/230106756 shutdown_connections 2024-09-16T02:05:14.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.879+0000 7f915ebbf640 1 --2- 172.21.15.105:0/230106756 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f91480a3390 0x7f91480a5850 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.879+0000 7f915ebbf640 1 --2- 172.21.15.105:0/230106756 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9158105560 0x7f91581966e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.879+0000 7f915ebbf640 1 --2- 172.21.15.105:0/230106756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9158100de0 0x7f91581961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.879+0000 7f915ebbf640 1 --2- 172.21.15.105:0/230106756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9158100430 0x7f915819d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.879+0000 7f915ebbf640 1 -- 172.21.15.105:0/230106756 >> 172.21.15.105:0/230106756 conn(0x7f9158077230 msgr2=0x7f91581037a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:14.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.879+0000 7f915ebbf640 1 -- 172.21.15.105:0/230106756 shutdown_connections 2024-09-16T02:05:14.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:14.879+0000 7f915ebbf640 1 -- 172.21.15.105:0/230106756 wait complete. 2024-09-16T02:05:14.920 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:14 smithi137 ceph-mon[34099]: pgmap v420: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 366 MiB/s wr, 102 op/s 2024-09-16T02:05:16.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:16 smithi190 ceph-mon[34762]: pgmap v421: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 622 MiB/s wr, 166 op/s 2024-09-16T02:05:16.872 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T02:05:16.872 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T02:05:16.912 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:16 smithi105 ceph-mon[31197]: pgmap v421: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 622 MiB/s wr, 166 op/s 2024-09-16T02:05:16.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:16 smithi137 ceph-mon[34099]: pgmap v421: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 622 MiB/s wr, 166 op/s 2024-09-16T02:05:17.298 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:17.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.930+0000 7fb6cb4a2640 1 -- 172.21.15.105:0/1635463241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6c410ea80 msgr2=0x7fb6c41154f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:17.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.930+0000 7fb6cb4a2640 1 --2- 172.21.15.105:0/1635463241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6c410ea80 0x7fb6c41154f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc005fd0 tx=0x7fb6bc02b3e0 comp rx=0 tx=0).stop 2024-09-16T02:05:17.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.930+0000 7fb6cb4a2640 1 -- 172.21.15.105:0/1635463241 shutdown_connections 2024-09-16T02:05:17.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.930+0000 7fb6cb4a2640 1 --2- 172.21.15.105:0/1635463241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6c410ea80 0x7fb6c41154f0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:17.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.930+0000 7fb6cb4a2640 1 --2- 172.21.15.105:0/1635463241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6c4109e00 0x7fb6c410e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:17.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.930+0000 7fb6cb4a2640 1 --2- 172.21.15.105:0/1635463241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6c4109450 0x7fb6c4109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:17.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.930+0000 7fb6cb4a2640 1 -- 172.21.15.105:0/1635463241 >> 172.21.15.105:0/1635463241 conn(0x7fb6c406e2d0 msgr2=0x7fb6c406e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:17.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.931+0000 7fb6cb4a2640 1 -- 172.21.15.105:0/1635463241 shutdown_connections 2024-09-16T02:05:17.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.931+0000 7fb6cb4a2640 1 -- 172.21.15.105:0/1635463241 wait complete. 2024-09-16T02:05:17.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.932+0000 7fb6cb4a2640 1 Processor -- start 2024-09-16T02:05:17.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.932+0000 7fb6cb4a2640 1 -- start start 2024-09-16T02:05:17.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.933+0000 7fb6cb4a2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6c4109450 0x7fb6c41a71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:17.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.933+0000 7fb6cb4a2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6c4109e00 0x7fb6c41a7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:17.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.933+0000 7fb6cb4a2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6c410ea80 0x7fb6c41aba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:17.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.933+0000 7fb6cb4a2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c4118440 con 0x7fb6c410ea80 2024-09-16T02:05:17.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.933+0000 7fb6cb4a2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c41182c0 con 0x7fb6c4109450 2024-09-16T02:05:17.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.933+0000 7fb6cb4a2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c41185c0 con 0x7fb6c4109e00 2024-09-16T02:05:17.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.933+0000 7fb6c9a3f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6c410ea80 0x7fb6c41aba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:17.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.933+0000 7fb6c9a3f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6c410ea80 0x7fb6c41aba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55428/0 (socket says 172.21.15.105:55428) 2024-09-16T02:05:17.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.933+0000 7fb6c9a3f640 1 -- 172.21.15.105:0/2573589532 learned_addr learned my addr 172.21.15.105:0/2573589532 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:17.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.933+0000 7fb6c923e640 1 --2- 172.21.15.105:0/2573589532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6c4109450 0x7fb6c41a71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:17.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.933+0000 7fb6c9a3f640 1 -- 172.21.15.105:0/2573589532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6c4109e00 msgr2=0x7fb6c41a7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:17.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.933+0000 7fb6c9a3f640 1 --2- 172.21.15.105:0/2573589532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6c4109e00 0x7fb6c41a7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:17.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.933+0000 7fb6c9a3f640 1 -- 172.21.15.105:0/2573589532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6c4109450 msgr2=0x7fb6c41a71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:17.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.933+0000 7fb6c9a3f640 1 --2- 172.21.15.105:0/2573589532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6c4109450 0x7fb6c41a71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:17.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.933+0000 7fb6c9a3f640 1 -- 172.21.15.105:0/2573589532 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c41ac170 con 0x7fb6c410ea80 2024-09-16T02:05:17.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.935+0000 7fb6c9a3f640 1 --2- 172.21.15.105:0/2573589532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6c410ea80 0x7fb6c41aba90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc005cc0 tx=0x7fb6bc033f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:17.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.935+0000 7fb6ba7fc640 1 -- 172.21.15.105:0/2573589532 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc037040 con 0x7fb6c410ea80 2024-09-16T02:05:17.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.935+0000 7fb6ba7fc640 1 -- 172.21.15.105:0/2573589532 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6bc007c50 con 0x7fb6c410ea80 2024-09-16T02:05:17.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.935+0000 7fb6ba7fc640 1 -- 172.21.15.105:0/2573589532 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc0042b0 con 0x7fb6c410ea80 2024-09-16T02:05:17.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.935+0000 7fb6cb4a2640 1 -- 172.21.15.105:0/2573589532 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6c41ac400 con 0x7fb6c410ea80 2024-09-16T02:05:17.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.935+0000 7fb6cb4a2640 1 -- 172.21.15.105:0/2573589532 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6c41b3d40 con 0x7fb6c410ea80 2024-09-16T02:05:17.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.939+0000 7fb6ba7fc640 1 -- 172.21.15.105:0/2573589532 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fb6bc004450 con 0x7fb6c410ea80 2024-09-16T02:05:17.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.940+0000 7fb6cd87b740 1 Processor -- start 2024-09-16T02:05:17.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.940+0000 7fb6cd87b740 1 -- start start 2024-09-16T02:05:17.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.941+0000 7fb6ba7fc640 1 --2- 172.21.15.105:0/2573589532 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb6b00a32c0 0x7fb6b00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:17.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.941+0000 7fb6cd87b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fd7f3b60 0x5555fd7f3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:17.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.941+0000 7fb6c8a3d640 1 --2- 172.21.15.105:0/2573589532 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb6b00a32c0 0x7fb6b00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:17.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.941+0000 7fb6cd87b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fd7f4480 0x5555fd7fe800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:17.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.941+0000 7fb6cd87b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555fd7fed40 0x5555fd801130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:17.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.941+0000 7fb6ba7fc640 1 -- 172.21.15.105:0/2573589532 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fb6bc0f0120 con 0x7fb6c410ea80 2024-09-16T02:05:17.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.941+0000 7fb6cd87b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5555fd759280 con 0x5555fd7fed40 2024-09-16T02:05:17.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.941+0000 7fb6cd87b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5555fd700b10 con 0x5555fd7f3b60 2024-09-16T02:05:17.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.941+0000 7fb6cd87b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5555fd755240 con 0x5555fd7f4480 2024-09-16T02:05:17.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.941+0000 7fb6c9a3f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555fd7fed40 0x5555fd801130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:17.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.941+0000 7fb6c9a3f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555fd7fed40 0x5555fd801130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55442/0 (socket says 172.21.15.105:55442) 2024-09-16T02:05:17.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.941+0000 7fb6c9a3f640 1 -- 172.21.15.105:0/571924261 learned_addr learned my addr 172.21.15.105:0/571924261 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:17.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.942+0000 7fb6c923e640 1 --2- 172.21.15.105:0/571924261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fd7f3b60 0x5555fd7f3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:17.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.942+0000 7fb6c8a3d640 1 --2- 172.21.15.105:0/571924261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fd7f4480 0x5555fd7fe800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:17.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.942+0000 7fb6c8a3d640 1 --2- 172.21.15.105:0/2573589532 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb6b00a32c0 0x7fb6b00a5780 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb6c40735c0 tx=0x7fb6c0009b90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:17.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.942+0000 7fb6c9a3f640 1 -- 172.21.15.105:0/571924261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fd7f4480 msgr2=0x5555fd7fe800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:17.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.942+0000 7fb6c9a3f640 1 --2- 172.21.15.105:0/571924261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fd7f4480 0x5555fd7fe800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:17.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.942+0000 7fb6c9a3f640 1 -- 172.21.15.105:0/571924261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fd7f3b60 msgr2=0x5555fd7f3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:17.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.942+0000 7fb6c9a3f640 1 --2- 172.21.15.105:0/571924261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fd7f3b60 0x5555fd7f3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:17.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.942+0000 7fb6c9a3f640 1 -- 172.21.15.105:0/571924261 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555fd70d1f0 con 0x5555fd7fed40 2024-09-16T02:05:17.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.942+0000 7fb6c9a3f640 1 --2- 172.21.15.105:0/571924261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555fd7fed40 0x5555fd801130 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc0bc380 tx=0x7fb6bc00a580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6aa7d558b6dc3c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:17.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.942+0000 7fb6af7fe640 1 -- 172.21.15.105:0/571924261 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc0bcd80 con 0x5555fd7fed40 2024-09-16T02:05:17.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.942+0000 7fb6c8a3d640 1 --2- 172.21.15.105:0/571924261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fd7f4480 0x5555fd7fe800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:05:17.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.942+0000 7fb6af7fe640 1 -- 172.21.15.105:0/571924261 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6bc0bc680 con 0x5555fd7fed40 2024-09-16T02:05:17.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.942+0000 7fb6af7fe640 1 -- 172.21.15.105:0/571924261 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc004740 con 0x5555fd7fed40 2024-09-16T02:05:17.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.943+0000 7fb6cd87b740 1 -- 172.21.15.105:0/571924261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555fd7fed40 msgr2=0x5555fd801130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:17.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.943+0000 7fb6cd87b740 1 --2- 172.21.15.105:0/571924261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555fd7fed40 0x5555fd801130 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc0bc380 tx=0x7fb6bc00a580 comp rx=0 tx=0).stop 2024-09-16T02:05:17.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.944+0000 7fb6cd87b740 1 -- 172.21.15.105:0/571924261 shutdown_connections 2024-09-16T02:05:17.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.944+0000 7fb6cd87b740 1 --2- 172.21.15.105:0/571924261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fd7f4480 0x5555fd7fe800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:17.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.944+0000 7fb6cd87b740 1 --2- 172.21.15.105:0/571924261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555fd7fed40 0x5555fd801130 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:17.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.944+0000 7fb6cd87b740 1 --2- 172.21.15.105:0/571924261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fd7f3b60 0x5555fd7f3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:17.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.944+0000 7fb6cd87b740 1 -- 172.21.15.105:0/571924261 >> 172.21.15.105:0/571924261 conn(0x5555fd7f00f0 msgr2=0x5555fd7f04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:17.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.944+0000 7fb6cd87b740 1 -- 172.21.15.105:0/571924261 shutdown_connections 2024-09-16T02:05:17.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.944+0000 7fb6cd87b740 1 -- 172.21.15.105:0/571924261 wait complete. 2024-09-16T02:05:17.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.944+0000 7fb6cd87b740 1 -- start start 2024-09-16T02:05:17.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.945+0000 7fb6cd87b740 1 Processor -- start 2024-09-16T02:05:17.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.945+0000 7fb6cd87b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fd7f3b60 0x5555fd7fbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:17.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.945+0000 7fb6cd87b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fd7f4480 0x5555fd7fc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:17.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.945+0000 7fb6cd87b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555fd7fed40 0x5555fd7fc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:17.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.945+0000 7fb6cd87b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5555fd8016d0 con 0x5555fd7fed40 2024-09-16T02:05:17.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.945+0000 7fb6cd87b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5555fd713f10 con 0x5555fd7f4480 2024-09-16T02:05:17.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.945+0000 7fb6cd87b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5555fd801850 con 0x5555fd7f3b60 2024-09-16T02:05:17.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.945+0000 7fb6c9a3f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555fd7fed40 0x5555fd7fc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:17.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.946+0000 7fb6c9a3f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555fd7fed40 0x5555fd7fc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55448/0 (socket says 172.21.15.105:55448) 2024-09-16T02:05:17.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.946+0000 7fb6cd87b740 10 client.0 _ll_register_callbacks cb 0x5555fd6dca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:17.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.946+0000 7fb6c9a3f640 1 -- 172.21.15.105:0/3610332523 learned_addr learned my addr 172.21.15.105:0/3610332523 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:17.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.946+0000 7fb6c923e640 1 --2- 172.21.15.105:0/3610332523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fd7f3b60 0x5555fd7fbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:17.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.946+0000 7fb6c9a3f640 1 -- 172.21.15.105:0/3610332523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fd7f3b60 msgr2=0x5555fd7fbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:17.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.946+0000 7fb6c9a3f640 1 --2- 172.21.15.105:0/3610332523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fd7f3b60 0x5555fd7fbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:17.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.946+0000 7fb6c9a3f640 1 -- 172.21.15.105:0/3610332523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fd7f4480 msgr2=0x5555fd7fc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:17.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.946+0000 7fb6c9a3f640 1 --2- 172.21.15.105:0/3610332523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fd7f4480 0x5555fd7fc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:17.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.946+0000 7fb6c9a3f640 1 -- 172.21.15.105:0/3610332523 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555fd4ca6a0 con 0x5555fd7fed40 2024-09-16T02:05:17.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.946+0000 7fb6c923e640 1 --2- 172.21.15.105:0/3610332523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fd7f3b60 0x5555fd7fbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:05:17.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.946+0000 7fb6c9a3f640 1 --2- 172.21.15.105:0/3610332523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555fd7fed40 0x5555fd7fc700 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc007490 tx=0x7fb6bc00aee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=608388cf350c799 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:17.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.946+0000 7fb6acff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:05:17.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.947+0000 7fb6acff9640 1 -- 172.21.15.105:0/3610332523 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc0c52f0 con 0x5555fd7fed40 2024-09-16T02:05:17.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.947+0000 7fb6acff9640 1 -- 172.21.15.105:0/3610332523 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fb6bc0b8cf0 con 0x5555fd7fed40 2024-09-16T02:05:17.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.947+0000 7fb6cd87b740 1 -- 172.21.15.105:0/3610332523 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6bc0bc680 con 0x5555fd7fed40 2024-09-16T02:05:17.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.947+0000 7fb6acff9640 1 -- 172.21.15.105:0/3610332523 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb6bc0ba720 con 0x5555fd7fed40 2024-09-16T02:05:17.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.947+0000 7fb6cd87b740 10 client.15768 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:17.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.947+0000 7fb6cd87b740 1 -- 172.21.15.105:0/3610332523 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5555fd8016d0 con 0x5555fd7fed40 2024-09-16T02:05:17.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.947+0000 7fb6acff9640 1 -- 172.21.15.105:0/3610332523 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6bc0bc680 con 0x5555fd7fed40 2024-09-16T02:05:17.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.948+0000 7fb6acff9640 1 -- 172.21.15.105:0/3610332523 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb6bc0bbad0 con 0x5555fd7fed40 2024-09-16T02:05:17.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.948+0000 7fb6cd87b740 10 client.15768 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:17.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.948+0000 7fb6cd87b740 10 client.15768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:05:17.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.948+0000 7fb6cd87b740 20 client.15768 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:17.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.948+0000 7fb6cd87b740 1 --2- 172.21.15.105:0/3610332523 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x5555fd80e4c0 0x5555fd8108b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:17.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.948+0000 7fb6cd87b740 4 client.15768 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:05:17.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.948+0000 7fb6cd87b740 1 -- 172.21.15.105:0/3610332523 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555fd7667f0 con 0x5555fd80e4c0 2024-09-16T02:05:17.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.949+0000 7fb6c923e640 1 --2- 172.21.15.105:0/3610332523 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x5555fd80e4c0 0x5555fd8108b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:17.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.952+0000 7fb6c923e640 1 --2- 172.21.15.105:0/3610332523 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x5555fd80e4c0 0x5555fd8108b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3835a1d4b1518387 server_cookie=15f53ae056043e92 in_seq=0 out_seq=0 2024-09-16T02:05:17.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.952+0000 7fb6acff9640 10 client.15768 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:05:17.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.953+0000 7fb6acff9640 1 -- 172.21.15.105:0/3610332523 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5555fd7667f0 con 0x5555fd80e4c0 2024-09-16T02:05:17.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.953+0000 7fb6acff9640 10 client.15768 handle_command_reply: tid=0 2024-09-16T02:05:17.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.953+0000 7fb6cd87b740 1 client.15768 shutdown 2024-09-16T02:05:17.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.954+0000 7fb6cd87b740 1 -- 172.21.15.105:0/3610332523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555fd7fed40 msgr2=0x5555fd7fc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:17.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.954+0000 7fb6cd87b740 1 --2- 172.21.15.105:0/3610332523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555fd7fed40 0x5555fd7fc700 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc007490 tx=0x7fb6bc00aee0 comp rx=0 tx=0).stop 2024-09-16T02:05:17.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.954+0000 7fb6cd87b740 1 -- 172.21.15.105:0/3610332523 shutdown_connections 2024-09-16T02:05:17.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.954+0000 7fb6cd87b740 1 --2- 172.21.15.105:0/3610332523 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x5555fd80e4c0 0x5555fd8108b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:17.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.954+0000 7fb6acff9640 0 client.15768 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:05:17.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.954+0000 7fb6cd87b740 1 --2- 172.21.15.105:0/3610332523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555fd7fed40 0x5555fd7fc700 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:17.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.954+0000 7fb6cd87b740 1 --2- 172.21.15.105:0/3610332523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fd7f4480 0x5555fd7fc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:17.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.954+0000 7fb6cd87b740 1 --2- 172.21.15.105:0/3610332523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fd7f3b60 0x5555fd7fbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:17.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.954+0000 7fb6cd87b740 1 -- 172.21.15.105:0/3610332523 >> 172.21.15.105:0/3610332523 conn(0x5555fd7f00f0 msgr2=0x5555fd7ef270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:17.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.954+0000 7fb6cd87b740 1 -- 172.21.15.105:0/3610332523 shutdown_connections 2024-09-16T02:05:17.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.954+0000 7fb6cd87b740 1 -- 172.21.15.105:0/3610332523 wait complete. 2024-09-16T02:05:17.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.954+0000 7fb6cd87b740 20 client.15768 trim_cache size 0 max 16384 2024-09-16T02:05:17.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.985+0000 7fb6cd87b740 1 Processor -- start 2024-09-16T02:05:17.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.985+0000 7fb6cd87b740 1 -- start start 2024-09-16T02:05:17.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.986+0000 7fb6cd87b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fd7f3b60 0x5555fd7f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:17.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.986+0000 7fb6cd87b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fd7f4480 0x5555fd816070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:17.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.986+0000 7fb6cd87b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555fd7fed40 0x5555fd8165b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:17.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.986+0000 7fb6cd87b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c41bc530 con 0x5555fd7fed40 2024-09-16T02:05:17.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.986+0000 7fb6cd87b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c41be070 con 0x5555fd7f3b60 2024-09-16T02:05:17.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.986+0000 7fb6cd87b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c41bcc00 con 0x5555fd7f4480 2024-09-16T02:05:17.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.987+0000 7fb6c8a3d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fd7f4480 0x5555fd816070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:17.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.987+0000 7fb6c9a3f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555fd7fed40 0x5555fd8165b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:17.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.987+0000 7fb6c9a3f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555fd7fed40 0x5555fd8165b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55450/0 (socket says 172.21.15.105:55450) 2024-09-16T02:05:17.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.987+0000 7fb6c9a3f640 1 -- 172.21.15.105:0/1478856521 learned_addr learned my addr 172.21.15.105:0/1478856521 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:17.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.987+0000 7fb6c9a3f640 1 -- 172.21.15.105:0/1478856521 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fd7f4480 msgr2=0x5555fd816070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:17.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.987+0000 7fb6c9a3f640 1 --2- 172.21.15.105:0/1478856521 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fd7f4480 0x5555fd816070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:17.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.987+0000 7fb6c9a3f640 1 -- 172.21.15.105:0/1478856521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fd7f3b60 msgr2=0x5555fd7f5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:17.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.987+0000 7fb6c9a3f640 1 --2- 172.21.15.105:0/1478856521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fd7f3b60 0x5555fd7f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:17.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.987+0000 7fb6c9a3f640 1 -- 172.21.15.105:0/1478856521 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555fd7fa1d0 con 0x5555fd7fed40 2024-09-16T02:05:17.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.987+0000 7fb6c9a3f640 1 --2- 172.21.15.105:0/1478856521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555fd7fed40 0x5555fd8165b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc005e30 tx=0x7fb6bc0ba9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8537cc988fd581c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:17.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.988+0000 7fb6aeffd640 1 -- 172.21.15.105:0/1478856521 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc0b8920 con 0x5555fd7fed40 2024-09-16T02:05:17.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.989+0000 7fb6aeffd640 1 -- 172.21.15.105:0/1478856521 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6bc0da070 con 0x5555fd7fed40 2024-09-16T02:05:17.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.989+0000 7fb6aeffd640 1 -- 172.21.15.105:0/1478856521 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc0c52f0 con 0x5555fd7fed40 2024-09-16T02:05:17.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.990+0000 7fb6cd87b740 1 -- 172.21.15.105:0/1478856521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555fd7fed40 msgr2=0x5555fd8165b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:17.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.990+0000 7fb6cd87b740 1 --2- 172.21.15.105:0/1478856521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555fd7fed40 0x5555fd8165b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc005e30 tx=0x7fb6bc0ba9f0 comp rx=0 tx=0).stop 2024-09-16T02:05:17.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.990+0000 7fb6cd87b740 1 -- 172.21.15.105:0/1478856521 shutdown_connections 2024-09-16T02:05:17.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.990+0000 7fb6cd87b740 1 --2- 172.21.15.105:0/1478856521 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fd7f4480 0x5555fd816070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:17.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.990+0000 7fb6cd87b740 1 --2- 172.21.15.105:0/1478856521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555fd7fed40 0x5555fd8165b0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:17.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.990+0000 7fb6cd87b740 1 --2- 172.21.15.105:0/1478856521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fd7f3b60 0x5555fd7f5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:17.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.990+0000 7fb6cd87b740 1 -- 172.21.15.105:0/1478856521 >> 172.21.15.105:0/1478856521 conn(0x5555fd7f00f0 msgr2=0x5555fd7f55b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:17.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.990+0000 7fb6cd87b740 1 -- 172.21.15.105:0/1478856521 shutdown_connections 2024-09-16T02:05:17.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.990+0000 7fb6cd87b740 1 -- 172.21.15.105:0/1478856521 wait complete. 2024-09-16T02:05:17.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.990+0000 7fb6cd87b740 1 -- start start 2024-09-16T02:05:17.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.991+0000 7fb6cd87b740 1 Processor -- start 2024-09-16T02:05:17.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.992+0000 7fb6cd87b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fd7f3b60 0x5555fd8029e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:17.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.992+0000 7fb6cd87b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555fd7f4480 0x5555fd802f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:17.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.992+0000 7fb6cd87b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fd7fed40 0x5555fd803480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:17.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.992+0000 7fb6cd87b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c41bd9a0 con 0x5555fd7f4480 2024-09-16T02:05:17.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.992+0000 7fb6cd87b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c41bd2d0 con 0x5555fd7fed40 2024-09-16T02:05:17.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.992+0000 7fb6cd87b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c41ba510 con 0x5555fd7f3b60 2024-09-16T02:05:17.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.992+0000 7fb6cd87b740 10 client.0 _ll_register_callbacks cb 0x7fb6b400b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:17.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.992+0000 7fb6c8a3d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555fd7f4480 0x5555fd802f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:17.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.993+0000 7fb6c8a3d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555fd7f4480 0x5555fd802f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55464/0 (socket says 172.21.15.105:55464) 2024-09-16T02:05:17.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.993+0000 7fb6c8a3d640 1 -- 172.21.15.105:0/3627903861 learned_addr learned my addr 172.21.15.105:0/3627903861 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:17.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.993+0000 7fb6c9a3f640 1 --2- 172.21.15.105:0/3627903861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fd7fed40 0x5555fd803480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:17.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.993+0000 7fb6c923e640 1 --2- 172.21.15.105:0/3627903861 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fd7f3b60 0x5555fd8029e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:17.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.993+0000 7fb6c8a3d640 1 -- 172.21.15.105:0/3627903861 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fd7f3b60 msgr2=0x5555fd8029e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:17.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.993+0000 7fb6c8a3d640 1 --2- 172.21.15.105:0/3627903861 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fd7f3b60 0x5555fd8029e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:17.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.993+0000 7fb6c8a3d640 1 -- 172.21.15.105:0/3627903861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fd7fed40 msgr2=0x5555fd803480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:17.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.993+0000 7fb6c8a3d640 1 --2- 172.21.15.105:0/3627903861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fd7fed40 0x5555fd803480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:17.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.993+0000 7fb6c8a3d640 1 -- 172.21.15.105:0/3627903861 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555fd878310 con 0x5555fd7f4480 2024-09-16T02:05:17.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.993+0000 7fb6c8a3d640 1 --2- 172.21.15.105:0/3627903861 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555fd7f4480 0x5555fd802f20 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x5555fd7fb510 tx=0x7fb6c00047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=832502c8d921b985 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:17.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.993+0000 7fb6acff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:05:17.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.994+0000 7fb6acff9640 1 -- 172.21.15.105:0/3627903861 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c001c040 con 0x5555fd7f4480 2024-09-16T02:05:17.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.994+0000 7fb6acff9640 1 -- 172.21.15.105:0/3627903861 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fb6c0016990 con 0x5555fd7f4480 2024-09-16T02:05:17.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.994+0000 7fb6cd87b740 1 -- 172.21.15.105:0/3627903861 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6bc0da070 con 0x5555fd7f4480 2024-09-16T02:05:17.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.994+0000 7fb6acff9640 1 -- 172.21.15.105:0/3627903861 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb6c000f040 con 0x5555fd7f4480 2024-09-16T02:05:17.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.994+0000 7fb6cd87b740 10 client.15774 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:17.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.994+0000 7fb6cd87b740 1 -- 172.21.15.105:0/3627903861 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5555fd8796e0 con 0x5555fd7f4480 2024-09-16T02:05:17.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.995+0000 7fb6acff9640 1 -- 172.21.15.105:0/3627903861 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6bc0da070 con 0x5555fd7f4480 2024-09-16T02:05:17.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.995+0000 7fb6acff9640 1 -- 172.21.15.105:0/3627903861 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb6c0012040 con 0x5555fd7f4480 2024-09-16T02:05:17.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.995+0000 7fb6cd87b740 10 client.15774 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:17.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.995+0000 7fb6cd87b740 10 client.15774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:05:17.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.995+0000 7fb6cd87b740 20 client.15774 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:17.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.995+0000 7fb6cd87b740 1 --2- 172.21.15.105:0/3627903861 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x5555fd80e4c0 0x5555fd87bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:17.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.995+0000 7fb6cd87b740 4 client.15774 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T02:05:17.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.995+0000 7fb6cd87b740 1 -- 172.21.15.105:0/3627903861 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5555fd760d00 con 0x5555fd80e4c0 2024-09-16T02:05:17.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.996+0000 7fb6c923e640 1 --2- 172.21.15.105:0/3627903861 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x5555fd80e4c0 0x5555fd87bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:17.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.999+0000 7fb6c923e640 1 --2- 172.21.15.105:0/3627903861 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x5555fd80e4c0 0x5555fd87bf20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a4d85b5a5d65d2 server_cookie=27dc8321dcd8b85e in_seq=0 out_seq=0 2024-09-16T02:05:18.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:17.999+0000 7fb6acff9640 10 client.15774 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:05:18.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:18.000+0000 7fb6acff9640 1 -- 172.21.15.105:0/3627903861 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5555fd760d00 con 0x5555fd80e4c0 2024-09-16T02:05:18.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:18.000+0000 7fb6acff9640 10 client.15774 handle_command_reply: tid=0 2024-09-16T02:05:18.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:18.000+0000 7fb6cd87b740 1 client.15774 shutdown 2024-09-16T02:05:18.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:18.001+0000 7fb6cd87b740 1 -- 172.21.15.105:0/3627903861 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555fd7f4480 msgr2=0x5555fd802f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:18.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:18.001+0000 7fb6cd87b740 1 --2- 172.21.15.105:0/3627903861 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555fd7f4480 0x5555fd802f20 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x5555fd7fb510 tx=0x7fb6c00047d0 comp rx=0 tx=0).stop 2024-09-16T02:05:18.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:18.001+0000 7fb6cd87b740 1 -- 172.21.15.105:0/3627903861 shutdown_connections 2024-09-16T02:05:18.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:18.001+0000 7fb6cd87b740 1 --2- 172.21.15.105:0/3627903861 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x5555fd80e4c0 0x5555fd87bf20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:18.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:18.001+0000 7fb6cd87b740 1 --2- 172.21.15.105:0/3627903861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fd7fed40 0x5555fd803480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:18.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:18.001+0000 7fb6cd87b740 1 --2- 172.21.15.105:0/3627903861 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555fd7f4480 0x5555fd802f20 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:18.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:18.001+0000 7fb6cd87b740 1 --2- 172.21.15.105:0/3627903861 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555fd7f3b60 0x5555fd8029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:18.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:18.001+0000 7fb6cd87b740 1 -- 172.21.15.105:0/3627903861 >> 172.21.15.105:0/3627903861 conn(0x5555fd7f00f0 msgr2=0x5555fd7fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:18.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:18.001+0000 7fb6acff9640 0 client.15774 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:05:18.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:18.001+0000 7fb6cd87b740 1 -- 172.21.15.105:0/3627903861 shutdown_connections 2024-09-16T02:05:18.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:18.002+0000 7fb6cd87b740 1 -- 172.21.15.105:0/3627903861 wait complete. 2024-09-16T02:05:18.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:18.002+0000 7fb6cd87b740 20 client.15774 trim_cache size 0 max 16384 2024-09-16T02:05:18.004 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-16T02:05:18.004 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-09-16T02:05:18.004 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "f56e0ed9-a93e-450f-b3f4-4877045ff236", 2024-09-16T02:05:18.004 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-09-16T02:05:18.004 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-16T02:05:18.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:18.003+0000 7fb6cb4a2640 1 -- 172.21.15.105:0/2573589532 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb6b00a32c0 msgr2=0x7fb6b00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:18.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:18.003+0000 7fb6cb4a2640 1 --2- 172.21.15.105:0/2573589532 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb6b00a32c0 0x7fb6b00a5780 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb6c40735c0 tx=0x7fb6c0009b90 comp rx=0 tx=0).stop 2024-09-16T02:05:18.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:18.003+0000 7fb6cb4a2640 1 -- 172.21.15.105:0/2573589532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6c410ea80 msgr2=0x7fb6c41aba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:18.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:18.003+0000 7fb6cb4a2640 1 --2- 172.21.15.105:0/2573589532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6c410ea80 0x7fb6c41aba90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc005cc0 tx=0x7fb6bc033f90 comp rx=0 tx=0).stop 2024-09-16T02:05:18.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:18.003+0000 7fb6cb4a2640 1 -- 172.21.15.105:0/2573589532 shutdown_connections 2024-09-16T02:05:18.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:18.003+0000 7fb6cb4a2640 1 --2- 172.21.15.105:0/2573589532 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb6b00a32c0 0x7fb6b00a5780 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:18.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:18.003+0000 7fb6cb4a2640 1 --2- 172.21.15.105:0/2573589532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6c4109e00 0x7fb6c41a7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:18.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:18.003+0000 7fb6cb4a2640 1 --2- 172.21.15.105:0/2573589532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6c410ea80 0x7fb6c41aba90 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:18.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:18.003+0000 7fb6cb4a2640 1 --2- 172.21.15.105:0/2573589532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6c4109450 0x7fb6c41a71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:18.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:18.003+0000 7fb6cb4a2640 1 -- 172.21.15.105:0/2573589532 >> 172.21.15.105:0/2573589532 conn(0x7fb6c406e2d0 msgr2=0x7fb6c410c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:18.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:18.003+0000 7fb6cb4a2640 1 -- 172.21.15.105:0/2573589532 shutdown_connections 2024-09-16T02:05:18.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:18.003+0000 7fb6cb4a2640 1 -- 172.21.15.105:0/2573589532 wait complete. 2024-09-16T02:05:18.137 INFO:tasks.workunit.client.0.smithi105.stdout: 2024-09-16T02:05:18.137 INFO:tasks.workunit.client.0.smithi105.stdout: Children see throughput for 1 rewriters = 770703.00 kB/sec 2024-09-16T02:05:18.138 INFO:tasks.workunit.client.0.smithi105.stdout: Parent sees throughput for 1 rewriters = 770687.06 kB/sec 2024-09-16T02:05:18.138 INFO:tasks.workunit.client.0.smithi105.stdout: Min throughput per process = 770703.00 kB/sec 2024-09-16T02:05:18.138 INFO:tasks.workunit.client.0.smithi105.stdout: Max throughput per process = 770703.00 kB/sec 2024-09-16T02:05:18.138 INFO:tasks.workunit.client.0.smithi105.stdout: Avg throughput per process = 770703.00 kB/sec 2024-09-16T02:05:18.138 INFO:tasks.workunit.client.0.smithi105.stdout: Min xfer = 10485760.00 kB 2024-09-16T02:05:18.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:18 smithi190 ceph-mon[34762]: pgmap v422: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 610 MiB/s wr, 161 op/s 2024-09-16T02:05:18.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:18 smithi137 ceph-mon[34099]: pgmap v422: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 610 MiB/s wr, 161 op/s 2024-09-16T02:05:19.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:18 smithi105 ceph-mon[31197]: pgmap v422: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 610 MiB/s wr, 161 op/s 2024-09-16T02:05:19.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:19 smithi190 ceph-mon[34762]: scrub queued for path: / 2024-09-16T02:05:19.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:19 smithi190 ceph-mon[34762]: scrub summary: idle+waiting paths [/] 2024-09-16T02:05:19.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:19 smithi190 ceph-mon[34762]: scrub summary: active paths [/] 2024-09-16T02:05:19.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:19 smithi190 ceph-mon[34762]: scrub summary: idle+waiting paths [/] 2024-09-16T02:05:19.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:19 smithi137 ceph-mon[34099]: scrub queued for path: / 2024-09-16T02:05:19.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:19 smithi137 ceph-mon[34099]: scrub summary: idle+waiting paths [/] 2024-09-16T02:05:19.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:19 smithi137 ceph-mon[34099]: scrub summary: active paths [/] 2024-09-16T02:05:19.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:19 smithi137 ceph-mon[34099]: scrub summary: idle+waiting paths [/] 2024-09-16T02:05:20.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:19 smithi105 ceph-mon[31197]: scrub queued for path: / 2024-09-16T02:05:20.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:19 smithi105 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-16T02:05:20.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:19 smithi105 ceph-mon[31197]: scrub summary: active paths [/] 2024-09-16T02:05:20.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:19 smithi105 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-16T02:05:20.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:20 smithi190 ceph-mon[34762]: pgmap v423: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 692 MiB/s wr, 184 op/s 2024-09-16T02:05:20.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:20 smithi190 ceph-mon[34762]: scrub summary: idle 2024-09-16T02:05:20.913 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:20 smithi105 ceph-mon[31197]: pgmap v423: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 692 MiB/s wr, 184 op/s 2024-09-16T02:05:20.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:20 smithi137 ceph-mon[34099]: pgmap v423: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 692 MiB/s wr, 184 op/s 2024-09-16T02:05:20.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:20 smithi137 ceph-mon[34099]: scrub summary: idle 2024-09-16T02:05:21.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:20 smithi105 ceph-mon[31197]: scrub summary: idle 2024-09-16T02:05:22.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:22 smithi190 ceph-mon[34762]: pgmap v424: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 752 MiB/s wr, 198 op/s 2024-09-16T02:05:22.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:22 smithi137 ceph-mon[34099]: pgmap v424: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 752 MiB/s wr, 198 op/s 2024-09-16T02:05:23.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:22 smithi105 ceph-mon[31197]: pgmap v424: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 752 MiB/s wr, 198 op/s 2024-09-16T02:05:24.440 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T02:05:24.734 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:24.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:24 smithi105 ceph-mon[31197]: pgmap v425: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 537 MiB/s wr, 144 op/s 2024-09-16T02:05:24.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:24 smithi137 ceph-mon[34099]: pgmap v425: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 537 MiB/s wr, 144 op/s 2024-09-16T02:05:25.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:24 smithi190 ceph-mon[34762]: pgmap v425: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 537 MiB/s wr, 144 op/s 2024-09-16T02:05:25.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.209+0000 7f20fb431640 1 -- 172.21.15.105:0/978553135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20f4102570 msgr2=0x7f20f41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:25.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.209+0000 7f20fb431640 1 --2- 172.21.15.105:0/978553135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20f4102570 0x7f20f41029b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f20f0009900 tx=0x7f20f002a480 comp rx=0 tx=0).stop 2024-09-16T02:05:25.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.209+0000 7f20fb431640 1 -- 172.21.15.105:0/978553135 shutdown_connections 2024-09-16T02:05:25.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.209+0000 7f20fb431640 1 --2- 172.21.15.105:0/978553135 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20f4102ef0 0x7f20f410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.209+0000 7f20fb431640 1 --2- 172.21.15.105:0/978553135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20f4102570 0x7f20f41029b0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.209+0000 7f20fb431640 1 --2- 172.21.15.105:0/978553135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20f4100e20 0x7f20f4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.209+0000 7f20fb431640 1 -- 172.21.15.105:0/978553135 >> 172.21.15.105:0/978553135 conn(0x7f20f40fc7d0 msgr2=0x7f20f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:25.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.209+0000 7f20fb431640 1 -- 172.21.15.105:0/978553135 shutdown_connections 2024-09-16T02:05:25.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.209+0000 7f20fb431640 1 -- 172.21.15.105:0/978553135 wait complete. 2024-09-16T02:05:25.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.209+0000 7f20fb431640 1 Processor -- start 2024-09-16T02:05:25.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.209+0000 7f20fb431640 1 -- start start 2024-09-16T02:05:25.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.210+0000 7f20fb431640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20f4100e20 0x7f20f4197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:25.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.210+0000 7f20fb431640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20f4102570 0x7f20f4197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:25.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.210+0000 7f20fb431640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20f4102ef0 0x7f20f419bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:25.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.210+0000 7f20fb431640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f20f4110e30 con 0x7f20f4102570 2024-09-16T02:05:25.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.210+0000 7f20fb431640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f20f4110cb0 con 0x7f20f4102ef0 2024-09-16T02:05:25.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.210+0000 7f20fb431640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f20f4110fb0 con 0x7f20f4100e20 2024-09-16T02:05:25.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.210+0000 7f20f89cc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20f4102570 0x7f20f4197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:25.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.210+0000 7f20f89cc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20f4102570 0x7f20f4197850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53236/0 (socket says 172.21.15.105:53236) 2024-09-16T02:05:25.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.210+0000 7f20f89cc640 1 -- 172.21.15.105:0/693643483 learned_addr learned my addr 172.21.15.105:0/693643483 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:25.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.210+0000 7f20f99ce640 1 --2- 172.21.15.105:0/693643483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20f4102ef0 0x7f20f419bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:25.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.210+0000 7f20f91cd640 1 --2- 172.21.15.105:0/693643483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20f4100e20 0x7f20f4197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:25.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.210+0000 7f20f89cc640 1 -- 172.21.15.105:0/693643483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20f4100e20 msgr2=0x7f20f4197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:25.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.210+0000 7f20f89cc640 1 --2- 172.21.15.105:0/693643483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20f4100e20 0x7f20f4197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.210+0000 7f20f89cc640 1 -- 172.21.15.105:0/693643483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20f4102ef0 msgr2=0x7f20f419bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:25.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.210+0000 7f20f89cc640 1 --2- 172.21.15.105:0/693643483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20f4102ef0 0x7f20f419bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.210+0000 7f20f89cc640 1 -- 172.21.15.105:0/693643483 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f419c2c0 con 0x7f20f4102570 2024-09-16T02:05:25.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.210+0000 7f20f89cc640 1 --2- 172.21.15.105:0/693643483 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20f4102570 0x7f20f4197850 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f20f0006010 tx=0x7f20f0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:25.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.210+0000 7f20e67fc640 1 -- 172.21.15.105:0/693643483 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f0046070 con 0x7f20f4102570 2024-09-16T02:05:25.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.210+0000 7f20e67fc640 1 -- 172.21.15.105:0/693643483 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20f0003aa0 con 0x7f20f4102570 2024-09-16T02:05:25.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.210+0000 7f20fb431640 1 -- 172.21.15.105:0/693643483 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20f419c550 con 0x7f20f4102570 2024-09-16T02:05:25.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.211+0000 7f20e67fc640 1 -- 172.21.15.105:0/693643483 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f0035050 con 0x7f20f4102570 2024-09-16T02:05:25.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.211+0000 7f20fb431640 1 -- 172.21.15.105:0/693643483 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20f4069f70 con 0x7f20f4102570 2024-09-16T02:05:25.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.213+0000 7f20e67fc640 1 -- 172.21.15.105:0/693643483 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f20f0049070 con 0x7f20f4102570 2024-09-16T02:05:25.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.213+0000 7f20fd80a740 1 Processor -- start 2024-09-16T02:05:25.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.213+0000 7f20fd80a740 1 -- start start 2024-09-16T02:05:25.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.213+0000 7f20e67fc640 1 --2- 172.21.15.105:0/693643483 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f20e00a3240 0x7f20e00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:25.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.213+0000 7f20fd80a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565425e5f4a0 0x565425e5f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:25.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.214+0000 7f20fd80a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565425e5fdc0 0x565425e6a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:25.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.214+0000 7f20fd80a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565425e6a680 0x565425e6ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:25.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.214+0000 7f20fd80a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x565425dc52e0 con 0x565425e6a680 2024-09-16T02:05:25.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.214+0000 7f20fd80a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x565425d6cce0 con 0x565425e5fdc0 2024-09-16T02:05:25.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.214+0000 7f20fd80a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x565425dc12a0 con 0x565425e5f4a0 2024-09-16T02:05:25.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.214+0000 7f20e67fc640 1 -- 172.21.15.105:0/693643483 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f20f00f1750 con 0x7f20f4102570 2024-09-16T02:05:25.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.214+0000 7f20f99ce640 1 --2- 172.21.15.105:0/693643483 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f20e00a3240 0x7f20e00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:25.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.214+0000 7f20f99ce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565425e6a680 0x565425e6ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:25.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.214+0000 7f20f99ce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565425e6a680 0x565425e6ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53242/0 (socket says 172.21.15.105:53242) 2024-09-16T02:05:25.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.214+0000 7f20f99ce640 1 -- 172.21.15.105:0/3540395248 learned_addr learned my addr 172.21.15.105:0/3540395248 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:25.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.214+0000 7f20f91cd640 1 --2- 172.21.15.105:0/3540395248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565425e5f4a0 0x565425e5f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:25.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.214+0000 7f20f89cc640 1 --2- 172.21.15.105:0/3540395248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565425e5fdc0 0x565425e6a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:25.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.214+0000 7f20f89cc640 1 -- 172.21.15.105:0/3540395248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565425e5f4a0 msgr2=0x565425e5f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:25.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.214+0000 7f20f89cc640 1 --2- 172.21.15.105:0/3540395248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565425e5f4a0 0x565425e5f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.214+0000 7f20f89cc640 1 -- 172.21.15.105:0/3540395248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565425e6a680 msgr2=0x565425e6ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:25.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.214+0000 7f20f89cc640 1 --2- 172.21.15.105:0/3540395248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565425e6a680 0x565425e6ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.214+0000 7f20f89cc640 1 -- 172.21.15.105:0/3540395248 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565425d51440 con 0x565425e5fdc0 2024-09-16T02:05:25.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.214+0000 7f20f99ce640 1 --2- 172.21.15.105:0/693643483 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f20e00a3240 0x7f20e00a5700 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f20ec012c80 tx=0x7f20ec009e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:25.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.214+0000 7f20f89cc640 1 --2- 172.21.15.105:0/3540395248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565425e5fdc0 0x565425e6a140 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f20f00ef240 tx=0x7f20f00bd910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56cc24b30dcb945f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:25.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.214+0000 7f20df7fe640 1 -- 172.21.15.105:0/3540395248 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f0034170 con 0x565425e5fdc0 2024-09-16T02:05:25.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.215+0000 7f20df7fe640 1 -- 172.21.15.105:0/3540395248 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20f00be410 con 0x565425e5fdc0 2024-09-16T02:05:25.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.215+0000 7f20df7fe640 1 -- 172.21.15.105:0/3540395248 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f0035690 con 0x565425e5fdc0 2024-09-16T02:05:25.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.215+0000 7f20fd80a740 1 -- 172.21.15.105:0/3540395248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565425e5fdc0 msgr2=0x565425e6a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:25.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.215+0000 7f20fd80a740 1 --2- 172.21.15.105:0/3540395248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565425e5fdc0 0x565425e6a140 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f20f00ef240 tx=0x7f20f00bd910 comp rx=0 tx=0).stop 2024-09-16T02:05:25.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.215+0000 7f20fd80a740 1 -- 172.21.15.105:0/3540395248 shutdown_connections 2024-09-16T02:05:25.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.215+0000 7f20fd80a740 1 --2- 172.21.15.105:0/3540395248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565425e6a680 0x565425e6ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.215+0000 7f20fd80a740 1 --2- 172.21.15.105:0/3540395248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565425e5fdc0 0x565425e6a140 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.215+0000 7f20fd80a740 1 --2- 172.21.15.105:0/3540395248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565425e5f4a0 0x565425e5f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.215+0000 7f20fd80a740 1 -- 172.21.15.105:0/3540395248 >> 172.21.15.105:0/3540395248 conn(0x565425e5ba30 msgr2=0x565425e5be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:25.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.215+0000 7f20fd80a740 1 -- 172.21.15.105:0/3540395248 shutdown_connections 2024-09-16T02:05:25.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.215+0000 7f20fd80a740 1 -- 172.21.15.105:0/3540395248 wait complete. 2024-09-16T02:05:25.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.216+0000 7f20fd80a740 1 -- start start 2024-09-16T02:05:25.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.216+0000 7f20fd80a740 1 Processor -- start 2024-09-16T02:05:25.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.216+0000 7f20fd80a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565425e5f4a0 0x565425e6a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:25.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.216+0000 7f20fd80a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565425e5fdc0 0x565425e672e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:25.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.216+0000 7f20fd80a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565425e6a680 0x565425e67880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:25.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.216+0000 7f20fd80a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x565425e6d010 con 0x565425e5f4a0 2024-09-16T02:05:25.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.216+0000 7f20fd80a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x565425d80050 con 0x565425e5fdc0 2024-09-16T02:05:25.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.216+0000 7f20fd80a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x565425e6d190 con 0x565425e6a680 2024-09-16T02:05:25.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.216+0000 7f20fd80a740 10 client.0 _ll_register_callbacks cb 0x565425dc0160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:25.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.216+0000 7f20f91cd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565425e5f4a0 0x565425e6a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:25.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.216+0000 7f20f91cd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565425e5f4a0 0x565425e6a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53246/0 (socket says 172.21.15.105:53246) 2024-09-16T02:05:25.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.216+0000 7f20f91cd640 1 -- 172.21.15.105:0/3760557462 learned_addr learned my addr 172.21.15.105:0/3760557462 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:25.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.216+0000 7f20f89cc640 1 --2- 172.21.15.105:0/3760557462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565425e5fdc0 0x565425e672e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:25.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.216+0000 7f20f99ce640 1 --2- 172.21.15.105:0/3760557462 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565425e6a680 0x565425e67880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:25.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.216+0000 7f20f91cd640 1 -- 172.21.15.105:0/3760557462 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565425e6a680 msgr2=0x565425e67880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:25.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.216+0000 7f20f91cd640 1 --2- 172.21.15.105:0/3760557462 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565425e6a680 0x565425e67880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.217+0000 7f20f91cd640 1 -- 172.21.15.105:0/3760557462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565425e5fdc0 msgr2=0x565425e672e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:25.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.217+0000 7f20f91cd640 1 --2- 172.21.15.105:0/3760557462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565425e5fdc0 0x565425e672e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.217+0000 7f20f91cd640 1 -- 172.21.15.105:0/3760557462 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565425b363c0 con 0x565425e5f4a0 2024-09-16T02:05:25.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.217+0000 7f20f89cc640 1 --2- 172.21.15.105:0/3760557462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565425e5fdc0 0x565425e672e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:05:25.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.217+0000 7f20f99ce640 1 --2- 172.21.15.105:0/3760557462 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565425e6a680 0x565425e67880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:05:25.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.217+0000 7f20f91cd640 1 --2- 172.21.15.105:0/3760557462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565425e5f4a0 0x565425e6a030 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f20e800bf60 tx=0x7f20e800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a9c4dbf8d02480 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:25.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.217+0000 7f20dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:05:25.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.217+0000 7f20dcff9640 1 -- 172.21.15.105:0/3760557462 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e801f060 con 0x565425e5f4a0 2024-09-16T02:05:25.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.217+0000 7f20fd80a740 1 -- 172.21.15.105:0/3760557462 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20f00be410 con 0x565425e5f4a0 2024-09-16T02:05:25.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.217+0000 7f20dcff9640 1 -- 172.21.15.105:0/3760557462 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f20e80108a0 con 0x565425e5f4a0 2024-09-16T02:05:25.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.217+0000 7f20dcff9640 1 -- 172.21.15.105:0/3760557462 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f20e8010aa0 con 0x565425e5f4a0 2024-09-16T02:05:25.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.217+0000 7f20fd80a740 10 client.15786 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:25.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.217+0000 7f20fd80a740 1 -- 172.21.15.105:0/3760557462 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x565425e78a40 con 0x565425e5f4a0 2024-09-16T02:05:25.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.217+0000 7f20dcff9640 1 -- 172.21.15.105:0/3760557462 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20f00be410 con 0x565425e5f4a0 2024-09-16T02:05:25.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.217+0000 7f20dcff9640 1 -- 172.21.15.105:0/3760557462 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f20e8023030 con 0x565425e5f4a0 2024-09-16T02:05:25.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.217+0000 7f20fd80a740 10 client.15786 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:25.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.217+0000 7f20fd80a740 10 client.15786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:05:25.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.217+0000 7f20fd80a740 20 client.15786 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:25.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.217+0000 7f20fd80a740 1 --2- 172.21.15.105:0/3760557462 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x565425e79f00 0x565425e7c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:25.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.217+0000 7f20fd80a740 4 client.15786 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:05:25.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.217+0000 7f20fd80a740 1 -- 172.21.15.105:0/3760557462 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565425dd2850 con 0x565425e79f00 2024-09-16T02:05:25.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.218+0000 7f20f89cc640 1 --2- 172.21.15.105:0/3760557462 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x565425e79f00 0x565425e7c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:25.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.218+0000 7f20dcff9640 10 client.15786 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:05:25.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.218+0000 7f20f89cc640 1 --2- 172.21.15.105:0/3760557462 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x565425e79f00 0x565425e7c2f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1aad62066f480665 server_cookie=3dce6bae7ec62ea3 in_seq=0 out_seq=0 2024-09-16T02:05:25.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.219+0000 7f20dcff9640 1 -- 172.21.15.105:0/3760557462 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565425dd2850 con 0x565425e79f00 2024-09-16T02:05:25.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.219+0000 7f20dcff9640 10 client.15786 handle_command_reply: tid=0 2024-09-16T02:05:25.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.219+0000 7f20fd80a740 1 client.15786 shutdown 2024-09-16T02:05:25.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.219+0000 7f20fd80a740 1 -- 172.21.15.105:0/3760557462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565425e5f4a0 msgr2=0x565425e6a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:25.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.219+0000 7f20fd80a740 1 --2- 172.21.15.105:0/3760557462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565425e5f4a0 0x565425e6a030 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f20e800bf60 tx=0x7f20e800bf90 comp rx=0 tx=0).stop 2024-09-16T02:05:25.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.219+0000 7f20fd80a740 1 -- 172.21.15.105:0/3760557462 shutdown_connections 2024-09-16T02:05:25.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.219+0000 7f20fd80a740 1 --2- 172.21.15.105:0/3760557462 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x565425e79f00 0x565425e7c2f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.219+0000 7f20fd80a740 1 --2- 172.21.15.105:0/3760557462 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565425e6a680 0x565425e67880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.219+0000 7f20fd80a740 1 --2- 172.21.15.105:0/3760557462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565425e5fdc0 0x565425e672e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.219+0000 7f20dcff9640 0 client.15786 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:05:25.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.219+0000 7f20fd80a740 1 --2- 172.21.15.105:0/3760557462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565425e5f4a0 0x565425e6a030 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.219+0000 7f20fd80a740 1 -- 172.21.15.105:0/3760557462 >> 172.21.15.105:0/3760557462 conn(0x565425e5ba30 msgr2=0x565425e61760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:25.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.219+0000 7f20fd80a740 1 -- 172.21.15.105:0/3760557462 shutdown_connections 2024-09-16T02:05:25.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.219+0000 7f20fd80a740 1 -- 172.21.15.105:0/3760557462 wait complete. 2024-09-16T02:05:25.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.220+0000 7f20fd80a740 20 client.15786 trim_cache size 0 max 16384 2024-09-16T02:05:25.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.237+0000 7f20fd80a740 1 Processor -- start 2024-09-16T02:05:25.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.237+0000 7f20fd80a740 1 -- start start 2024-09-16T02:05:25.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.237+0000 7f20fd80a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565425e5f4a0 0x565425e65bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:25.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.237+0000 7f20fd80a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565425e5fdc0 0x565425e76130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:25.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.237+0000 7f20fd80a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565425e6a680 0x565425e81b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:25.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.237+0000 7f20fd80a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f20f41ab330 con 0x565425e5fdc0 2024-09-16T02:05:25.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.237+0000 7f20fd80a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f20f41ace70 con 0x565425e6a680 2024-09-16T02:05:25.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.237+0000 7f20fd80a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f20f41aba00 con 0x565425e5f4a0 2024-09-16T02:05:25.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.237+0000 7f20f89cc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565425e5fdc0 0x565425e76130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:25.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.237+0000 7f20f89cc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565425e5fdc0 0x565425e76130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53260/0 (socket says 172.21.15.105:53260) 2024-09-16T02:05:25.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.237+0000 7f20f89cc640 1 -- 172.21.15.105:0/2508153473 learned_addr learned my addr 172.21.15.105:0/2508153473 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:25.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.237+0000 7f20f99ce640 1 --2- 172.21.15.105:0/2508153473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565425e6a680 0x565425e81b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:25.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.237+0000 7f20f91cd640 1 --2- 172.21.15.105:0/2508153473 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565425e5f4a0 0x565425e65bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:25.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.237+0000 7f20f89cc640 1 -- 172.21.15.105:0/2508153473 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565425e5f4a0 msgr2=0x565425e65bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:25.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.237+0000 7f20f89cc640 1 --2- 172.21.15.105:0/2508153473 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565425e5f4a0 0x565425e65bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.237+0000 7f20f89cc640 1 -- 172.21.15.105:0/2508153473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565425e6a680 msgr2=0x565425e81b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:25.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.237+0000 7f20f89cc640 1 --2- 172.21.15.105:0/2508153473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565425e6a680 0x565425e81b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.237+0000 7f20f89cc640 1 -- 172.21.15.105:0/2508153473 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565425e66290 con 0x565425e5fdc0 2024-09-16T02:05:25.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.237+0000 7f20f91cd640 1 --2- 172.21.15.105:0/2508153473 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565425e5f4a0 0x565425e65bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:05:25.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.237+0000 7f20f89cc640 1 --2- 172.21.15.105:0/2508153473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565425e5fdc0 0x565425e76130 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f20f00058d0 tx=0x7f20f0007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2efe69c3c5e2799 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:25.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.237+0000 7f20deffd640 1 -- 172.21.15.105:0/2508153473 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f00db070 con 0x565425e5fdc0 2024-09-16T02:05:25.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.238+0000 7f20deffd640 1 -- 172.21.15.105:0/2508153473 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20f0034170 con 0x565425e5fdc0 2024-09-16T02:05:25.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.238+0000 7f20deffd640 1 -- 172.21.15.105:0/2508153473 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f00bc010 con 0x565425e5fdc0 2024-09-16T02:05:25.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.238+0000 7f20fd80a740 1 -- 172.21.15.105:0/2508153473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565425e5fdc0 msgr2=0x565425e76130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:25.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.238+0000 7f20fd80a740 1 --2- 172.21.15.105:0/2508153473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565425e5fdc0 0x565425e76130 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f20f00058d0 tx=0x7f20f0007f90 comp rx=0 tx=0).stop 2024-09-16T02:05:25.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.238+0000 7f20fd80a740 1 -- 172.21.15.105:0/2508153473 shutdown_connections 2024-09-16T02:05:25.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.238+0000 7f20fd80a740 1 --2- 172.21.15.105:0/2508153473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565425e6a680 0x565425e81b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.238+0000 7f20fd80a740 1 --2- 172.21.15.105:0/2508153473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565425e5fdc0 0x565425e76130 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.238+0000 7f20fd80a740 1 --2- 172.21.15.105:0/2508153473 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565425e5f4a0 0x565425e65bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.238+0000 7f20fd80a740 1 -- 172.21.15.105:0/2508153473 >> 172.21.15.105:0/2508153473 conn(0x565425e5ba30 msgr2=0x565425e62870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:25.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.238+0000 7f20fd80a740 1 -- 172.21.15.105:0/2508153473 shutdown_connections 2024-09-16T02:05:25.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.238+0000 7f20fd80a740 1 -- 172.21.15.105:0/2508153473 wait complete. 2024-09-16T02:05:25.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.239+0000 7f20fd80a740 1 -- start start 2024-09-16T02:05:25.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.239+0000 7f20fd80a740 1 Processor -- start 2024-09-16T02:05:25.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.239+0000 7f20fd80a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565425e5f4a0 0x565425e6e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:25.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.239+0000 7f20fd80a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565425e5fdc0 0x565425e6e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:25.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.239+0000 7f20fd80a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565425e6a680 0x565425e6ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:25.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.239+0000 7f20fd80a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f20f41ac7a0 con 0x565425e5f4a0 2024-09-16T02:05:25.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.239+0000 7f20fd80a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f20f41ac0d0 con 0x565425e5fdc0 2024-09-16T02:05:25.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.239+0000 7f20fd80a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f20f41a9310 con 0x565425e6a680 2024-09-16T02:05:25.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.239+0000 7f20fd80a740 10 client.0 _ll_register_callbacks cb 0x7f20e800b790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:25.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.239+0000 7f20f91cd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565425e5f4a0 0x565425e6e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:25.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.239+0000 7f20f91cd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565425e5f4a0 0x565425e6e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53270/0 (socket says 172.21.15.105:53270) 2024-09-16T02:05:25.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.239+0000 7f20f91cd640 1 -- 172.21.15.105:0/1126791867 learned_addr learned my addr 172.21.15.105:0/1126791867 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:25.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.240+0000 7f20f89cc640 1 --2- 172.21.15.105:0/1126791867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565425e5fdc0 0x565425e6e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:25.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.240+0000 7f20f99ce640 1 --2- 172.21.15.105:0/1126791867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565425e6a680 0x565425e6ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:25.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.240+0000 7f20f91cd640 1 -- 172.21.15.105:0/1126791867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565425e6a680 msgr2=0x565425e6ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:25.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.240+0000 7f20f91cd640 1 --2- 172.21.15.105:0/1126791867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565425e6a680 0x565425e6ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.240+0000 7f20f91cd640 1 -- 172.21.15.105:0/1126791867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565425e5fdc0 msgr2=0x565425e6e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:25.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.240+0000 7f20f91cd640 1 --2- 172.21.15.105:0/1126791867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565425e5fdc0 0x565425e6e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.240+0000 7f20f91cd640 1 -- 172.21.15.105:0/1126791867 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565425e6a3a0 con 0x565425e5f4a0 2024-09-16T02:05:25.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.240+0000 7f20f91cd640 1 --2- 172.21.15.105:0/1126791867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565425e5f4a0 0x565425e6e270 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f20e800b760 tx=0x7f20e8010cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77b1851a70a192a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:25.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.240+0000 7f20dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:05:25.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.240+0000 7f20dcff9640 1 -- 172.21.15.105:0/1126791867 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e800da70 con 0x565425e5f4a0 2024-09-16T02:05:25.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.240+0000 7f20dcff9640 1 -- 172.21.15.105:0/1126791867 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f20e800dc10 con 0x565425e5f4a0 2024-09-16T02:05:25.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.240+0000 7f20fd80a740 1 -- 172.21.15.105:0/1126791867 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20f0034170 con 0x565425e5f4a0 2024-09-16T02:05:25.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.240+0000 7f20dcff9640 1 -- 172.21.15.105:0/1126791867 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f20e80108b0 con 0x565425e5f4a0 2024-09-16T02:05:25.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.240+0000 7f20fd80a740 10 client.15792 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:25.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.240+0000 7f20fd80a740 1 -- 172.21.15.105:0/1126791867 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x565425e7c660 con 0x565425e5f4a0 2024-09-16T02:05:25.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.240+0000 7f20dcff9640 1 -- 172.21.15.105:0/1126791867 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20f0034170 con 0x565425e5f4a0 2024-09-16T02:05:25.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.241+0000 7f20dcff9640 1 -- 172.21.15.105:0/1126791867 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f20e8035020 con 0x565425e5f4a0 2024-09-16T02:05:25.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.241+0000 7f20fd80a740 10 client.15792 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:25.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.241+0000 7f20fd80a740 10 client.15792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:05:25.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.241+0000 7f20fd80a740 20 client.15792 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:25.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.241+0000 7f20fd80a740 1 --2- 172.21.15.105:0/1126791867 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x565425ee5420 0x565425ee7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:25.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.241+0000 7f20fd80a740 4 client.15792 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T02:05:25.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.241+0000 7f20fd80a740 1 -- 172.21.15.105:0/1126791867 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "scrub status"}) -- 0x565425dccd60 con 0x565425ee5420 2024-09-16T02:05:25.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.241+0000 7f20f89cc640 1 --2- 172.21.15.105:0/1126791867 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x565425ee5420 0x565425ee7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:25.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.241+0000 7f20f89cc640 1 --2- 172.21.15.105:0/1126791867 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x565425ee5420 0x565425ee7840 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55ca7b65e9ae3c15 server_cookie=4d524e72f7f0dc27 in_seq=0 out_seq=0 2024-09-16T02:05:25.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.241+0000 7f20dcff9640 10 client.15792 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:05:25.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.241+0000 7f20dcff9640 1 -- 172.21.15.105:0/1126791867 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x565425dccd60 con 0x565425ee5420 2024-09-16T02:05:25.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.241+0000 7f20dcff9640 10 client.15792 handle_command_reply: tid=0 2024-09-16T02:05:25.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.241+0000 7f20fd80a740 1 client.15792 shutdown 2024-09-16T02:05:25.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.242+0000 7f20fd80a740 1 -- 172.21.15.105:0/1126791867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565425e5f4a0 msgr2=0x565425e6e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:25.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.242+0000 7f20fd80a740 1 --2- 172.21.15.105:0/1126791867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565425e5f4a0 0x565425e6e270 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f20e800b760 tx=0x7f20e8010cc0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.242+0000 7f20fd80a740 1 -- 172.21.15.105:0/1126791867 shutdown_connections 2024-09-16T02:05:25.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.242+0000 7f20fd80a740 1 --2- 172.21.15.105:0/1126791867 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x565425ee5420 0x565425ee7840 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.242+0000 7f20fd80a740 1 --2- 172.21.15.105:0/1126791867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565425e6a680 0x565425e6ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.242+0000 7f20fd80a740 1 --2- 172.21.15.105:0/1126791867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565425e5fdc0 0x565425e6e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.242+0000 7f20fd80a740 1 --2- 172.21.15.105:0/1126791867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565425e5f4a0 0x565425e6e270 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.242+0000 7f20dcff9640 0 client.15792 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:05:25.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.242+0000 7f20fd80a740 1 -- 172.21.15.105:0/1126791867 >> 172.21.15.105:0/1126791867 conn(0x565425e5ba30 msgr2=0x565425e61f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:25.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.242+0000 7f20fd80a740 1 -- 172.21.15.105:0/1126791867 shutdown_connections 2024-09-16T02:05:25.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.242+0000 7f20fd80a740 1 -- 172.21.15.105:0/1126791867 wait complete. 2024-09-16T02:05:25.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.242+0000 7f20fd80a740 20 client.15792 trim_cache size 0 max 16384 2024-09-16T02:05:25.260 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-16T02:05:25.260 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-09-16T02:05:25.260 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-09-16T02:05:25.260 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-16T02:05:25.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.242+0000 7f20fb431640 1 -- 172.21.15.105:0/693643483 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f20e00a3240 msgr2=0x7f20e00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:25.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.242+0000 7f20fb431640 1 --2- 172.21.15.105:0/693643483 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f20e00a3240 0x7f20e00a5700 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f20ec012c80 tx=0x7f20ec009e80 comp rx=0 tx=0).stop 2024-09-16T02:05:25.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.242+0000 7f20fb431640 1 -- 172.21.15.105:0/693643483 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20f4102570 msgr2=0x7f20f4197850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:25.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.242+0000 7f20fb431640 1 --2- 172.21.15.105:0/693643483 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20f4102570 0x7f20f4197850 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f20f0006010 tx=0x7f20f0004270 comp rx=0 tx=0).stop 2024-09-16T02:05:25.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.242+0000 7f20fb431640 1 -- 172.21.15.105:0/693643483 shutdown_connections 2024-09-16T02:05:25.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.242+0000 7f20fb431640 1 --2- 172.21.15.105:0/693643483 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f20e00a3240 0x7f20e00a5700 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.242+0000 7f20fb431640 1 --2- 172.21.15.105:0/693643483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20f4102ef0 0x7f20f419bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.242+0000 7f20fb431640 1 --2- 172.21.15.105:0/693643483 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20f4102570 0x7f20f4197850 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.242+0000 7f20fb431640 1 --2- 172.21.15.105:0/693643483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20f4100e20 0x7f20f4197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.242+0000 7f20fb431640 1 -- 172.21.15.105:0/693643483 >> 172.21.15.105:0/693643483 conn(0x7f20f40fc7d0 msgr2=0x7f20f40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:25.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.243+0000 7f20fb431640 1 -- 172.21.15.105:0/693643483 shutdown_connections 2024-09-16T02:05:25.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:25.243+0000 7f20fb431640 1 -- 172.21.15.105:0/693643483 wait complete. 2024-09-16T02:05:25.995 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T02:05:25.995 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:05:26.263 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:26.698 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:26 smithi105 ceph-mon[31197]: pgmap v426: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 487 MiB/s wr, 132 op/s 2024-09-16T02:05:26.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.784+0000 7ff741651640 1 -- 172.21.15.105:0/3082896580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff73c102570 msgr2=0x7ff73c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.784+0000 7ff741651640 1 --2- 172.21.15.105:0/3082896580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff73c102570 0x7ff73c1029b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff730009900 tx=0x7ff73002a480 comp rx=0 tx=0).stop 2024-09-16T02:05:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.784+0000 7ff741651640 1 -- 172.21.15.105:0/3082896580 shutdown_connections 2024-09-16T02:05:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.784+0000 7ff741651640 1 --2- 172.21.15.105:0/3082896580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff73c102ef0 0x7ff73c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.784+0000 7ff741651640 1 --2- 172.21.15.105:0/3082896580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff73c102570 0x7ff73c1029b0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.784+0000 7ff741651640 1 --2- 172.21.15.105:0/3082896580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff73c100e20 0x7ff73c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.784+0000 7ff741651640 1 -- 172.21.15.105:0/3082896580 >> 172.21.15.105:0/3082896580 conn(0x7ff73c0fc7d0 msgr2=0x7ff73c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:26.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.784+0000 7ff741651640 1 -- 172.21.15.105:0/3082896580 shutdown_connections 2024-09-16T02:05:26.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.784+0000 7ff741651640 1 -- 172.21.15.105:0/3082896580 wait complete. 2024-09-16T02:05:26.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.784+0000 7ff741651640 1 Processor -- start 2024-09-16T02:05:26.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.785+0000 7ff741651640 1 -- start start 2024-09-16T02:05:26.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.785+0000 7ff741651640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff73c100e20 0x7ff73c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:26.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.785+0000 7ff741651640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff73c102570 0x7ff73c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:26.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.785+0000 7ff741651640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff73c102ef0 0x7ff73c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:26.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.785+0000 7ff741651640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff73c111f50 con 0x7ff73c102570 2024-09-16T02:05:26.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.785+0000 7ff741651640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff73c111dd0 con 0x7ff73c100e20 2024-09-16T02:05:26.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.785+0000 7ff741651640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff73c1120d0 con 0x7ff73c102ef0 2024-09-16T02:05:26.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.785+0000 7ff73a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff73c102570 0x7ff73c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:26.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.785+0000 7ff73a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff73c102570 0x7ff73c198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53278/0 (socket says 172.21.15.105:53278) 2024-09-16T02:05:26.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.785+0000 7ff73a7fc640 1 -- 172.21.15.105:0/2868240966 learned_addr learned my addr 172.21.15.105:0/2868240966 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:26.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.785+0000 7ff73affd640 1 --2- 172.21.15.105:0/2868240966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff73c100e20 0x7ff73c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:26.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.785+0000 7ff73b7fe640 1 --2- 172.21.15.105:0/2868240966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff73c102ef0 0x7ff73c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:26.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.785+0000 7ff73a7fc640 1 -- 172.21.15.105:0/2868240966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff73c102ef0 msgr2=0x7ff73c19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:26.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.785+0000 7ff73a7fc640 1 --2- 172.21.15.105:0/2868240966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff73c102ef0 0x7ff73c19cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:26.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.785+0000 7ff73a7fc640 1 -- 172.21.15.105:0/2868240966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff73c100e20 msgr2=0x7ff73c1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:26.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.785+0000 7ff73a7fc640 1 --2- 172.21.15.105:0/2868240966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff73c100e20 0x7ff73c1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:26.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.785+0000 7ff73a7fc640 1 -- 172.21.15.105:0/2868240966 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff73c19d380 con 0x7ff73c102570 2024-09-16T02:05:26.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.786+0000 7ff73a7fc640 1 --2- 172.21.15.105:0/2868240966 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff73c102570 0x7ff73c198910 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff730006010 tx=0x7ff7300042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:26.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.786+0000 7ff723fff640 1 -- 172.21.15.105:0/2868240966 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73003f030 con 0x7ff73c102570 2024-09-16T02:05:26.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.786+0000 7ff741651640 1 -- 172.21.15.105:0/2868240966 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff73c19d610 con 0x7ff73c102570 2024-09-16T02:05:26.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.786+0000 7ff723fff640 1 -- 172.21.15.105:0/2868240966 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff730003cb0 con 0x7ff73c102570 2024-09-16T02:05:26.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.786+0000 7ff723fff640 1 -- 172.21.15.105:0/2868240966 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff730035070 con 0x7ff73c102570 2024-09-16T02:05:26.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.786+0000 7ff741651640 1 -- 172.21.15.105:0/2868240966 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff73c1a4f50 con 0x7ff73c102570 2024-09-16T02:05:26.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.787+0000 7ff741651640 1 -- 172.21.15.105:0/2868240966 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff73c1021b0 con 0x7ff73c102570 2024-09-16T02:05:26.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.788+0000 7ff723fff640 1 -- 172.21.15.105:0/2868240966 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7ff730042030 con 0x7ff73c102570 2024-09-16T02:05:26.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.789+0000 7ff723fff640 1 --2- 172.21.15.105:0/2868240966 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff7240a3240 0x7ff7240a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:26.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.789+0000 7ff723fff640 1 -- 172.21.15.105:0/2868240966 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7ff7300efda0 con 0x7ff73c102570 2024-09-16T02:05:26.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.789+0000 7ff73affd640 1 --2- 172.21.15.105:0/2868240966 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff7240a3240 0x7ff7240a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:26.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.790+0000 7ff73affd640 1 --2- 172.21.15.105:0/2868240966 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff7240a3240 0x7ff7240a5700 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff72c00bbe0 tx=0x7ff72c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:26.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:26.794+0000 7ff723fff640 1 -- 172.21.15.105:0/2868240966 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff7300bc410 con 0x7ff73c102570 2024-09-16T02:05:26.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:26 smithi137 ceph-mon[34099]: pgmap v426: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 487 MiB/s wr, 132 op/s 2024-09-16T02:05:27.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:27.003+0000 7ff741651640 1 -- 172.21.15.105:0/2868240966 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff73c109090 con 0x7ff73c102570 2024-09-16T02:05:27.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:27.004+0000 7ff723fff640 1 -- 172.21.15.105:0/2868240966 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11991 (secure 0 0 0) 0x7ff7300c0710 con 0x7ff73c102570 2024-09-16T02:05:27.005 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:05:27.005 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":28,"btime":"2024-09-16T02:02:41:554178+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2057699920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2057699920},{"type":"v1","addr":"172.21.15.105:6835","nonce":2057699920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1764886948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1764886948},{"type":"v1","addr":"172.21.15.105:6837","nonce":1764886948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T02:00:35.112266+0000","modified":"2024-09-16T02:02:40.885614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24469,"mds_2":24461,"mds_3":24457,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/4112062361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":4112062361},{"type":"v1","addr":"172.21.15.105:6839","nonce":4112062361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6841/163156514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":163156514},{"type":"v1","addr":"172.21.15.105:6841","nonce":163156514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/683519097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":683519097},{"type":"v1","addr":"172.21.15.190:6833","nonce":683519097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3025442862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3025442862},{"type":"v1","addr":"172.21.15.137:6833","nonce":3025442862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/763783602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":763783602},{"type":"v1","addr":"172.21.15.137:6835","nonce":763783602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.190:6835/1851042281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1851042281},{"type":"v1","addr":"172.21.15.190:6835","nonce":1851042281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/3181257549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3181257549},{"type":"v1","addr":"172.21.15.190:6837","nonce":3181257549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6837/2788541030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2788541030},{"type":"v1","addr":"172.21.15.137:6837","nonce":2788541030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.190:6839/1222573537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":1222573537},{"type":"v1","addr":"172.21.15.190:6839","nonce":1222573537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/3454924009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3454924009},{"type":"v1","addr":"172.21.15.137:6839","nonce":3454924009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24445,24457,24461,24469,24482]},"id":1}]} 2024-09-16T02:05:27.006 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 28 2024-09-16T02:05:27.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:27.007+0000 7ff741651640 1 -- 172.21.15.105:0/2868240966 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff7240a3240 msgr2=0x7ff7240a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:27.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:27.007+0000 7ff741651640 1 --2- 172.21.15.105:0/2868240966 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff7240a3240 0x7ff7240a5700 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff72c00bbe0 tx=0x7ff72c00b390 comp rx=0 tx=0).stop 2024-09-16T02:05:27.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:27.007+0000 7ff741651640 1 -- 172.21.15.105:0/2868240966 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff73c102570 msgr2=0x7ff73c198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:27.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:27.007+0000 7ff741651640 1 --2- 172.21.15.105:0/2868240966 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff73c102570 0x7ff73c198910 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff730006010 tx=0x7ff7300042f0 comp rx=0 tx=0).stop 2024-09-16T02:05:27.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:27.007+0000 7ff741651640 1 -- 172.21.15.105:0/2868240966 shutdown_connections 2024-09-16T02:05:27.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:27.007+0000 7ff741651640 1 --2- 172.21.15.105:0/2868240966 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff7240a3240 0x7ff7240a5700 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:27.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:27.007+0000 7ff741651640 1 --2- 172.21.15.105:0/2868240966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff73c102ef0 0x7ff73c19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:27.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:27.007+0000 7ff741651640 1 --2- 172.21.15.105:0/2868240966 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff73c102570 0x7ff73c198910 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:27.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:27.007+0000 7ff741651640 1 --2- 172.21.15.105:0/2868240966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff73c100e20 0x7ff73c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:27.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:27.007+0000 7ff741651640 1 -- 172.21.15.105:0/2868240966 >> 172.21.15.105:0/2868240966 conn(0x7ff73c0fc7d0 msgr2=0x7ff73c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:27.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:27.007+0000 7ff741651640 1 -- 172.21.15.105:0/2868240966 shutdown_connections 2024-09-16T02:05:27.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:27.007+0000 7ff741651640 1 -- 172.21.15.105:0/2868240966 wait complete. 2024-09-16T02:05:27.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:26 smithi190 ceph-mon[34762]: pgmap v426: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 487 MiB/s wr, 132 op/s 2024-09-16T02:05:27.365 INFO:tasks.workunit.client.0.smithi105.stdout: 2024-09-16T02:05:27.365 INFO:tasks.workunit.client.0.smithi105.stdout: Children see throughput for 1 readers = 7805453.00 kB/sec 2024-09-16T02:05:27.365 INFO:tasks.workunit.client.0.smithi105.stdout: Parent sees throughput for 1 readers = 7803914.55 kB/sec 2024-09-16T02:05:27.365 INFO:tasks.workunit.client.0.smithi105.stdout: Min throughput per process = 7805453.00 kB/sec 2024-09-16T02:05:27.365 INFO:tasks.workunit.client.0.smithi105.stdout: Max throughput per process = 7805453.00 kB/sec 2024-09-16T02:05:27.366 INFO:tasks.workunit.client.0.smithi105.stdout: Avg throughput per process = 7805453.00 kB/sec 2024-09-16T02:05:27.366 INFO:tasks.workunit.client.0.smithi105.stdout: Min xfer = 10485760.00 kB 2024-09-16T02:05:27.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:27 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2868240966' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:05:27.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:27 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/2868240966' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:05:28.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:27 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/2868240966' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:05:28.166 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-16T02:05:28.410 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:28.797 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:28 smithi105 ceph-mon[31197]: pgmap v427: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 231 MiB/s wr, 67 op/s 2024-09-16T02:05:28.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:28 smithi137 ceph-mon[34099]: pgmap v427: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 231 MiB/s wr, 67 op/s 2024-09-16T02:05:28.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.915+0000 7ff5c2e0c640 1 -- 172.21.15.105:0/862272922 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5bc102780 msgr2=0x7ff5bc1067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:28.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.915+0000 7ff5c2e0c640 1 --2- 172.21.15.105:0/862272922 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5bc102780 0x7ff5bc1067d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8009900 tx=0x7ff5a802a410 comp rx=0 tx=0).stop 2024-09-16T02:05:28.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.915+0000 7ff5c2e0c640 1 -- 172.21.15.105:0/862272922 shutdown_connections 2024-09-16T02:05:28.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.915+0000 7ff5c2e0c640 1 --2- 172.21.15.105:0/862272922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5bc106f00 0x7ff5bc10ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:28.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.915+0000 7ff5c2e0c640 1 --2- 172.21.15.105:0/862272922 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5bc102780 0x7ff5bc1067d0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:28.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.915+0000 7ff5c2e0c640 1 --2- 172.21.15.105:0/862272922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5bc101dd0 0x7ff5bc1021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:28.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.915+0000 7ff5c2e0c640 1 -- 172.21.15.105:0/862272922 >> 172.21.15.105:0/862272922 conn(0x7ff5bc0fd890 msgr2=0x7ff5bc0ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:28.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.915+0000 7ff5c2e0c640 1 -- 172.21.15.105:0/862272922 shutdown_connections 2024-09-16T02:05:28.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.915+0000 7ff5c2e0c640 1 -- 172.21.15.105:0/862272922 wait complete. 2024-09-16T02:05:28.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.916+0000 7ff5c2e0c640 1 Processor -- start 2024-09-16T02:05:28.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.916+0000 7ff5c2e0c640 1 -- start start 2024-09-16T02:05:28.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.916+0000 7ff5c2e0c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5bc101dd0 0x7ff5bc198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:28.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.916+0000 7ff5c2e0c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5bc102780 0x7ff5bc198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:28.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.916+0000 7ff5c2e0c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5bc106f00 0x7ff5bc19cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:28.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.916+0000 7ff5c2e0c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc111eb0 con 0x7ff5bc101dd0 2024-09-16T02:05:28.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.916+0000 7ff5c2e0c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc111d30 con 0x7ff5bc102780 2024-09-16T02:05:28.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.916+0000 7ff5c2e0c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc112030 con 0x7ff5bc106f00 2024-09-16T02:05:28.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.916+0000 7ff5c0ba8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5bc101dd0 0x7ff5bc198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:28.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.916+0000 7ff5c0ba8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5bc101dd0 0x7ff5bc198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45876/0 (socket says 172.21.15.105:45876) 2024-09-16T02:05:28.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.916+0000 7ff5c0ba8640 1 -- 172.21.15.105:0/1056152935 learned_addr learned my addr 172.21.15.105:0/1056152935 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:28.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.917+0000 7ff5c13a9640 1 --2- 172.21.15.105:0/1056152935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5bc106f00 0x7ff5bc19cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:28.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.917+0000 7ff5bbfff640 1 --2- 172.21.15.105:0/1056152935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5bc102780 0x7ff5bc198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:28.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.917+0000 7ff5c0ba8640 1 -- 172.21.15.105:0/1056152935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5bc106f00 msgr2=0x7ff5bc19cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:28.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.917+0000 7ff5c0ba8640 1 --2- 172.21.15.105:0/1056152935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5bc106f00 0x7ff5bc19cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:28.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.917+0000 7ff5c0ba8640 1 -- 172.21.15.105:0/1056152935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5bc102780 msgr2=0x7ff5bc198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:28.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.917+0000 7ff5c0ba8640 1 --2- 172.21.15.105:0/1056152935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5bc102780 0x7ff5bc198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:28.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.917+0000 7ff5c0ba8640 1 -- 172.21.15.105:0/1056152935 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5bc19d5c0 con 0x7ff5bc101dd0 2024-09-16T02:05:28.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.917+0000 7ff5bbfff640 1 --2- 172.21.15.105:0/1056152935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5bc102780 0x7ff5bc198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:05:28.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.917+0000 7ff5c0ba8640 1 --2- 172.21.15.105:0/1056152935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5bc101dd0 0x7ff5bc198610 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff5b000b4a0 tx=0x7ff5b000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:28.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.917+0000 7ff5b9ffb640 1 -- 172.21.15.105:0/1056152935 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0014070 con 0x7ff5bc101dd0 2024-09-16T02:05:28.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.917+0000 7ff5c2e0c640 1 -- 172.21.15.105:0/1056152935 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5bc19d8b0 con 0x7ff5bc101dd0 2024-09-16T02:05:28.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.917+0000 7ff5b9ffb640 1 -- 172.21.15.105:0/1056152935 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5b0010ac0 con 0x7ff5bc101dd0 2024-09-16T02:05:28.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.917+0000 7ff5b9ffb640 1 -- 172.21.15.105:0/1056152935 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0017c30 con 0x7ff5bc101dd0 2024-09-16T02:05:28.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.917+0000 7ff5c2e0c640 1 -- 172.21.15.105:0/1056152935 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5bc1a5190 con 0x7ff5bc101dd0 2024-09-16T02:05:28.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.919+0000 7ff5b9ffb640 1 -- 172.21.15.105:0/1056152935 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7ff5b0017070 con 0x7ff5bc101dd0 2024-09-16T02:05:28.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.920+0000 7ff5c51e5740 1 Processor -- start 2024-09-16T02:05:28.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.920+0000 7ff5c51e5740 1 -- start start 2024-09-16T02:05:28.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.920+0000 7ff5b9ffb640 1 --2- 172.21.15.105:0/1056152935 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff5ac0a3240 0x7ff5ac0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:28.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.920+0000 7ff5c51e5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a43da5b40 0x557a43da4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:28.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.920+0000 7ff5c13a9640 1 --2- 172.21.15.105:0/1056152935 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff5ac0a3240 0x7ff5ac0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:28.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.920+0000 7ff5b9ffb640 1 -- 172.21.15.105:0/1056152935 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7ff5b00cee10 con 0x7ff5bc101dd0 2024-09-16T02:05:28.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.920+0000 7ff5c51e5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a43da45d0 0x557a43da4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:28.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.920+0000 7ff5c51e5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a43daf370 0x557a43db1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:28.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.920+0000 7ff5c51e5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x557a43d0a2f0 con 0x557a43da5b40 2024-09-16T02:05:28.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.920+0000 7ff5c51e5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557a43cb1cf0 con 0x557a43daf370 2024-09-16T02:05:28.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.920+0000 7ff5c51e5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x557a43d062b0 con 0x557a43da45d0 2024-09-16T02:05:28.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.920+0000 7ff5c0ba8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a43da5b40 0x557a43da4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:28.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.920+0000 7ff5c0ba8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a43da5b40 0x557a43da4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45878/0 (socket says 172.21.15.105:45878) 2024-09-16T02:05:28.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.920+0000 7ff5c0ba8640 1 -- 172.21.15.105:0/3373747880 learned_addr learned my addr 172.21.15.105:0/3373747880 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:28.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.920+0000 7ff5bbfff640 1 --2- 172.21.15.105:0/3373747880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a43da45d0 0x557a43da4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:28.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.920+0000 7ff5c13a9640 1 --2- 172.21.15.105:0/3373747880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a43daf370 0x557a43db1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:28.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.921+0000 7ff5c0ba8640 1 -- 172.21.15.105:0/3373747880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a43da45d0 msgr2=0x557a43da4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:28.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.921+0000 7ff5c0ba8640 1 --2- 172.21.15.105:0/3373747880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a43da45d0 0x557a43da4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:28.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.921+0000 7ff5c0ba8640 1 -- 172.21.15.105:0/3373747880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a43daf370 msgr2=0x557a43db1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:28.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.921+0000 7ff5c0ba8640 1 --2- 172.21.15.105:0/3373747880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a43daf370 0x557a43db1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:28.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.921+0000 7ff5c0ba8640 1 -- 172.21.15.105:0/3373747880 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a43c963a0 con 0x557a43da5b40 2024-09-16T02:05:28.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.921+0000 7ff5c13a9640 1 --2- 172.21.15.105:0/1056152935 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff5ac0a3240 0x7ff5ac0a5700 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4009eb0 tx=0x7ff5b400c7f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:28.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.921+0000 7ff5c0ba8640 1 --2- 172.21.15.105:0/3373747880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a43da5b40 0x557a43da4090 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff5b009b0d0 tx=0x7ff5b0097780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e25f9f16616aca server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:28.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.921+0000 7ff5a6ffd640 1 -- 172.21.15.105:0/3373747880 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b00b4070 con 0x557a43da5b40 2024-09-16T02:05:28.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.921+0000 7ff5a6ffd640 1 -- 172.21.15.105:0/3373747880 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5b009f780 con 0x557a43da5b40 2024-09-16T02:05:28.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.921+0000 7ff5a6ffd640 1 -- 172.21.15.105:0/3373747880 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b001e070 con 0x557a43da5b40 2024-09-16T02:05:28.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.922+0000 7ff5c51e5740 1 -- 172.21.15.105:0/3373747880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a43da5b40 msgr2=0x557a43da4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:28.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.922+0000 7ff5c51e5740 1 --2- 172.21.15.105:0/3373747880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a43da5b40 0x557a43da4090 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff5b009b0d0 tx=0x7ff5b0097780 comp rx=0 tx=0).stop 2024-09-16T02:05:28.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.922+0000 7ff5c51e5740 1 -- 172.21.15.105:0/3373747880 shutdown_connections 2024-09-16T02:05:28.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.922+0000 7ff5c51e5740 1 --2- 172.21.15.105:0/3373747880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a43da45d0 0x557a43da4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:28.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.922+0000 7ff5c51e5740 1 --2- 172.21.15.105:0/3373747880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a43daf370 0x557a43db1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:28.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.922+0000 7ff5c51e5740 1 --2- 172.21.15.105:0/3373747880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a43da5b40 0x557a43da4090 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:28.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.922+0000 7ff5c51e5740 1 -- 172.21.15.105:0/3373747880 >> 172.21.15.105:0/3373747880 conn(0x557a43d6d310 msgr2=0x557a43da0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:28.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.922+0000 7ff5c51e5740 1 -- 172.21.15.105:0/3373747880 shutdown_connections 2024-09-16T02:05:28.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.922+0000 7ff5c51e5740 1 -- 172.21.15.105:0/3373747880 wait complete. 2024-09-16T02:05:28.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.922+0000 7ff5c51e5740 1 -- start start 2024-09-16T02:05:28.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.922+0000 7ff5c51e5740 1 Processor -- start 2024-09-16T02:05:28.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.923+0000 7ff5c51e5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a43da45d0 0x557a43daed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:28.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.923+0000 7ff5c51e5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a43da5b40 0x557a43dac090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:28.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.923+0000 7ff5c51e5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a43daf370 0x557a43dac6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:28.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.923+0000 7ff5c51e5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x557a43db2070 con 0x557a43da45d0 2024-09-16T02:05:28.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.923+0000 7ff5c51e5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557a43cc5060 con 0x557a43da5b40 2024-09-16T02:05:28.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.923+0000 7ff5c51e5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x557a43db21f0 con 0x557a43daf370 2024-09-16T02:05:28.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.923+0000 7ff5c51e5740 10 client.0 _ll_register_callbacks cb 0x557a43a188c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:28.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.923+0000 7ff5c0ba8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a43da45d0 0x557a43daed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:28.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.923+0000 7ff5c0ba8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a43da45d0 0x557a43daed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45890/0 (socket says 172.21.15.105:45890) 2024-09-16T02:05:28.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.923+0000 7ff5c0ba8640 1 -- 172.21.15.105:0/3755068507 learned_addr learned my addr 172.21.15.105:0/3755068507 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:28.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.923+0000 7ff5bbfff640 1 --2- 172.21.15.105:0/3755068507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a43da5b40 0x557a43dac090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:28.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.923+0000 7ff5c13a9640 1 --2- 172.21.15.105:0/3755068507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a43daf370 0x557a43dac6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:28.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.923+0000 7ff5c0ba8640 1 -- 172.21.15.105:0/3755068507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a43daf370 msgr2=0x557a43dac6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:28.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.923+0000 7ff5c0ba8640 1 --2- 172.21.15.105:0/3755068507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a43daf370 0x557a43dac6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:28.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.923+0000 7ff5c0ba8640 1 -- 172.21.15.105:0/3755068507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a43da5b40 msgr2=0x557a43dac090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:28.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.923+0000 7ff5c0ba8640 1 --2- 172.21.15.105:0/3755068507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a43da5b40 0x557a43dac090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:28.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.923+0000 7ff5c0ba8640 1 -- 172.21.15.105:0/3755068507 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a43a7b3c0 con 0x557a43da45d0 2024-09-16T02:05:28.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.923+0000 7ff5bbfff640 1 --2- 172.21.15.105:0/3755068507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a43da5b40 0x557a43dac090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:05:28.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.923+0000 7ff5c13a9640 1 --2- 172.21.15.105:0/3755068507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a43daf370 0x557a43dac6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:05:28.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.923+0000 7ff5c0ba8640 1 --2- 172.21.15.105:0/3755068507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a43da45d0 0x557a43daed90 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0097830 tx=0x7ff5b00a40c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6582e481d2c6024d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:28.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.923+0000 7ff59bfff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:05:28.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.923+0000 7ff59bfff640 1 -- 172.21.15.105:0/3755068507 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b00b9060 con 0x557a43da45d0 2024-09-16T02:05:28.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.923+0000 7ff59bfff640 1 -- 172.21.15.105:0/3755068507 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7ff5b00b41e0 con 0x557a43da45d0 2024-09-16T02:05:28.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.923+0000 7ff5c51e5740 1 -- 172.21.15.105:0/3755068507 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5b009f780 con 0x557a43da45d0 2024-09-16T02:05:28.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.923+0000 7ff59bfff640 1 -- 172.21.15.105:0/3755068507 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff5b001e070 con 0x557a43da45d0 2024-09-16T02:05:28.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.923+0000 7ff5c51e5740 10 client.15810 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:28.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.923+0000 7ff5c51e5740 1 -- 172.21.15.105:0/3755068507 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557a43dbdb80 con 0x557a43da45d0 2024-09-16T02:05:28.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.924+0000 7ff59bfff640 1 -- 172.21.15.105:0/3755068507 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5b009f780 con 0x557a43da45d0 2024-09-16T02:05:28.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.924+0000 7ff59bfff640 1 -- 172.21.15.105:0/3755068507 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff5b009a920 con 0x557a43da45d0 2024-09-16T02:05:28.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.924+0000 7ff5c51e5740 10 client.15810 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:28.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.924+0000 7ff5c51e5740 10 client.15810 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T02:05:28.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.924+0000 7ff5c51e5740 20 client.15810 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:28.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.924+0000 7ff5c51e5740 1 --2- 172.21.15.105:0/3755068507 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x557a43dbeed0 0x557a43dc12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:28.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.924+0000 7ff5c51e5740 4 client.15810 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:05:28.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.924+0000 7ff5c51e5740 1 -- 172.21.15.105:0/3755068507 --> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a43d17860 con 0x557a43dbeed0 2024-09-16T02:05:28.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.924+0000 7ff5bbfff640 1 --2- 172.21.15.105:0/3755068507 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x557a43dbeed0 0x557a43dc12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:28.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.925+0000 7ff5bbfff640 1 --2- 172.21.15.105:0/3755068507 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x557a43dbeed0 0x557a43dc12c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=37a30f0185d0d74d server_cookie=4e1b55e310db5355 in_seq=0 out_seq=0 2024-09-16T02:05:28.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.925+0000 7ff59bfff640 10 client.15810 ms_handle_connect on v2:172.21.15.190:6834/1851042281 2024-09-16T02:05:28.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.925+0000 7ff59bfff640 1 -- 172.21.15.105:0/3755068507 <== mds.4 v2:172.21.15.190:6834/1851042281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557a43d17860 con 0x557a43dbeed0 2024-09-16T02:05:28.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.925+0000 7ff59bfff640 10 client.15810 handle_command_reply: tid=0 2024-09-16T02:05:28.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.925+0000 7ff5c51e5740 1 client.15810 shutdown 2024-09-16T02:05:28.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.926+0000 7ff5c51e5740 1 -- 172.21.15.105:0/3755068507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a43da45d0 msgr2=0x557a43daed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:28.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.926+0000 7ff5c51e5740 1 --2- 172.21.15.105:0/3755068507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a43da45d0 0x557a43daed90 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0097830 tx=0x7ff5b00a40c0 comp rx=0 tx=0).stop 2024-09-16T02:05:28.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.926+0000 7ff5c51e5740 1 -- 172.21.15.105:0/3755068507 shutdown_connections 2024-09-16T02:05:28.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.926+0000 7ff5c51e5740 1 --2- 172.21.15.105:0/3755068507 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x557a43dbeed0 0x557a43dc12c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:28.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.926+0000 7ff5c51e5740 1 --2- 172.21.15.105:0/3755068507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a43daf370 0x557a43dac6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:28.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.926+0000 7ff5c51e5740 1 --2- 172.21.15.105:0/3755068507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a43da5b40 0x557a43dac090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:28.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.926+0000 7ff59bfff640 0 client.15810 ms_handle_reset on v2:172.21.15.190:6834/1851042281 2024-09-16T02:05:28.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.926+0000 7ff5c51e5740 1 --2- 172.21.15.105:0/3755068507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a43da45d0 0x557a43daed90 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:28.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.926+0000 7ff5c51e5740 1 -- 172.21.15.105:0/3755068507 >> 172.21.15.105:0/3755068507 conn(0x557a43d6d310 msgr2=0x557a43da6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:28.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.926+0000 7ff5c51e5740 1 -- 172.21.15.105:0/3755068507 shutdown_connections 2024-09-16T02:05:28.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.926+0000 7ff5c51e5740 1 -- 172.21.15.105:0/3755068507 wait complete. 2024-09-16T02:05:28.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.926+0000 7ff5c51e5740 20 client.15810 trim_cache size 0 max 16384 2024-09-16T02:05:28.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.939+0000 7ff5c51e5740 1 Processor -- start 2024-09-16T02:05:28.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.939+0000 7ff5c51e5740 1 -- start start 2024-09-16T02:05:28.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.939+0000 7ff5c51e5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a43da45d0 0x557a43dbb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:28.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.939+0000 7ff5c51e5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a43da5b40 0x557a43dae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:28.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.939+0000 7ff5c51e5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a43daf370 0x557a43dae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:28.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.939+0000 7ff5c51e5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc1ad7e0 con 0x557a43da5b40 2024-09-16T02:05:28.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.939+0000 7ff5c51e5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc1af320 con 0x557a43da45d0 2024-09-16T02:05:28.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.939+0000 7ff5c51e5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc1adeb0 con 0x557a43daf370 2024-09-16T02:05:28.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.939+0000 7ff5bbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a43da5b40 0x557a43dae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:28.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.939+0000 7ff5bbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a43da5b40 0x557a43dae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45904/0 (socket says 172.21.15.105:45904) 2024-09-16T02:05:28.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.939+0000 7ff5bbfff640 1 -- 172.21.15.105:0/3832182083 learned_addr learned my addr 172.21.15.105:0/3832182083 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:28.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.939+0000 7ff5c0ba8640 1 --2- 172.21.15.105:0/3832182083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a43da45d0 0x557a43dbb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:28.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.939+0000 7ff5c13a9640 1 --2- 172.21.15.105:0/3832182083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a43daf370 0x557a43dae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:28.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.939+0000 7ff5bbfff640 1 -- 172.21.15.105:0/3832182083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a43daf370 msgr2=0x557a43dae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:28.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.940+0000 7ff5bbfff640 1 --2- 172.21.15.105:0/3832182083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a43daf370 0x557a43dae540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:28.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.940+0000 7ff5bbfff640 1 -- 172.21.15.105:0/3832182083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a43da45d0 msgr2=0x557a43dbb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:28.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.940+0000 7ff5bbfff640 1 --2- 172.21.15.105:0/3832182083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a43da45d0 0x557a43dbb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:28.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.940+0000 7ff5bbfff640 1 -- 172.21.15.105:0/3832182083 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a43d9fbe0 con 0x557a43da5b40 2024-09-16T02:05:28.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.940+0000 7ff5bbfff640 1 --2- 172.21.15.105:0/3832182083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a43da5b40 0x557a43dae000 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8032fd0 tx=0x7ff5a8005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1951972111ebee29 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:28.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.940+0000 7ff5a67fc640 1 -- 172.21.15.105:0/3832182083 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8046020 con 0x557a43da5b40 2024-09-16T02:05:28.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.940+0000 7ff5a67fc640 1 -- 172.21.15.105:0/3832182083 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5a8032900 con 0x557a43da5b40 2024-09-16T02:05:28.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.940+0000 7ff5a67fc640 1 -- 172.21.15.105:0/3832182083 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8035040 con 0x557a43da5b40 2024-09-16T02:05:28.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.941+0000 7ff5c51e5740 1 -- 172.21.15.105:0/3832182083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a43da5b40 msgr2=0x557a43dae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:28.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.941+0000 7ff5c51e5740 1 --2- 172.21.15.105:0/3832182083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a43da5b40 0x557a43dae000 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8032fd0 tx=0x7ff5a8005a20 comp rx=0 tx=0).stop 2024-09-16T02:05:28.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.941+0000 7ff5c51e5740 1 -- 172.21.15.105:0/3832182083 shutdown_connections 2024-09-16T02:05:28.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.941+0000 7ff5c51e5740 1 --2- 172.21.15.105:0/3832182083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a43daf370 0x557a43dae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:28.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.941+0000 7ff5c51e5740 1 --2- 172.21.15.105:0/3832182083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a43da5b40 0x557a43dae000 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:28.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.941+0000 7ff5c51e5740 1 --2- 172.21.15.105:0/3832182083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a43da45d0 0x557a43dbb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:28.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.941+0000 7ff5c51e5740 1 -- 172.21.15.105:0/3832182083 >> 172.21.15.105:0/3832182083 conn(0x557a43d6d310 msgr2=0x557a43dad8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:28.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.941+0000 7ff5c51e5740 1 -- 172.21.15.105:0/3832182083 shutdown_connections 2024-09-16T02:05:28.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.941+0000 7ff5c51e5740 1 -- 172.21.15.105:0/3832182083 wait complete. 2024-09-16T02:05:28.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.941+0000 7ff5c51e5740 1 -- start start 2024-09-16T02:05:28.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.941+0000 7ff5c51e5740 1 Processor -- start 2024-09-16T02:05:28.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.942+0000 7ff5c51e5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a43da45d0 0x557a43dc4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:28.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.942+0000 7ff5c51e5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a43da5b40 0x557a43dc1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:28.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.942+0000 7ff5c51e5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a43daf370 0x557a43dc2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:28.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.942+0000 7ff5c51e5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc1aec50 con 0x557a43da45d0 2024-09-16T02:05:28.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.942+0000 7ff5c51e5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc1ae580 con 0x557a43daf370 2024-09-16T02:05:28.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.942+0000 7ff5c51e5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc1ab7c0 con 0x557a43da5b40 2024-09-16T02:05:28.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.942+0000 7ff5c51e5740 10 client.0 _ll_register_callbacks cb 0x7ff5b00b2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:28.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.942+0000 7ff5c0ba8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a43da45d0 0x557a43dc4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:28.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.942+0000 7ff5c0ba8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a43da45d0 0x557a43dc4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45920/0 (socket says 172.21.15.105:45920) 2024-09-16T02:05:28.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.942+0000 7ff5c0ba8640 1 -- 172.21.15.105:0/1053138849 learned_addr learned my addr 172.21.15.105:0/1053138849 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:28.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.942+0000 7ff5bbfff640 1 --2- 172.21.15.105:0/1053138849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a43da5b40 0x557a43dc1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:28.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.942+0000 7ff5c13a9640 1 --2- 172.21.15.105:0/1053138849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a43daf370 0x557a43dc2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:28.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.942+0000 7ff5c0ba8640 1 -- 172.21.15.105:0/1053138849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a43da5b40 msgr2=0x557a43dc1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:28.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.942+0000 7ff5c0ba8640 1 --2- 172.21.15.105:0/1053138849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a43da5b40 0x557a43dc1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:28.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.942+0000 7ff5c0ba8640 1 -- 172.21.15.105:0/1053138849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a43daf370 msgr2=0x557a43dc2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:28.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.942+0000 7ff5c0ba8640 1 --2- 172.21.15.105:0/1053138849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a43daf370 0x557a43dc2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:28.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.942+0000 7ff5c0ba8640 1 -- 172.21.15.105:0/1053138849 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a43dc2a30 con 0x557a43da45d0 2024-09-16T02:05:28.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.942+0000 7ff5bbfff640 1 --2- 172.21.15.105:0/1053138849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a43da5b40 0x557a43dc1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:05:28.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.942+0000 7ff5c13a9640 1 --2- 172.21.15.105:0/1053138849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a43daf370 0x557a43dc2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:05:28.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.942+0000 7ff5c0ba8640 1 --2- 172.21.15.105:0/1053138849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a43da45d0 0x557a43dc4970 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff5b009b360 tx=0x7ff5b00663a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f76d80b68e9be58 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:28.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.942+0000 7ff59bfff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:05:28.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.942+0000 7ff59bfff640 1 -- 172.21.15.105:0/1053138849 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b00b9040 con 0x557a43da45d0 2024-09-16T02:05:28.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.942+0000 7ff59bfff640 1 -- 172.21.15.105:0/1053138849 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7ff5b00b41e0 con 0x557a43da45d0 2024-09-16T02:05:28.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.942+0000 7ff5c51e5740 1 -- 172.21.15.105:0/1053138849 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5a8032900 con 0x557a43da45d0 2024-09-16T02:05:28.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.943+0000 7ff59bfff640 1 -- 172.21.15.105:0/1053138849 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff5b00b4040 con 0x557a43da45d0 2024-09-16T02:05:28.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.943+0000 7ff5c51e5740 10 client.15816 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:28.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.943+0000 7ff5c51e5740 1 -- 172.21.15.105:0/1053138849 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557a43e29750 con 0x557a43da45d0 2024-09-16T02:05:28.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.943+0000 7ff59bfff640 1 -- 172.21.15.105:0/1053138849 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5a8032900 con 0x557a43da45d0 2024-09-16T02:05:28.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.943+0000 7ff59bfff640 1 -- 172.21.15.105:0/1053138849 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff5b009a820 con 0x557a43da45d0 2024-09-16T02:05:28.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.943+0000 7ff5c51e5740 10 client.15816 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:28.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.943+0000 7ff5c51e5740 10 client.15816 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T02:05:28.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.943+0000 7ff5c51e5740 20 client.15816 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:28.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.943+0000 7ff5c51e5740 1 --2- 172.21.15.105:0/1053138849 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x557a43e29c20 0x557a43e2c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:28.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.943+0000 7ff5c51e5740 4 client.15816 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T02:05:28.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.943+0000 7ff5c51e5740 1 -- 172.21.15.105:0/1053138849 --> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557a43d11d70 con 0x557a43e29c20 2024-09-16T02:05:28.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.943+0000 7ff5bbfff640 1 --2- 172.21.15.105:0/1053138849 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x557a43e29c20 0x557a43e2c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:28.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.944+0000 7ff5bbfff640 1 --2- 172.21.15.105:0/1053138849 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x557a43e29c20 0x557a43e2c010 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=33c7a394206abd31 server_cookie=94994012601860d1 in_seq=0 out_seq=0 2024-09-16T02:05:28.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.944+0000 7ff59bfff640 10 client.15816 ms_handle_connect on v2:172.21.15.190:6834/1851042281 2024-09-16T02:05:28.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.944+0000 7ff59bfff640 1 -- 172.21.15.105:0/1053138849 <== mds.4 v2:172.21.15.190:6834/1851042281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557a43d11d70 con 0x557a43e29c20 2024-09-16T02:05:28.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.944+0000 7ff59bfff640 10 client.15816 handle_command_reply: tid=0 2024-09-16T02:05:28.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.944+0000 7ff5c51e5740 1 client.15816 shutdown 2024-09-16T02:05:28.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.944+0000 7ff5c51e5740 1 -- 172.21.15.105:0/1053138849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a43da45d0 msgr2=0x557a43dc4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:28.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.944+0000 7ff5c51e5740 1 --2- 172.21.15.105:0/1053138849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a43da45d0 0x557a43dc4970 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff5b009b360 tx=0x7ff5b00663a0 comp rx=0 tx=0).stop 2024-09-16T02:05:28.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.944+0000 7ff5c51e5740 1 -- 172.21.15.105:0/1053138849 shutdown_connections 2024-09-16T02:05:28.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.944+0000 7ff5c51e5740 1 --2- 172.21.15.105:0/1053138849 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x557a43e29c20 0x557a43e2c010 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:28.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.944+0000 7ff59bfff640 0 client.15816 ms_handle_reset on v2:172.21.15.190:6834/1851042281 2024-09-16T02:05:28.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.944+0000 7ff5c51e5740 1 --2- 172.21.15.105:0/1053138849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a43da5b40 0x557a43dc1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:28.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.944+0000 7ff5c51e5740 1 --2- 172.21.15.105:0/1053138849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a43daf370 0x557a43dc2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:28.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.944+0000 7ff5c51e5740 1 --2- 172.21.15.105:0/1053138849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a43da45d0 0x557a43dc4970 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:28.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.944+0000 7ff5c51e5740 1 -- 172.21.15.105:0/1053138849 >> 172.21.15.105:0/1053138849 conn(0x557a43d6d310 msgr2=0x557a43dae110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:28.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.944+0000 7ff5c51e5740 1 -- 172.21.15.105:0/1053138849 shutdown_connections 2024-09-16T02:05:28.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.944+0000 7ff5c51e5740 1 -- 172.21.15.105:0/1053138849 wait complete. 2024-09-16T02:05:28.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.944+0000 7ff5c51e5740 20 client.15816 trim_cache size 0 max 16384 2024-09-16T02:05:28.954 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-09-16T02:05:28.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.945+0000 7ff5c2e0c640 1 -- 172.21.15.105:0/1056152935 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff5ac0a3240 msgr2=0x7ff5ac0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:28.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.945+0000 7ff5c2e0c640 1 --2- 172.21.15.105:0/1056152935 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff5ac0a3240 0x7ff5ac0a5700 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4009eb0 tx=0x7ff5b400c7f0 comp rx=0 tx=0).stop 2024-09-16T02:05:28.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.945+0000 7ff5c2e0c640 1 -- 172.21.15.105:0/1056152935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5bc101dd0 msgr2=0x7ff5bc198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:28.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.945+0000 7ff5c2e0c640 1 --2- 172.21.15.105:0/1056152935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5bc101dd0 0x7ff5bc198610 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff5b000b4a0 tx=0x7ff5b000b960 comp rx=0 tx=0).stop 2024-09-16T02:05:28.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.945+0000 7ff5c2e0c640 1 -- 172.21.15.105:0/1056152935 shutdown_connections 2024-09-16T02:05:28.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.945+0000 7ff5c2e0c640 1 --2- 172.21.15.105:0/1056152935 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff5ac0a3240 0x7ff5ac0a5700 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:28.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.945+0000 7ff5c2e0c640 1 --2- 172.21.15.105:0/1056152935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5bc106f00 0x7ff5bc19cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:28.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.945+0000 7ff5c2e0c640 1 --2- 172.21.15.105:0/1056152935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5bc102780 0x7ff5bc198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:28.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.945+0000 7ff5c2e0c640 1 --2- 172.21.15.105:0/1056152935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5bc101dd0 0x7ff5bc198610 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:28.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.945+0000 7ff5c2e0c640 1 -- 172.21.15.105:0/1056152935 >> 172.21.15.105:0/1056152935 conn(0x7ff5bc0fd890 msgr2=0x7ff5bc0fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:28.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.945+0000 7ff5c2e0c640 1 -- 172.21.15.105:0/1056152935 shutdown_connections 2024-09-16T02:05:28.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:28.945+0000 7ff5c2e0c640 1 -- 172.21.15.105:0/1056152935 wait complete. 2024-09-16T02:05:29.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:28 smithi190 ceph-mon[34762]: pgmap v427: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 231 MiB/s wr, 67 op/s 2024-09-16T02:05:29.848 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-16T02:05:30.121 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:30.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.645+0000 7fb55df6d640 1 -- 172.21.15.105:0/3535594701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb558101dd0 msgr2=0x7fb5581021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:30.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.645+0000 7fb55df6d640 1 --2- 172.21.15.105:0/3535594701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb558101dd0 0x7fb5581021b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb548009900 tx=0x7fb54802a3f0 comp rx=0 tx=0).stop 2024-09-16T02:05:30.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.645+0000 7fb55df6d640 1 -- 172.21.15.105:0/3535594701 shutdown_connections 2024-09-16T02:05:30.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.645+0000 7fb55df6d640 1 --2- 172.21.15.105:0/3535594701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb558102780 0x7fb5581067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:30.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.645+0000 7fb55df6d640 1 --2- 172.21.15.105:0/3535594701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb558106f00 0x7fb55810ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:30.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.645+0000 7fb55df6d640 1 --2- 172.21.15.105:0/3535594701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb558101dd0 0x7fb5581021b0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:30.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.645+0000 7fb55df6d640 1 -- 172.21.15.105:0/3535594701 >> 172.21.15.105:0/3535594701 conn(0x7fb5580fd8b0 msgr2=0x7fb5580ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:30.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.645+0000 7fb55df6d640 1 -- 172.21.15.105:0/3535594701 shutdown_connections 2024-09-16T02:05:30.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.645+0000 7fb55df6d640 1 -- 172.21.15.105:0/3535594701 wait complete. 2024-09-16T02:05:30.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.646+0000 7fb55df6d640 1 Processor -- start 2024-09-16T02:05:30.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.646+0000 7fb55df6d640 1 -- start start 2024-09-16T02:05:30.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.646+0000 7fb55df6d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb558101dd0 0x7fb558198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:30.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.646+0000 7fb55df6d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb558102780 0x7fb5581988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:30.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.646+0000 7fb55df6d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb558106f00 0x7fb55819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:30.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.646+0000 7fb55df6d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb558111dd0 con 0x7fb558102780 2024-09-16T02:05:30.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.646+0000 7fb55df6d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb558111c50 con 0x7fb558106f00 2024-09-16T02:05:30.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.646+0000 7fb55df6d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb558111f50 con 0x7fb558101dd0 2024-09-16T02:05:30.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.646+0000 7fb54ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb558102780 0x7fb5581988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:30.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.646+0000 7fb54ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb558102780 0x7fb5581988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45936/0 (socket says 172.21.15.105:45936) 2024-09-16T02:05:30.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.646+0000 7fb54ffff640 1 -- 172.21.15.105:0/4007370562 learned_addr learned my addr 172.21.15.105:0/4007370562 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:30.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.646+0000 7fb5577fe640 1 --2- 172.21.15.105:0/4007370562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb558101dd0 0x7fb558198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:30.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.646+0000 7fb557fff640 1 --2- 172.21.15.105:0/4007370562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb558106f00 0x7fb55819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:30.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.647+0000 7fb54ffff640 1 -- 172.21.15.105:0/4007370562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb558101dd0 msgr2=0x7fb558198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:30.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.647+0000 7fb54ffff640 1 --2- 172.21.15.105:0/4007370562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb558101dd0 0x7fb558198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:30.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.647+0000 7fb54ffff640 1 -- 172.21.15.105:0/4007370562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb558106f00 msgr2=0x7fb55819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:30.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.647+0000 7fb54ffff640 1 --2- 172.21.15.105:0/4007370562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb558106f00 0x7fb55819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:30.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.647+0000 7fb54ffff640 1 -- 172.21.15.105:0/4007370562 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb55819d310 con 0x7fb558102780 2024-09-16T02:05:30.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.647+0000 7fb557fff640 1 --2- 172.21.15.105:0/4007370562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb558106f00 0x7fb55819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:05:30.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.647+0000 7fb5577fe640 1 --2- 172.21.15.105:0/4007370562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb558101dd0 0x7fb558198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:05:30.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.647+0000 7fb54ffff640 1 --2- 172.21.15.105:0/4007370562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb558102780 0x7fb5581988a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb54000d950 tx=0x7fb54000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:30.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.647+0000 7fb5557fa640 1 -- 172.21.15.105:0/4007370562 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb540017070 con 0x7fb558102780 2024-09-16T02:05:30.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.647+0000 7fb55df6d640 1 -- 172.21.15.105:0/4007370562 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb55819d600 con 0x7fb558102780 2024-09-16T02:05:30.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.647+0000 7fb5557fa640 1 -- 172.21.15.105:0/4007370562 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb540009c30 con 0x7fb558102780 2024-09-16T02:05:30.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.647+0000 7fb5557fa640 1 -- 172.21.15.105:0/4007370562 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54000f620 con 0x7fb558102780 2024-09-16T02:05:30.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.647+0000 7fb55df6d640 1 -- 172.21.15.105:0/4007370562 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5581a4ee0 con 0x7fb558102780 2024-09-16T02:05:30.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.650+0000 7fb560346740 1 Processor -- start 2024-09-16T02:05:30.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.650+0000 7fb560346740 1 -- start start 2024-09-16T02:05:30.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.650+0000 7fb560346740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b1e306b40 0x556b1e305090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:30.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.650+0000 7fb560346740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1e3055d0 0x556b1e305a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:30.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.650+0000 7fb560346740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b1e314780 0x556b1e316b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:30.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.650+0000 7fb560346740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556b1e26b2f0 con 0x556b1e314780 2024-09-16T02:05:30.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.650+0000 7fb560346740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556b1e212cf0 con 0x556b1e306b40 2024-09-16T02:05:30.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.650+0000 7fb560346740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556b1e2672b0 con 0x556b1e3055d0 2024-09-16T02:05:30.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.651+0000 7fb5557fa640 1 -- 172.21.15.105:0/4007370562 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fb5400109a0 con 0x7fb558102780 2024-09-16T02:05:30.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.651+0000 7fb5577fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1e3055d0 0x556b1e305a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:30.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.651+0000 7fb5577fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1e3055d0 0x556b1e305a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.105:60550/0 (socket says 172.21.15.105:60550) 2024-09-16T02:05:30.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.651+0000 7fb5577fe640 1 -- 172.21.15.105:0/3370450388 learned_addr learned my addr 172.21.15.105:0/3370450388 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:30.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.651+0000 7fb557fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b1e306b40 0x556b1e305090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:30.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.651+0000 7fb54ffff640 1 --2- 172.21.15.105:0/3370450388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b1e314780 0x556b1e316b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:30.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.651+0000 7fb5577fe640 1 -- 172.21.15.105:0/3370450388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b1e306b40 msgr2=0x556b1e305090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:30.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.651+0000 7fb5577fe640 1 --2- 172.21.15.105:0/3370450388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b1e306b40 0x556b1e305090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:30.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.651+0000 7fb5577fe640 1 -- 172.21.15.105:0/3370450388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b1e314780 msgr2=0x556b1e316b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:30.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.651+0000 7fb5577fe640 1 --2- 172.21.15.105:0/3370450388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b1e314780 0x556b1e316b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:30.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.651+0000 7fb5577fe640 1 -- 172.21.15.105:0/3370450388 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b1e1f73a0 con 0x556b1e3055d0 2024-09-16T02:05:30.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.651+0000 7fb557fff640 1 --2- 172.21.15.105:0/3370450388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b1e306b40 0x556b1e305090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:05:30.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.652+0000 7fb5577fe640 1 --2- 172.21.15.105:0/3370450388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1e3055d0 0x556b1e305a10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb548009900 tx=0x7fb5480314a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ef457c13b8b2ce7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:30.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.652+0000 7fb5557fa640 1 --2- 172.21.15.105:0/4007370562 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb5440a3240 0x7fb5440a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:30.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.652+0000 7fb557fff640 1 --2- 172.21.15.105:0/4007370562 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb5440a3240 0x7fb5440a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:30.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.652+0000 7fb54dffb640 1 -- 172.21.15.105:0/3370450388 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb548040070 con 0x556b1e3055d0 2024-09-16T02:05:30.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.652+0000 7fb5557fa640 1 -- 172.21.15.105:0/4007370562 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fb5400cf5b0 con 0x7fb558102780 2024-09-16T02:05:30.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.652+0000 7fb54dffb640 1 -- 172.21.15.105:0/3370450388 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb548031890 con 0x556b1e3055d0 2024-09-16T02:05:30.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.652+0000 7fb54dffb640 1 -- 172.21.15.105:0/3370450388 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb548035040 con 0x556b1e3055d0 2024-09-16T02:05:30.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.653+0000 7fb560346740 1 -- 172.21.15.105:0/3370450388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1e3055d0 msgr2=0x556b1e305a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:30.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.653+0000 7fb560346740 1 --2- 172.21.15.105:0/3370450388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1e3055d0 0x556b1e305a10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb548009900 tx=0x7fb5480314a0 comp rx=0 tx=0).stop 2024-09-16T02:05:30.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.653+0000 7fb557fff640 1 --2- 172.21.15.105:0/4007370562 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb5440a3240 0x7fb5440a5700 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb550005fd0 tx=0x7fb550009780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:30.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.653+0000 7fb560346740 1 -- 172.21.15.105:0/3370450388 shutdown_connections 2024-09-16T02:05:30.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.653+0000 7fb560346740 1 --2- 172.21.15.105:0/3370450388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1e3055d0 0x556b1e305a10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:30.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.653+0000 7fb560346740 1 --2- 172.21.15.105:0/3370450388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b1e314780 0x556b1e316b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:30.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.653+0000 7fb560346740 1 --2- 172.21.15.105:0/3370450388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b1e306b40 0x556b1e305090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:30.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.653+0000 7fb560346740 1 -- 172.21.15.105:0/3370450388 >> 172.21.15.105:0/3370450388 conn(0x556b1e2ce310 msgr2=0x556b1e301c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:30.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.653+0000 7fb560346740 1 -- 172.21.15.105:0/3370450388 shutdown_connections 2024-09-16T02:05:30.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.653+0000 7fb560346740 1 -- 172.21.15.105:0/3370450388 wait complete. 2024-09-16T02:05:30.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.653+0000 7fb560346740 1 -- start start 2024-09-16T02:05:30.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.653+0000 7fb560346740 1 Processor -- start 2024-09-16T02:05:30.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.654+0000 7fb560346740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1e3055d0 0x556b1e3141a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:30.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.654+0000 7fb560346740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b1e306b40 0x556b1e311450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:30.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.654+0000 7fb560346740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b1e314780 0x556b1e311a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:30.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.654+0000 7fb560346740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556b1e317420 con 0x556b1e306b40 2024-09-16T02:05:30.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.654+0000 7fb560346740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556b1e226060 con 0x556b1e314780 2024-09-16T02:05:30.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.654+0000 7fb560346740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556b1e3175a0 con 0x556b1e3055d0 2024-09-16T02:05:30.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.654+0000 7fb560346740 10 client.0 _ll_register_callbacks cb 0x556b1df798c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:30.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.654+0000 7fb54ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b1e306b40 0x556b1e311450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:30.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.654+0000 7fb54ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b1e306b40 0x556b1e311450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45960/0 (socket says 172.21.15.105:45960) 2024-09-16T02:05:30.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.654+0000 7fb54ffff640 1 -- 172.21.15.105:0/2314724308 learned_addr learned my addr 172.21.15.105:0/2314724308 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:30.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.654+0000 7fb557fff640 1 --2- 172.21.15.105:0/2314724308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b1e314780 0x556b1e311a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:30.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.654+0000 7fb5577fe640 1 --2- 172.21.15.105:0/2314724308 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1e3055d0 0x556b1e3141a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:30.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.654+0000 7fb54ffff640 1 -- 172.21.15.105:0/2314724308 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1e3055d0 msgr2=0x556b1e3141a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:30.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.654+0000 7fb54ffff640 1 --2- 172.21.15.105:0/2314724308 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1e3055d0 0x556b1e3141a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:30.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.654+0000 7fb54ffff640 1 -- 172.21.15.105:0/2314724308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b1e314780 msgr2=0x556b1e311a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:30.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.654+0000 7fb54ffff640 1 --2- 172.21.15.105:0/2314724308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b1e314780 0x556b1e311a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:30.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.654+0000 7fb54ffff640 1 -- 172.21.15.105:0/2314724308 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b1dfdc3c0 con 0x556b1e306b40 2024-09-16T02:05:30.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.654+0000 7fb557fff640 1 --2- 172.21.15.105:0/2314724308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b1e314780 0x556b1e311a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:05:30.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.655+0000 7fb54ffff640 1 --2- 172.21.15.105:0/2314724308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b1e306b40 0x556b1e311450 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb540097fd0 tx=0x7fb540005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a07b00de55028bf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:30.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.655+0000 7fb536ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:05:30.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.655+0000 7fb536ffd640 1 -- 172.21.15.105:0/2314724308 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb540020070 con 0x556b1e306b40 2024-09-16T02:05:30.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.655+0000 7fb536ffd640 1 -- 172.21.15.105:0/2314724308 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fb5400860b0 con 0x556b1e306b40 2024-09-16T02:05:30.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.655+0000 7fb560346740 1 -- 172.21.15.105:0/2314724308 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb548031890 con 0x556b1e306b40 2024-09-16T02:05:30.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.655+0000 7fb536ffd640 1 -- 172.21.15.105:0/2314724308 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb540009de0 con 0x556b1e306b40 2024-09-16T02:05:30.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.655+0000 7fb560346740 10 client.15828 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:30.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.655+0000 7fb560346740 1 -- 172.21.15.105:0/2314724308 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556b1e322f50 con 0x556b1e306b40 2024-09-16T02:05:30.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.655+0000 7fb536ffd640 1 -- 172.21.15.105:0/2314724308 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb548031890 con 0x556b1e306b40 2024-09-16T02:05:30.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.655+0000 7fb536ffd640 1 -- 172.21.15.105:0/2314724308 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb54009afc0 con 0x556b1e306b40 2024-09-16T02:05:30.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.655+0000 7fb560346740 10 client.15828 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:30.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.655+0000 7fb560346740 10 client.15828 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T02:05:30.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.655+0000 7fb560346740 20 client.15828 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:30.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.655+0000 7fb560346740 1 --2- 172.21.15.105:0/2314724308 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x556b1e324360 0x556b1e326750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:30.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.655+0000 7fb560346740 4 client.15828 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:05:30.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.655+0000 7fb560346740 1 -- 172.21.15.105:0/2314724308 --> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b1e278860 con 0x556b1e324360 2024-09-16T02:05:30.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.656+0000 7fb5577fe640 1 --2- 172.21.15.105:0/2314724308 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x556b1e324360 0x556b1e326750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:30.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.656+0000 7fb536ffd640 10 client.15828 ms_handle_connect on v2:172.21.15.190:6836/3181257549 2024-09-16T02:05:30.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.656+0000 7fb5577fe640 1 --2- 172.21.15.105:0/2314724308 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x556b1e324360 0x556b1e326750 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1c05b8c507349657 server_cookie=c31c3c2ad8f67c68 in_seq=0 out_seq=0 2024-09-16T02:05:30.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.657+0000 7fb536ffd640 1 -- 172.21.15.105:0/2314724308 <== mds.3 v2:172.21.15.190:6836/3181257549 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556b1e278860 con 0x556b1e324360 2024-09-16T02:05:30.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.657+0000 7fb536ffd640 10 client.15828 handle_command_reply: tid=0 2024-09-16T02:05:30.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.657+0000 7fb560346740 1 client.15828 shutdown 2024-09-16T02:05:30.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.657+0000 7fb560346740 1 -- 172.21.15.105:0/2314724308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b1e306b40 msgr2=0x556b1e311450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:30.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.657+0000 7fb560346740 1 --2- 172.21.15.105:0/2314724308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b1e306b40 0x556b1e311450 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb540097fd0 tx=0x7fb540005bd0 comp rx=0 tx=0).stop 2024-09-16T02:05:30.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.657+0000 7fb560346740 1 -- 172.21.15.105:0/2314724308 shutdown_connections 2024-09-16T02:05:30.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.657+0000 7fb560346740 1 --2- 172.21.15.105:0/2314724308 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x556b1e324360 0x556b1e326750 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:30.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.657+0000 7fb560346740 1 --2- 172.21.15.105:0/2314724308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b1e314780 0x556b1e311a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:30.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.657+0000 7fb560346740 1 --2- 172.21.15.105:0/2314724308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b1e306b40 0x556b1e311450 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:30.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.657+0000 7fb536ffd640 0 client.15828 ms_handle_reset on v2:172.21.15.190:6836/3181257549 2024-09-16T02:05:30.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.657+0000 7fb560346740 1 --2- 172.21.15.105:0/2314724308 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1e3055d0 0x556b1e3141a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:30.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.657+0000 7fb560346740 1 -- 172.21.15.105:0/2314724308 >> 172.21.15.105:0/2314724308 conn(0x556b1e2ce310 msgr2=0x556b1e307960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:30.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.657+0000 7fb560346740 1 -- 172.21.15.105:0/2314724308 shutdown_connections 2024-09-16T02:05:30.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.657+0000 7fb560346740 1 -- 172.21.15.105:0/2314724308 wait complete. 2024-09-16T02:05:30.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.658+0000 7fb560346740 20 client.15828 trim_cache size 0 max 16384 2024-09-16T02:05:30.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.671+0000 7fb560346740 1 Processor -- start 2024-09-16T02:05:30.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.671+0000 7fb560346740 1 -- start start 2024-09-16T02:05:30.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.671+0000 7fb560346740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b1e3055d0 0x556b1e3206c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:30.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.671+0000 7fb560346740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1e306b40 0x556b1e3133c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:30.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.671+0000 7fb560346740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b1e314780 0x556b1e315b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:30.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.671+0000 7fb560346740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb5581ab520 con 0x556b1e314780 2024-09-16T02:05:30.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.672+0000 7fb560346740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb5581ad060 con 0x556b1e3055d0 2024-09-16T02:05:30.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.672+0000 7fb560346740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb5581abbf0 con 0x556b1e306b40 2024-09-16T02:05:30.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.672+0000 7fb557fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b1e314780 0x556b1e315b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:30.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.672+0000 7fb557fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b1e314780 0x556b1e315b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45974/0 (socket says 172.21.15.105:45974) 2024-09-16T02:05:30.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.672+0000 7fb557fff640 1 -- 172.21.15.105:0/4258378706 learned_addr learned my addr 172.21.15.105:0/4258378706 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:30.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.672+0000 7fb54ffff640 1 --2- 172.21.15.105:0/4258378706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1e306b40 0x556b1e3133c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:30.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.672+0000 7fb5577fe640 1 --2- 172.21.15.105:0/4258378706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b1e3055d0 0x556b1e3206c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:30.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.672+0000 7fb557fff640 1 -- 172.21.15.105:0/4258378706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1e306b40 msgr2=0x556b1e3133c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:30.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.672+0000 7fb557fff640 1 --2- 172.21.15.105:0/4258378706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1e306b40 0x556b1e3133c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:30.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.672+0000 7fb557fff640 1 -- 172.21.15.105:0/4258378706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b1e3055d0 msgr2=0x556b1e3206c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:30.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.672+0000 7fb557fff640 1 --2- 172.21.15.105:0/4258378706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b1e3055d0 0x556b1e3206c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:30.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.672+0000 7fb557fff640 1 -- 172.21.15.105:0/4258378706 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b1e3014f0 con 0x556b1e314780 2024-09-16T02:05:30.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.672+0000 7fb5577fe640 1 --2- 172.21.15.105:0/4258378706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b1e3055d0 0x556b1e3206c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:05:30.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.672+0000 7fb54ffff640 1 --2- 172.21.15.105:0/4258378706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1e306b40 0x556b1e3133c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:05:30.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.672+0000 7fb557fff640 1 --2- 172.21.15.105:0/4258378706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b1e314780 0x556b1e315b70 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb550011a10 tx=0x7fb550016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9e570f6c54d73c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:30.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.672+0000 7fb54d7fa640 1 -- 172.21.15.105:0/4258378706 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55002d070 con 0x556b1e314780 2024-09-16T02:05:30.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.673+0000 7fb54d7fa640 1 -- 172.21.15.105:0/4258378706 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb55001f980 con 0x556b1e314780 2024-09-16T02:05:30.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.673+0000 7fb54d7fa640 1 -- 172.21.15.105:0/4258378706 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb550028b20 con 0x556b1e314780 2024-09-16T02:05:30.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.673+0000 7fb560346740 1 -- 172.21.15.105:0/4258378706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b1e314780 msgr2=0x556b1e315b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:30.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.673+0000 7fb560346740 1 --2- 172.21.15.105:0/4258378706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b1e314780 0x556b1e315b70 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb550011a10 tx=0x7fb550016000 comp rx=0 tx=0).stop 2024-09-16T02:05:30.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.673+0000 7fb560346740 1 -- 172.21.15.105:0/4258378706 shutdown_connections 2024-09-16T02:05:30.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.673+0000 7fb560346740 1 --2- 172.21.15.105:0/4258378706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1e306b40 0x556b1e3133c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:30.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.673+0000 7fb560346740 1 --2- 172.21.15.105:0/4258378706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b1e314780 0x556b1e315b70 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:30.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.673+0000 7fb560346740 1 --2- 172.21.15.105:0/4258378706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b1e3055d0 0x556b1e3206c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:30.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.673+0000 7fb560346740 1 -- 172.21.15.105:0/4258378706 >> 172.21.15.105:0/4258378706 conn(0x556b1e2ce310 msgr2=0x556b1e312cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:30.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.674+0000 7fb560346740 1 -- 172.21.15.105:0/4258378706 shutdown_connections 2024-09-16T02:05:30.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.674+0000 7fb560346740 1 -- 172.21.15.105:0/4258378706 wait complete. 2024-09-16T02:05:30.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.674+0000 7fb560346740 1 -- start start 2024-09-16T02:05:30.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.674+0000 7fb560346740 1 Processor -- start 2024-09-16T02:05:30.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.675+0000 7fb560346740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1e3055d0 0x556b1e31c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:30.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.675+0000 7fb560346740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b1e306b40 0x556b1e31cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:30.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.675+0000 7fb560346740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b1e314780 0x556b1e3177f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:30.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.675+0000 7fb560346740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb5581ac990 con 0x556b1e314780 2024-09-16T02:05:30.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.675+0000 7fb560346740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb5581ac2c0 con 0x556b1e306b40 2024-09-16T02:05:30.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.675+0000 7fb560346740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb5581a9500 con 0x556b1e3055d0 2024-09-16T02:05:30.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.675+0000 7fb54ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b1e306b40 0x556b1e31cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:30.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.675+0000 7fb557fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b1e314780 0x556b1e3177f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:30.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.675+0000 7fb557fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b1e314780 0x556b1e3177f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45980/0 (socket says 172.21.15.105:45980) 2024-09-16T02:05:30.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.675+0000 7fb54ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b1e306b40 0x556b1e31cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.105:49658/0 (socket says 172.21.15.105:49658) 2024-09-16T02:05:30.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.675+0000 7fb557fff640 1 -- 172.21.15.105:0/1855297306 learned_addr learned my addr 172.21.15.105:0/1855297306 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:30.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.675+0000 7fb560346740 10 client.0 _ll_register_callbacks cb 0x7fb54000df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:30.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.675+0000 7fb5577fe640 1 --2- 172.21.15.105:0/1855297306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1e3055d0 0x556b1e31c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:30.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.675+0000 7fb54ffff640 1 -- 172.21.15.105:0/1855297306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1e3055d0 msgr2=0x556b1e31c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:30.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.675+0000 7fb54ffff640 1 --2- 172.21.15.105:0/1855297306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1e3055d0 0x556b1e31c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:30.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.675+0000 7fb54ffff640 1 -- 172.21.15.105:0/1855297306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b1e314780 msgr2=0x556b1e3177f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:30.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.675+0000 7fb54ffff640 1 --2- 172.21.15.105:0/1855297306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b1e314780 0x556b1e3177f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:30.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.675+0000 7fb54ffff640 1 -- 172.21.15.105:0/1855297306 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b1e31d4c0 con 0x556b1e306b40 2024-09-16T02:05:30.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.675+0000 7fb557fff640 1 --2- 172.21.15.105:0/1855297306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b1e314780 0x556b1e3177f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:05:30.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.675+0000 7fb5577fe640 1 --2- 172.21.15.105:0/1855297306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1e3055d0 0x556b1e31c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:05:30.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.676+0000 7fb54ffff640 1 --2- 172.21.15.105:0/1855297306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b1e306b40 0x556b1e31cd80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x556b1e302e90 tx=0x7fb540005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25cac9c9b1e11acf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:30.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.676+0000 7fb536ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-16T02:05:30.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.676+0000 7fb536ffd640 1 -- 172.21.15.105:0/1855297306 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb540020070 con 0x556b1e306b40 2024-09-16T02:05:30.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.676+0000 7fb560346740 1 -- 172.21.15.105:0/1855297306 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb55001f980 con 0x556b1e306b40 2024-09-16T02:05:30.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.676+0000 7fb536ffd640 1 -- 172.21.15.105:0/1855297306 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fb5400cf2d0 con 0x556b1e306b40 2024-09-16T02:05:30.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.676+0000 7fb536ffd640 1 -- 172.21.15.105:0/1855297306 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb54009b6b0 con 0x556b1e306b40 2024-09-16T02:05:30.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.676+0000 7fb560346740 10 client.25082 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:30.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.676+0000 7fb560346740 1 -- 172.21.15.105:0/1855297306 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556b1e38f5a0 con 0x556b1e306b40 2024-09-16T02:05:30.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.676+0000 7fb536ffd640 1 -- 172.21.15.105:0/1855297306 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb55001f980 con 0x556b1e306b40 2024-09-16T02:05:30.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.677+0000 7fb536ffd640 1 -- 172.21.15.105:0/1855297306 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb5400d7060 con 0x556b1e306b40 2024-09-16T02:05:30.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.677+0000 7fb560346740 10 client.25082 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:30.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.677+0000 7fb560346740 10 client.25082 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T02:05:30.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.677+0000 7fb560346740 20 client.25082 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:30.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.677+0000 7fb560346740 1 --2- 172.21.15.105:0/1855297306 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x556b1e38fd40 0x556b1e392160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:30.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.677+0000 7fb560346740 4 client.25082 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T02:05:30.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.677+0000 7fb560346740 1 -- 172.21.15.105:0/1855297306 --> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556b1e272d70 con 0x556b1e38fd40 2024-09-16T02:05:30.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.678+0000 7fb5577fe640 1 --2- 172.21.15.105:0/1855297306 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x556b1e38fd40 0x556b1e392160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:30.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.678+0000 7fb536ffd640 10 client.25082 ms_handle_connect on v2:172.21.15.190:6836/3181257549 2024-09-16T02:05:30.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.678+0000 7fb5577fe640 1 --2- 172.21.15.105:0/1855297306 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x556b1e38fd40 0x556b1e392160 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=74b86f4e1e46d684 server_cookie=9a10a0ff8b33b1ee in_seq=0 out_seq=0 2024-09-16T02:05:30.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.678+0000 7fb536ffd640 1 -- 172.21.15.105:0/1855297306 <== mds.3 v2:172.21.15.190:6836/3181257549 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556b1e272d70 con 0x556b1e38fd40 2024-09-16T02:05:30.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.678+0000 7fb536ffd640 10 client.25082 handle_command_reply: tid=0 2024-09-16T02:05:30.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.678+0000 7fb560346740 1 client.25082 shutdown 2024-09-16T02:05:30.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.679+0000 7fb560346740 1 -- 172.21.15.105:0/1855297306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b1e306b40 msgr2=0x556b1e31cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:30.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.679+0000 7fb560346740 1 --2- 172.21.15.105:0/1855297306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b1e306b40 0x556b1e31cd80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x556b1e302e90 tx=0x7fb540005880 comp rx=0 tx=0).stop 2024-09-16T02:05:30.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.679+0000 7fb560346740 1 -- 172.21.15.105:0/1855297306 shutdown_connections 2024-09-16T02:05:30.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.679+0000 7fb560346740 1 --2- 172.21.15.105:0/1855297306 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x556b1e38fd40 0x556b1e392160 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:30.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.679+0000 7fb560346740 1 --2- 172.21.15.105:0/1855297306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b1e314780 0x556b1e3177f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:30.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.679+0000 7fb560346740 1 --2- 172.21.15.105:0/1855297306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b1e306b40 0x556b1e31cd80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:30.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.679+0000 7fb560346740 1 --2- 172.21.15.105:0/1855297306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1e3055d0 0x556b1e31c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:30.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.679+0000 7fb560346740 1 -- 172.21.15.105:0/1855297306 >> 172.21.15.105:0/1855297306 conn(0x556b1e2ce310 msgr2=0x556b1e3004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:30.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.679+0000 7fb536ffd640 0 client.25082 ms_handle_reset on v2:172.21.15.190:6836/3181257549 2024-09-16T02:05:30.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.679+0000 7fb560346740 1 -- 172.21.15.105:0/1855297306 shutdown_connections 2024-09-16T02:05:30.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.679+0000 7fb560346740 1 -- 172.21.15.105:0/1855297306 wait complete. 2024-09-16T02:05:30.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.679+0000 7fb560346740 20 client.25082 trim_cache size 0 max 16384 2024-09-16T02:05:30.694 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-09-16T02:05:30.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.680+0000 7fb55df6d640 1 -- 172.21.15.105:0/4007370562 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb5440a3240 msgr2=0x7fb5440a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:30.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.680+0000 7fb55df6d640 1 --2- 172.21.15.105:0/4007370562 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb5440a3240 0x7fb5440a5700 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb550005fd0 tx=0x7fb550009780 comp rx=0 tx=0).stop 2024-09-16T02:05:30.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.680+0000 7fb55df6d640 1 -- 172.21.15.105:0/4007370562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb558102780 msgr2=0x7fb5581988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:30.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.680+0000 7fb55df6d640 1 --2- 172.21.15.105:0/4007370562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb558102780 0x7fb5581988a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb54000d950 tx=0x7fb54000de10 comp rx=0 tx=0).stop 2024-09-16T02:05:30.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.680+0000 7fb55df6d640 1 -- 172.21.15.105:0/4007370562 shutdown_connections 2024-09-16T02:05:30.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.680+0000 7fb55df6d640 1 --2- 172.21.15.105:0/4007370562 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb5440a3240 0x7fb5440a5700 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:30.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.680+0000 7fb55df6d640 1 --2- 172.21.15.105:0/4007370562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb558106f00 0x7fb55819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:30.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.680+0000 7fb55df6d640 1 --2- 172.21.15.105:0/4007370562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb558102780 0x7fb5581988a0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:30.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.680+0000 7fb55df6d640 1 --2- 172.21.15.105:0/4007370562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb558101dd0 0x7fb558198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:30.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.680+0000 7fb55df6d640 1 -- 172.21.15.105:0/4007370562 >> 172.21.15.105:0/4007370562 conn(0x7fb5580fd8b0 msgr2=0x7fb5580fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:30.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.680+0000 7fb55df6d640 1 -- 172.21.15.105:0/4007370562 shutdown_connections 2024-09-16T02:05:30.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:30.680+0000 7fb55df6d640 1 -- 172.21.15.105:0/4007370562 wait complete. 2024-09-16T02:05:30.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:30 smithi137 ceph-mon[34099]: pgmap v428: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 224 MiB/s wr, 67 op/s 2024-09-16T02:05:30.970 INFO:tasks.workunit.client.0.smithi105.stdout: 2024-09-16T02:05:30.970 INFO:tasks.workunit.client.0.smithi105.stdout: Children see throughput for 1 re-readers = 7782563.50 kB/sec 2024-09-16T02:05:30.970 INFO:tasks.workunit.client.0.smithi105.stdout: Parent sees throughput for 1 re-readers = 7781206.17 kB/sec 2024-09-16T02:05:30.970 INFO:tasks.workunit.client.0.smithi105.stdout: Min throughput per process = 7782563.50 kB/sec 2024-09-16T02:05:30.971 INFO:tasks.workunit.client.0.smithi105.stdout: Max throughput per process = 7782563.50 kB/sec 2024-09-16T02:05:30.971 INFO:tasks.workunit.client.0.smithi105.stdout: Avg throughput per process = 7782563.50 kB/sec 2024-09-16T02:05:30.971 INFO:tasks.workunit.client.0.smithi105.stdout: Min xfer = 10485760.00 kB 2024-09-16T02:05:31.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:30 smithi105 ceph-mon[31197]: pgmap v428: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 224 MiB/s wr, 67 op/s 2024-09-16T02:05:31.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:30 smithi190 ceph-mon[34762]: pgmap v428: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 224 MiB/s wr, 67 op/s 2024-09-16T02:05:31.876 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T02:05:32.124 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:32.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.611+0000 7fac5c55e640 1 -- 172.21.15.105:0/1168784961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac541002f0 msgr2=0x7fac541006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.611+0000 7fac5c55e640 1 --2- 172.21.15.105:0/1168784961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac541002f0 0x7fac541006d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fac50009930 tx=0x7fac5002a420 comp rx=0 tx=0).stop 2024-09-16T02:05:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.611+0000 7fac5c55e640 1 -- 172.21.15.105:0/1168784961 shutdown_connections 2024-09-16T02:05:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.611+0000 7fac5c55e640 1 --2- 172.21.15.105:0/1168784961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac54106f50 0x7fac5410eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.611+0000 7fac5c55e640 1 --2- 172.21.15.105:0/1168784961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac54100ca0 0x7fac54106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.611+0000 7fac5c55e640 1 --2- 172.21.15.105:0/1168784961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac541002f0 0x7fac541006d0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.611+0000 7fac5c55e640 1 -- 172.21.15.105:0/1168784961 >> 172.21.15.105:0/1168784961 conn(0x7fac54077210 msgr2=0x7fac540fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.612+0000 7fac5c55e640 1 -- 172.21.15.105:0/1168784961 shutdown_connections 2024-09-16T02:05:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.612+0000 7fac5c55e640 1 -- 172.21.15.105:0/1168784961 wait complete. 2024-09-16T02:05:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.612+0000 7fac5c55e640 1 Processor -- start 2024-09-16T02:05:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.612+0000 7fac5c55e640 1 -- start start 2024-09-16T02:05:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.612+0000 7fac5c55e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac541002f0 0x7fac5419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.612+0000 7fac5c55e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac54100ca0 0x7fac5419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:32.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.612+0000 7fac5c55e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac54106f50 0x7fac5419ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:32.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.612+0000 7fac5c55e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fac54111e50 con 0x7fac541002f0 2024-09-16T02:05:32.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.612+0000 7fac5c55e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fac54111cd0 con 0x7fac54100ca0 2024-09-16T02:05:32.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.612+0000 7fac5c55e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fac54111fd0 con 0x7fac54106f50 2024-09-16T02:05:32.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.612+0000 7fac5a2fa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac541002f0 0x7fac5419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:32.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.612+0000 7fac5a2fa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac541002f0 0x7fac5419a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46000/0 (socket says 172.21.15.105:46000) 2024-09-16T02:05:32.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.612+0000 7fac5a2fa640 1 -- 172.21.15.105:0/3793096894 learned_addr learned my addr 172.21.15.105:0/3793096894 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:32.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.613+0000 7fac5aafb640 1 --2- 172.21.15.105:0/3793096894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac54106f50 0x7fac5419ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:32.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.613+0000 7fac59af9640 1 --2- 172.21.15.105:0/3793096894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac54100ca0 0x7fac5419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:32.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.613+0000 7fac5a2fa640 1 -- 172.21.15.105:0/3793096894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac54106f50 msgr2=0x7fac5419ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:32.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.613+0000 7fac5a2fa640 1 --2- 172.21.15.105:0/3793096894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac54106f50 0x7fac5419ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.613+0000 7fac5a2fa640 1 -- 172.21.15.105:0/3793096894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac54100ca0 msgr2=0x7fac5419aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:32.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.613+0000 7fac5a2fa640 1 --2- 172.21.15.105:0/3793096894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac54100ca0 0x7fac5419aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.613+0000 7fac5a2fa640 1 -- 172.21.15.105:0/3793096894 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac5419f560 con 0x7fac541002f0 2024-09-16T02:05:32.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.613+0000 7fac59af9640 1 --2- 172.21.15.105:0/3793096894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac54100ca0 0x7fac5419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:05:32.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.613+0000 7fac5a2fa640 1 --2- 172.21.15.105:0/3793096894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac541002f0 0x7fac5419a5b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fac50004770 tx=0x7fac500042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:32.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.613+0000 7fac4b7fe640 1 -- 172.21.15.105:0/3793096894 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5003f070 con 0x7fac541002f0 2024-09-16T02:05:32.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.613+0000 7fac4b7fe640 1 -- 172.21.15.105:0/3793096894 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac5003b460 con 0x7fac541002f0 2024-09-16T02:05:32.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.613+0000 7fac5c55e640 1 -- 172.21.15.105:0/3793096894 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac5419f7f0 con 0x7fac541002f0 2024-09-16T02:05:32.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.613+0000 7fac4b7fe640 1 -- 172.21.15.105:0/3793096894 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac50035070 con 0x7fac541002f0 2024-09-16T02:05:32.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.613+0000 7fac5c55e640 1 -- 172.21.15.105:0/3793096894 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac541a7130 con 0x7fac541002f0 2024-09-16T02:05:32.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.616+0000 7fac4b7fe640 1 -- 172.21.15.105:0/3793096894 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fac5003b610 con 0x7fac541002f0 2024-09-16T02:05:32.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.616+0000 7fac5e937740 1 Processor -- start 2024-09-16T02:05:32.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.616+0000 7fac5e937740 1 -- start start 2024-09-16T02:05:32.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.616+0000 7fac5e937740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eca0cc0b40 0x55eca0cbf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:32.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.616+0000 7fac5e937740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eca0cbf5d0 0x55eca0cbfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:32.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.616+0000 7fac5e937740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eca0cca370 0x55eca0ccc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:32.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.616+0000 7fac5e937740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55eca0c252f0 con 0x55eca0cbf5d0 2024-09-16T02:05:32.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.616+0000 7fac5e937740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55eca0bcccf0 con 0x55eca0cca370 2024-09-16T02:05:32.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.616+0000 7fac5e937740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55eca0c212b0 con 0x55eca0cc0b40 2024-09-16T02:05:32.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.616+0000 7fac5a2fa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eca0cbf5d0 0x55eca0cbfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:32.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.616+0000 7fac5aafb640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eca0cc0b40 0x55eca0cbf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:32.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.616+0000 7fac5aafb640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eca0cc0b40 0x55eca0cbf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.105:60614/0 (socket says 172.21.15.105:60614) 2024-09-16T02:05:32.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.616+0000 7fac5aafb640 1 -- 172.21.15.105:0/1067932850 learned_addr learned my addr 172.21.15.105:0/1067932850 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:32.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.616+0000 7fac59af9640 1 --2- 172.21.15.105:0/1067932850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eca0cca370 0x55eca0ccc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:32.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.617+0000 7fac5aafb640 1 -- 172.21.15.105:0/1067932850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eca0cca370 msgr2=0x55eca0ccc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:32.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.617+0000 7fac5aafb640 1 --2- 172.21.15.105:0/1067932850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eca0cca370 0x55eca0ccc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.617+0000 7fac5aafb640 1 -- 172.21.15.105:0/1067932850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eca0cbf5d0 msgr2=0x55eca0cbfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:32.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.617+0000 7fac5aafb640 1 --2- 172.21.15.105:0/1067932850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eca0cbf5d0 0x55eca0cbfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.617+0000 7fac5aafb640 1 -- 172.21.15.105:0/1067932850 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eca0bb13a0 con 0x55eca0cc0b40 2024-09-16T02:05:32.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.617+0000 7fac4b7fe640 1 --2- 172.21.15.105:0/3793096894 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fac400a32c0 0x7fac400a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:32.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.617+0000 7fac5a2fa640 1 --2- 172.21.15.105:0/3793096894 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fac400a32c0 0x7fac400a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:32.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.617+0000 7fac5aafb640 1 --2- 172.21.15.105:0/1067932850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eca0cc0b40 0x55eca0cbf090 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fac4c009f70 tx=0x7fac4c010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f961be291a940670 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:32.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.617+0000 7fac4b7fe640 1 -- 172.21.15.105:0/3793096894 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fac500efc90 con 0x7fac541002f0 2024-09-16T02:05:32.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.617+0000 7fac48ff9640 1 -- 172.21.15.105:0/1067932850 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4c016070 con 0x55eca0cc0b40 2024-09-16T02:05:32.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.617+0000 7fac48ff9640 1 -- 172.21.15.105:0/1067932850 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac4c004d20 con 0x55eca0cc0b40 2024-09-16T02:05:32.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.617+0000 7fac48ff9640 1 -- 172.21.15.105:0/1067932850 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4c012770 con 0x55eca0cc0b40 2024-09-16T02:05:32.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.618+0000 7fac5e937740 1 -- 172.21.15.105:0/1067932850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eca0cc0b40 msgr2=0x55eca0cbf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:32.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.618+0000 7fac5a2fa640 1 --2- 172.21.15.105:0/3793096894 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fac400a32c0 0x7fac400a5780 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fac500edaa0 tx=0x7fac500dc040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:32.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.618+0000 7fac5e937740 1 --2- 172.21.15.105:0/1067932850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eca0cc0b40 0x55eca0cbf090 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fac4c009f70 tx=0x7fac4c010800 comp rx=0 tx=0).stop 2024-09-16T02:05:32.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.618+0000 7fac5e937740 1 -- 172.21.15.105:0/1067932850 shutdown_connections 2024-09-16T02:05:32.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.618+0000 7fac5e937740 1 --2- 172.21.15.105:0/1067932850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eca0cca370 0x55eca0ccc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.618+0000 7fac5e937740 1 --2- 172.21.15.105:0/1067932850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eca0cbf5d0 0x55eca0cbfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.618+0000 7fac5e937740 1 --2- 172.21.15.105:0/1067932850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eca0cc0b40 0x55eca0cbf090 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.618+0000 7fac5e937740 1 -- 172.21.15.105:0/1067932850 >> 172.21.15.105:0/1067932850 conn(0x55eca0c88310 msgr2=0x55eca0cbbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:32.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.618+0000 7fac5e937740 1 -- 172.21.15.105:0/1067932850 shutdown_connections 2024-09-16T02:05:32.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.618+0000 7fac5e937740 1 -- 172.21.15.105:0/1067932850 wait complete. 2024-09-16T02:05:32.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.618+0000 7fac5e937740 1 -- start start 2024-09-16T02:05:32.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.618+0000 7fac5e937740 1 Processor -- start 2024-09-16T02:05:32.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.619+0000 7fac5e937740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eca0cbf5d0 0x55eca0cc9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:32.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.619+0000 7fac5e937740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eca0cc0b40 0x55eca0cc7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:32.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.619+0000 7fac5e937740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eca0cca370 0x55eca0cc7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:32.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.619+0000 7fac5e937740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55eca0ccd070 con 0x55eca0cca370 2024-09-16T02:05:32.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.619+0000 7fac5e937740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55eca0be0060 con 0x55eca0cc0b40 2024-09-16T02:05:32.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.619+0000 7fac5e937740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55eca0ccd1f0 con 0x55eca0cbf5d0 2024-09-16T02:05:32.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.619+0000 7fac5e937740 10 client.0 _ll_register_callbacks cb 0x55eca09338c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:32.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.619+0000 7fac5aafb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eca0cca370 0x55eca0cc7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:32.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.619+0000 7fac5aafb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eca0cca370 0x55eca0cc7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46026/0 (socket says 172.21.15.105:46026) 2024-09-16T02:05:32.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.619+0000 7fac5aafb640 1 -- 172.21.15.105:0/4070021709 learned_addr learned my addr 172.21.15.105:0/4070021709 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:32.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.619+0000 7fac59af9640 1 --2- 172.21.15.105:0/4070021709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eca0cc0b40 0x55eca0cc7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:32.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.619+0000 7fac5aafb640 1 --2- 172.21.15.105:0/4070021709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eca0cbf5d0 0x55eca0cc9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:32.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.619+0000 7fac5aafb640 1 -- 172.21.15.105:0/4070021709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eca0cbf5d0 msgr2=0x55eca0cc9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:32.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.619+0000 7fac5aafb640 1 --2- 172.21.15.105:0/4070021709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eca0cbf5d0 0x55eca0cc9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.619+0000 7fac5aafb640 1 -- 172.21.15.105:0/4070021709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eca0cc0b40 msgr2=0x55eca0cc7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:32.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.619+0000 7fac5aafb640 1 --2- 172.21.15.105:0/4070021709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eca0cc0b40 0x55eca0cc7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.619+0000 7fac5aafb640 1 -- 172.21.15.105:0/4070021709 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eca09963c0 con 0x55eca0cca370 2024-09-16T02:05:32.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.619+0000 7fac5aafb640 1 --2- 172.21.15.105:0/4070021709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eca0cca370 0x55eca0cc7640 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fac4c004bf0 tx=0x7fac4c012450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1935c8a1cf71769 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:32.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.619+0000 7fac3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:05:32.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.619+0000 7fac3dffb640 1 -- 172.21.15.105:0/4070021709 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4c016070 con 0x55eca0cca370 2024-09-16T02:05:32.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.619+0000 7fac3dffb640 1 -- 172.21.15.105:0/4070021709 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fac4c012ba0 con 0x55eca0cca370 2024-09-16T02:05:32.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.619+0000 7fac5e937740 1 -- 172.21.15.105:0/4070021709 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac4c004d20 con 0x55eca0cca370 2024-09-16T02:05:32.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.620+0000 7fac3dffb640 1 -- 172.21.15.105:0/4070021709 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fac4c003970 con 0x55eca0cca370 2024-09-16T02:05:32.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.620+0000 7fac5e937740 10 client.15846 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:32.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.620+0000 7fac5e937740 1 -- 172.21.15.105:0/4070021709 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eca0cd8b40 con 0x55eca0cca370 2024-09-16T02:05:32.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.620+0000 7fac3dffb640 1 -- 172.21.15.105:0/4070021709 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac4c004d20 con 0x55eca0cca370 2024-09-16T02:05:32.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.620+0000 7fac3dffb640 1 -- 172.21.15.105:0/4070021709 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fac4c00f040 con 0x55eca0cca370 2024-09-16T02:05:32.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.620+0000 7fac5e937740 10 client.15846 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:32.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.620+0000 7fac5e937740 10 client.15846 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T02:05:32.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.620+0000 7fac5e937740 20 client.15846 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:32.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.620+0000 7fac5e937740 1 --2- 172.21.15.105:0/4070021709 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x55eca0cd9fa0 0x55eca0cdc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:32.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.620+0000 7fac5e937740 4 client.15846 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:05:32.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.620+0000 7fac5e937740 1 -- 172.21.15.105:0/4070021709 --> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eca0c32860 con 0x55eca0cd9fa0 2024-09-16T02:05:32.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.620+0000 7fac59af9640 1 --2- 172.21.15.105:0/4070021709 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x55eca0cd9fa0 0x55eca0cdc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:32.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.621+0000 7fac3dffb640 10 client.15846 ms_handle_connect on v2:172.21.15.137:6836/2788541030 2024-09-16T02:05:32.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.621+0000 7fac59af9640 1 --2- 172.21.15.105:0/4070021709 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x55eca0cd9fa0 0x55eca0cdc390 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d9539eb61c66b3ab server_cookie=c0ae07ce405001ad in_seq=0 out_seq=0 2024-09-16T02:05:32.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.622+0000 7fac3dffb640 1 -- 172.21.15.105:0/4070021709 <== mds.2 v2:172.21.15.137:6836/2788541030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eca0c32860 con 0x55eca0cd9fa0 2024-09-16T02:05:32.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.622+0000 7fac3dffb640 10 client.15846 handle_command_reply: tid=0 2024-09-16T02:05:32.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.622+0000 7fac5e937740 1 client.15846 shutdown 2024-09-16T02:05:32.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.622+0000 7fac5e937740 1 -- 172.21.15.105:0/4070021709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eca0cca370 msgr2=0x55eca0cc7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:32.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.622+0000 7fac5e937740 1 --2- 172.21.15.105:0/4070021709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eca0cca370 0x55eca0cc7640 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fac4c004bf0 tx=0x7fac4c012450 comp rx=0 tx=0).stop 2024-09-16T02:05:32.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.622+0000 7fac5e937740 1 -- 172.21.15.105:0/4070021709 shutdown_connections 2024-09-16T02:05:32.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.622+0000 7fac5e937740 1 --2- 172.21.15.105:0/4070021709 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x55eca0cd9fa0 0x55eca0cdc390 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.622+0000 7fac5e937740 1 --2- 172.21.15.105:0/4070021709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eca0cca370 0x55eca0cc7640 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.622+0000 7fac5e937740 1 --2- 172.21.15.105:0/4070021709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eca0cc0b40 0x55eca0cc7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.622+0000 7fac3dffb640 0 client.15846 ms_handle_reset on v2:172.21.15.137:6836/2788541030 2024-09-16T02:05:32.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.622+0000 7fac5e937740 1 --2- 172.21.15.105:0/4070021709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eca0cbf5d0 0x55eca0cc9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.622+0000 7fac5e937740 1 -- 172.21.15.105:0/4070021709 >> 172.21.15.105:0/4070021709 conn(0x55eca0c88310 msgr2=0x55eca0cc1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:32.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.622+0000 7fac5e937740 1 -- 172.21.15.105:0/4070021709 shutdown_connections 2024-09-16T02:05:32.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.622+0000 7fac5e937740 1 -- 172.21.15.105:0/4070021709 wait complete. 2024-09-16T02:05:32.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.622+0000 7fac5e937740 20 client.15846 trim_cache size 0 max 16384 2024-09-16T02:05:32.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.635+0000 7fac5e937740 1 Processor -- start 2024-09-16T02:05:32.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.635+0000 7fac5e937740 1 -- start start 2024-09-16T02:05:32.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.635+0000 7fac5e937740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eca0cbf5d0 0x55eca0cd6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:32.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.635+0000 7fac5e937740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eca0cc0b40 0x55eca0cc6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:32.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.635+0000 7fac5e937740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eca0cca370 0x55eca0ce1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:32.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.635+0000 7fac5e937740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fac541af900 con 0x55eca0cc0b40 2024-09-16T02:05:32.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.635+0000 7fac5e937740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fac541b1440 con 0x55eca0cca370 2024-09-16T02:05:32.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.635+0000 7fac5e937740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fac541affd0 con 0x55eca0cbf5d0 2024-09-16T02:05:32.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.635+0000 7fac59af9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eca0cc0b40 0x55eca0cc6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:32.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.635+0000 7fac59af9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eca0cc0b40 0x55eca0cc6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46036/0 (socket says 172.21.15.105:46036) 2024-09-16T02:05:32.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.635+0000 7fac59af9640 1 -- 172.21.15.105:0/1031839075 learned_addr learned my addr 172.21.15.105:0/1031839075 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:32.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.635+0000 7fac59af9640 1 -- 172.21.15.105:0/1031839075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eca0cbf5d0 msgr2=0x55eca0cd6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:32.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.635+0000 7fac5aafb640 1 --2- 172.21.15.105:0/1031839075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eca0cbf5d0 0x55eca0cd6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:32.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.636+0000 7fac59af9640 1 --2- 172.21.15.105:0/1031839075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eca0cbf5d0 0x55eca0cd6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.636+0000 7fac5aafb640 1 --2- 172.21.15.105:0/1031839075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eca0cca370 0x55eca0ce1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:32.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.636+0000 7fac59af9640 1 -- 172.21.15.105:0/1031839075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eca0cca370 msgr2=0x55eca0ce1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:32.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.636+0000 7fac59af9640 1 --2- 172.21.15.105:0/1031839075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eca0cca370 0x55eca0ce1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.636+0000 7fac59af9640 1 -- 172.21.15.105:0/1031839075 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eca0cdc920 con 0x55eca0cc0b40 2024-09-16T02:05:32.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.636+0000 7fac59af9640 1 --2- 172.21.15.105:0/1031839075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eca0cc0b40 0x55eca0cc6010 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fac4400bee0 tx=0x7fac4400b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=287a9b95710f2689 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:32.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.636+0000 7fac3ffff640 1 -- 172.21.15.105:0/1031839075 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac44005050 con 0x55eca0cc0b40 2024-09-16T02:05:32.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.636+0000 7fac3ffff640 1 -- 172.21.15.105:0/1031839075 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac44004730 con 0x55eca0cc0b40 2024-09-16T02:05:32.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.636+0000 7fac3ffff640 1 -- 172.21.15.105:0/1031839075 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac44011680 con 0x55eca0cc0b40 2024-09-16T02:05:32.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.636+0000 7fac5e937740 1 -- 172.21.15.105:0/1031839075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eca0cc0b40 msgr2=0x55eca0cc6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:32.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.636+0000 7fac5e937740 1 --2- 172.21.15.105:0/1031839075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eca0cc0b40 0x55eca0cc6010 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fac4400bee0 tx=0x7fac4400b5a0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.637+0000 7fac5e937740 1 -- 172.21.15.105:0/1031839075 shutdown_connections 2024-09-16T02:05:32.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.637+0000 7fac5e937740 1 --2- 172.21.15.105:0/1031839075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eca0cca370 0x55eca0ce1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.637+0000 7fac5e937740 1 --2- 172.21.15.105:0/1031839075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eca0cc0b40 0x55eca0cc6010 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.637+0000 7fac5e937740 1 --2- 172.21.15.105:0/1031839075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eca0cbf5d0 0x55eca0cd6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.637+0000 7fac5e937740 1 -- 172.21.15.105:0/1031839075 >> 172.21.15.105:0/1031839075 conn(0x55eca0c88310 msgr2=0x55eca0cc5900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:32.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.637+0000 7fac5e937740 1 -- 172.21.15.105:0/1031839075 shutdown_connections 2024-09-16T02:05:32.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.637+0000 7fac5e937740 1 -- 172.21.15.105:0/1031839075 wait complete. 2024-09-16T02:05:32.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.637+0000 7fac5e937740 1 -- start start 2024-09-16T02:05:32.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.637+0000 7fac5e937740 1 Processor -- start 2024-09-16T02:05:32.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.637+0000 7fac5e937740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eca0cbf5d0 0x55eca0ccd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:32.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.637+0000 7fac5e937740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eca0cc0b40 0x55eca0ccdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:32.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.637+0000 7fac5e937740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eca0cca370 0x55eca0d42b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:32.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.637+0000 7fac5e937740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fac541b0d70 con 0x55eca0cc0b40 2024-09-16T02:05:32.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.637+0000 7fac5e937740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fac541b06a0 con 0x55eca0cca370 2024-09-16T02:05:32.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.637+0000 7fac5e937740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fac541ad8e0 con 0x55eca0cbf5d0 2024-09-16T02:05:32.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.637+0000 7fac5e937740 10 client.0 _ll_register_callbacks cb 0x7fac4400b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:32.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.637+0000 7fac59af9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eca0cc0b40 0x55eca0ccdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:32.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.637+0000 7fac59af9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eca0cc0b40 0x55eca0ccdce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46048/0 (socket says 172.21.15.105:46048) 2024-09-16T02:05:32.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.637+0000 7fac59af9640 1 -- 172.21.15.105:0/2593278522 learned_addr learned my addr 172.21.15.105:0/2593278522 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:32.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.637+0000 7fac5aafb640 1 --2- 172.21.15.105:0/2593278522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eca0cbf5d0 0x55eca0ccd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:32.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.638+0000 7fac5aafb640 1 --2- 172.21.15.105:0/2593278522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eca0cca370 0x55eca0d42b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:32.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.638+0000 7fac59af9640 1 -- 172.21.15.105:0/2593278522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eca0cbf5d0 msgr2=0x55eca0ccd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:32.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.638+0000 7fac59af9640 1 --2- 172.21.15.105:0/2593278522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eca0cbf5d0 0x55eca0ccd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.638+0000 7fac59af9640 1 -- 172.21.15.105:0/2593278522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eca0cca370 msgr2=0x55eca0d42b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:32.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.638+0000 7fac59af9640 1 --2- 172.21.15.105:0/2593278522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eca0cca370 0x55eca0d42b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.638+0000 7fac59af9640 1 -- 172.21.15.105:0/2593278522 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eca0d43050 con 0x55eca0cc0b40 2024-09-16T02:05:32.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.638+0000 7fac5aafb640 1 --2- 172.21.15.105:0/2593278522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eca0cca370 0x55eca0d42b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:05:32.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.638+0000 7fac59af9640 1 --2- 172.21.15.105:0/2593278522 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eca0cc0b40 0x55eca0ccdce0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fac44000c00 tx=0x7fac440132f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cabb38b4682b84c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:32.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.638+0000 7fac3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:05:32.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.638+0000 7fac3dffb640 1 -- 172.21.15.105:0/2593278522 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac44005050 con 0x55eca0cc0b40 2024-09-16T02:05:32.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.638+0000 7fac3dffb640 1 -- 172.21.15.105:0/2593278522 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fac440051f0 con 0x55eca0cc0b40 2024-09-16T02:05:32.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.638+0000 7fac5e937740 1 -- 172.21.15.105:0/2593278522 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac44004730 con 0x55eca0cc0b40 2024-09-16T02:05:32.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.638+0000 7fac3dffb640 1 -- 172.21.15.105:0/2593278522 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fac44036050 con 0x55eca0cc0b40 2024-09-16T02:05:32.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.638+0000 7fac5e937740 10 client.15852 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:32.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.638+0000 7fac5e937740 1 -- 172.21.15.105:0/2593278522 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eca0d44250 con 0x55eca0cc0b40 2024-09-16T02:05:32.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.638+0000 7fac3dffb640 1 -- 172.21.15.105:0/2593278522 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac44004730 con 0x55eca0cc0b40 2024-09-16T02:05:32.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.639+0000 7fac3dffb640 1 -- 172.21.15.105:0/2593278522 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fac4401c420 con 0x55eca0cc0b40 2024-09-16T02:05:32.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.639+0000 7fac5e937740 10 client.15852 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:32.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.639+0000 7fac5e937740 10 client.15852 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T02:05:32.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.639+0000 7fac5e937740 20 client.15852 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:32.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.639+0000 7fac5e937740 1 --2- 172.21.15.105:0/2593278522 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x55eca0d44940 0x55eca0d46d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:32.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.639+0000 7fac5e937740 4 client.15852 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T02:05:32.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.639+0000 7fac5e937740 1 -- 172.21.15.105:0/2593278522 --> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eca0c2cd70 con 0x55eca0d44940 2024-09-16T02:05:32.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.639+0000 7fac5aafb640 1 --2- 172.21.15.105:0/2593278522 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x55eca0d44940 0x55eca0d46d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:32.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.639+0000 7fac5aafb640 1 --2- 172.21.15.105:0/2593278522 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x55eca0d44940 0x55eca0d46d60 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=18f7075120272d52 server_cookie=8a3268b5be297e43 in_seq=0 out_seq=0 2024-09-16T02:05:32.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.639+0000 7fac3dffb640 10 client.15852 ms_handle_connect on v2:172.21.15.137:6836/2788541030 2024-09-16T02:05:32.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.639+0000 7fac3dffb640 1 -- 172.21.15.105:0/2593278522 <== mds.2 v2:172.21.15.137:6836/2788541030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eca0c2cd70 con 0x55eca0d44940 2024-09-16T02:05:32.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.639+0000 7fac3dffb640 10 client.15852 handle_command_reply: tid=0 2024-09-16T02:05:32.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.639+0000 7fac5e937740 1 client.15852 shutdown 2024-09-16T02:05:32.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.639+0000 7fac5e937740 1 -- 172.21.15.105:0/2593278522 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eca0cc0b40 msgr2=0x55eca0ccdce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:32.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.639+0000 7fac5e937740 1 --2- 172.21.15.105:0/2593278522 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eca0cc0b40 0x55eca0ccdce0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fac44000c00 tx=0x7fac440132f0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.639+0000 7fac5e937740 1 -- 172.21.15.105:0/2593278522 shutdown_connections 2024-09-16T02:05:32.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.639+0000 7fac5e937740 1 --2- 172.21.15.105:0/2593278522 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x55eca0d44940 0x55eca0d46d60 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.639+0000 7fac5e937740 1 --2- 172.21.15.105:0/2593278522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eca0cca370 0x55eca0d42b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.639+0000 7fac5e937740 1 --2- 172.21.15.105:0/2593278522 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eca0cc0b40 0x55eca0ccdce0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.640+0000 7fac5e937740 1 --2- 172.21.15.105:0/2593278522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eca0cbf5d0 0x55eca0ccd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.640+0000 7fac3dffb640 0 client.15852 ms_handle_reset on v2:172.21.15.137:6836/2788541030 2024-09-16T02:05:32.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.640+0000 7fac5e937740 1 -- 172.21.15.105:0/2593278522 >> 172.21.15.105:0/2593278522 conn(0x55eca0c88310 msgr2=0x55eca0cc6120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:32.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.640+0000 7fac5e937740 1 -- 172.21.15.105:0/2593278522 shutdown_connections 2024-09-16T02:05:32.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.640+0000 7fac5e937740 1 -- 172.21.15.105:0/2593278522 wait complete. 2024-09-16T02:05:32.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.640+0000 7fac5e937740 20 client.15852 trim_cache size 0 max 16384 2024-09-16T02:05:32.654 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-09-16T02:05:32.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.640+0000 7fac5c55e640 1 -- 172.21.15.105:0/3793096894 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fac400a32c0 msgr2=0x7fac400a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:32.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.640+0000 7fac5c55e640 1 --2- 172.21.15.105:0/3793096894 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fac400a32c0 0x7fac400a5780 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fac500edaa0 tx=0x7fac500dc040 comp rx=0 tx=0).stop 2024-09-16T02:05:32.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.640+0000 7fac5c55e640 1 -- 172.21.15.105:0/3793096894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac541002f0 msgr2=0x7fac5419a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:32.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.640+0000 7fac5c55e640 1 --2- 172.21.15.105:0/3793096894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac541002f0 0x7fac5419a5b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fac50004770 tx=0x7fac500042f0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.640+0000 7fac5c55e640 1 -- 172.21.15.105:0/3793096894 shutdown_connections 2024-09-16T02:05:32.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.640+0000 7fac5c55e640 1 --2- 172.21.15.105:0/3793096894 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fac400a32c0 0x7fac400a5780 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.640+0000 7fac5c55e640 1 --2- 172.21.15.105:0/3793096894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac54106f50 0x7fac5419ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.640+0000 7fac5c55e640 1 --2- 172.21.15.105:0/3793096894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac54100ca0 0x7fac5419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.640+0000 7fac5c55e640 1 --2- 172.21.15.105:0/3793096894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac541002f0 0x7fac5419a5b0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.640+0000 7fac5c55e640 1 -- 172.21.15.105:0/3793096894 >> 172.21.15.105:0/3793096894 conn(0x7fac54077210 msgr2=0x7fac54105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:32.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.640+0000 7fac5c55e640 1 -- 172.21.15.105:0/3793096894 shutdown_connections 2024-09-16T02:05:32.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:32.640+0000 7fac5c55e640 1 -- 172.21.15.105:0/3793096894 wait complete. 2024-09-16T02:05:32.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:32 smithi105 ceph-mon[31197]: pgmap v429: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 139 MiB/s wr, 45 op/s 2024-09-16T02:05:32.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:32 smithi137 ceph-mon[34099]: pgmap v429: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 139 MiB/s wr, 45 op/s 2024-09-16T02:05:33.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:32 smithi190 ceph-mon[34762]: pgmap v429: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 139 MiB/s wr, 45 op/s 2024-09-16T02:05:33.457 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T02:05:33.710 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:34.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.204+0000 7f7785228640 1 -- 172.21.15.105:0/546768713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7780102d40 msgr2=0x7f7780103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:34.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.204+0000 7f7785228640 1 --2- 172.21.15.105:0/546768713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7780102d40 0x7f7780103120 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f7774009900 tx=0x7f777402a480 comp rx=0 tx=0).stop 2024-09-16T02:05:34.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.204+0000 7f7785228640 1 -- 172.21.15.105:0/546768713 shutdown_connections 2024-09-16T02:05:34.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.204+0000 7f7785228640 1 --2- 172.21.15.105:0/546768713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7780103660 0x7f778010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.204+0000 7f7785228640 1 --2- 172.21.15.105:0/546768713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f778010c170 0x7f778010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.204+0000 7f7785228640 1 --2- 172.21.15.105:0/546768713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7780102d40 0x7f7780103120 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.204+0000 7f7785228640 1 -- 172.21.15.105:0/546768713 >> 172.21.15.105:0/546768713 conn(0x7f77800fc7a0 msgr2=0x7f77800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:34.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.204+0000 7f7785228640 1 -- 172.21.15.105:0/546768713 shutdown_connections 2024-09-16T02:05:34.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.204+0000 7f7785228640 1 -- 172.21.15.105:0/546768713 wait complete. 2024-09-16T02:05:34.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.204+0000 7f7785228640 1 Processor -- start 2024-09-16T02:05:34.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.205+0000 7f7785228640 1 -- start start 2024-09-16T02:05:34.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.205+0000 7f7785228640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7780102d40 0x7f77801982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:34.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.205+0000 7f7785228640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7780103660 0x7f7780198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:34.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.205+0000 7f7785228640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f778010c170 0x7f778019cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:34.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.205+0000 7f7785228640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7780111e20 con 0x7f778010c170 2024-09-16T02:05:34.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.205+0000 7f7785228640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7780111ca0 con 0x7f7780102d40 2024-09-16T02:05:34.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.205+0000 7f7785228640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7780111fa0 con 0x7f7780103660 2024-09-16T02:05:34.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.205+0000 7f777f59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f778010c170 0x7f778019cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:34.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.205+0000 7f777f59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f778010c170 0x7f778019cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46068/0 (socket says 172.21.15.105:46068) 2024-09-16T02:05:34.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.205+0000 7f777f59e640 1 -- 172.21.15.105:0/3858863470 learned_addr learned my addr 172.21.15.105:0/3858863470 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:34.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.205+0000 7f777ed9d640 1 --2- 172.21.15.105:0/3858863470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7780102d40 0x7f77801982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:34.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.205+0000 7f777e59c640 1 --2- 172.21.15.105:0/3858863470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7780103660 0x7f7780198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:34.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.205+0000 7f777f59e640 1 -- 172.21.15.105:0/3858863470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7780103660 msgr2=0x7f7780198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:34.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.205+0000 7f777f59e640 1 --2- 172.21.15.105:0/3858863470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7780103660 0x7f7780198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.205+0000 7f777f59e640 1 -- 172.21.15.105:0/3858863470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7780102d40 msgr2=0x7f77801982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:34.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.205+0000 7f777f59e640 1 --2- 172.21.15.105:0/3858863470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7780102d40 0x7f77801982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.205+0000 7f777f59e640 1 -- 172.21.15.105:0/3858863470 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f778019d290 con 0x7f778010c170 2024-09-16T02:05:34.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.206+0000 7f777f59e640 1 --2- 172.21.15.105:0/3858863470 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f778010c170 0x7f778019cbb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f777800cce0 tx=0x7f777800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:34.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.206+0000 7f7767fff640 1 -- 172.21.15.105:0/3858863470 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f777800f450 con 0x7f778010c170 2024-09-16T02:05:34.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.206+0000 7f7785228640 1 -- 172.21.15.105:0/3858863470 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f778019d580 con 0x7f778010c170 2024-09-16T02:05:34.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.206+0000 7f7767fff640 1 -- 172.21.15.105:0/3858863470 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f777800faa0 con 0x7f778010c170 2024-09-16T02:05:34.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.206+0000 7f7767fff640 1 -- 172.21.15.105:0/3858863470 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7778018b70 con 0x7f778010c170 2024-09-16T02:05:34.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.206+0000 7f7785228640 1 -- 172.21.15.105:0/3858863470 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77801a4e60 con 0x7f778010c170 2024-09-16T02:05:34.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.208+0000 7f7767fff640 1 -- 172.21.15.105:0/3858863470 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f7778018d10 con 0x7f778010c170 2024-09-16T02:05:34.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.208+0000 7f7787601740 1 Processor -- start 2024-09-16T02:05:34.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.208+0000 7f7787601740 1 -- start start 2024-09-16T02:05:34.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.208+0000 7f7787601740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d7df240b40 0x55d7df23f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:34.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.208+0000 7f7787601740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7df23f5d0 0x55d7df23fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:34.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.208+0000 7f7787601740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7df24a370 0x55d7df24c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:34.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.208+0000 7f7787601740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d7df1a52f0 con 0x55d7df240b40 2024-09-16T02:05:34.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.208+0000 7f7787601740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d7df14ccf0 con 0x55d7df23f5d0 2024-09-16T02:05:34.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.208+0000 7f7787601740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d7df1a12b0 con 0x55d7df24a370 2024-09-16T02:05:34.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.209+0000 7f7767fff640 1 --2- 172.21.15.105:0/3858863470 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f77680a3240 0x7f77680a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:34.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.209+0000 7f777e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d7df240b40 0x55d7df23f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:34.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.209+0000 7f777e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d7df240b40 0x55d7df23f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46072/0 (socket says 172.21.15.105:46072) 2024-09-16T02:05:34.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.209+0000 7f777e59c640 1 -- 172.21.15.105:0/3178447485 learned_addr learned my addr 172.21.15.105:0/3178447485 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:34.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.209+0000 7f777e59c640 1 --2- 172.21.15.105:0/3178447485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7df24a370 0x55d7df24c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:34.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.209+0000 7f777f59e640 1 --2- 172.21.15.105:0/3858863470 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f77680a3240 0x7f77680a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:34.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.209+0000 7f7767fff640 1 -- 172.21.15.105:0/3858863470 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f77780cfc90 con 0x7f778010c170 2024-09-16T02:05:34.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.209+0000 7f777ed9d640 1 --2- 172.21.15.105:0/3178447485 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7df23f5d0 0x55d7df23fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:34.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.209+0000 7f777e59c640 1 -- 172.21.15.105:0/3178447485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7df24a370 msgr2=0x55d7df24c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:34.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.209+0000 7f777e59c640 1 --2- 172.21.15.105:0/3178447485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7df24a370 0x55d7df24c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.209+0000 7f777e59c640 1 -- 172.21.15.105:0/3178447485 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7df23f5d0 msgr2=0x55d7df23fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:34.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.209+0000 7f777e59c640 1 --2- 172.21.15.105:0/3178447485 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7df23f5d0 0x55d7df23fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.209+0000 7f777e59c640 1 -- 172.21.15.105:0/3178447485 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7df1313a0 con 0x55d7df240b40 2024-09-16T02:05:34.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.209+0000 7f777ed9d640 1 --2- 172.21.15.105:0/3178447485 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7df23f5d0 0x55d7df23fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:05:34.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.209+0000 7f777e59c640 1 --2- 172.21.15.105:0/3178447485 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d7df240b40 0x55d7df23f090 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f777000bde0 tx=0x7f7770004160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1deeaeadca9a11b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:34.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.209+0000 7f777f59e640 1 --2- 172.21.15.105:0/3858863470 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f77680a3240 0x7f77680a5700 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7778098c20 tx=0x7f7778086b90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:34.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.209+0000 7f77657fa640 1 -- 172.21.15.105:0/3178447485 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7770003e80 con 0x55d7df240b40 2024-09-16T02:05:34.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.209+0000 7f77657fa640 1 -- 172.21.15.105:0/3178447485 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77700054d0 con 0x55d7df240b40 2024-09-16T02:05:34.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.209+0000 7f77657fa640 1 -- 172.21.15.105:0/3178447485 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7770004e60 con 0x55d7df240b40 2024-09-16T02:05:34.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.210+0000 7f7787601740 1 -- 172.21.15.105:0/3178447485 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d7df240b40 msgr2=0x55d7df23f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:34.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.210+0000 7f7787601740 1 --2- 172.21.15.105:0/3178447485 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d7df240b40 0x55d7df23f090 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f777000bde0 tx=0x7f7770004160 comp rx=0 tx=0).stop 2024-09-16T02:05:34.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.210+0000 7f7787601740 1 -- 172.21.15.105:0/3178447485 shutdown_connections 2024-09-16T02:05:34.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.210+0000 7f7787601740 1 --2- 172.21.15.105:0/3178447485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7df24a370 0x55d7df24c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.210+0000 7f7787601740 1 --2- 172.21.15.105:0/3178447485 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7df23f5d0 0x55d7df23fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.210+0000 7f7787601740 1 --2- 172.21.15.105:0/3178447485 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d7df240b40 0x55d7df23f090 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.210+0000 7f7787601740 1 -- 172.21.15.105:0/3178447485 >> 172.21.15.105:0/3178447485 conn(0x55d7df208310 msgr2=0x55d7df23bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:34.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.210+0000 7f7787601740 1 -- 172.21.15.105:0/3178447485 shutdown_connections 2024-09-16T02:05:34.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.210+0000 7f7787601740 1 -- 172.21.15.105:0/3178447485 wait complete. 2024-09-16T02:05:34.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.210+0000 7f7787601740 1 -- start start 2024-09-16T02:05:34.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.211+0000 7f7787601740 1 Processor -- start 2024-09-16T02:05:34.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.211+0000 7f7787601740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7df23f5d0 0x55d7df249cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:34.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.211+0000 7f7787601740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7df240b40 0x55d7df246f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:34.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.211+0000 7f7787601740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d7df24a370 0x55d7df2474e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:34.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.211+0000 7f7787601740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d7df24d010 con 0x55d7df24a370 2024-09-16T02:05:34.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.211+0000 7f7787601740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d7df160060 con 0x55d7df23f5d0 2024-09-16T02:05:34.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.211+0000 7f7787601740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d7df24d190 con 0x55d7df240b40 2024-09-16T02:05:34.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.211+0000 7f7787601740 10 client.0 _ll_register_callbacks cb 0x55d7deeb38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:34.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.211+0000 7f777e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d7df24a370 0x55d7df2474e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:34.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.211+0000 7f777e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d7df24a370 0x55d7df2474e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46080/0 (socket says 172.21.15.105:46080) 2024-09-16T02:05:34.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.211+0000 7f777e59c640 1 -- 172.21.15.105:0/1490741898 learned_addr learned my addr 172.21.15.105:0/1490741898 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:34.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.211+0000 7f777ed9d640 1 --2- 172.21.15.105:0/1490741898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7df240b40 0x55d7df246f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:34.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.211+0000 7f777e59c640 1 --2- 172.21.15.105:0/1490741898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7df23f5d0 0x55d7df249cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:34.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.211+0000 7f777e59c640 1 -- 172.21.15.105:0/1490741898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7df240b40 msgr2=0x55d7df246f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:34.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.211+0000 7f777e59c640 1 --2- 172.21.15.105:0/1490741898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7df240b40 0x55d7df246f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.211+0000 7f777e59c640 1 -- 172.21.15.105:0/1490741898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7df23f5d0 msgr2=0x55d7df249cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:34.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.211+0000 7f777e59c640 1 --2- 172.21.15.105:0/1490741898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7df23f5d0 0x55d7df249cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.211+0000 7f777e59c640 1 -- 172.21.15.105:0/1490741898 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7def163c0 con 0x55d7df24a370 2024-09-16T02:05:34.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.211+0000 7f777ed9d640 1 --2- 172.21.15.105:0/1490741898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7df240b40 0x55d7df246f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:05:34.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.211+0000 7f777e59c640 1 --2- 172.21.15.105:0/1490741898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d7df24a370 0x55d7df2474e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f77700057a0 tx=0x7f7770014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109715d01bd610cb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:34.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.211+0000 7f7762ff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:05:34.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.211+0000 7f7762ff5640 1 -- 172.21.15.105:0/1490741898 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77700051e0 con 0x55d7df24a370 2024-09-16T02:05:34.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.211+0000 7f7762ff5640 1 -- 172.21.15.105:0/1490741898 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f7770012450 con 0x55d7df24a370 2024-09-16T02:05:34.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.211+0000 7f7787601740 1 -- 172.21.15.105:0/1490741898 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77700054d0 con 0x55d7df24a370 2024-09-16T02:05:34.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.212+0000 7f7762ff5640 1 -- 172.21.15.105:0/1490741898 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f77700048d0 con 0x55d7df24a370 2024-09-16T02:05:34.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.212+0000 7f7787601740 10 client.15864 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:34.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.212+0000 7f7787601740 1 -- 172.21.15.105:0/1490741898 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d7df258990 con 0x55d7df24a370 2024-09-16T02:05:34.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.212+0000 7f7762ff5640 1 -- 172.21.15.105:0/1490741898 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77700054d0 con 0x55d7df24a370 2024-09-16T02:05:34.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.212+0000 7f7762ff5640 1 -- 172.21.15.105:0/1490741898 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f777002f020 con 0x55d7df24a370 2024-09-16T02:05:34.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.212+0000 7f7787601740 10 client.15864 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:34.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.212+0000 7f7787601740 10 client.15864 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T02:05:34.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.212+0000 7f7787601740 20 client.15864 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:34.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.212+0000 7f7787601740 1 --2- 172.21.15.105:0/1490741898 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55d7df259ce0 0x55d7df25c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:34.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.212+0000 7f7787601740 4 client.15864 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:05:34.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.212+0000 7f7787601740 1 -- 172.21.15.105:0/1490741898 --> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7df1b2860 con 0x55d7df259ce0 2024-09-16T02:05:34.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.212+0000 7f777ed9d640 1 --2- 172.21.15.105:0/1490741898 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55d7df259ce0 0x55d7df25c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:34.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.213+0000 7f7762ff5640 10 client.15864 ms_handle_connect on v2:172.21.15.190:6838/1222573537 2024-09-16T02:05:34.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.213+0000 7f777ed9d640 1 --2- 172.21.15.105:0/1490741898 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55d7df259ce0 0x55d7df25c0d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=38679bb2d2a057e6 server_cookie=6b281acdff50dfb5 in_seq=0 out_seq=0 2024-09-16T02:05:34.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.213+0000 7f7762ff5640 1 -- 172.21.15.105:0/1490741898 <== mds.1 v2:172.21.15.190:6838/1222573537 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d7df1b2860 con 0x55d7df259ce0 2024-09-16T02:05:34.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.213+0000 7f7762ff5640 10 client.15864 handle_command_reply: tid=0 2024-09-16T02:05:34.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.214+0000 7f7787601740 1 client.15864 shutdown 2024-09-16T02:05:34.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.214+0000 7f7787601740 1 -- 172.21.15.105:0/1490741898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d7df24a370 msgr2=0x55d7df2474e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:34.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.214+0000 7f7787601740 1 --2- 172.21.15.105:0/1490741898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d7df24a370 0x55d7df2474e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f77700057a0 tx=0x7f7770014040 comp rx=0 tx=0).stop 2024-09-16T02:05:34.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.214+0000 7f7787601740 1 -- 172.21.15.105:0/1490741898 shutdown_connections 2024-09-16T02:05:34.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.214+0000 7f7787601740 1 --2- 172.21.15.105:0/1490741898 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55d7df259ce0 0x55d7df25c0d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.214+0000 7f7787601740 1 --2- 172.21.15.105:0/1490741898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7df240b40 0x55d7df246f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.214+0000 7f7762ff5640 0 client.15864 ms_handle_reset on v2:172.21.15.190:6838/1222573537 2024-09-16T02:05:34.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.214+0000 7f7787601740 1 --2- 172.21.15.105:0/1490741898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d7df24a370 0x55d7df2474e0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.214+0000 7f7787601740 1 --2- 172.21.15.105:0/1490741898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7df23f5d0 0x55d7df249cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.214+0000 7f7787601740 1 -- 172.21.15.105:0/1490741898 >> 172.21.15.105:0/1490741898 conn(0x55d7df208310 msgr2=0x55d7df241960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:34.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.214+0000 7f7787601740 1 -- 172.21.15.105:0/1490741898 shutdown_connections 2024-09-16T02:05:34.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.214+0000 7f7787601740 1 -- 172.21.15.105:0/1490741898 wait complete. 2024-09-16T02:05:34.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.214+0000 7f7787601740 20 client.15864 trim_cache size 0 max 16384 2024-09-16T02:05:34.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.227+0000 7f7787601740 1 Processor -- start 2024-09-16T02:05:34.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.227+0000 7f7787601740 1 -- start start 2024-09-16T02:05:34.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.227+0000 7f7787601740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d7df23f5d0 0x55d7df2562b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:34.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.227+0000 7f7787601740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7df240b40 0x55d7df242120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:34.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.227+0000 7f7787601740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7df24a370 0x55d7df242660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:34.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.227+0000 7f7787601740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f77801ad5f0 con 0x55d7df23f5d0 2024-09-16T02:05:34.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.227+0000 7f7787601740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f77801af130 con 0x55d7df24a370 2024-09-16T02:05:34.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.227+0000 7f7787601740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f77801adcc0 con 0x55d7df240b40 2024-09-16T02:05:34.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.227+0000 7f777e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d7df23f5d0 0x55d7df2562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:34.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.227+0000 7f777e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d7df23f5d0 0x55d7df2562b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46088/0 (socket says 172.21.15.105:46088) 2024-09-16T02:05:34.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.227+0000 7f777e59c640 1 -- 172.21.15.105:0/1222886391 learned_addr learned my addr 172.21.15.105:0/1222886391 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:34.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.227+0000 7f777ed9d640 1 --2- 172.21.15.105:0/1222886391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7df240b40 0x55d7df242120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:34.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.227+0000 7f777e59c640 1 -- 172.21.15.105:0/1222886391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7df240b40 msgr2=0x55d7df242120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:34.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.227+0000 7f777e59c640 1 --2- 172.21.15.105:0/1222886391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7df240b40 0x55d7df242120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.227+0000 7f777e59c640 1 -- 172.21.15.105:0/1222886391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7df24a370 msgr2=0x55d7df242660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:34.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.227+0000 7f777e59c640 1 --2- 172.21.15.105:0/1222886391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7df24a370 0x55d7df242660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.227+0000 7f777e59c640 1 -- 172.21.15.105:0/1222886391 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7df24cec0 con 0x55d7df23f5d0 2024-09-16T02:05:34.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.228+0000 7f777e59c640 1 --2- 172.21.15.105:0/1222886391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d7df23f5d0 0x55d7df2562b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f7770004ab0 tx=0x7f7770031890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3321881870569854 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:34.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.228+0000 7f7764ff9640 1 -- 172.21.15.105:0/1222886391 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7770031460 con 0x55d7df23f5d0 2024-09-16T02:05:34.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.228+0000 7f7764ff9640 1 -- 172.21.15.105:0/1222886391 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7770016070 con 0x55d7df23f5d0 2024-09-16T02:05:34.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.228+0000 7f7764ff9640 1 -- 172.21.15.105:0/1222886391 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77700126b0 con 0x55d7df23f5d0 2024-09-16T02:05:34.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.228+0000 7f7787601740 1 -- 172.21.15.105:0/1222886391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d7df23f5d0 msgr2=0x55d7df2562b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:34.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.228+0000 7f7787601740 1 --2- 172.21.15.105:0/1222886391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d7df23f5d0 0x55d7df2562b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f7770004ab0 tx=0x7f7770031890 comp rx=0 tx=0).stop 2024-09-16T02:05:34.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.228+0000 7f7787601740 1 -- 172.21.15.105:0/1222886391 shutdown_connections 2024-09-16T02:05:34.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.228+0000 7f7787601740 1 --2- 172.21.15.105:0/1222886391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7df240b40 0x55d7df242120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.228+0000 7f7787601740 1 --2- 172.21.15.105:0/1222886391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7df24a370 0x55d7df242660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.228+0000 7f7787601740 1 --2- 172.21.15.105:0/1222886391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d7df23f5d0 0x55d7df2562b0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.228+0000 7f7787601740 1 -- 172.21.15.105:0/1222886391 >> 172.21.15.105:0/1222886391 conn(0x55d7df208310 msgr2=0x55d7df23ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:34.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.228+0000 7f7787601740 1 -- 172.21.15.105:0/1222886391 shutdown_connections 2024-09-16T02:05:34.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.228+0000 7f7787601740 1 -- 172.21.15.105:0/1222886391 wait complete. 2024-09-16T02:05:34.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.229+0000 7f7787601740 1 -- start start 2024-09-16T02:05:34.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.229+0000 7f7787601740 1 Processor -- start 2024-09-16T02:05:34.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.229+0000 7f7787601740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7df23f5d0 0x55d7df252b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:34.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.229+0000 7f7787601740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d7df240b40 0x55d7df24dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:34.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.229+0000 7f7787601740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7df24a370 0x55d7df24e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:34.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.229+0000 7f7787601740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f77801aea60 con 0x55d7df240b40 2024-09-16T02:05:34.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.229+0000 7f7787601740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f77801ae390 con 0x55d7df24a370 2024-09-16T02:05:34.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.229+0000 7f7787601740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f77801ab5d0 con 0x55d7df23f5d0 2024-09-16T02:05:34.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.229+0000 7f7787601740 10 client.0 _ll_register_callbacks cb 0x7f777000b8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:34.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.229+0000 7f777ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d7df240b40 0x55d7df24dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:34.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.229+0000 7f777ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d7df240b40 0x55d7df24dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46090/0 (socket says 172.21.15.105:46090) 2024-09-16T02:05:34.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.229+0000 7f777ed9d640 1 -- 172.21.15.105:0/2031185492 learned_addr learned my addr 172.21.15.105:0/2031185492 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:34.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.229+0000 7f777e59c640 1 --2- 172.21.15.105:0/2031185492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7df23f5d0 0x55d7df252b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:34.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.229+0000 7f777e59c640 1 --2- 172.21.15.105:0/2031185492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7df24a370 0x55d7df24e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:34.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.229+0000 7f777ed9d640 1 -- 172.21.15.105:0/2031185492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7df23f5d0 msgr2=0x55d7df252b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:34.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.229+0000 7f777ed9d640 1 --2- 172.21.15.105:0/2031185492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7df23f5d0 0x55d7df252b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.229+0000 7f777ed9d640 1 -- 172.21.15.105:0/2031185492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7df24a370 msgr2=0x55d7df24e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:34.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.229+0000 7f777ed9d640 1 --2- 172.21.15.105:0/2031185492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7df24a370 0x55d7df24e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.229+0000 7f777ed9d640 1 -- 172.21.15.105:0/2031185492 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7df24ebc0 con 0x55d7df240b40 2024-09-16T02:05:34.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.230+0000 7f777ed9d640 1 --2- 172.21.15.105:0/2031185492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d7df240b40 0x55d7df24dea0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f777402a9a0 tx=0x7f7774032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee1a9bebf17e9c97 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:34.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.230+0000 7f7762ff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:05:34.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.230+0000 7f7762ff5640 1 -- 172.21.15.105:0/2031185492 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7774035040 con 0x55d7df240b40 2024-09-16T02:05:34.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.230+0000 7f7762ff5640 1 -- 172.21.15.105:0/2031185492 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f7774005040 con 0x55d7df240b40 2024-09-16T02:05:34.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.230+0000 7f7787601740 1 -- 172.21.15.105:0/2031185492 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7770016070 con 0x55d7df240b40 2024-09-16T02:05:34.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.230+0000 7f7762ff5640 1 -- 172.21.15.105:0/2031185492 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7774052050 con 0x55d7df240b40 2024-09-16T02:05:34.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.230+0000 7f7787601740 10 client.15870 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:34.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.230+0000 7f7787601740 1 -- 172.21.15.105:0/2031185492 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d7df2c47c0 con 0x55d7df240b40 2024-09-16T02:05:34.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.230+0000 7f7762ff5640 1 -- 172.21.15.105:0/2031185492 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7770016070 con 0x55d7df240b40 2024-09-16T02:05:34.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.230+0000 7f7762ff5640 1 -- 172.21.15.105:0/2031185492 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f77740035f0 con 0x55d7df240b40 2024-09-16T02:05:34.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.230+0000 7f7787601740 10 client.15870 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:34.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.230+0000 7f7787601740 10 client.15870 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T02:05:34.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.230+0000 7f7787601740 20 client.15870 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:34.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.230+0000 7f7787601740 1 --2- 172.21.15.105:0/2031185492 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55d7df2c4f30 0x55d7df2c7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:34.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.230+0000 7f7787601740 4 client.15870 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T02:05:34.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.230+0000 7f7787601740 1 -- 172.21.15.105:0/2031185492 --> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d7df1acd70 con 0x55d7df2c4f30 2024-09-16T02:05:34.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.231+0000 7f777e59c640 1 --2- 172.21.15.105:0/2031185492 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55d7df2c4f30 0x55d7df2c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:34.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.231+0000 7f777e59c640 1 --2- 172.21.15.105:0/2031185492 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55d7df2c4f30 0x55d7df2c7320 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=85e31da46bd413ee server_cookie=52881697fee7ab31 in_seq=0 out_seq=0 2024-09-16T02:05:34.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.231+0000 7f7762ff5640 10 client.15870 ms_handle_connect on v2:172.21.15.190:6838/1222573537 2024-09-16T02:05:34.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.231+0000 7f7762ff5640 1 -- 172.21.15.105:0/2031185492 <== mds.1 v2:172.21.15.190:6838/1222573537 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d7df1acd70 con 0x55d7df2c4f30 2024-09-16T02:05:34.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.231+0000 7f7762ff5640 10 client.15870 handle_command_reply: tid=0 2024-09-16T02:05:34.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.231+0000 7f7787601740 1 client.15870 shutdown 2024-09-16T02:05:34.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.232+0000 7f7787601740 1 -- 172.21.15.105:0/2031185492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d7df240b40 msgr2=0x55d7df24dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:34.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.232+0000 7f7787601740 1 --2- 172.21.15.105:0/2031185492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d7df240b40 0x55d7df24dea0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f777402a9a0 tx=0x7f7774032d10 comp rx=0 tx=0).stop 2024-09-16T02:05:34.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.232+0000 7f7787601740 1 -- 172.21.15.105:0/2031185492 shutdown_connections 2024-09-16T02:05:34.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.232+0000 7f7787601740 1 --2- 172.21.15.105:0/2031185492 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55d7df2c4f30 0x55d7df2c7320 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.232+0000 7f7787601740 1 --2- 172.21.15.105:0/2031185492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7df24a370 0x55d7df24e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.232+0000 7f7787601740 1 --2- 172.21.15.105:0/2031185492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d7df240b40 0x55d7df24dea0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.232+0000 7f7787601740 1 --2- 172.21.15.105:0/2031185492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7df23f5d0 0x55d7df252b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.232+0000 7f7762ff5640 0 client.15870 ms_handle_reset on v2:172.21.15.190:6838/1222573537 2024-09-16T02:05:34.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.232+0000 7f7787601740 1 -- 172.21.15.105:0/2031185492 >> 172.21.15.105:0/2031185492 conn(0x55d7df208310 msgr2=0x55d7df2487e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:34.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.232+0000 7f7787601740 1 -- 172.21.15.105:0/2031185492 shutdown_connections 2024-09-16T02:05:34.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.232+0000 7f7787601740 1 -- 172.21.15.105:0/2031185492 wait complete. 2024-09-16T02:05:34.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.232+0000 7f7787601740 20 client.15870 trim_cache size 0 max 16384 2024-09-16T02:05:34.251 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-09-16T02:05:34.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.232+0000 7f7785228640 1 -- 172.21.15.105:0/3858863470 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f77680a3240 msgr2=0x7f77680a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:34.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.232+0000 7f7785228640 1 --2- 172.21.15.105:0/3858863470 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f77680a3240 0x7f77680a5700 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7778098c20 tx=0x7f7778086b90 comp rx=0 tx=0).stop 2024-09-16T02:05:34.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.232+0000 7f7785228640 1 -- 172.21.15.105:0/3858863470 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f778010c170 msgr2=0x7f778019cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:34.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.232+0000 7f7785228640 1 --2- 172.21.15.105:0/3858863470 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f778010c170 0x7f778019cbb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f777800cce0 tx=0x7f777800a620 comp rx=0 tx=0).stop 2024-09-16T02:05:34.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.233+0000 7f7785228640 1 -- 172.21.15.105:0/3858863470 shutdown_connections 2024-09-16T02:05:34.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.233+0000 7f7785228640 1 --2- 172.21.15.105:0/3858863470 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f77680a3240 0x7f77680a5700 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.233+0000 7f7785228640 1 --2- 172.21.15.105:0/3858863470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7780103660 0x7f7780198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.233+0000 7f7785228640 1 --2- 172.21.15.105:0/3858863470 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f778010c170 0x7f778019cbb0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.233+0000 7f7785228640 1 --2- 172.21.15.105:0/3858863470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7780102d40 0x7f77801982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.233+0000 7f7785228640 1 -- 172.21.15.105:0/3858863470 >> 172.21.15.105:0/3858863470 conn(0x7f77800fc7a0 msgr2=0x7f77800fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:34.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.233+0000 7f7785228640 1 -- 172.21.15.105:0/3858863470 shutdown_connections 2024-09-16T02:05:34.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:34.233+0000 7f7785228640 1 -- 172.21.15.105:0/3858863470 wait complete. 2024-09-16T02:05:34.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:34 smithi137 ceph-mon[34099]: pgmap v430: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.0 MiB/s wr, 11 op/s 2024-09-16T02:05:34.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:34 smithi105 ceph-mon[31197]: pgmap v430: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.0 MiB/s wr, 11 op/s 2024-09-16T02:05:35.020 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T02:05:35.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:34 smithi190 ceph-mon[34762]: pgmap v430: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.0 MiB/s wr, 11 op/s 2024-09-16T02:05:35.266 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:35.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.751+0000 7f6beee9d640 1 -- 172.21.15.105:0/2426743767 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6be8100e20 msgr2=0x7f6be8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:35.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.751+0000 7f6beee9d640 1 --2- 172.21.15.105:0/2426743767 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6be8100e20 0x7f6be8101200 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc009900 tx=0x7f6bdc02a480 comp rx=0 tx=0).stop 2024-09-16T02:05:35.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.751+0000 7f6beee9d640 1 -- 172.21.15.105:0/2426743767 shutdown_connections 2024-09-16T02:05:35.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.751+0000 7f6beee9d640 1 --2- 172.21.15.105:0/2426743767 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6be8102570 0x7f6be81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.751+0000 7f6beee9d640 1 --2- 172.21.15.105:0/2426743767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6be8102ef0 0x7f6be810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.751+0000 7f6beee9d640 1 --2- 172.21.15.105:0/2426743767 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6be8100e20 0x7f6be8101200 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.751+0000 7f6beee9d640 1 -- 172.21.15.105:0/2426743767 >> 172.21.15.105:0/2426743767 conn(0x7f6be80fc7d0 msgr2=0x7f6be80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:35.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.751+0000 7f6beee9d640 1 -- 172.21.15.105:0/2426743767 shutdown_connections 2024-09-16T02:05:35.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.751+0000 7f6beee9d640 1 -- 172.21.15.105:0/2426743767 wait complete. 2024-09-16T02:05:35.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.752+0000 7f6beee9d640 1 Processor -- start 2024-09-16T02:05:35.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.752+0000 7f6beee9d640 1 -- start start 2024-09-16T02:05:35.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.752+0000 7f6beee9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6be8100e20 0x7f6be8199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:35.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.752+0000 7f6beee9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6be8102570 0x7f6be81996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:35.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.752+0000 7f6beee9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6be8102ef0 0x7f6be81a41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:35.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.752+0000 7f6beee9d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6be8110e30 con 0x7f6be8100e20 2024-09-16T02:05:35.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.752+0000 7f6beee9d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6be8110cb0 con 0x7f6be8102ef0 2024-09-16T02:05:35.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.752+0000 7f6beee9d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6be8110fb0 con 0x7f6be8102570 2024-09-16T02:05:35.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.752+0000 7f6becc39640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6be8100e20 0x7f6be8199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:35.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.752+0000 7f6becc39640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6be8100e20 0x7f6be8199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46106/0 (socket says 172.21.15.105:46106) 2024-09-16T02:05:35.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.752+0000 7f6becc39640 1 -- 172.21.15.105:0/696039166 learned_addr learned my addr 172.21.15.105:0/696039166 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:35.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.752+0000 7f6bed43a640 1 --2- 172.21.15.105:0/696039166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6be8102ef0 0x7f6be81a41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:35.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.752+0000 7f6be7fff640 1 --2- 172.21.15.105:0/696039166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6be8102570 0x7f6be81996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:35.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.752+0000 7f6becc39640 1 -- 172.21.15.105:0/696039166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6be8102570 msgr2=0x7f6be81996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:35.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.752+0000 7f6becc39640 1 --2- 172.21.15.105:0/696039166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6be8102570 0x7f6be81996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.752+0000 7f6becc39640 1 -- 172.21.15.105:0/696039166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6be8102ef0 msgr2=0x7f6be81a41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:35.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.752+0000 7f6becc39640 1 --2- 172.21.15.105:0/696039166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6be8102ef0 0x7f6be81a41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.752+0000 7f6becc39640 1 -- 172.21.15.105:0/696039166 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be81a4720 con 0x7f6be8100e20 2024-09-16T02:05:35.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.752+0000 7f6bed43a640 1 --2- 172.21.15.105:0/696039166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6be8102ef0 0x7f6be81a41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:05:35.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.752+0000 7f6be7fff640 1 --2- 172.21.15.105:0/696039166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6be8102570 0x7f6be81996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:05:35.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.753+0000 7f6becc39640 1 --2- 172.21.15.105:0/696039166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6be8100e20 0x7f6be8199160 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc006010 tx=0x7f6bdc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:35.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.753+0000 7f6be5ffb640 1 -- 172.21.15.105:0/696039166 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc03f070 con 0x7f6be8100e20 2024-09-16T02:05:35.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.753+0000 7f6beee9d640 1 -- 172.21.15.105:0/696039166 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6be81a48f0 con 0x7f6be8100e20 2024-09-16T02:05:35.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.753+0000 7f6be5ffb640 1 -- 172.21.15.105:0/696039166 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6bdc003aa0 con 0x7f6be8100e20 2024-09-16T02:05:35.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.753+0000 7f6be5ffb640 1 -- 172.21.15.105:0/696039166 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc035030 con 0x7f6be8100e20 2024-09-16T02:05:35.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.753+0000 7f6beee9d640 1 -- 172.21.15.105:0/696039166 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6be81a4da0 con 0x7f6be8100e20 2024-09-16T02:05:35.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.755+0000 7f6be5ffb640 1 -- 172.21.15.105:0/696039166 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f6bdc042070 con 0x7f6be8100e20 2024-09-16T02:05:35.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.756+0000 7f6bf1276740 1 Processor -- start 2024-09-16T02:05:35.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.756+0000 7f6bf1276740 1 -- start start 2024-09-16T02:05:35.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.756+0000 7f6bf1276740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56139c59db40 0x56139c59c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:35.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.756+0000 7f6bf1276740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139c59c5d0 0x56139c59ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:35.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.756+0000 7f6bf1276740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56139c5a7370 0x56139c5a9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:35.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.756+0000 7f6bf1276740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56139c5022f0 con 0x56139c59c5d0 2024-09-16T02:05:35.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.756+0000 7f6bf1276740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56139c4a9cf0 con 0x56139c59db40 2024-09-16T02:05:35.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.756+0000 7f6bf1276740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56139c4fe2b0 con 0x56139c5a7370 2024-09-16T02:05:35.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.756+0000 7f6becc39640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139c59c5d0 0x56139c59ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:35.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.756+0000 7f6bed43a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56139c59db40 0x56139c59c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:35.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.756+0000 7f6be5ffb640 1 --2- 172.21.15.105:0/696039166 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f6bd80a3240 0x7f6bd80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:35.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.756+0000 7f6becc39640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139c59c5d0 0x56139c59ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46122/0 (socket says 172.21.15.105:46122) 2024-09-16T02:05:35.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.756+0000 7f6becc39640 1 -- 172.21.15.105:0/4240065944 learned_addr learned my addr 172.21.15.105:0/4240065944 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:35.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.756+0000 7f6be7fff640 1 --2- 172.21.15.105:0/4240065944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56139c5a7370 0x56139c5a9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:35.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.756+0000 7f6bed43a640 1 --2- 172.21.15.105:0/696039166 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f6bd80a3240 0x7f6bd80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:35.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.756+0000 7f6be5ffb640 1 -- 172.21.15.105:0/696039166 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f6bdc0f0370 con 0x7f6be8100e20 2024-09-16T02:05:35.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.757+0000 7f6becc39640 1 -- 172.21.15.105:0/4240065944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56139c5a7370 msgr2=0x56139c5a9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:35.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.757+0000 7f6becc39640 1 --2- 172.21.15.105:0/4240065944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56139c5a7370 0x56139c5a9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.757+0000 7f6becc39640 1 -- 172.21.15.105:0/4240065944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56139c59db40 msgr2=0x56139c59c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:35.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.757+0000 7f6becc39640 1 --2- 172.21.15.105:0/4240065944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56139c59db40 0x56139c59c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.757+0000 7f6becc39640 1 -- 172.21.15.105:0/4240065944 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56139c48e3a0 con 0x56139c59c5d0 2024-09-16T02:05:35.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.757+0000 7f6becc39640 1 --2- 172.21.15.105:0/4240065944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139c59c5d0 0x56139c59ca10 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc0bc870 tx=0x7f6bdc0319b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda1659787a747b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:35.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.757+0000 7f6bed43a640 1 --2- 172.21.15.105:0/696039166 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f6bd80a3240 0x7f6bd80a5700 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6be00127b0 tx=0x7f6be0004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:35.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.757+0000 7f6bd2ffd640 1 -- 172.21.15.105:0/4240065944 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc049070 con 0x56139c59c5d0 2024-09-16T02:05:35.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.757+0000 7f6bd2ffd640 1 -- 172.21.15.105:0/4240065944 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6bdc0bd030 con 0x56139c59c5d0 2024-09-16T02:05:35.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.757+0000 7f6bd2ffd640 1 -- 172.21.15.105:0/4240065944 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc0bac30 con 0x56139c59c5d0 2024-09-16T02:05:35.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.758+0000 7f6bf1276740 1 -- 172.21.15.105:0/4240065944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139c59c5d0 msgr2=0x56139c59ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:35.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.758+0000 7f6bf1276740 1 --2- 172.21.15.105:0/4240065944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139c59c5d0 0x56139c59ca10 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc0bc870 tx=0x7f6bdc0319b0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.758+0000 7f6bf1276740 1 -- 172.21.15.105:0/4240065944 shutdown_connections 2024-09-16T02:05:35.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.758+0000 7f6bf1276740 1 --2- 172.21.15.105:0/4240065944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56139c5a7370 0x56139c5a9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.758+0000 7f6bf1276740 1 --2- 172.21.15.105:0/4240065944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139c59c5d0 0x56139c59ca10 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.758+0000 7f6bf1276740 1 --2- 172.21.15.105:0/4240065944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56139c59db40 0x56139c59c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.758+0000 7f6bf1276740 1 -- 172.21.15.105:0/4240065944 >> 172.21.15.105:0/4240065944 conn(0x56139c565310 msgr2=0x56139c598c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:35.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.758+0000 7f6bf1276740 1 -- 172.21.15.105:0/4240065944 shutdown_connections 2024-09-16T02:05:35.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.758+0000 7f6bf1276740 1 -- 172.21.15.105:0/4240065944 wait complete. 2024-09-16T02:05:35.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.758+0000 7f6bf1276740 1 -- start start 2024-09-16T02:05:35.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.758+0000 7f6bf1276740 1 Processor -- start 2024-09-16T02:05:35.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.759+0000 7f6bf1276740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139c59c5d0 0x56139c5a6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:35.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.759+0000 7f6bf1276740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56139c59db40 0x56139c59fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:35.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.759+0000 7f6bf1276740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56139c5a7370 0x56139c5a0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:35.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.759+0000 7f6bf1276740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56139c5a9d70 con 0x56139c59c5d0 2024-09-16T02:05:35.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.759+0000 7f6bf1276740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56139c4bd060 con 0x56139c5a7370 2024-09-16T02:05:35.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.759+0000 7f6bf1276740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56139c5a9ef0 con 0x56139c59db40 2024-09-16T02:05:35.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.759+0000 7f6bf1276740 10 client.0 _ll_register_callbacks cb 0x56139c2108c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:35.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.759+0000 7f6becc39640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139c59c5d0 0x56139c5a6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:35.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.759+0000 7f6becc39640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139c59c5d0 0x56139c5a6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46132/0 (socket says 172.21.15.105:46132) 2024-09-16T02:05:35.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.759+0000 7f6becc39640 1 -- 172.21.15.105:0/642596802 learned_addr learned my addr 172.21.15.105:0/642596802 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:35.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.759+0000 7f6be7fff640 1 --2- 172.21.15.105:0/642596802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56139c59db40 0x56139c59fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:35.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.759+0000 7f6bed43a640 1 --2- 172.21.15.105:0/642596802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56139c5a7370 0x56139c5a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:35.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.759+0000 7f6becc39640 1 -- 172.21.15.105:0/642596802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56139c59db40 msgr2=0x56139c59fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:35.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.759+0000 7f6becc39640 1 --2- 172.21.15.105:0/642596802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56139c59db40 0x56139c59fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.759+0000 7f6becc39640 1 -- 172.21.15.105:0/642596802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56139c5a7370 msgr2=0x56139c5a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:35.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.759+0000 7f6becc39640 1 --2- 172.21.15.105:0/642596802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56139c5a7370 0x56139c5a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.759+0000 7f6becc39640 1 -- 172.21.15.105:0/642596802 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56139c2733c0 con 0x56139c59c5d0 2024-09-16T02:05:35.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.759+0000 7f6bed43a640 1 --2- 172.21.15.105:0/642596802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56139c5a7370 0x56139c5a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:05:35.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.759+0000 7f6becc39640 1 --2- 172.21.15.105:0/642596802 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139c59c5d0 0x56139c5a6c50 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc0b9300 tx=0x7f6bdc0b8c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37178c3cf9927c71 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:35.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.759+0000 7f6bc7fff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:05:35.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.760+0000 7f6bc7fff640 1 -- 172.21.15.105:0/642596802 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc049070 con 0x56139c59c5d0 2024-09-16T02:05:35.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.760+0000 7f6bc7fff640 1 -- 172.21.15.105:0/642596802 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f6bdc003c50 con 0x56139c59c5d0 2024-09-16T02:05:35.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.760+0000 7f6bf1276740 1 -- 172.21.15.105:0/642596802 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bdc0bd030 con 0x56139c59c5d0 2024-09-16T02:05:35.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.760+0000 7f6bc7fff640 1 -- 172.21.15.105:0/642596802 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6bdc0bc930 con 0x56139c59c5d0 2024-09-16T02:05:35.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.760+0000 7f6bf1276740 10 client.15882 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:35.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.760+0000 7f6bf1276740 1 -- 172.21.15.105:0/642596802 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56139c5a9d70 con 0x56139c59c5d0 2024-09-16T02:05:35.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.760+0000 7f6bc7fff640 1 -- 172.21.15.105:0/642596802 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6bdc0bd030 con 0x56139c59c5d0 2024-09-16T02:05:35.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.760+0000 7f6bc7fff640 1 -- 172.21.15.105:0/642596802 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6bdc032530 con 0x56139c59c5d0 2024-09-16T02:05:35.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.760+0000 7f6bf1276740 10 client.15882 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:35.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.760+0000 7f6bf1276740 10 client.15882 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:05:35.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.760+0000 7f6bf1276740 20 client.15882 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:35.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.760+0000 7f6bf1276740 1 --2- 172.21.15.105:0/642596802 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x56139c5b6a40 0x56139c5b8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:35.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.760+0000 7f6bf1276740 4 client.15882 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:05:35.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.760+0000 7f6bf1276740 1 -- 172.21.15.105:0/642596802 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56139c50f860 con 0x56139c5b6a40 2024-09-16T02:05:35.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.761+0000 7f6be7fff640 1 --2- 172.21.15.105:0/642596802 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x56139c5b6a40 0x56139c5b8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:35.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.761+0000 7f6be7fff640 1 --2- 172.21.15.105:0/642596802 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x56139c5b6a40 0x56139c5b8e30 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa08cd83c7b9c6c4 server_cookie=878acc16b490d894 in_seq=0 out_seq=0 2024-09-16T02:05:35.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.761+0000 7f6bc7fff640 10 client.15882 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:05:35.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.762+0000 7f6bc7fff640 1 -- 172.21.15.105:0/642596802 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56139c50f860 con 0x56139c5b6a40 2024-09-16T02:05:35.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.762+0000 7f6bc7fff640 10 client.15882 handle_command_reply: tid=0 2024-09-16T02:05:35.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.762+0000 7f6bf1276740 1 client.15882 shutdown 2024-09-16T02:05:35.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.762+0000 7f6bf1276740 1 -- 172.21.15.105:0/642596802 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139c59c5d0 msgr2=0x56139c5a6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:35.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.762+0000 7f6bf1276740 1 --2- 172.21.15.105:0/642596802 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139c59c5d0 0x56139c5a6c50 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc0b9300 tx=0x7f6bdc0b8c30 comp rx=0 tx=0).stop 2024-09-16T02:05:35.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.762+0000 7f6bf1276740 1 -- 172.21.15.105:0/642596802 shutdown_connections 2024-09-16T02:05:35.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.762+0000 7f6bf1276740 1 --2- 172.21.15.105:0/642596802 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x56139c5b6a40 0x56139c5b8e30 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.762+0000 7f6bc7fff640 0 client.15882 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:05:35.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.762+0000 7f6bf1276740 1 --2- 172.21.15.105:0/642596802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56139c59db40 0x56139c59fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.762+0000 7f6bf1276740 1 --2- 172.21.15.105:0/642596802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56139c5a7370 0x56139c5a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.762+0000 7f6bf1276740 1 --2- 172.21.15.105:0/642596802 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139c59c5d0 0x56139c5a6c50 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.762+0000 7f6bf1276740 1 -- 172.21.15.105:0/642596802 >> 172.21.15.105:0/642596802 conn(0x56139c565310 msgr2=0x56139c59e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:35.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.762+0000 7f6bf1276740 1 -- 172.21.15.105:0/642596802 shutdown_connections 2024-09-16T02:05:35.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.762+0000 7f6bf1276740 1 -- 172.21.15.105:0/642596802 wait complete. 2024-09-16T02:05:35.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.762+0000 7f6bf1276740 20 client.15882 trim_cache size 0 max 16384 2024-09-16T02:05:35.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.775+0000 7f6bf1276740 1 Processor -- start 2024-09-16T02:05:35.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.775+0000 7f6bf1276740 1 -- start start 2024-09-16T02:05:35.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.775+0000 7f6bf1276740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56139c59c5d0 0x56139c597fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:35.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.775+0000 7f6bf1276740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139c59db40 0x56139c5be660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:35.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.775+0000 7f6bf1276740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56139c5a7370 0x56139c5beba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:35.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.775+0000 7f6bf1276740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6be81ab2f0 con 0x56139c59db40 2024-09-16T02:05:35.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.775+0000 7f6bf1276740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6be81ace30 con 0x56139c5a7370 2024-09-16T02:05:35.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.775+0000 7f6bf1276740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6be81ab9c0 con 0x56139c59c5d0 2024-09-16T02:05:35.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.776+0000 7f6be7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139c59db40 0x56139c5be660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:35.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.776+0000 7f6be7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139c59db40 0x56139c5be660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46148/0 (socket says 172.21.15.105:46148) 2024-09-16T02:05:35.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.776+0000 7f6be7fff640 1 -- 172.21.15.105:0/3996687513 learned_addr learned my addr 172.21.15.105:0/3996687513 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:35.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.776+0000 7f6becc39640 1 --2- 172.21.15.105:0/3996687513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56139c59c5d0 0x56139c597fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:35.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.776+0000 7f6be7fff640 1 -- 172.21.15.105:0/3996687513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56139c59c5d0 msgr2=0x56139c597fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:35.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.776+0000 7f6be7fff640 1 --2- 172.21.15.105:0/3996687513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56139c59c5d0 0x56139c597fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.776+0000 7f6be7fff640 1 -- 172.21.15.105:0/3996687513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56139c5a7370 msgr2=0x56139c5beba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:35.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.776+0000 7f6be7fff640 1 --2- 172.21.15.105:0/3996687513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56139c5a7370 0x56139c5beba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.776+0000 7f6be7fff640 1 -- 172.21.15.105:0/3996687513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56139c5991a0 con 0x56139c59db40 2024-09-16T02:05:35.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.776+0000 7f6be7fff640 1 --2- 172.21.15.105:0/3996687513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139c59db40 0x56139c5be660 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4005fd0 tx=0x7f6bd4015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7dffb24a444ff70 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:35.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.776+0000 7f6bd27fc640 1 -- 172.21.15.105:0/3996687513 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4030070 con 0x56139c59db40 2024-09-16T02:05:35.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.776+0000 7f6bd27fc640 1 -- 172.21.15.105:0/3996687513 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6bd4011cb0 con 0x56139c59db40 2024-09-16T02:05:35.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.776+0000 7f6bd27fc640 1 -- 172.21.15.105:0/3996687513 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd40136d0 con 0x56139c59db40 2024-09-16T02:05:35.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.777+0000 7f6bf1276740 1 -- 172.21.15.105:0/3996687513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139c59db40 msgr2=0x56139c5be660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:35.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.777+0000 7f6bf1276740 1 --2- 172.21.15.105:0/3996687513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139c59db40 0x56139c5be660 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4005fd0 tx=0x7f6bd4015040 comp rx=0 tx=0).stop 2024-09-16T02:05:35.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.777+0000 7f6bf1276740 1 -- 172.21.15.105:0/3996687513 shutdown_connections 2024-09-16T02:05:35.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.777+0000 7f6bf1276740 1 --2- 172.21.15.105:0/3996687513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56139c5a7370 0x56139c5beba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.777+0000 7f6bf1276740 1 --2- 172.21.15.105:0/3996687513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139c59db40 0x56139c5be660 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.777+0000 7f6bf1276740 1 --2- 172.21.15.105:0/3996687513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56139c59c5d0 0x56139c597fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.777+0000 7f6bf1276740 1 -- 172.21.15.105:0/3996687513 >> 172.21.15.105:0/3996687513 conn(0x56139c565310 msgr2=0x56139c59e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:35.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.777+0000 7f6bf1276740 1 -- 172.21.15.105:0/3996687513 shutdown_connections 2024-09-16T02:05:35.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.777+0000 7f6bf1276740 1 -- 172.21.15.105:0/3996687513 wait complete. 2024-09-16T02:05:35.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.778+0000 7f6bf1276740 1 -- start start 2024-09-16T02:05:35.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.778+0000 7f6bf1276740 1 Processor -- start 2024-09-16T02:05:35.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.778+0000 7f6bf1276740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56139c59c5d0 0x56139c5af440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:35.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.778+0000 7f6bf1276740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56139c59db40 0x56139c5af980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:35.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.778+0000 7f6bf1276740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139c5a7370 0x56139c5afee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:35.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.778+0000 7f6bf1276740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6be81ac760 con 0x56139c5a7370 2024-09-16T02:05:35.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.778+0000 7f6bf1276740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6be81ac090 con 0x56139c59db40 2024-09-16T02:05:35.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.778+0000 7f6bf1276740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6be81a92d0 con 0x56139c59c5d0 2024-09-16T02:05:35.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.778+0000 7f6bf1276740 10 client.0 _ll_register_callbacks cb 0x7f6be0007600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:35.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.778+0000 7f6bed43a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139c5a7370 0x56139c5afee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:35.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.778+0000 7f6bed43a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139c5a7370 0x56139c5afee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46164/0 (socket says 172.21.15.105:46164) 2024-09-16T02:05:35.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.778+0000 7f6bed43a640 1 -- 172.21.15.105:0/1468865503 learned_addr learned my addr 172.21.15.105:0/1468865503 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:35.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.778+0000 7f6becc39640 1 --2- 172.21.15.105:0/1468865503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56139c59c5d0 0x56139c5af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:35.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.779+0000 7f6be7fff640 1 --2- 172.21.15.105:0/1468865503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56139c59db40 0x56139c5af980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:35.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.779+0000 7f6bed43a640 1 -- 172.21.15.105:0/1468865503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56139c59c5d0 msgr2=0x56139c5af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:35.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.779+0000 7f6bed43a640 1 --2- 172.21.15.105:0/1468865503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56139c59c5d0 0x56139c5af440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.779+0000 7f6bed43a640 1 -- 172.21.15.105:0/1468865503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56139c59db40 msgr2=0x56139c5af980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:35.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.779+0000 7f6bed43a640 1 --2- 172.21.15.105:0/1468865503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56139c59db40 0x56139c5af980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.779+0000 7f6bed43a640 1 -- 172.21.15.105:0/1468865503 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56139c620900 con 0x56139c5a7370 2024-09-16T02:05:35.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.779+0000 7f6be7fff640 1 --2- 172.21.15.105:0/1468865503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56139c59db40 0x56139c5af980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:05:35.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.779+0000 7f6bed43a640 1 --2- 172.21.15.105:0/1468865503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139c5a7370 0x56139c5afee0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6be00124b0 tx=0x7f6be0007cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd41ebdb70d9220 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:35.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.779+0000 7f6bc7fff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:05:35.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.779+0000 7f6bc7fff640 1 -- 172.21.15.105:0/1468865503 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be0025020 con 0x56139c5a7370 2024-09-16T02:05:35.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.779+0000 7f6bc7fff640 1 -- 172.21.15.105:0/1468865503 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f6be000fe10 con 0x56139c5a7370 2024-09-16T02:05:35.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.779+0000 7f6bf1276740 1 -- 172.21.15.105:0/1468865503 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bd4011cb0 con 0x56139c5a7370 2024-09-16T02:05:35.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.779+0000 7f6bc7fff640 1 -- 172.21.15.105:0/1468865503 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6be0030050 con 0x56139c5a7370 2024-09-16T02:05:35.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.779+0000 7f6bf1276740 10 client.15888 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:35.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.779+0000 7f6bf1276740 1 -- 172.21.15.105:0/1468865503 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56139c621c60 con 0x56139c5a7370 2024-09-16T02:05:35.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.780+0000 7f6bc7fff640 1 -- 172.21.15.105:0/1468865503 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6bd4011cb0 con 0x56139c5a7370 2024-09-16T02:05:35.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.780+0000 7f6bc7fff640 1 -- 172.21.15.105:0/1468865503 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6be001e050 con 0x56139c5a7370 2024-09-16T02:05:35.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.780+0000 7f6bf1276740 10 client.15888 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:35.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.780+0000 7f6bf1276740 10 client.15888 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:05:35.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.780+0000 7f6bf1276740 20 client.15888 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:35.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.780+0000 7f6bf1276740 1 --2- 172.21.15.105:0/1468865503 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x56139c5b6a40 0x56139c624470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:35.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.780+0000 7f6bf1276740 4 client.15888 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T02:05:35.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.780+0000 7f6bf1276740 1 -- 172.21.15.105:0/1468865503 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56139c509d70 con 0x56139c5b6a40 2024-09-16T02:05:35.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.780+0000 7f6becc39640 1 --2- 172.21.15.105:0/1468865503 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x56139c5b6a40 0x56139c624470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:35.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.780+0000 7f6bc7fff640 10 client.15888 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:05:35.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.780+0000 7f6becc39640 1 --2- 172.21.15.105:0/1468865503 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x56139c5b6a40 0x56139c624470 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d7ca30b2d771842 server_cookie=b2c9129112a2557e in_seq=0 out_seq=0 2024-09-16T02:05:35.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.781+0000 7f6bc7fff640 1 -- 172.21.15.105:0/1468865503 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56139c509d70 con 0x56139c5b6a40 2024-09-16T02:05:35.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.781+0000 7f6bc7fff640 10 client.15888 handle_command_reply: tid=0 2024-09-16T02:05:35.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.781+0000 7f6bf1276740 1 client.15888 shutdown 2024-09-16T02:05:35.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.781+0000 7f6bf1276740 1 -- 172.21.15.105:0/1468865503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139c5a7370 msgr2=0x56139c5afee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:35.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.781+0000 7f6bf1276740 1 --2- 172.21.15.105:0/1468865503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139c5a7370 0x56139c5afee0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6be00124b0 tx=0x7f6be0007cb0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.781+0000 7f6bf1276740 1 -- 172.21.15.105:0/1468865503 shutdown_connections 2024-09-16T02:05:35.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.781+0000 7f6bf1276740 1 --2- 172.21.15.105:0/1468865503 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x56139c5b6a40 0x56139c624470 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.781+0000 7f6bc7fff640 0 client.15888 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:05:35.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.781+0000 7f6bf1276740 1 --2- 172.21.15.105:0/1468865503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139c5a7370 0x56139c5afee0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.781+0000 7f6bf1276740 1 --2- 172.21.15.105:0/1468865503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56139c59db40 0x56139c5af980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.781+0000 7f6bf1276740 1 --2- 172.21.15.105:0/1468865503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56139c59c5d0 0x56139c5af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.781+0000 7f6bf1276740 1 -- 172.21.15.105:0/1468865503 >> 172.21.15.105:0/1468865503 conn(0x56139c565310 msgr2=0x56139c59f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:35.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.781+0000 7f6bf1276740 1 -- 172.21.15.105:0/1468865503 shutdown_connections 2024-09-16T02:05:35.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.781+0000 7f6bf1276740 1 -- 172.21.15.105:0/1468865503 wait complete. 2024-09-16T02:05:35.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.781+0000 7f6bf1276740 20 client.15888 trim_cache size 0 max 16384 2024-09-16T02:05:35.804 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-09-16T02:05:35.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.782+0000 7f6beee9d640 1 -- 172.21.15.105:0/696039166 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f6bd80a3240 msgr2=0x7f6bd80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:35.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.782+0000 7f6beee9d640 1 --2- 172.21.15.105:0/696039166 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f6bd80a3240 0x7f6bd80a5700 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6be00127b0 tx=0x7f6be0004ab0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.782+0000 7f6beee9d640 1 -- 172.21.15.105:0/696039166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6be8100e20 msgr2=0x7f6be8199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:35.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.782+0000 7f6beee9d640 1 --2- 172.21.15.105:0/696039166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6be8100e20 0x7f6be8199160 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc006010 tx=0x7f6bdc0042f0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.782+0000 7f6beee9d640 1 -- 172.21.15.105:0/696039166 shutdown_connections 2024-09-16T02:05:35.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.782+0000 7f6beee9d640 1 --2- 172.21.15.105:0/696039166 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f6bd80a3240 0x7f6bd80a5700 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.782+0000 7f6beee9d640 1 --2- 172.21.15.105:0/696039166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6be8102570 0x7f6be81996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.782+0000 7f6beee9d640 1 --2- 172.21.15.105:0/696039166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6be8102ef0 0x7f6be81a41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.782+0000 7f6beee9d640 1 --2- 172.21.15.105:0/696039166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6be8100e20 0x7f6be8199160 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.782+0000 7f6beee9d640 1 -- 172.21.15.105:0/696039166 >> 172.21.15.105:0/696039166 conn(0x7f6be80fc7d0 msgr2=0x7f6be80fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:35.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.782+0000 7f6beee9d640 1 -- 172.21.15.105:0/696039166 shutdown_connections 2024-09-16T02:05:35.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:35.782+0000 7f6beee9d640 1 -- 172.21.15.105:0/696039166 wait complete. 2024-09-16T02:05:36.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:36 smithi105 ceph-mon[31197]: pgmap v431: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-16T02:05:36.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:36 smithi137 ceph-mon[34099]: pgmap v431: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-16T02:05:37.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:36 smithi190 ceph-mon[34762]: pgmap v431: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-16T02:05:37.903 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T02:05:37.904 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T02:05:38.163 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:38.532 INFO:tasks.workunit.client.0.smithi105.stdout: 2024-09-16T02:05:38.532 INFO:tasks.workunit.client.0.smithi105.stdout: 2024-09-16T02:05:38.532 INFO:tasks.workunit.client.0.smithi105.stdout: 2024-09-16T02:05:38.532 INFO:tasks.workunit.client.0.smithi105.stdout:iozone test complete. 2024-09-16T02:05:38.532 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-16T02:05:38.533 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-16T02:05:38.572 INFO:tasks.workunit:Stopping ['suites/iozone.sh'] on client.0... 2024-09-16T02:05:38.572 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-16T02:05:38.864 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:38 smithi105 ceph-mon[31197]: pgmap v432: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-16T02:05:38.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:38 smithi137 ceph-mon[34099]: pgmap v432: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-16T02:05:38.933 DEBUG:teuthology.parallel:result is None 2024-09-16T02:05:38.934 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-16T02:05:38.973 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-16T02:05:38.973 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-16T02:05:38.983 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-16T02:05:38.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.947+0000 7fe1c8fe1640 1 -- 172.21.15.105:0/890061246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1c4105de0 msgr2=0x7fe1c410d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:38.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.947+0000 7fe1c8fe1640 1 --2- 172.21.15.105:0/890061246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1c4105de0 0x7fe1c410d960 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc009ca0 tx=0x7fe1bc02a8c0 comp rx=0 tx=0).stop 2024-09-16T02:05:38.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.947+0000 7fe1c8fe1640 1 -- 172.21.15.105:0/890061246 shutdown_connections 2024-09-16T02:05:38.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.947+0000 7fe1c8fe1640 1 --2- 172.21.15.105:0/890061246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1c4105de0 0x7fe1c410d960 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:38.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.947+0000 7fe1c8fe1640 1 --2- 172.21.15.105:0/890061246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1c4101e20 0x7fe1c41056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:38.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.947+0000 7fe1c8fe1640 1 --2- 172.21.15.105:0/890061246 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1c4101470 0x7fe1c4101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:38.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.947+0000 7fe1c8fe1640 1 -- 172.21.15.105:0/890061246 >> 172.21.15.105:0/890061246 conn(0x7fe1c4077230 msgr2=0x7fe1c40fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:38.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.947+0000 7fe1c8fe1640 1 -- 172.21.15.105:0/890061246 shutdown_connections 2024-09-16T02:05:38.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.947+0000 7fe1c8fe1640 1 -- 172.21.15.105:0/890061246 wait complete. 2024-09-16T02:05:38.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.948+0000 7fe1c8fe1640 1 Processor -- start 2024-09-16T02:05:38.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.948+0000 7fe1c8fe1640 1 -- start start 2024-09-16T02:05:38.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.948+0000 7fe1c8fe1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1c4101470 0x7fe1c40fc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:38.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.948+0000 7fe1c8fe1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1c4101e20 0x7fe1c40fcc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:38.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.948+0000 7fe1c8fe1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1c4105de0 0x7fe1c40fd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:38.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.948+0000 7fe1c8fe1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c4110cf0 con 0x7fe1c4101e20 2024-09-16T02:05:38.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.948+0000 7fe1c8fe1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c4110b70 con 0x7fe1c4101470 2024-09-16T02:05:38.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.948+0000 7fe1c8fe1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c4110e70 con 0x7fe1c4105de0 2024-09-16T02:05:38.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.948+0000 7fe1c1d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1c4101e20 0x7fe1c40fcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:38.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.948+0000 7fe1c1d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1c4101e20 0x7fe1c40fcc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49492/0 (socket says 172.21.15.105:49492) 2024-09-16T02:05:38.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.948+0000 7fe1c1d9b640 1 -- 172.21.15.105:0/2246122055 learned_addr learned my addr 172.21.15.105:0/2246122055 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:38.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.949+0000 7fe1c1d9b640 1 -- 172.21.15.105:0/2246122055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1c4105de0 msgr2=0x7fe1c40fd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:38.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.949+0000 7fe1c2d9d640 1 --2- 172.21.15.105:0/2246122055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1c4105de0 0x7fe1c40fd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:38.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.949+0000 7fe1c1d9b640 1 --2- 172.21.15.105:0/2246122055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1c4105de0 0x7fe1c40fd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:38.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.949+0000 7fe1c1d9b640 1 -- 172.21.15.105:0/2246122055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1c4101470 msgr2=0x7fe1c40fc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:38.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.949+0000 7fe1c1d9b640 1 --2- 172.21.15.105:0/2246122055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1c4101470 0x7fe1c40fc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:38.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.949+0000 7fe1c1d9b640 1 -- 172.21.15.105:0/2246122055 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1c41a67b0 con 0x7fe1c4101e20 2024-09-16T02:05:38.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.949+0000 7fe1c2d9d640 1 --2- 172.21.15.105:0/2246122055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1c4105de0 0x7fe1c40fd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:05:38.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.949+0000 7fe1c1d9b640 1 --2- 172.21.15.105:0/2246122055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1c4101e20 0x7fe1c40fcc40 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe1b400e9f0 tx=0x7fe1b400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:38.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.949+0000 7fe1bb7fe640 1 -- 172.21.15.105:0/2246122055 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b400cd80 con 0x7fe1c4101e20 2024-09-16T02:05:38.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.949+0000 7fe1bb7fe640 1 -- 172.21.15.105:0/2246122055 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1b40108e0 con 0x7fe1c4101e20 2024-09-16T02:05:38.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.949+0000 7fe1c8fe1640 1 -- 172.21.15.105:0/2246122055 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1c41a6aa0 con 0x7fe1c4101e20 2024-09-16T02:05:38.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.949+0000 7fe1bb7fe640 1 -- 172.21.15.105:0/2246122055 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4018920 con 0x7fe1c4101e20 2024-09-16T02:05:38.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.949+0000 7fe1c8fe1640 1 -- 172.21.15.105:0/2246122055 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1c41a6fe0 con 0x7fe1c4101e20 2024-09-16T02:05:38.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.951+0000 7fe1bb7fe640 1 -- 172.21.15.105:0/2246122055 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fe1b4010a90 con 0x7fe1c4101e20 2024-09-16T02:05:38.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.952+0000 7fe1bb7fe640 1 --2- 172.21.15.105:0/2246122055 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe1b00a3240 0x7fe1b00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:38.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.952+0000 7fe1cb3ba740 1 Processor -- start 2024-09-16T02:05:38.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.952+0000 7fe1c259c640 1 --2- 172.21.15.105:0/2246122055 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe1b00a3240 0x7fe1b00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:38.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.952+0000 7fe1bb7fe640 1 -- 172.21.15.105:0/2246122055 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fe1b40cfd60 con 0x7fe1c4101e20 2024-09-16T02:05:38.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.952+0000 7fe1cb3ba740 1 -- start start 2024-09-16T02:05:38.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.953+0000 7fe1cb3ba740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fafa33cb60 0x55fafa33cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:38.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.953+0000 7fe1cb3ba740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fafa33d480 0x55fafa347800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:38.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.953+0000 7fe1cb3ba740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fafa347d40 0x55fafa34a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:38.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.953+0000 7fe1cb3ba740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55fafa2a2280 con 0x55fafa33d480 2024-09-16T02:05:38.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.953+0000 7fe1cb3ba740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fafa249b10 con 0x55fafa347d40 2024-09-16T02:05:38.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.953+0000 7fe1cb3ba740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55fafa29e240 con 0x55fafa33cb60 2024-09-16T02:05:38.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.953+0000 7fe1c259c640 1 --2- 172.21.15.105:0/2246122055 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe1b00a3240 0x7fe1b00a5700 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac0058c0 tx=0x7fe1ac005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:38.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.953+0000 7fe1c1d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fafa33d480 0x55fafa347800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:38.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.953+0000 7fe1c1d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fafa33d480 0x55fafa347800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49508/0 (socket says 172.21.15.105:49508) 2024-09-16T02:05:38.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.953+0000 7fe1c1d9b640 1 -- 172.21.15.105:0/550973283 learned_addr learned my addr 172.21.15.105:0/550973283 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:38.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.953+0000 7fe1c2d9d640 1 --2- 172.21.15.105:0/550973283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fafa347d40 0x55fafa34a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:38.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.953+0000 7fe1c259c640 1 --2- 172.21.15.105:0/550973283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fafa33cb60 0x55fafa33cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:38.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.953+0000 7fe1c1d9b640 1 -- 172.21.15.105:0/550973283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fafa33cb60 msgr2=0x55fafa33cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:38.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.953+0000 7fe1c1d9b640 1 --2- 172.21.15.105:0/550973283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fafa33cb60 0x55fafa33cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:38.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.953+0000 7fe1c1d9b640 1 -- 172.21.15.105:0/550973283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fafa347d40 msgr2=0x55fafa34a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:38.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.953+0000 7fe1c1d9b640 1 --2- 172.21.15.105:0/550973283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fafa347d40 0x55fafa34a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:38.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.953+0000 7fe1c1d9b640 1 -- 172.21.15.105:0/550973283 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fafa2561f0 con 0x55fafa33d480 2024-09-16T02:05:38.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.953+0000 7fe1c2d9d640 1 --2- 172.21.15.105:0/550973283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fafa347d40 0x55fafa34a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:05:38.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.953+0000 7fe1c259c640 1 --2- 172.21.15.105:0/550973283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fafa33cb60 0x55fafa33cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:05:38.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.953+0000 7fe1c1d9b640 1 --2- 172.21.15.105:0/550973283 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fafa33d480 0x55fafa347800 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe1b409c2b0 tx=0x7fe1b409bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ac7e5d82867ae36 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:38.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.954+0000 7fe1b8ff9640 1 -- 172.21.15.105:0/550973283 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b40a06d0 con 0x55fafa33d480 2024-09-16T02:05:38.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.954+0000 7fe1b8ff9640 1 -- 172.21.15.105:0/550973283 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1b40a4f90 con 0x55fafa33d480 2024-09-16T02:05:38.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.954+0000 7fe1b8ff9640 1 -- 172.21.15.105:0/550973283 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b40bc0c0 con 0x55fafa33d480 2024-09-16T02:05:38.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.955+0000 7fe1cb3ba740 1 -- 172.21.15.105:0/550973283 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fafa33d480 msgr2=0x55fafa347800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:38.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.955+0000 7fe1cb3ba740 1 --2- 172.21.15.105:0/550973283 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fafa33d480 0x55fafa347800 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe1b409c2b0 tx=0x7fe1b409bda0 comp rx=0 tx=0).stop 2024-09-16T02:05:38.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.955+0000 7fe1cb3ba740 1 -- 172.21.15.105:0/550973283 shutdown_connections 2024-09-16T02:05:38.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.955+0000 7fe1cb3ba740 1 --2- 172.21.15.105:0/550973283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fafa347d40 0x55fafa34a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:38.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.955+0000 7fe1cb3ba740 1 --2- 172.21.15.105:0/550973283 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fafa33d480 0x55fafa347800 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:38.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.955+0000 7fe1cb3ba740 1 --2- 172.21.15.105:0/550973283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fafa33cb60 0x55fafa33cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:38.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.955+0000 7fe1cb3ba740 1 -- 172.21.15.105:0/550973283 >> 172.21.15.105:0/550973283 conn(0x55fafa3390f0 msgr2=0x55fafa3394d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:38.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.955+0000 7fe1cb3ba740 1 -- 172.21.15.105:0/550973283 shutdown_connections 2024-09-16T02:05:38.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.955+0000 7fe1cb3ba740 1 -- 172.21.15.105:0/550973283 wait complete. 2024-09-16T02:05:38.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.955+0000 7fe1cb3ba740 1 -- start start 2024-09-16T02:05:38.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.955+0000 7fe1cb3ba740 1 Processor -- start 2024-09-16T02:05:38.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.955+0000 7fe1cb3ba740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fafa33cb60 0x55fafa344c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:38.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.956+0000 7fe1cb3ba740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fafa33d480 0x55fafa345170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:38.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.956+0000 7fe1cb3ba740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fafa347d40 0x55fafa3456b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:38.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.956+0000 7fe1cb3ba740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55fafa34a8b0 con 0x55fafa347d40 2024-09-16T02:05:38.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.956+0000 7fe1cb3ba740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fafa25cf10 con 0x55fafa33d480 2024-09-16T02:05:38.999 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-16T02:05:38.999 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-09-16T02:05:38.999 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "2036e3e7-96d8-4d98-9b4d-360307b6f5f5", 2024-09-16T02:05:38.999 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-09-16T02:05:38.999 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-16T02:05:38.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.956+0000 7fe1cb3ba740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55fafa34aa30 con 0x55fafa33cb60 2024-09-16T02:05:39.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.956+0000 7fe1cb3ba740 10 client.0 _ll_register_callbacks cb 0x55fafa225a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:39.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.956+0000 7fe1c2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fafa347d40 0x55fafa3456b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:39.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.956+0000 7fe1c2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fafa347d40 0x55fafa3456b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49510/0 (socket says 172.21.15.105:49510) 2024-09-16T02:05:39.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.956+0000 7fe1c2d9d640 1 -- 172.21.15.105:0/475930329 learned_addr learned my addr 172.21.15.105:0/475930329 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:39.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.956+0000 7fe1c259c640 1 --2- 172.21.15.105:0/475930329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fafa33cb60 0x55fafa344c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:39.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.956+0000 7fe1c1d9b640 1 --2- 172.21.15.105:0/475930329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fafa33d480 0x55fafa345170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:39.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.956+0000 7fe1c2d9d640 1 -- 172.21.15.105:0/475930329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fafa33cb60 msgr2=0x55fafa344c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:39.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.956+0000 7fe1c2d9d640 1 --2- 172.21.15.105:0/475930329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fafa33cb60 0x55fafa344c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:39.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.956+0000 7fe1c2d9d640 1 -- 172.21.15.105:0/475930329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fafa33d480 msgr2=0x55fafa345170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:39.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.956+0000 7fe1c2d9d640 1 --2- 172.21.15.105:0/475930329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fafa33d480 0x55fafa345170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:39.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.956+0000 7fe1c2d9d640 1 -- 172.21.15.105:0/475930329 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fafa0136a0 con 0x55fafa347d40 2024-09-16T02:05:39.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.956+0000 7fe1c2d9d640 1 --2- 172.21.15.105:0/475930329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fafa347d40 0x55fafa3456b0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc009a60 tx=0x7fe1bc031f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45aca9187e71b8a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:39.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.956+0000 7fe1a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:05:39.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.956+0000 7fe1a9ffb640 1 -- 172.21.15.105:0/475930329 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc003970 con 0x55fafa347d40 2024-09-16T02:05:39.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.956+0000 7fe1a9ffb640 1 -- 172.21.15.105:0/475930329 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fe1bc003b10 con 0x55fafa347d40 2024-09-16T02:05:39.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.956+0000 7fe1cb3ba740 1 -- 172.21.15.105:0/475930329 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1b40a4f90 con 0x55fafa347d40 2024-09-16T02:05:39.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.957+0000 7fe1a9ffb640 1 -- 172.21.15.105:0/475930329 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe1bc04e050 con 0x55fafa347d40 2024-09-16T02:05:39.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.957+0000 7fe1cb3ba740 10 client.15900 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:39.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.957+0000 7fe1cb3ba740 1 -- 172.21.15.105:0/475930329 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fafa356480 con 0x55fafa347d40 2024-09-16T02:05:39.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.957+0000 7fe1a9ffb640 1 -- 172.21.15.105:0/475930329 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1bc036040 con 0x55fafa347d40 2024-09-16T02:05:39.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.957+0000 7fe1a9ffb640 1 -- 172.21.15.105:0/475930329 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe1bc030650 con 0x55fafa347d40 2024-09-16T02:05:39.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.957+0000 7fe1cb3ba740 10 client.15900 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:39.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.957+0000 7fe1cb3ba740 10 client.15900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:05:39.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.957+0000 7fe1cb3ba740 20 client.15900 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:39.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.957+0000 7fe1cb3ba740 1 --2- 172.21.15.105:0/475930329 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55fafa357870 0x55fafa359c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:39.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.957+0000 7fe1cb3ba740 4 client.15900 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:05:39.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.957+0000 7fe1cb3ba740 1 -- 172.21.15.105:0/475930329 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe1b40a4f90 con 0x55fafa357870 2024-09-16T02:05:39.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.958+0000 7fe1c259c640 1 --2- 172.21.15.105:0/475930329 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55fafa357870 0x55fafa359c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:39.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.958+0000 7fe1c259c640 1 --2- 172.21.15.105:0/475930329 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55fafa357870 0x55fafa359c60 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfb396686fe4593f server_cookie=54873721204d2fd7 in_seq=0 out_seq=0 2024-09-16T02:05:39.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.958+0000 7fe1a9ffb640 10 client.15900 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:05:39.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.959+0000 7fe1a9ffb640 1 -- 172.21.15.105:0/475930329 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fe1b40a4f90 con 0x55fafa357870 2024-09-16T02:05:39.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.959+0000 7fe1a9ffb640 10 client.15900 handle_command_reply: tid=0 2024-09-16T02:05:39.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.959+0000 7fe1cb3ba740 1 client.15900 shutdown 2024-09-16T02:05:39.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.959+0000 7fe1cb3ba740 1 -- 172.21.15.105:0/475930329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fafa347d40 msgr2=0x55fafa3456b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:39.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.959+0000 7fe1cb3ba740 1 --2- 172.21.15.105:0/475930329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fafa347d40 0x55fafa3456b0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc009a60 tx=0x7fe1bc031f70 comp rx=0 tx=0).stop 2024-09-16T02:05:39.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.959+0000 7fe1cb3ba740 1 -- 172.21.15.105:0/475930329 shutdown_connections 2024-09-16T02:05:39.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.959+0000 7fe1cb3ba740 1 --2- 172.21.15.105:0/475930329 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55fafa357870 0x55fafa359c60 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:39.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.959+0000 7fe1cb3ba740 1 --2- 172.21.15.105:0/475930329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fafa347d40 0x55fafa3456b0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:39.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.959+0000 7fe1a9ffb640 0 client.15900 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:05:39.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.959+0000 7fe1cb3ba740 1 --2- 172.21.15.105:0/475930329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fafa33d480 0x55fafa345170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:39.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.959+0000 7fe1cb3ba740 1 --2- 172.21.15.105:0/475930329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fafa33cb60 0x55fafa344c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:39.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.959+0000 7fe1cb3ba740 1 -- 172.21.15.105:0/475930329 >> 172.21.15.105:0/475930329 conn(0x55fafa3390f0 msgr2=0x55fafa33edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:39.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.959+0000 7fe1cb3ba740 1 -- 172.21.15.105:0/475930329 shutdown_connections 2024-09-16T02:05:39.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.959+0000 7fe1cb3ba740 1 -- 172.21.15.105:0/475930329 wait complete. 2024-09-16T02:05:39.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.959+0000 7fe1cb3ba740 20 client.15900 trim_cache size 0 max 16384 2024-09-16T02:05:39.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.976+0000 7fe1cb3ba740 1 Processor -- start 2024-09-16T02:05:39.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.976+0000 7fe1cb3ba740 1 -- start start 2024-09-16T02:05:39.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.977+0000 7fe1cb3ba740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fafa33cb60 0x55fafa346a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:39.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.977+0000 7fe1cb3ba740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fafa33d480 0x55fafa35f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:39.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.977+0000 7fe1cb3ba740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fafa347d40 0x55fafa35fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:39.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.977+0000 7fe1cb3ba740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c41af7e0 con 0x55fafa347d40 2024-09-16T02:05:39.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.977+0000 7fe1cb3ba740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c41b1320 con 0x55fafa33d480 2024-09-16T02:05:39.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.977+0000 7fe1cb3ba740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c41afeb0 con 0x55fafa33cb60 2024-09-16T02:05:39.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.977+0000 7fe1c2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fafa347d40 0x55fafa35fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:39.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.977+0000 7fe1c2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fafa347d40 0x55fafa35fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49524/0 (socket says 172.21.15.105:49524) 2024-09-16T02:05:39.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.977+0000 7fe1c2d9d640 1 -- 172.21.15.105:0/3787224805 learned_addr learned my addr 172.21.15.105:0/3787224805 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:39.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.977+0000 7fe1c1d9b640 1 --2- 172.21.15.105:0/3787224805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fafa33d480 0x55fafa35f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:39.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.977+0000 7fe1c259c640 1 --2- 172.21.15.105:0/3787224805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fafa33cb60 0x55fafa346a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:39.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.977+0000 7fe1c2d9d640 1 -- 172.21.15.105:0/3787224805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fafa33cb60 msgr2=0x55fafa346a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:39.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.977+0000 7fe1c2d9d640 1 --2- 172.21.15.105:0/3787224805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fafa33cb60 0x55fafa346a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:39.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.977+0000 7fe1c2d9d640 1 -- 172.21.15.105:0/3787224805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fafa33d480 msgr2=0x55fafa35f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:39.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.977+0000 7fe1c2d9d640 1 --2- 172.21.15.105:0/3787224805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fafa33d480 0x55fafa35f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:39.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.977+0000 7fe1c2d9d640 1 -- 172.21.15.105:0/3787224805 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fafa343be0 con 0x55fafa347d40 2024-09-16T02:05:39.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.977+0000 7fe1c2d9d640 1 --2- 172.21.15.105:0/3787224805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fafa347d40 0x55fafa35fa30 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc009ca0 tx=0x7fe1bc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c52ddbf3794819 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:39.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.977+0000 7fe1abfff640 1 -- 172.21.15.105:0/3787224805 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc057070 con 0x55fafa347d40 2024-09-16T02:05:39.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.977+0000 7fe1abfff640 1 -- 172.21.15.105:0/3787224805 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1bc036070 con 0x55fafa347d40 2024-09-16T02:05:39.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.977+0000 7fe1abfff640 1 -- 172.21.15.105:0/3787224805 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc045930 con 0x55fafa347d40 2024-09-16T02:05:39.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.978+0000 7fe1cb3ba740 1 -- 172.21.15.105:0/3787224805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fafa347d40 msgr2=0x55fafa35fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:39.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.978+0000 7fe1cb3ba740 1 --2- 172.21.15.105:0/3787224805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fafa347d40 0x55fafa35fa30 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc009ca0 tx=0x7fe1bc009b20 comp rx=0 tx=0).stop 2024-09-16T02:05:39.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.978+0000 7fe1cb3ba740 1 -- 172.21.15.105:0/3787224805 shutdown_connections 2024-09-16T02:05:39.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.978+0000 7fe1cb3ba740 1 --2- 172.21.15.105:0/3787224805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fafa347d40 0x55fafa35fa30 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:39.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.978+0000 7fe1cb3ba740 1 --2- 172.21.15.105:0/3787224805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fafa33d480 0x55fafa35f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:39.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.978+0000 7fe1cb3ba740 1 --2- 172.21.15.105:0/3787224805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fafa33cb60 0x55fafa346a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:39.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.978+0000 7fe1cb3ba740 1 -- 172.21.15.105:0/3787224805 >> 172.21.15.105:0/3787224805 conn(0x55fafa3390f0 msgr2=0x55fafa346380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:39.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.978+0000 7fe1cb3ba740 1 -- 172.21.15.105:0/3787224805 shutdown_connections 2024-09-16T02:05:39.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.978+0000 7fe1cb3ba740 1 -- 172.21.15.105:0/3787224805 wait complete. 2024-09-16T02:05:39.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.978+0000 7fe1cb3ba740 1 -- start start 2024-09-16T02:05:39.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.979+0000 7fe1cb3ba740 1 Processor -- start 2024-09-16T02:05:39.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.979+0000 7fe1cb3ba740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fafa33cb60 0x55fafa34c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:39.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.979+0000 7fe1cb3ba740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fafa33d480 0x55fafa34ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:39.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.979+0000 7fe1cb3ba740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fafa347d40 0x55fafa34fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:39.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.979+0000 7fe1cb3ba740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c41b0c50 con 0x55fafa33cb60 2024-09-16T02:05:39.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.979+0000 7fe1cb3ba740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c41b0580 con 0x55fafa347d40 2024-09-16T02:05:39.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.979+0000 7fe1cb3ba740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c41ad7c0 con 0x55fafa33d480 2024-09-16T02:05:39.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.979+0000 7fe1cb3ba740 10 client.0 _ll_register_callbacks cb 0x7fe1b4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:39.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.979+0000 7fe1c259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fafa33cb60 0x55fafa34c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:39.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.979+0000 7fe1c259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fafa33cb60 0x55fafa34c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49530/0 (socket says 172.21.15.105:49530) 2024-09-16T02:05:39.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.979+0000 7fe1c259c640 1 -- 172.21.15.105:0/1589633797 learned_addr learned my addr 172.21.15.105:0/1589633797 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:39.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.979+0000 7fe1c2d9d640 1 --2- 172.21.15.105:0/1589633797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fafa347d40 0x55fafa34fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:39.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.979+0000 7fe1c1d9b640 1 --2- 172.21.15.105:0/1589633797 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fafa33d480 0x55fafa34ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:39.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.979+0000 7fe1c259c640 1 -- 172.21.15.105:0/1589633797 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fafa33d480 msgr2=0x55fafa34ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:39.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.979+0000 7fe1c259c640 1 --2- 172.21.15.105:0/1589633797 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fafa33d480 0x55fafa34ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:39.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.979+0000 7fe1c259c640 1 -- 172.21.15.105:0/1589633797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fafa347d40 msgr2=0x55fafa34fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:39.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.979+0000 7fe1c259c640 1 --2- 172.21.15.105:0/1589633797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fafa347d40 0x55fafa34fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:39.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.979+0000 7fe1c259c640 1 -- 172.21.15.105:0/1589633797 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fafa3501c0 con 0x55fafa33cb60 2024-09-16T02:05:39.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.979+0000 7fe1c259c640 1 --2- 172.21.15.105:0/1589633797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fafa33cb60 0x55fafa34c520 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac00ed80 tx=0x7fe1ac011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0722cdacdf87f05 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:39.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.979+0000 7fe1a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:05:39.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.980+0000 7fe1a9ffb640 1 -- 172.21.15.105:0/1589633797 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac035060 con 0x55fafa33cb60 2024-09-16T02:05:39.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.980+0000 7fe1a9ffb640 1 -- 172.21.15.105:0/1589633797 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fe1ac010c70 con 0x55fafa33cb60 2024-09-16T02:05:39.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.980+0000 7fe1cb3ba740 1 -- 172.21.15.105:0/1589633797 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1bc036070 con 0x55fafa33cb60 2024-09-16T02:05:39.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.980+0000 7fe1a9ffb640 1 -- 172.21.15.105:0/1589633797 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe1ac0283f0 con 0x55fafa33cb60 2024-09-16T02:05:39.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.980+0000 7fe1cb3ba740 10 client.15906 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:39.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.980+0000 7fe1cb3ba740 1 -- 172.21.15.105:0/1589633797 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fafa3c3130 con 0x55fafa33cb60 2024-09-16T02:05:39.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.980+0000 7fe1a9ffb640 1 -- 172.21.15.105:0/1589633797 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1bc036070 con 0x55fafa33cb60 2024-09-16T02:05:39.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.980+0000 7fe1a9ffb640 1 -- 172.21.15.105:0/1589633797 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe1ac039030 con 0x55fafa33cb60 2024-09-16T02:05:39.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.980+0000 7fe1cb3ba740 10 client.15906 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:39.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.980+0000 7fe1cb3ba740 10 client.15906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:05:39.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.980+0000 7fe1cb3ba740 20 client.15906 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:39.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.980+0000 7fe1cb3ba740 1 --2- 172.21.15.105:0/1589633797 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55fafa3c3860 0x55fafa3c5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:39.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.980+0000 7fe1cb3ba740 4 client.15906 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T02:05:39.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.980+0000 7fe1cb3ba740 1 -- 172.21.15.105:0/1589633797 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fafa2af7f0 con 0x55fafa3c3860 2024-09-16T02:05:39.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.980+0000 7fe1c1d9b640 1 --2- 172.21.15.105:0/1589633797 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55fafa3c3860 0x55fafa3c5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:39.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.981+0000 7fe1a9ffb640 10 client.15906 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:05:39.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.981+0000 7fe1c1d9b640 1 --2- 172.21.15.105:0/1589633797 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55fafa3c3860 0x55fafa3c5c80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=708b3e09490086ef server_cookie=98cf57ae45d3201 in_seq=0 out_seq=0 2024-09-16T02:05:39.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.982+0000 7fe1a9ffb640 1 -- 172.21.15.105:0/1589633797 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fafa2af7f0 con 0x55fafa3c3860 2024-09-16T02:05:39.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.982+0000 7fe1a9ffb640 10 client.15906 handle_command_reply: tid=0 2024-09-16T02:05:39.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.982+0000 7fe1cb3ba740 1 client.15906 shutdown 2024-09-16T02:05:39.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.982+0000 7fe1cb3ba740 1 -- 172.21.15.105:0/1589633797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fafa33cb60 msgr2=0x55fafa34c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:39.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.982+0000 7fe1cb3ba740 1 --2- 172.21.15.105:0/1589633797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fafa33cb60 0x55fafa34c520 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac00ed80 tx=0x7fe1ac011000 comp rx=0 tx=0).stop 2024-09-16T02:05:39.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.982+0000 7fe1cb3ba740 1 -- 172.21.15.105:0/1589633797 shutdown_connections 2024-09-16T02:05:39.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.982+0000 7fe1cb3ba740 1 --2- 172.21.15.105:0/1589633797 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55fafa3c3860 0x55fafa3c5c80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:39.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.982+0000 7fe1cb3ba740 1 --2- 172.21.15.105:0/1589633797 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fafa33d480 0x55fafa34ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:39.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.982+0000 7fe1cb3ba740 1 --2- 172.21.15.105:0/1589633797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fafa347d40 0x55fafa34fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:39.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.982+0000 7fe1cb3ba740 1 --2- 172.21.15.105:0/1589633797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fafa33cb60 0x55fafa34c520 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:39.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.982+0000 7fe1cb3ba740 1 -- 172.21.15.105:0/1589633797 >> 172.21.15.105:0/1589633797 conn(0x55fafa3390f0 msgr2=0x55fafa3453f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:39.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.982+0000 7fe1a9ffb640 0 client.15906 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:05:39.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.982+0000 7fe1cb3ba740 1 -- 172.21.15.105:0/1589633797 shutdown_connections 2024-09-16T02:05:39.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.982+0000 7fe1cb3ba740 1 -- 172.21.15.105:0/1589633797 wait complete. 2024-09-16T02:05:39.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.982+0000 7fe1cb3ba740 20 client.15906 trim_cache size 0 max 16384 2024-09-16T02:05:39.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.983+0000 7fe1c8fe1640 1 -- 172.21.15.105:0/2246122055 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe1b00a3240 msgr2=0x7fe1b00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:39.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.983+0000 7fe1c8fe1640 1 --2- 172.21.15.105:0/2246122055 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe1b00a3240 0x7fe1b00a5700 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac0058c0 tx=0x7fe1ac005850 comp rx=0 tx=0).stop 2024-09-16T02:05:39.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.983+0000 7fe1c8fe1640 1 -- 172.21.15.105:0/2246122055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1c4101e20 msgr2=0x7fe1c40fcc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:39.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.983+0000 7fe1c8fe1640 1 --2- 172.21.15.105:0/2246122055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1c4101e20 0x7fe1c40fcc40 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe1b400e9f0 tx=0x7fe1b400eeb0 comp rx=0 tx=0).stop 2024-09-16T02:05:39.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.983+0000 7fe1c8fe1640 1 -- 172.21.15.105:0/2246122055 shutdown_connections 2024-09-16T02:05:39.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.983+0000 7fe1c8fe1640 1 --2- 172.21.15.105:0/2246122055 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fe1b00a3240 0x7fe1b00a5700 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:39.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.983+0000 7fe1c8fe1640 1 --2- 172.21.15.105:0/2246122055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1c4105de0 0x7fe1c40fd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:39.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.983+0000 7fe1c8fe1640 1 --2- 172.21.15.105:0/2246122055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1c4101e20 0x7fe1c40fcc40 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:39.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.983+0000 7fe1c8fe1640 1 --2- 172.21.15.105:0/2246122055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1c4101470 0x7fe1c40fc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:39.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.983+0000 7fe1c8fe1640 1 -- 172.21.15.105:0/2246122055 >> 172.21.15.105:0/2246122055 conn(0x7fe1c4077230 msgr2=0x7fe1c41003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:39.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.983+0000 7fe1c8fe1640 1 -- 172.21.15.105:0/2246122055 shutdown_connections 2024-09-16T02:05:39.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:38.983+0000 7fe1c8fe1640 1 -- 172.21.15.105:0/2246122055 wait complete. 2024-09-16T02:05:39.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:38 smithi190 ceph-mon[34762]: pgmap v432: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-16T02:05:39.827 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T02:05:40.111 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:40.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.680+0000 7f541ad8c640 1 -- 172.21.15.105:0/968260265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5414101420 msgr2=0x7f541410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:40.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.680+0000 7f541ad8c640 1 --2- 172.21.15.105:0/968260265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5414101420 0x7f541410d950 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f540c009cd0 tx=0x7f540c02a8e0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.681+0000 7f541ad8c640 1 -- 172.21.15.105:0/968260265 shutdown_connections 2024-09-16T02:05:40.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.681+0000 7f541ad8c640 1 --2- 172.21.15.105:0/968260265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5414101420 0x7f541410d950 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.681+0000 7f541ad8c640 1 --2- 172.21.15.105:0/968260265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5414100a80 0x7f5414100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.681+0000 7f541ad8c640 1 --2- 172.21.15.105:0/968260265 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5414106a80 0x7f5414106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.681+0000 7f541ad8c640 1 -- 172.21.15.105:0/968260265 >> 172.21.15.105:0/968260265 conn(0x7f54140fc7a0 msgr2=0x7f54140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:40.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.681+0000 7f541ad8c640 1 -- 172.21.15.105:0/968260265 shutdown_connections 2024-09-16T02:05:40.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.681+0000 7f541ad8c640 1 -- 172.21.15.105:0/968260265 wait complete. 2024-09-16T02:05:40.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.682+0000 7f541ad8c640 1 Processor -- start 2024-09-16T02:05:40.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.682+0000 7f541ad8c640 1 -- start start 2024-09-16T02:05:40.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.683+0000 7f541ad8c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5414100a80 0x7f54141079d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:40.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.683+0000 7f541ad8c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5414101420 0x7f5414107f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:40.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.683+0000 7f541ad8c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5414106a80 0x7f5414108450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:40.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.683+0000 7f541ad8c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f541410fcc0 con 0x7f5414101420 2024-09-16T02:05:40.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.683+0000 7f541ad8c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f541410fb40 con 0x7f5414100a80 2024-09-16T02:05:40.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.683+0000 7f541ad8c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f541410fe40 con 0x7f5414106a80 2024-09-16T02:05:40.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.683+0000 7f5418b28640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5414100a80 0x7f54141079d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:40.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.683+0000 7f5419329640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5414106a80 0x7f5414108450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:40.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.683+0000 7f5419329640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5414106a80 0x7f5414108450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.105:55330/0 (socket says 172.21.15.105:55330) 2024-09-16T02:05:40.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.683+0000 7f5418b28640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5414100a80 0x7f54141079d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.105:48922/0 (socket says 172.21.15.105:48922) 2024-09-16T02:05:40.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.683+0000 7f5419329640 1 -- 172.21.15.105:0/2560476357 learned_addr learned my addr 172.21.15.105:0/2560476357 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:40.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.684+0000 7f5413fff640 1 --2- 172.21.15.105:0/2560476357 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5414101420 0x7f5414107f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:40.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.684+0000 7f5419329640 1 -- 172.21.15.105:0/2560476357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5414100a80 msgr2=0x7f54141079d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:40.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.684+0000 7f5419329640 1 --2- 172.21.15.105:0/2560476357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5414100a80 0x7f54141079d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.684+0000 7f5419329640 1 -- 172.21.15.105:0/2560476357 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5414101420 msgr2=0x7f5414107f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:40.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.684+0000 7f5419329640 1 --2- 172.21.15.105:0/2560476357 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5414101420 0x7f5414107f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.684+0000 7f5419329640 1 -- 172.21.15.105:0/2560476357 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f541419a220 con 0x7f5414106a80 2024-09-16T02:05:40.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.685+0000 7f5419329640 1 --2- 172.21.15.105:0/2560476357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5414106a80 0x7f5414108450 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f540c009bf0 tx=0x7f540c003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:40.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.685+0000 7f5411ffb640 1 -- 172.21.15.105:0/2560476357 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540c004240 con 0x7f5414106a80 2024-09-16T02:05:40.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.685+0000 7f5411ffb640 1 -- 172.21.15.105:0/2560476357 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f540c0305a0 con 0x7f5414106a80 2024-09-16T02:05:40.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.685+0000 7f541ad8c640 1 -- 172.21.15.105:0/2560476357 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f541419a4b0 con 0x7f5414106a80 2024-09-16T02:05:40.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.686+0000 7f5411ffb640 1 -- 172.21.15.105:0/2560476357 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540c0466e0 con 0x7f5414106a80 2024-09-16T02:05:40.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.686+0000 7f541ad8c640 1 -- 172.21.15.105:0/2560476357 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f541419a9f0 con 0x7f5414106a80 2024-09-16T02:05:40.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.690+0000 7f5411ffb640 1 -- 172.21.15.105:0/2560476357 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f540c0043e0 con 0x7f5414106a80 2024-09-16T02:05:40.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.691+0000 7f541d165740 1 Processor -- start 2024-09-16T02:05:40.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.691+0000 7f541d165740 1 -- start start 2024-09-16T02:05:40.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.692+0000 7f5411ffb640 1 --2- 172.21.15.105:0/2560476357 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f54040a3390 0x7f54040a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:40.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.692+0000 7f541d165740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b6778c4a0 0x557b6778c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:40.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.692+0000 7f541d165740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557b6778cdc0 0x557b67797140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:40.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.692+0000 7f5411ffb640 1 -- 172.21.15.105:0/2560476357 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f540c0f0d70 con 0x7f5414106a80 2024-09-16T02:05:40.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.692+0000 7f541d165740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b67797680 0x557b67799a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:40.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.692+0000 7f541d165740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x557b676f22e0 con 0x557b6778cdc0 2024-09-16T02:05:40.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.692+0000 7f541d165740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557b67699ce0 con 0x557b6778c4a0 2024-09-16T02:05:40.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.692+0000 7f541d165740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x557b676ee2a0 con 0x557b67797680 2024-09-16T02:05:40.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.692+0000 7f5418b28640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b6778c4a0 0x557b6778c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:40.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.692+0000 7f5418b28640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b6778c4a0 0x557b6778c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.105:48938/0 (socket says 172.21.15.105:48938) 2024-09-16T02:05:40.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.692+0000 7f5418b28640 1 -- 172.21.15.105:0/867876354 learned_addr learned my addr 172.21.15.105:0/867876354 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:40.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.692+0000 7f5413fff640 1 --2- 172.21.15.105:0/867876354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557b6778cdc0 0x557b67797140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:40.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.692+0000 7f5419329640 1 --2- 172.21.15.105:0/867876354 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b67797680 0x557b67799a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:40.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.692+0000 7f5413fff640 1 --2- 172.21.15.105:0/2560476357 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f54040a3390 0x7f54040a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:40.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.693+0000 7f5419329640 1 -- 172.21.15.105:0/867876354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b6778c4a0 msgr2=0x557b6778c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:40.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.693+0000 7f5419329640 1 --2- 172.21.15.105:0/867876354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b6778c4a0 0x557b6778c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.693+0000 7f5419329640 1 -- 172.21.15.105:0/867876354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557b6778cdc0 msgr2=0x557b67797140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:40.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.693+0000 7f5419329640 1 --2- 172.21.15.105:0/867876354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557b6778cdc0 0x557b67797140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.693+0000 7f5419329640 1 -- 172.21.15.105:0/867876354 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b6767e440 con 0x557b67797680 2024-09-16T02:05:40.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.693+0000 7f5419329640 1 --2- 172.21.15.105:0/867876354 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b67797680 0x557b67799a70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f540c0ee860 tx=0x7f540c0b95d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb8453daeb2c1bd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:40.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.693+0000 7f5413fff640 1 --2- 172.21.15.105:0/2560476357 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f54040a3390 0x7f54040a5850 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x557b67789890 tx=0x7f5400009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:40.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.694+0000 7f53feffd640 1 -- 172.21.15.105:0/867876354 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540c0bd330 con 0x557b67797680 2024-09-16T02:05:40.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.694+0000 7f53feffd640 1 -- 172.21.15.105:0/867876354 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f540c036030 con 0x557b67797680 2024-09-16T02:05:40.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.694+0000 7f53feffd640 1 -- 172.21.15.105:0/867876354 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540c0bc780 con 0x557b67797680 2024-09-16T02:05:40.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.695+0000 7f541d165740 1 -- 172.21.15.105:0/867876354 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b67797680 msgr2=0x557b67799a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:40.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.695+0000 7f541d165740 1 --2- 172.21.15.105:0/867876354 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b67797680 0x557b67799a70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f540c0ee860 tx=0x7f540c0b95d0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.695+0000 7f541d165740 1 -- 172.21.15.105:0/867876354 shutdown_connections 2024-09-16T02:05:40.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.695+0000 7f541d165740 1 --2- 172.21.15.105:0/867876354 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b67797680 0x557b67799a70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.695+0000 7f541d165740 1 --2- 172.21.15.105:0/867876354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557b6778cdc0 0x557b67797140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.695+0000 7f541d165740 1 --2- 172.21.15.105:0/867876354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b6778c4a0 0x557b6778c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.695+0000 7f541d165740 1 -- 172.21.15.105:0/867876354 >> 172.21.15.105:0/867876354 conn(0x557b67788a30 msgr2=0x557b67788e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:40.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.695+0000 7f541d165740 1 -- 172.21.15.105:0/867876354 shutdown_connections 2024-09-16T02:05:40.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.695+0000 7f541d165740 1 -- 172.21.15.105:0/867876354 wait complete. 2024-09-16T02:05:40.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.695+0000 7f541d165740 1 -- start start 2024-09-16T02:05:40.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.695+0000 7f541d165740 1 Processor -- start 2024-09-16T02:05:40.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.696+0000 7f541d165740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557b6778c4a0 0x557b67797000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:40.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.696+0000 7f541d165740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b6778cdc0 0x557b677942b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:40.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.696+0000 7f541d165740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b67797680 0x557b67794830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:40.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.696+0000 7f541d165740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x557b6779a010 con 0x557b6778c4a0 2024-09-16T02:05:40.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.696+0000 7f541d165740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557b676ad050 con 0x557b67797680 2024-09-16T02:05:40.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.696+0000 7f541d165740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x557b6779a190 con 0x557b6778cdc0 2024-09-16T02:05:40.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.696+0000 7f5418b28640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557b6778c4a0 0x557b67797000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:40.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.696+0000 7f541d165740 10 client.0 _ll_register_callbacks cb 0x557b676ed160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:40.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.696+0000 7f5418b28640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557b6778c4a0 0x557b67797000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49566/0 (socket says 172.21.15.105:49566) 2024-09-16T02:05:40.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.696+0000 7f5418b28640 1 -- 172.21.15.105:0/3780289383 learned_addr learned my addr 172.21.15.105:0/3780289383 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:40.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.696+0000 7f5413fff640 1 --2- 172.21.15.105:0/3780289383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b6778cdc0 0x557b677942b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:40.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.697+0000 7f5418b28640 1 -- 172.21.15.105:0/3780289383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b6778cdc0 msgr2=0x557b677942b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:40.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.697+0000 7f5418b28640 1 --2- 172.21.15.105:0/3780289383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b6778cdc0 0x557b677942b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.697+0000 7f5418b28640 1 -- 172.21.15.105:0/3780289383 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b67797680 msgr2=0x557b67794830 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T02:05:40.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.697+0000 7f5418b28640 1 --2- 172.21.15.105:0/3780289383 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b67797680 0x557b67794830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.697+0000 7f5418b28640 1 -- 172.21.15.105:0/3780289383 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b674633c0 con 0x557b6778c4a0 2024-09-16T02:05:40.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.697+0000 7f5413fff640 1 --2- 172.21.15.105:0/3780289383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b6778cdc0 0x557b677942b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:05:40.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.697+0000 7f5418b28640 1 --2- 172.21.15.105:0/3780289383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557b6778c4a0 0x557b67797000 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5408012840 tx=0x7f5408012d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=638c94adb6b97102 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:40.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.697+0000 7f53f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:05:40.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.697+0000 7f53f3fff640 1 -- 172.21.15.105:0/3780289383 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540800f920 con 0x557b6778c4a0 2024-09-16T02:05:40.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.697+0000 7f53f3fff640 1 -- 172.21.15.105:0/3780289383 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f540800fac0 con 0x557b6778c4a0 2024-09-16T02:05:40.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.697+0000 7f541d165740 1 -- 172.21.15.105:0/3780289383 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f540c036030 con 0x557b6778c4a0 2024-09-16T02:05:40.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.697+0000 7f53f3fff640 1 -- 172.21.15.105:0/3780289383 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f540800fcc0 con 0x557b6778c4a0 2024-09-16T02:05:40.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.697+0000 7f541d165740 10 client.15918 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:40.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.697+0000 7f541d165740 1 -- 172.21.15.105:0/3780289383 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557b677a59d0 con 0x557b6778c4a0 2024-09-16T02:05:40.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.698+0000 7f53f3fff640 1 -- 172.21.15.105:0/3780289383 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f540c036030 con 0x557b6778c4a0 2024-09-16T02:05:40.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.698+0000 7f53f3fff640 1 -- 172.21.15.105:0/3780289383 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5408024020 con 0x557b6778c4a0 2024-09-16T02:05:40.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.698+0000 7f541d165740 10 client.15918 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:40.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.698+0000 7f541d165740 10 client.15918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:05:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.698+0000 7f541d165740 20 client.15918 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.698+0000 7f541d165740 1 --2- 172.21.15.105:0/3780289383 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x557b677a6e50 0x557b677a9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.698+0000 7f541d165740 4 client.15918 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:05:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.698+0000 7f541d165740 1 -- 172.21.15.105:0/3780289383 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b676ff850 con 0x557b677a6e50 2024-09-16T02:05:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.699+0000 7f5413fff640 1 --2- 172.21.15.105:0/3780289383 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x557b677a6e50 0x557b677a9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.701+0000 7f5413fff640 1 --2- 172.21.15.105:0/3780289383 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x557b677a6e50 0x557b677a9310 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ef37a4489f51589 server_cookie=9feafc33f8fdf29c in_seq=0 out_seq=0 2024-09-16T02:05:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.701+0000 7f53f3fff640 10 client.15918 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:05:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.702+0000 7f53f3fff640 1 -- 172.21.15.105:0/3780289383 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557b676ff850 con 0x557b677a6e50 2024-09-16T02:05:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.702+0000 7f53f3fff640 10 client.15918 handle_command_reply: tid=0 2024-09-16T02:05:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.702+0000 7f541d165740 1 client.15918 shutdown 2024-09-16T02:05:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.702+0000 7f541d165740 1 -- 172.21.15.105:0/3780289383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557b6778c4a0 msgr2=0x557b67797000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.702+0000 7f541d165740 1 --2- 172.21.15.105:0/3780289383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557b6778c4a0 0x557b67797000 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5408012840 tx=0x7f5408012d00 comp rx=0 tx=0).stop 2024-09-16T02:05:40.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.703+0000 7f541d165740 1 -- 172.21.15.105:0/3780289383 shutdown_connections 2024-09-16T02:05:40.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.703+0000 7f541d165740 1 --2- 172.21.15.105:0/3780289383 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x557b677a6e50 0x557b677a9310 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.703+0000 7f541d165740 1 --2- 172.21.15.105:0/3780289383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b6778cdc0 0x557b677942b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.703+0000 7f541d165740 1 --2- 172.21.15.105:0/3780289383 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b67797680 0x557b67794830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.703+0000 7f541d165740 1 --2- 172.21.15.105:0/3780289383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557b6778c4a0 0x557b67797000 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.703+0000 7f541d165740 1 -- 172.21.15.105:0/3780289383 >> 172.21.15.105:0/3780289383 conn(0x557b67788a30 msgr2=0x557b6778e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:40.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.703+0000 7f53f3fff640 0 client.15918 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:05:40.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.703+0000 7f541d165740 1 -- 172.21.15.105:0/3780289383 shutdown_connections 2024-09-16T02:05:40.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.703+0000 7f541d165740 1 -- 172.21.15.105:0/3780289383 wait complete. 2024-09-16T02:05:40.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.703+0000 7f541d165740 20 client.15918 trim_cache size 0 max 16384 2024-09-16T02:05:40.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.734+0000 7f541d165740 1 Processor -- start 2024-09-16T02:05:40.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.734+0000 7f541d165740 1 -- start start 2024-09-16T02:05:40.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.734+0000 7f541d165740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557b6778c4a0 0x557b677a31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:40.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.735+0000 7f541d165740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b6778cdc0 0x557b67793290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:40.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.735+0000 7f541d165740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b67797680 0x557b677aec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:40.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.735+0000 7f541d165740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f53f800e6c0 con 0x557b6778c4a0 2024-09-16T02:05:40.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.735+0000 7f541d165740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f53f8010200 con 0x557b6778cdc0 2024-09-16T02:05:40.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.735+0000 7f5418b28640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557b6778c4a0 0x557b677a31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:40.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.735+0000 7f541d165740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f53f800ed90 con 0x557b67797680 2024-09-16T02:05:40.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.735+0000 7f5418b28640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557b6778c4a0 0x557b677a31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49576/0 (socket says 172.21.15.105:49576) 2024-09-16T02:05:40.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.735+0000 7f5418b28640 1 -- 172.21.15.105:0/4220583621 learned_addr learned my addr 172.21.15.105:0/4220583621 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:40.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.735+0000 7f5419329640 1 --2- 172.21.15.105:0/4220583621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b67797680 0x557b677aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:40.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.735+0000 7f5413fff640 1 --2- 172.21.15.105:0/4220583621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b6778cdc0 0x557b67793290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:40.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.735+0000 7f5419329640 1 -- 172.21.15.105:0/4220583621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b6778cdc0 msgr2=0x557b67793290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:40.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.735+0000 7f5419329640 1 --2- 172.21.15.105:0/4220583621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b6778cdc0 0x557b67793290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.735+0000 7f5419329640 1 -- 172.21.15.105:0/4220583621 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557b6778c4a0 msgr2=0x557b677a31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:40.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.735+0000 7f5419329640 1 --2- 172.21.15.105:0/4220583621 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557b6778c4a0 0x557b677a31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.735+0000 7f5419329640 1 -- 172.21.15.105:0/4220583621 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b67794ba0 con 0x557b67797680 2024-09-16T02:05:40.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.736+0000 7f5419329640 1 --2- 172.21.15.105:0/4220583621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b67797680 0x557b677aec70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f540c0eeb80 tx=0x7f540c02a8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24057ffd3a161e61 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:40.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.736+0000 7f53fe7fc640 1 -- 172.21.15.105:0/4220583621 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540c0bd330 con 0x557b67797680 2024-09-16T02:05:40.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.736+0000 7f53fe7fc640 1 -- 172.21.15.105:0/4220583621 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f540c036050 con 0x557b67797680 2024-09-16T02:05:40.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.736+0000 7f5418b28640 1 --2- 172.21.15.105:0/4220583621 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557b6778c4a0 0x557b677a31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:05:40.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.736+0000 7f53fe7fc640 1 -- 172.21.15.105:0/4220583621 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540c0b99a0 con 0x557b67797680 2024-09-16T02:05:40.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.737+0000 7f541d165740 1 -- 172.21.15.105:0/4220583621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b67797680 msgr2=0x557b677aec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:40.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.737+0000 7f541d165740 1 --2- 172.21.15.105:0/4220583621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b67797680 0x557b677aec70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f540c0eeb80 tx=0x7f540c02a8e0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.737+0000 7f541d165740 1 -- 172.21.15.105:0/4220583621 shutdown_connections 2024-09-16T02:05:40.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.737+0000 7f541d165740 1 --2- 172.21.15.105:0/4220583621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b67797680 0x557b677aec70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.737+0000 7f541d165740 1 --2- 172.21.15.105:0/4220583621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b6778cdc0 0x557b67793290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.737+0000 7f541d165740 1 --2- 172.21.15.105:0/4220583621 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557b6778c4a0 0x557b677a31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.737+0000 7f541d165740 1 -- 172.21.15.105:0/4220583621 >> 172.21.15.105:0/4220583621 conn(0x557b67788a30 msgr2=0x557b67792b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:40.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.737+0000 7f541d165740 1 -- 172.21.15.105:0/4220583621 shutdown_connections 2024-09-16T02:05:40.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.738+0000 7f541d165740 1 -- 172.21.15.105:0/4220583621 wait complete. 2024-09-16T02:05:40.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.738+0000 7f541d165740 1 -- start start 2024-09-16T02:05:40.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.738+0000 7f541d165740 1 Processor -- start 2024-09-16T02:05:40.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.739+0000 7f541d165740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557b6778c4a0 0x557b6779f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:40.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.739+0000 7f541d165740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b6778cdc0 0x557b6779ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:40.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.739+0000 7f541d165740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b67797680 0x557b6779b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:40.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.739+0000 7f541d165740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f53f800fb30 con 0x557b6778c4a0 2024-09-16T02:05:40.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.739+0000 7f541d165740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f53f800f460 con 0x557b6778cdc0 2024-09-16T02:05:40.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.739+0000 7f541d165740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f53f800c6a0 con 0x557b67797680 2024-09-16T02:05:40.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.739+0000 7f541d165740 10 client.0 _ll_register_callbacks cb 0x7f54080098d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:40.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.739+0000 7f5419329640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b67797680 0x557b6779b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:40.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.739+0000 7f5419329640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b67797680 0x557b6779b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.105:55372/0 (socket says 172.21.15.105:55372) 2024-09-16T02:05:40.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.739+0000 7f5419329640 1 -- 172.21.15.105:0/3714118371 learned_addr learned my addr 172.21.15.105:0/3714118371 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:40.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.739+0000 7f5418b28640 1 --2- 172.21.15.105:0/3714118371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557b6778c4a0 0x557b6779f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:40.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.740+0000 7f5419329640 1 -- 172.21.15.105:0/3714118371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b6778cdc0 msgr2=0x557b6779ad70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T02:05:40.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.740+0000 7f5419329640 1 --2- 172.21.15.105:0/3714118371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b6778cdc0 0x557b6779ad70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.740+0000 7f5419329640 1 -- 172.21.15.105:0/3714118371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557b6778c4a0 msgr2=0x557b6779f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:40.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.740+0000 7f5419329640 1 --2- 172.21.15.105:0/3714118371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557b6778c4a0 0x557b6779f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.740+0000 7f5419329640 1 -- 172.21.15.105:0/3714118371 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b6779bae0 con 0x557b67797680 2024-09-16T02:05:40.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.740+0000 7f5419329640 1 --2- 172.21.15.105:0/3714118371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b67797680 0x557b6779b2b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f540c0b9010 tx=0x7f540c0bb780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23216c42420fb869 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:40.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.740+0000 7f53f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-09-16T02:05:40.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.741+0000 7f53f3fff640 1 -- 172.21.15.105:0/3714118371 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540c0c5fa0 con 0x557b67797680 2024-09-16T02:05:40.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.741+0000 7f53f3fff640 1 -- 172.21.15.105:0/3714118371 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f540c0b99a0 con 0x557b67797680 2024-09-16T02:05:40.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.741+0000 7f541d165740 1 -- 172.21.15.105:0/3714118371 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f540c036050 con 0x557b67797680 2024-09-16T02:05:40.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.742+0000 7f53f3fff640 1 -- 172.21.15.105:0/3714118371 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f540c0bd330 con 0x557b67797680 2024-09-16T02:05:40.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.742+0000 7f541d165740 10 client.25153 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:40.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.742+0000 7f541d165740 1 -- 172.21.15.105:0/3714118371 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557b67811b30 con 0x557b67797680 2024-09-16T02:05:40.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.743+0000 7f53f3fff640 1 -- 172.21.15.105:0/3714118371 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f540c0bc780 con 0x557b67797680 2024-09-16T02:05:40.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.744+0000 7f53f3fff640 1 -- 172.21.15.105:0/3714118371 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f540c0c9020 con 0x557b67797680 2024-09-16T02:05:40.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.744+0000 7f541d165740 10 client.25153 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:40.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.744+0000 7f541d165740 10 client.25153 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:05:40.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.744+0000 7f541d165740 20 client.25153 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:40.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.744+0000 7f541d165740 1 --2- 172.21.15.105:0/3714118371 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x557b67811ce0 0x557b678140d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:40.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.744+0000 7f541d165740 4 client.25153 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T02:05:40.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.744+0000 7f541d165740 1 -- 172.21.15.105:0/3714118371 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f540c036050 con 0x557b67811ce0 2024-09-16T02:05:40.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.745+0000 7f5418b28640 1 --2- 172.21.15.105:0/3714118371 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x557b67811ce0 0x557b678140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:40.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.747+0000 7f5418b28640 1 --2- 172.21.15.105:0/3714118371 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x557b67811ce0 0x557b678140d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb18921787e00494 server_cookie=c4274c174f4f1d57 in_seq=0 out_seq=0 2024-09-16T02:05:40.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.747+0000 7f53f3fff640 10 client.25153 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:05:40.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.747+0000 7f53f3fff640 1 -- 172.21.15.105:0/3714118371 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f540c036050 con 0x557b67811ce0 2024-09-16T02:05:40.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.747+0000 7f53f3fff640 10 client.25153 handle_command_reply: tid=0 2024-09-16T02:05:40.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.747+0000 7f541d165740 1 client.25153 shutdown 2024-09-16T02:05:40.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.747+0000 7f541d165740 1 -- 172.21.15.105:0/3714118371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b67797680 msgr2=0x557b6779b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:40.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.747+0000 7f541d165740 1 --2- 172.21.15.105:0/3714118371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b67797680 0x557b6779b2b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f540c0b9010 tx=0x7f540c0bb780 comp rx=0 tx=0).stop 2024-09-16T02:05:40.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.748+0000 7f541d165740 1 -- 172.21.15.105:0/3714118371 shutdown_connections 2024-09-16T02:05:40.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.749+0000 7f541d165740 1 --2- 172.21.15.105:0/3714118371 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x557b67811ce0 0x557b678140d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.749+0000 7f541d165740 1 --2- 172.21.15.105:0/3714118371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b67797680 0x557b6779b2b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.749+0000 7f541d165740 1 --2- 172.21.15.105:0/3714118371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b6778cdc0 0x557b6779ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.749+0000 7f541d165740 1 --2- 172.21.15.105:0/3714118371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557b6778c4a0 0x557b6779f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.749+0000 7f541d165740 1 -- 172.21.15.105:0/3714118371 >> 172.21.15.105:0/3714118371 conn(0x557b67788a30 msgr2=0x557b677933a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:40.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.749+0000 7f53f3fff640 0 client.25153 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:05:40.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.749+0000 7f541d165740 1 -- 172.21.15.105:0/3714118371 shutdown_connections 2024-09-16T02:05:40.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.749+0000 7f541d165740 1 -- 172.21.15.105:0/3714118371 wait complete. 2024-09-16T02:05:40.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.749+0000 7f541d165740 20 client.25153 trim_cache size 0 max 16384 2024-09-16T02:05:40.753 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-16T02:05:40.753 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-09-16T02:05:40.754 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-09-16T02:05:40.754 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-16T02:05:40.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.752+0000 7f541ad8c640 1 -- 172.21.15.105:0/2560476357 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f54040a3390 msgr2=0x7f54040a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:40.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.752+0000 7f541ad8c640 1 --2- 172.21.15.105:0/2560476357 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f54040a3390 0x7f54040a5850 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x557b67789890 tx=0x7f5400009110 comp rx=0 tx=0).stop 2024-09-16T02:05:40.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.752+0000 7f541ad8c640 1 -- 172.21.15.105:0/2560476357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5414106a80 msgr2=0x7f5414108450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:40.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.752+0000 7f541ad8c640 1 --2- 172.21.15.105:0/2560476357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5414106a80 0x7f5414108450 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f540c009bf0 tx=0x7f540c003ba0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.752+0000 7f541ad8c640 1 -- 172.21.15.105:0/2560476357 shutdown_connections 2024-09-16T02:05:40.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.752+0000 7f541ad8c640 1 --2- 172.21.15.105:0/2560476357 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f54040a3390 0x7f54040a5850 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.752+0000 7f541ad8c640 1 --2- 172.21.15.105:0/2560476357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5414106a80 0x7f5414108450 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.752+0000 7f541ad8c640 1 --2- 172.21.15.105:0/2560476357 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5414101420 0x7f5414107f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.752+0000 7f541ad8c640 1 --2- 172.21.15.105:0/2560476357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5414100a80 0x7f54141079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.752+0000 7f541ad8c640 1 -- 172.21.15.105:0/2560476357 >> 172.21.15.105:0/2560476357 conn(0x7f54140fc7a0 msgr2=0x7f54140fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:40.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.753+0000 7f541ad8c640 1 -- 172.21.15.105:0/2560476357 shutdown_connections 2024-09-16T02:05:40.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:40.753+0000 7f541ad8c640 1 -- 172.21.15.105:0/2560476357 wait complete. 2024-09-16T02:05:40.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:40 smithi137 ceph-mon[34099]: scrub queued for path: / 2024-09-16T02:05:40.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:40 smithi137 ceph-mon[34099]: scrub summary: idle+waiting paths [/] 2024-09-16T02:05:40.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:40 smithi137 ceph-mon[34099]: scrub summary: active paths [/] 2024-09-16T02:05:40.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:40 smithi137 ceph-mon[34099]: scrub summary: idle+waiting paths [/] 2024-09-16T02:05:40.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:40 smithi137 ceph-mon[34099]: pgmap v433: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-16T02:05:40.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:40 smithi137 ceph-mon[34099]: scrub summary: idle 2024-09-16T02:05:41.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:40 smithi105 ceph-mon[31197]: scrub queued for path: / 2024-09-16T02:05:41.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:40 smithi105 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-16T02:05:41.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:40 smithi105 ceph-mon[31197]: scrub summary: active paths [/] 2024-09-16T02:05:41.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:40 smithi105 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-16T02:05:41.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:40 smithi105 ceph-mon[31197]: pgmap v433: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-16T02:05:41.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:40 smithi105 ceph-mon[31197]: scrub summary: idle 2024-09-16T02:05:41.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:40 smithi190 ceph-mon[34762]: scrub queued for path: / 2024-09-16T02:05:41.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:40 smithi190 ceph-mon[34762]: scrub summary: idle+waiting paths [/] 2024-09-16T02:05:41.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:40 smithi190 ceph-mon[34762]: scrub summary: active paths [/] 2024-09-16T02:05:41.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:40 smithi190 ceph-mon[34762]: scrub summary: idle+waiting paths [/] 2024-09-16T02:05:41.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:40 smithi190 ceph-mon[34762]: pgmap v433: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-16T02:05:41.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:40 smithi190 ceph-mon[34762]: scrub summary: idle 2024-09-16T02:05:41.598 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T02:05:41.598 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:05:41.843 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:41.963 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:41 smithi105 ceph-mon[31197]: pgmap v434: 97 pgs: 97 active+clean; 9.8 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 14 op/s 2024-09-16T02:05:42.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:41 smithi190 ceph-mon[34762]: pgmap v434: 97 pgs: 97 active+clean; 9.8 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 14 op/s 2024-09-16T02:05:42.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:41 smithi137 ceph-mon[34099]: pgmap v434: 97 pgs: 97 active+clean; 9.8 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 14 op/s 2024-09-16T02:05:42.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.350+0000 7fac5aa6c640 1 -- 172.21.15.105:0/644360838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac54103780 msgr2=0x7fac54107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:42.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.350+0000 7fac5aa6c640 1 --2- 172.21.15.105:0/644360838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac54103780 0x7fac54107bc0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fac3c009930 tx=0x7fac3c02a420 comp rx=0 tx=0).stop 2024-09-16T02:05:42.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.350+0000 7fac5aa6c640 1 -- 172.21.15.105:0/644360838 shutdown_connections 2024-09-16T02:05:42.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.350+0000 7fac5aa6c640 1 --2- 172.21.15.105:0/644360838 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac54108300 0x7fac5410ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.350+0000 7fac5aa6c640 1 --2- 172.21.15.105:0/644360838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac54103780 0x7fac54107bc0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.350+0000 7fac5aa6c640 1 --2- 172.21.15.105:0/644360838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac54102dd0 0x7fac541031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.350+0000 7fac5aa6c640 1 -- 172.21.15.105:0/644360838 >> 172.21.15.105:0/644360838 conn(0x7fac540fc7d0 msgr2=0x7fac540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:42.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.351+0000 7fac5aa6c640 1 -- 172.21.15.105:0/644360838 shutdown_connections 2024-09-16T02:05:42.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.351+0000 7fac5aa6c640 1 -- 172.21.15.105:0/644360838 wait complete. 2024-09-16T02:05:42.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.351+0000 7fac5aa6c640 1 Processor -- start 2024-09-16T02:05:42.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.351+0000 7fac5aa6c640 1 -- start start 2024-09-16T02:05:42.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.351+0000 7fac5aa6c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac54102dd0 0x7fac54198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:42.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.351+0000 7fac5aa6c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac54103780 0x7fac54198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:42.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.351+0000 7fac5aa6c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac54108300 0x7fac5419cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:42.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.351+0000 7fac5aa6c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fac54111f20 con 0x7fac54103780 2024-09-16T02:05:42.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.351+0000 7fac5aa6c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fac54111da0 con 0x7fac54102dd0 2024-09-16T02:05:42.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.351+0000 7fac5aa6c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fac541120a0 con 0x7fac54108300 2024-09-16T02:05:42.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.352+0000 7fac53fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac54103780 0x7fac54198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:42.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.352+0000 7fac53fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac54103780 0x7fac54198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49602/0 (socket says 172.21.15.105:49602) 2024-09-16T02:05:42.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.352+0000 7fac53fff640 1 -- 172.21.15.105:0/1638412298 learned_addr learned my addr 172.21.15.105:0/1638412298 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:42.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.352+0000 7fac58808640 1 --2- 172.21.15.105:0/1638412298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac54102dd0 0x7fac54198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:42.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.352+0000 7fac59009640 1 --2- 172.21.15.105:0/1638412298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac54108300 0x7fac5419cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:42.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.352+0000 7fac53fff640 1 -- 172.21.15.105:0/1638412298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac54108300 msgr2=0x7fac5419cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:42.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.352+0000 7fac53fff640 1 --2- 172.21.15.105:0/1638412298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac54108300 0x7fac5419cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.352+0000 7fac53fff640 1 -- 172.21.15.105:0/1638412298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac54102dd0 msgr2=0x7fac54198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:42.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.352+0000 7fac53fff640 1 --2- 172.21.15.105:0/1638412298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac54102dd0 0x7fac54198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.352+0000 7fac53fff640 1 -- 172.21.15.105:0/1638412298 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac5419d4d0 con 0x7fac54103780 2024-09-16T02:05:42.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.352+0000 7fac59009640 1 --2- 172.21.15.105:0/1638412298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac54108300 0x7fac5419cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:05:42.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.352+0000 7fac53fff640 1 --2- 172.21.15.105:0/1638412298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac54103780 0x7fac54198a60 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fac3c031440 tx=0x7fac3c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:42.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.352+0000 7fac51ffb640 1 -- 172.21.15.105:0/1638412298 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c03f030 con 0x7fac54103780 2024-09-16T02:05:42.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.352+0000 7fac5aa6c640 1 -- 172.21.15.105:0/1638412298 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac5419d760 con 0x7fac54103780 2024-09-16T02:05:42.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.352+0000 7fac51ffb640 1 -- 172.21.15.105:0/1638412298 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac3c003aa0 con 0x7fac54103780 2024-09-16T02:05:42.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.352+0000 7fac51ffb640 1 -- 172.21.15.105:0/1638412298 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c035070 con 0x7fac54103780 2024-09-16T02:05:42.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.353+0000 7fac5aa6c640 1 -- 172.21.15.105:0/1638412298 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac541a50a0 con 0x7fac54103780 2024-09-16T02:05:42.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.353+0000 7fac5aa6c640 1 -- 172.21.15.105:0/1638412298 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac541031b0 con 0x7fac54103780 2024-09-16T02:05:42.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.355+0000 7fac51ffb640 1 -- 172.21.15.105:0/1638412298 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fac3c042030 con 0x7fac54103780 2024-09-16T02:05:42.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.355+0000 7fac51ffb640 1 --2- 172.21.15.105:0/1638412298 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fac400a3240 0x7fac400a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:42.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.355+0000 7fac58808640 1 --2- 172.21.15.105:0/1638412298 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fac400a3240 0x7fac400a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:42.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.355+0000 7fac51ffb640 1 -- 172.21.15.105:0/1638412298 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fac3c0f03f0 con 0x7fac54103780 2024-09-16T02:05:42.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.356+0000 7fac58808640 1 --2- 172.21.15.105:0/1638412298 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fac400a3240 0x7fac400a5700 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fac44005fd0 tx=0x7fac44009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:42.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.358+0000 7fac51ffb640 1 -- 172.21.15.105:0/1638412298 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fac3c0bd0b0 con 0x7fac54103780 2024-09-16T02:05:42.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.558+0000 7fac5aa6c640 1 -- 172.21.15.105:0/1638412298 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac54199880 con 0x7fac54103780 2024-09-16T02:05:42.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.559+0000 7fac51ffb640 1 -- 172.21.15.105:0/1638412298 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11991 (secure 0 0 0) 0x7fac54199880 con 0x7fac54103780 2024-09-16T02:05:42.561 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:05:42.562 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":28,"btime":"2024-09-16T02:02:41:554178+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2057699920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2057699920},{"type":"v1","addr":"172.21.15.105:6835","nonce":2057699920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1764886948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1764886948},{"type":"v1","addr":"172.21.15.105:6837","nonce":1764886948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T02:00:35.112266+0000","modified":"2024-09-16T02:02:40.885614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24469,"mds_2":24461,"mds_3":24457,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/4112062361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":4112062361},{"type":"v1","addr":"172.21.15.105:6839","nonce":4112062361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6841/163156514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":163156514},{"type":"v1","addr":"172.21.15.105:6841","nonce":163156514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/683519097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":683519097},{"type":"v1","addr":"172.21.15.190:6833","nonce":683519097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3025442862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3025442862},{"type":"v1","addr":"172.21.15.137:6833","nonce":3025442862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/763783602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":763783602},{"type":"v1","addr":"172.21.15.137:6835","nonce":763783602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.190:6835/1851042281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1851042281},{"type":"v1","addr":"172.21.15.190:6835","nonce":1851042281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/3181257549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3181257549},{"type":"v1","addr":"172.21.15.190:6837","nonce":3181257549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6837/2788541030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2788541030},{"type":"v1","addr":"172.21.15.137:6837","nonce":2788541030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.190:6839/1222573537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":1222573537},{"type":"v1","addr":"172.21.15.190:6839","nonce":1222573537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/3454924009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3454924009},{"type":"v1","addr":"172.21.15.137:6839","nonce":3454924009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24445,24457,24461,24469,24482]},"id":1}]} 2024-09-16T02:05:42.562 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 28 2024-09-16T02:05:42.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.561+0000 7fac5aa6c640 1 -- 172.21.15.105:0/1638412298 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fac400a3240 msgr2=0x7fac400a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:42.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.561+0000 7fac5aa6c640 1 --2- 172.21.15.105:0/1638412298 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fac400a3240 0x7fac400a5700 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fac44005fd0 tx=0x7fac44009500 comp rx=0 tx=0).stop 2024-09-16T02:05:42.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.561+0000 7fac5aa6c640 1 -- 172.21.15.105:0/1638412298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac54103780 msgr2=0x7fac54198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:42.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.561+0000 7fac5aa6c640 1 --2- 172.21.15.105:0/1638412298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac54103780 0x7fac54198a60 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fac3c031440 tx=0x7fac3c004270 comp rx=0 tx=0).stop 2024-09-16T02:05:42.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.561+0000 7fac5aa6c640 1 -- 172.21.15.105:0/1638412298 shutdown_connections 2024-09-16T02:05:42.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.561+0000 7fac5aa6c640 1 --2- 172.21.15.105:0/1638412298 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fac400a3240 0x7fac400a5700 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.561+0000 7fac5aa6c640 1 --2- 172.21.15.105:0/1638412298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac54108300 0x7fac5419cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.561+0000 7fac5aa6c640 1 --2- 172.21.15.105:0/1638412298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac54103780 0x7fac54198a60 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.561+0000 7fac5aa6c640 1 --2- 172.21.15.105:0/1638412298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac54102dd0 0x7fac54198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.561+0000 7fac5aa6c640 1 -- 172.21.15.105:0/1638412298 >> 172.21.15.105:0/1638412298 conn(0x7fac540fc7d0 msgr2=0x7fac54105ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:42.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.561+0000 7fac5aa6c640 1 -- 172.21.15.105:0/1638412298 shutdown_connections 2024-09-16T02:05:42.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:42.561+0000 7fac5aa6c640 1 -- 172.21.15.105:0/1638412298 wait complete. 2024-09-16T02:05:43.007 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:42 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/1638412298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:05:43.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:42 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/1638412298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:05:43.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:42 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/1638412298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:05:43.397 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-16T02:05:43.645 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:43.950 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:43 smithi105 ceph-mon[31197]: pgmap v435: 97 pgs: 97 active+clean; 8.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.9 KiB/s wr, 48 op/s 2024-09-16T02:05:44.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:43 smithi190 ceph-mon[34762]: pgmap v435: 97 pgs: 97 active+clean; 8.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.9 KiB/s wr, 48 op/s 2024-09-16T02:05:44.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:43 smithi137 ceph-mon[34099]: pgmap v435: 97 pgs: 97 active+clean; 8.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.9 KiB/s wr, 48 op/s 2024-09-16T02:05:44.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.162+0000 7f7b3b3bd640 1 -- 172.21.15.105:0/1429046124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b34100e20 msgr2=0x7f7b34101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:44.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.162+0000 7f7b3b3bd640 1 --2- 172.21.15.105:0/1429046124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b34100e20 0x7f7b34101200 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f7b30009900 tx=0x7f7b3002a480 comp rx=0 tx=0).stop 2024-09-16T02:05:44.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.162+0000 7f7b3b3bd640 1 -- 172.21.15.105:0/1429046124 shutdown_connections 2024-09-16T02:05:44.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.162+0000 7f7b3b3bd640 1 --2- 172.21.15.105:0/1429046124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b34102ef0 0x7f7b3410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.162+0000 7f7b3b3bd640 1 --2- 172.21.15.105:0/1429046124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b34102570 0x7f7b341029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.162+0000 7f7b3b3bd640 1 --2- 172.21.15.105:0/1429046124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b34100e20 0x7f7b34101200 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.162+0000 7f7b3b3bd640 1 -- 172.21.15.105:0/1429046124 >> 172.21.15.105:0/1429046124 conn(0x7f7b340fc7d0 msgr2=0x7f7b340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:44.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.162+0000 7f7b3b3bd640 1 -- 172.21.15.105:0/1429046124 shutdown_connections 2024-09-16T02:05:44.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.162+0000 7f7b3b3bd640 1 -- 172.21.15.105:0/1429046124 wait complete. 2024-09-16T02:05:44.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.163+0000 7f7b3b3bd640 1 Processor -- start 2024-09-16T02:05:44.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.163+0000 7f7b3b3bd640 1 -- start start 2024-09-16T02:05:44.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.163+0000 7f7b3b3bd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b34100e20 0x7f7b340fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:44.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.163+0000 7f7b3b3bd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b34102570 0x7f7b340fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:44.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.163+0000 7f7b3b3bd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b34102ef0 0x7f7b340fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:44.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.163+0000 7f7b3b3bd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7b34110ea0 con 0x7f7b34102570 2024-09-16T02:05:44.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.163+0000 7f7b3b3bd640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7b34110d20 con 0x7f7b34100e20 2024-09-16T02:05:44.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.163+0000 7f7b3b3bd640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7b34111020 con 0x7f7b34102ef0 2024-09-16T02:05:44.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.163+0000 7f7b38958640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b34102570 0x7f7b340fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:44.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.163+0000 7f7b38958640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b34102570 0x7f7b340fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49628/0 (socket says 172.21.15.105:49628) 2024-09-16T02:05:44.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.163+0000 7f7b38958640 1 -- 172.21.15.105:0/342108945 learned_addr learned my addr 172.21.15.105:0/342108945 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:44.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.163+0000 7f7b39159640 1 --2- 172.21.15.105:0/342108945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b34100e20 0x7f7b340fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:44.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.163+0000 7f7b3995a640 1 --2- 172.21.15.105:0/342108945 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b34102ef0 0x7f7b340fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:44.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.164+0000 7f7b38958640 1 -- 172.21.15.105:0/342108945 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b34102ef0 msgr2=0x7f7b340fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:44.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.164+0000 7f7b38958640 1 --2- 172.21.15.105:0/342108945 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b34102ef0 0x7f7b340fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.164+0000 7f7b38958640 1 -- 172.21.15.105:0/342108945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b34100e20 msgr2=0x7f7b340fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:44.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.164+0000 7f7b38958640 1 --2- 172.21.15.105:0/342108945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b34100e20 0x7f7b340fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.164+0000 7f7b38958640 1 -- 172.21.15.105:0/342108945 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b341a68f0 con 0x7f7b34102570 2024-09-16T02:05:44.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.164+0000 7f7b38958640 1 --2- 172.21.15.105:0/342108945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b34102570 0x7f7b340fd1c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f7b2400d950 tx=0x7f7b2400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:44.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.164+0000 7f7b2a7fc640 1 -- 172.21.15.105:0/342108945 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b24017070 con 0x7f7b34102570 2024-09-16T02:05:44.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.164+0000 7f7b2a7fc640 1 -- 172.21.15.105:0/342108945 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b24009c30 con 0x7f7b34102570 2024-09-16T02:05:44.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.164+0000 7f7b3b3bd640 1 -- 172.21.15.105:0/342108945 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b341a6be0 con 0x7f7b34102570 2024-09-16T02:05:44.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.164+0000 7f7b2a7fc640 1 -- 172.21.15.105:0/342108945 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2400f620 con 0x7f7b34102570 2024-09-16T02:05:44.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.164+0000 7f7b3b3bd640 1 -- 172.21.15.105:0/342108945 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b341a7170 con 0x7f7b34102570 2024-09-16T02:05:44.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.166+0000 7f7b2a7fc640 1 -- 172.21.15.105:0/342108945 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f7b240109a0 con 0x7f7b34102570 2024-09-16T02:05:44.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.167+0000 7f7b3d796740 1 Processor -- start 2024-09-16T02:05:44.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.167+0000 7f7b3d796740 1 -- start start 2024-09-16T02:05:44.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.167+0000 7f7b3d796740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ad5610b40 0x558ad560f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:44.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.167+0000 7f7b3d796740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558ad560f5d0 0x558ad560fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:44.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.167+0000 7f7b3d796740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ad561a370 0x558ad561c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:44.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.167+0000 7f7b3d796740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558ad55752f0 con 0x558ad560f5d0 2024-09-16T02:05:44.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.167+0000 7f7b3d796740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558ad551ccf0 con 0x558ad5610b40 2024-09-16T02:05:44.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.167+0000 7f7b3d796740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558ad55712b0 con 0x558ad561a370 2024-09-16T02:05:44.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.167+0000 7f7b2a7fc640 1 --2- 172.21.15.105:0/342108945 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7b200a32c0 0x7f7b200a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:44.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.167+0000 7f7b39159640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558ad560f5d0 0x558ad560fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:44.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.167+0000 7f7b39159640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558ad560f5d0 0x558ad560fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49630/0 (socket says 172.21.15.105:49630) 2024-09-16T02:05:44.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.167+0000 7f7b39159640 1 -- 172.21.15.105:0/2405957085 learned_addr learned my addr 172.21.15.105:0/2405957085 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:44.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.167+0000 7f7b3995a640 1 --2- 172.21.15.105:0/2405957085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ad5610b40 0x558ad560f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:44.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.167+0000 7f7b3995a640 1 --2- 172.21.15.105:0/342108945 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7b200a32c0 0x7f7b200a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:44.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.167+0000 7f7b38958640 1 --2- 172.21.15.105:0/2405957085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ad561a370 0x558ad561c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:44.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.167+0000 7f7b2a7fc640 1 -- 172.21.15.105:0/342108945 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f7b240cf530 con 0x7f7b34102570 2024-09-16T02:05:44.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.167+0000 7f7b39159640 1 -- 172.21.15.105:0/2405957085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ad561a370 msgr2=0x558ad561c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:44.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.167+0000 7f7b39159640 1 --2- 172.21.15.105:0/2405957085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ad561a370 0x558ad561c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.167+0000 7f7b39159640 1 -- 172.21.15.105:0/2405957085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ad5610b40 msgr2=0x558ad560f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:44.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.167+0000 7f7b39159640 1 --2- 172.21.15.105:0/2405957085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ad5610b40 0x558ad560f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.167+0000 7f7b39159640 1 -- 172.21.15.105:0/2405957085 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ad55013a0 con 0x558ad560f5d0 2024-09-16T02:05:44.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.167+0000 7f7b3995a640 1 --2- 172.21.15.105:0/2405957085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ad5610b40 0x558ad560f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:05:44.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.168+0000 7f7b39159640 1 --2- 172.21.15.105:0/2405957085 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558ad560f5d0 0x558ad560fa10 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7b30009900 tx=0x7f7b30004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c403c6a69b1182 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:44.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.168+0000 7f7b3995a640 1 --2- 172.21.15.105:0/342108945 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7b200a32c0 0x7f7b200a5780 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c0070b0 tx=0x7f7b2c0093d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:44.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.168+0000 7f7b1f7fe640 1 -- 172.21.15.105:0/2405957085 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b30035040 con 0x558ad560f5d0 2024-09-16T02:05:44.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.168+0000 7f7b1f7fe640 1 -- 172.21.15.105:0/2405957085 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b30004740 con 0x558ad560f5d0 2024-09-16T02:05:44.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.168+0000 7f7b1f7fe640 1 -- 172.21.15.105:0/2405957085 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b30003880 con 0x558ad560f5d0 2024-09-16T02:05:44.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.168+0000 7f7b3d796740 1 -- 172.21.15.105:0/2405957085 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558ad560f5d0 msgr2=0x558ad560fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:44.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.168+0000 7f7b3d796740 1 --2- 172.21.15.105:0/2405957085 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558ad560f5d0 0x558ad560fa10 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7b30009900 tx=0x7f7b30004310 comp rx=0 tx=0).stop 2024-09-16T02:05:44.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.168+0000 7f7b3d796740 1 -- 172.21.15.105:0/2405957085 shutdown_connections 2024-09-16T02:05:44.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.168+0000 7f7b3d796740 1 --2- 172.21.15.105:0/2405957085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ad561a370 0x558ad561c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.168+0000 7f7b3d796740 1 --2- 172.21.15.105:0/2405957085 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558ad560f5d0 0x558ad560fa10 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.169+0000 7f7b3d796740 1 --2- 172.21.15.105:0/2405957085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ad5610b40 0x558ad560f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.169+0000 7f7b3d796740 1 -- 172.21.15.105:0/2405957085 >> 172.21.15.105:0/2405957085 conn(0x558ad55d8310 msgr2=0x558ad560bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:44.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.169+0000 7f7b3d796740 1 -- 172.21.15.105:0/2405957085 shutdown_connections 2024-09-16T02:05:44.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.169+0000 7f7b3d796740 1 -- 172.21.15.105:0/2405957085 wait complete. 2024-09-16T02:05:44.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.169+0000 7f7b3d796740 1 -- start start 2024-09-16T02:05:44.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.169+0000 7f7b3d796740 1 Processor -- start 2024-09-16T02:05:44.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.169+0000 7f7b3d796740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558ad560f5d0 0x558ad5619d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:44.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.169+0000 7f7b3d796740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ad5610b40 0x558ad5616fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:44.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.169+0000 7f7b3d796740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ad561a370 0x558ad5617560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:44.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.169+0000 7f7b3d796740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558ad561d010 con 0x558ad560f5d0 2024-09-16T02:05:44.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.169+0000 7f7b3d796740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558ad5530060 con 0x558ad561a370 2024-09-16T02:05:44.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.169+0000 7f7b3d796740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558ad561d190 con 0x558ad5610b40 2024-09-16T02:05:44.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.169+0000 7f7b3d796740 10 client.0 _ll_register_callbacks cb 0x558ad52838c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:44.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.169+0000 7f7b39159640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558ad560f5d0 0x558ad5619d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:44.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.169+0000 7f7b39159640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558ad560f5d0 0x558ad5619d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49646/0 (socket says 172.21.15.105:49646) 2024-09-16T02:05:44.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.169+0000 7f7b39159640 1 -- 172.21.15.105:0/1952887041 learned_addr learned my addr 172.21.15.105:0/1952887041 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:44.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.170+0000 7f7b38958640 1 --2- 172.21.15.105:0/1952887041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ad5610b40 0x558ad5616fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:44.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.170+0000 7f7b3995a640 1 --2- 172.21.15.105:0/1952887041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ad561a370 0x558ad5617560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:44.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.170+0000 7f7b39159640 1 -- 172.21.15.105:0/1952887041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ad5610b40 msgr2=0x558ad5616fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:44.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.170+0000 7f7b39159640 1 --2- 172.21.15.105:0/1952887041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ad5610b40 0x558ad5616fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.170+0000 7f7b39159640 1 -- 172.21.15.105:0/1952887041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ad561a370 msgr2=0x558ad5617560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:44.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.170+0000 7f7b39159640 1 --2- 172.21.15.105:0/1952887041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ad561a370 0x558ad5617560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.170+0000 7f7b39159640 1 -- 172.21.15.105:0/1952887041 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ad52e63c0 con 0x558ad560f5d0 2024-09-16T02:05:44.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.170+0000 7f7b38958640 1 --2- 172.21.15.105:0/1952887041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ad5610b40 0x558ad5616fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:05:44.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.170+0000 7f7b3995a640 1 --2- 172.21.15.105:0/1952887041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ad561a370 0x558ad5617560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:05:44.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.170+0000 7f7b39159640 1 --2- 172.21.15.105:0/1952887041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558ad560f5d0 0x558ad5619d10 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f7b30004b20 tx=0x7f7b3002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad950d2572e67b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:44.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.170+0000 7f7b1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:05:44.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.170+0000 7f7b1cff9640 1 -- 172.21.15.105:0/1952887041 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b30052060 con 0x558ad560f5d0 2024-09-16T02:05:44.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.170+0000 7f7b1cff9640 1 -- 172.21.15.105:0/1952887041 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f7b30005130 con 0x558ad560f5d0 2024-09-16T02:05:44.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.170+0000 7f7b3d796740 1 -- 172.21.15.105:0/1952887041 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b30004740 con 0x558ad560f5d0 2024-09-16T02:05:44.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.170+0000 7f7b1cff9640 1 -- 172.21.15.105:0/1952887041 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7b30035050 con 0x558ad560f5d0 2024-09-16T02:05:44.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.170+0000 7f7b3d796740 10 client.15939 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:44.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.170+0000 7f7b3d796740 1 -- 172.21.15.105:0/1952887041 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558ad5628a60 con 0x558ad560f5d0 2024-09-16T02:05:44.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.170+0000 7f7b1cff9640 1 -- 172.21.15.105:0/1952887041 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b30004740 con 0x558ad560f5d0 2024-09-16T02:05:44.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.171+0000 7f7b1cff9640 1 -- 172.21.15.105:0/1952887041 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7b30056030 con 0x558ad560f5d0 2024-09-16T02:05:44.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.171+0000 7f7b3d796740 10 client.15939 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:44.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.171+0000 7f7b3d796740 10 client.15939 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T02:05:44.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.171+0000 7f7b3d796740 20 client.15939 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:44.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.171+0000 7f7b3d796740 1 --2- 172.21.15.105:0/1952887041 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x558ad5629db0 0x558ad562c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:44.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.171+0000 7f7b3d796740 4 client.15939 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:05:44.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.171+0000 7f7b3d796740 1 -- 172.21.15.105:0/1952887041 --> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ad5582860 con 0x558ad5629db0 2024-09-16T02:05:44.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.171+0000 7f7b38958640 1 --2- 172.21.15.105:0/1952887041 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x558ad5629db0 0x558ad562c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:44.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.171+0000 7f7b1cff9640 10 client.15939 ms_handle_connect on v2:172.21.15.190:6834/1851042281 2024-09-16T02:05:44.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.171+0000 7f7b38958640 1 --2- 172.21.15.105:0/1952887041 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x558ad5629db0 0x558ad562c1a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bae6925203890e9b server_cookie=acf846758bd53d14 in_seq=0 out_seq=0 2024-09-16T02:05:44.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.172+0000 7f7b1cff9640 1 -- 172.21.15.105:0/1952887041 <== mds.4 v2:172.21.15.190:6834/1851042281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558ad5582860 con 0x558ad5629db0 2024-09-16T02:05:44.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.172+0000 7f7b1cff9640 10 client.15939 handle_command_reply: tid=0 2024-09-16T02:05:44.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.172+0000 7f7b3d796740 1 client.15939 shutdown 2024-09-16T02:05:44.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.172+0000 7f7b3d796740 1 -- 172.21.15.105:0/1952887041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558ad560f5d0 msgr2=0x558ad5619d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:44.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.172+0000 7f7b3d796740 1 --2- 172.21.15.105:0/1952887041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558ad560f5d0 0x558ad5619d10 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f7b30004b20 tx=0x7f7b3002a480 comp rx=0 tx=0).stop 2024-09-16T02:05:44.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.172+0000 7f7b3d796740 1 -- 172.21.15.105:0/1952887041 shutdown_connections 2024-09-16T02:05:44.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.172+0000 7f7b3d796740 1 --2- 172.21.15.105:0/1952887041 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x558ad5629db0 0x558ad562c1a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.172+0000 7f7b3d796740 1 --2- 172.21.15.105:0/1952887041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ad5610b40 0x558ad5616fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.172+0000 7f7b3d796740 1 --2- 172.21.15.105:0/1952887041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ad561a370 0x558ad5617560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.172+0000 7f7b1cff9640 0 client.15939 ms_handle_reset on v2:172.21.15.190:6834/1851042281 2024-09-16T02:05:44.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.172+0000 7f7b3d796740 1 --2- 172.21.15.105:0/1952887041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558ad560f5d0 0x558ad5619d10 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.172+0000 7f7b3d796740 1 -- 172.21.15.105:0/1952887041 >> 172.21.15.105:0/1952887041 conn(0x558ad55d8310 msgr2=0x558ad5611930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:44.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.172+0000 7f7b3d796740 1 -- 172.21.15.105:0/1952887041 shutdown_connections 2024-09-16T02:05:44.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.172+0000 7f7b3d796740 1 -- 172.21.15.105:0/1952887041 wait complete. 2024-09-16T02:05:44.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.173+0000 7f7b3d796740 20 client.15939 trim_cache size 0 max 16384 2024-09-16T02:05:44.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.185+0000 7f7b3d796740 1 Processor -- start 2024-09-16T02:05:44.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.185+0000 7f7b3d796740 1 -- start start 2024-09-16T02:05:44.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.185+0000 7f7b3d796740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ad560f5d0 0x558ad5612120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:44.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.185+0000 7f7b3d796740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558ad5610b40 0x558ad5612660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:44.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.185+0000 7f7b3d796740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ad561a370 0x558ad5614280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:44.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.185+0000 7f7b3d796740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7b341af810 con 0x558ad5610b40 2024-09-16T02:05:44.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.185+0000 7f7b3d796740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7b341b1350 con 0x558ad560f5d0 2024-09-16T02:05:44.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.185+0000 7f7b3d796740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7b341afee0 con 0x558ad561a370 2024-09-16T02:05:44.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.186+0000 7f7b38958640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558ad5610b40 0x558ad5612660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:44.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.186+0000 7f7b38958640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558ad5610b40 0x558ad5612660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49662/0 (socket says 172.21.15.105:49662) 2024-09-16T02:05:44.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.186+0000 7f7b38958640 1 -- 172.21.15.105:0/2725708318 learned_addr learned my addr 172.21.15.105:0/2725708318 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:44.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.186+0000 7f7b39159640 1 --2- 172.21.15.105:0/2725708318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ad560f5d0 0x558ad5612120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:44.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.186+0000 7f7b3995a640 1 --2- 172.21.15.105:0/2725708318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ad561a370 0x558ad5614280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:44.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.186+0000 7f7b38958640 1 -- 172.21.15.105:0/2725708318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ad561a370 msgr2=0x558ad5614280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:44.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.186+0000 7f7b38958640 1 --2- 172.21.15.105:0/2725708318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ad561a370 0x558ad5614280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.186+0000 7f7b38958640 1 -- 172.21.15.105:0/2725708318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ad560f5d0 msgr2=0x558ad5612120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:44.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.186+0000 7f7b38958640 1 --2- 172.21.15.105:0/2725708318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ad560f5d0 0x558ad5612120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.186+0000 7f7b38958640 1 -- 172.21.15.105:0/2725708318 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ad5615550 con 0x558ad5610b40 2024-09-16T02:05:44.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.186+0000 7f7b38958640 1 --2- 172.21.15.105:0/2725708318 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558ad5610b40 0x558ad5612660 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f7b24010fd0 tx=0x7f7b24086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b990bebcb5ec26 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:44.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.186+0000 7f7b1effd640 1 -- 172.21.15.105:0/2725708318 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b24020040 con 0x558ad5610b40 2024-09-16T02:05:44.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.186+0000 7f7b1effd640 1 -- 172.21.15.105:0/2725708318 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b2409fea0 con 0x558ad5610b40 2024-09-16T02:05:44.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.186+0000 7f7b1effd640 1 -- 172.21.15.105:0/2725708318 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b240a42d0 con 0x558ad5610b40 2024-09-16T02:05:44.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.187+0000 7f7b3d796740 1 -- 172.21.15.105:0/2725708318 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558ad5610b40 msgr2=0x558ad5612660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:44.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.187+0000 7f7b3d796740 1 --2- 172.21.15.105:0/2725708318 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558ad5610b40 0x558ad5612660 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f7b24010fd0 tx=0x7f7b24086040 comp rx=0 tx=0).stop 2024-09-16T02:05:44.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.187+0000 7f7b3d796740 1 -- 172.21.15.105:0/2725708318 shutdown_connections 2024-09-16T02:05:44.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.187+0000 7f7b3d796740 1 --2- 172.21.15.105:0/2725708318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ad561a370 0x558ad5614280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.187+0000 7f7b3d796740 1 --2- 172.21.15.105:0/2725708318 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558ad5610b40 0x558ad5612660 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.187+0000 7f7b3d796740 1 --2- 172.21.15.105:0/2725708318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ad560f5d0 0x558ad5612120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.187+0000 7f7b3d796740 1 -- 172.21.15.105:0/2725708318 >> 172.21.15.105:0/2725708318 conn(0x558ad55d8310 msgr2=0x558ad5619040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:44.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.187+0000 7f7b3d796740 1 -- 172.21.15.105:0/2725708318 shutdown_connections 2024-09-16T02:05:44.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.187+0000 7f7b3d796740 1 -- 172.21.15.105:0/2725708318 wait complete. 2024-09-16T02:05:44.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.187+0000 7f7b3d796740 1 -- start start 2024-09-16T02:05:44.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.188+0000 7f7b3d796740 1 Processor -- start 2024-09-16T02:05:44.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.188+0000 7f7b3d796740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558ad560f5d0 0x558ad562d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:44.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.188+0000 7f7b3d796740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ad5610b40 0x558ad562db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:44.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.188+0000 7f7b3d796740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ad561a370 0x558ad56936f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:44.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.188+0000 7f7b3d796740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7b341b0c80 con 0x558ad560f5d0 2024-09-16T02:05:44.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.188+0000 7f7b3d796740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7b341b05b0 con 0x558ad5610b40 2024-09-16T02:05:44.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.188+0000 7f7b3d796740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7b341ad7f0 con 0x558ad561a370 2024-09-16T02:05:44.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.188+0000 7f7b3d796740 10 client.0 _ll_register_callbacks cb 0x7f7b2c009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:44.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.188+0000 7f7b39159640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558ad560f5d0 0x558ad562d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:44.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.188+0000 7f7b39159640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558ad560f5d0 0x558ad562d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49676/0 (socket says 172.21.15.105:49676) 2024-09-16T02:05:44.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.188+0000 7f7b39159640 1 -- 172.21.15.105:0/640895349 learned_addr learned my addr 172.21.15.105:0/640895349 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:44.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.188+0000 7f7b38958640 1 --2- 172.21.15.105:0/640895349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ad5610b40 0x558ad562db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:44.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.188+0000 7f7b3995a640 1 --2- 172.21.15.105:0/640895349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ad561a370 0x558ad56936f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:44.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.188+0000 7f7b39159640 1 -- 172.21.15.105:0/640895349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ad561a370 msgr2=0x558ad56936f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:44.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.188+0000 7f7b39159640 1 --2- 172.21.15.105:0/640895349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ad561a370 0x558ad56936f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.188+0000 7f7b39159640 1 -- 172.21.15.105:0/640895349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ad5610b40 msgr2=0x558ad562db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:44.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.188+0000 7f7b39159640 1 --2- 172.21.15.105:0/640895349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ad5610b40 0x558ad562db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.188+0000 7f7b39159640 1 -- 172.21.15.105:0/640895349 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ad5693c30 con 0x558ad560f5d0 2024-09-16T02:05:44.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.188+0000 7f7b3995a640 1 --2- 172.21.15.105:0/640895349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ad561a370 0x558ad56936f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:05:44.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.189+0000 7f7b39159640 1 --2- 172.21.15.105:0/640895349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558ad560f5d0 0x558ad562d5e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f7b30009900 tx=0x7f7b300045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c44e19ecb5da18d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:44.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.189+0000 7f7b1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:05:44.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.189+0000 7f7b1cff9640 1 -- 172.21.15.105:0/640895349 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b30004740 con 0x558ad560f5d0 2024-09-16T02:05:44.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.189+0000 7f7b1cff9640 1 -- 172.21.15.105:0/640895349 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f7b30005130 con 0x558ad560f5d0 2024-09-16T02:05:44.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.189+0000 7f7b3d796740 1 -- 172.21.15.105:0/640895349 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b2409fea0 con 0x558ad560f5d0 2024-09-16T02:05:44.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.189+0000 7f7b1cff9640 1 -- 172.21.15.105:0/640895349 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7b30005330 con 0x558ad560f5d0 2024-09-16T02:05:44.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.189+0000 7f7b3d796740 10 client.15945 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:44.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.189+0000 7f7b3d796740 1 -- 172.21.15.105:0/640895349 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558ad5694e90 con 0x558ad560f5d0 2024-09-16T02:05:44.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.189+0000 7f7b1cff9640 1 -- 172.21.15.105:0/640895349 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b2409fea0 con 0x558ad560f5d0 2024-09-16T02:05:44.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.189+0000 7f7b1cff9640 1 -- 172.21.15.105:0/640895349 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7b30056040 con 0x558ad560f5d0 2024-09-16T02:05:44.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.189+0000 7f7b3d796740 10 client.15945 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:44.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.189+0000 7f7b3d796740 10 client.15945 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T02:05:44.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.189+0000 7f7b3d796740 20 client.15945 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:44.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.189+0000 7f7b3d796740 1 --2- 172.21.15.105:0/640895349 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x558ad56955a0 0x558ad56979c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:44.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.189+0000 7f7b3d796740 4 client.15945 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T02:05:44.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.189+0000 7f7b3d796740 1 -- 172.21.15.105:0/640895349 --> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558ad557cd70 con 0x558ad56955a0 2024-09-16T02:05:44.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.190+0000 7f7b38958640 1 --2- 172.21.15.105:0/640895349 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x558ad56955a0 0x558ad56979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:44.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.190+0000 7f7b1cff9640 10 client.15945 ms_handle_connect on v2:172.21.15.190:6834/1851042281 2024-09-16T02:05:44.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.190+0000 7f7b38958640 1 --2- 172.21.15.105:0/640895349 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x558ad56955a0 0x558ad56979c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7341666d4853629f server_cookie=e15a485a5722b14e in_seq=0 out_seq=0 2024-09-16T02:05:44.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.190+0000 7f7b1cff9640 1 -- 172.21.15.105:0/640895349 <== mds.4 v2:172.21.15.190:6834/1851042281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558ad557cd70 con 0x558ad56955a0 2024-09-16T02:05:44.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.190+0000 7f7b1cff9640 10 client.15945 handle_command_reply: tid=0 2024-09-16T02:05:44.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.190+0000 7f7b3d796740 1 client.15945 shutdown 2024-09-16T02:05:44.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.190+0000 7f7b3d796740 1 -- 172.21.15.105:0/640895349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558ad560f5d0 msgr2=0x558ad562d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:44.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.190+0000 7f7b3d796740 1 --2- 172.21.15.105:0/640895349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558ad560f5d0 0x558ad562d5e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f7b30009900 tx=0x7f7b300045b0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.190+0000 7f7b3d796740 1 -- 172.21.15.105:0/640895349 shutdown_connections 2024-09-16T02:05:44.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.190+0000 7f7b3d796740 1 --2- 172.21.15.105:0/640895349 >> [v2:172.21.15.190:6834/1851042281,v1:172.21.15.190:6835/1851042281] conn(0x558ad56955a0 0x558ad56979c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.190+0000 7f7b3d796740 1 --2- 172.21.15.105:0/640895349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ad561a370 0x558ad56936f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.190+0000 7f7b3d796740 1 --2- 172.21.15.105:0/640895349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ad5610b40 0x558ad562db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.190+0000 7f7b1cff9640 0 client.15945 ms_handle_reset on v2:172.21.15.190:6834/1851042281 2024-09-16T02:05:44.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.190+0000 7f7b3d796740 1 --2- 172.21.15.105:0/640895349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558ad560f5d0 0x558ad562d5e0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.190+0000 7f7b3d796740 1 -- 172.21.15.105:0/640895349 >> 172.21.15.105:0/640895349 conn(0x558ad55d8310 msgr2=0x558ad5625420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:44.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.190+0000 7f7b3d796740 1 -- 172.21.15.105:0/640895349 shutdown_connections 2024-09-16T02:05:44.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.190+0000 7f7b3d796740 1 -- 172.21.15.105:0/640895349 wait complete. 2024-09-16T02:05:44.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.191+0000 7f7b3d796740 20 client.15945 trim_cache size 0 max 16384 2024-09-16T02:05:44.204 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-09-16T02:05:44.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.191+0000 7f7b3b3bd640 1 -- 172.21.15.105:0/342108945 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7b200a32c0 msgr2=0x7f7b200a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:44.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.191+0000 7f7b3b3bd640 1 --2- 172.21.15.105:0/342108945 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7b200a32c0 0x7f7b200a5780 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c0070b0 tx=0x7f7b2c0093d0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.191+0000 7f7b3b3bd640 1 -- 172.21.15.105:0/342108945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b34102570 msgr2=0x7f7b340fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:44.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.191+0000 7f7b3b3bd640 1 --2- 172.21.15.105:0/342108945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b34102570 0x7f7b340fd1c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f7b2400d950 tx=0x7f7b2400de10 comp rx=0 tx=0).stop 2024-09-16T02:05:44.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.191+0000 7f7b3b3bd640 1 -- 172.21.15.105:0/342108945 shutdown_connections 2024-09-16T02:05:44.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.191+0000 7f7b3b3bd640 1 --2- 172.21.15.105:0/342108945 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7b200a32c0 0x7f7b200a5780 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.191+0000 7f7b3b3bd640 1 --2- 172.21.15.105:0/342108945 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b34102ef0 0x7f7b340fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.191+0000 7f7b3b3bd640 1 --2- 172.21.15.105:0/342108945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b34102570 0x7f7b340fd1c0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.191+0000 7f7b3b3bd640 1 --2- 172.21.15.105:0/342108945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b34100e20 0x7f7b340fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.191+0000 7f7b3b3bd640 1 -- 172.21.15.105:0/342108945 >> 172.21.15.105:0/342108945 conn(0x7f7b340fc7d0 msgr2=0x7f7b3406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:44.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.191+0000 7f7b3b3bd640 1 -- 172.21.15.105:0/342108945 shutdown_connections 2024-09-16T02:05:44.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:44.191+0000 7f7b3b3bd640 1 -- 172.21.15.105:0/342108945 wait complete. 2024-09-16T02:05:45.016 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-16T02:05:45.260 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:45.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.747+0000 7f031d468640 1 -- 172.21.15.105:0/542848266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03181027a0 msgr2=0x7f03181067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:45.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.747+0000 7f031d468640 1 --2- 172.21.15.105:0/542848266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03181027a0 0x7f03181067f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f0300009900 tx=0x7f030002a410 comp rx=0 tx=0).stop 2024-09-16T02:05:45.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.747+0000 7f031d468640 1 -- 172.21.15.105:0/542848266 shutdown_connections 2024-09-16T02:05:45.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.747+0000 7f031d468640 1 --2- 172.21.15.105:0/542848266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0318106f20 0x7f031810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:45.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.747+0000 7f031d468640 1 --2- 172.21.15.105:0/542848266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03181027a0 0x7f03181067f0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:45.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.747+0000 7f031d468640 1 --2- 172.21.15.105:0/542848266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0318101df0 0x7f03181021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:45.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.747+0000 7f031d468640 1 -- 172.21.15.105:0/542848266 >> 172.21.15.105:0/542848266 conn(0x7f03180fd8f0 msgr2=0x7f03180ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:45.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.747+0000 7f031d468640 1 -- 172.21.15.105:0/542848266 shutdown_connections 2024-09-16T02:05:45.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.747+0000 7f031d468640 1 -- 172.21.15.105:0/542848266 wait complete. 2024-09-16T02:05:45.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.747+0000 7f031d468640 1 Processor -- start 2024-09-16T02:05:45.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.748+0000 7f031d468640 1 -- start start 2024-09-16T02:05:45.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.748+0000 7f031d468640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0318101df0 0x7f0318198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:45.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.748+0000 7f031d468640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03181027a0 0x7f0318198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:45.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.748+0000 7f031d468640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0318106f20 0x7f031819cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:45.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.748+0000 7f031d468640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0318111ed0 con 0x7f0318106f20 2024-09-16T02:05:45.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.748+0000 7f031d468640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0318111d50 con 0x7f03181027a0 2024-09-16T02:05:45.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.748+0000 7f031d468640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0318112050 con 0x7f0318101df0 2024-09-16T02:05:45.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.748+0000 7f03177fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0318106f20 0x7f031819cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:45.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.748+0000 7f03177fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0318106f20 0x7f031819cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49696/0 (socket says 172.21.15.105:49696) 2024-09-16T02:05:45.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.748+0000 7f03177fe640 1 -- 172.21.15.105:0/2892596489 learned_addr learned my addr 172.21.15.105:0/2892596489 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:45.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.748+0000 7f0316ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0318101df0 0x7f0318198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:45.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.748+0000 7f03167fc640 1 --2- 172.21.15.105:0/2892596489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03181027a0 0x7f0318198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:45.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.748+0000 7f03177fe640 1 -- 172.21.15.105:0/2892596489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0318101df0 msgr2=0x7f0318198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:45.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.748+0000 7f03177fe640 1 --2- 172.21.15.105:0/2892596489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0318101df0 0x7f0318198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:45.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.748+0000 7f03177fe640 1 -- 172.21.15.105:0/2892596489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03181027a0 msgr2=0x7f0318198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:45.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.748+0000 7f03177fe640 1 --2- 172.21.15.105:0/2892596489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03181027a0 0x7f0318198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:45.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.748+0000 7f03177fe640 1 -- 172.21.15.105:0/2892596489 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f031819d5e0 con 0x7f0318106f20 2024-09-16T02:05:45.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.748+0000 7f03167fc640 1 --2- 172.21.15.105:0/2892596489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03181027a0 0x7f0318198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:05:45.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.749+0000 7f03177fe640 1 --2- 172.21.15.105:0/2892596489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0318106f20 0x7f031819cf00 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f031000b970 tx=0x7f031000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:45.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.749+0000 7f030ffff640 1 -- 172.21.15.105:0/2892596489 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0310019070 con 0x7f0318106f20 2024-09-16T02:05:45.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.749+0000 7f030ffff640 1 -- 172.21.15.105:0/2892596489 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0310014440 con 0x7f0318106f20 2024-09-16T02:05:45.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.749+0000 7f031d468640 1 -- 172.21.15.105:0/2892596489 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f031819d8d0 con 0x7f0318106f20 2024-09-16T02:05:45.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.749+0000 7f030ffff640 1 -- 172.21.15.105:0/2892596489 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03100075d0 con 0x7f0318106f20 2024-09-16T02:05:45.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.749+0000 7f031d468640 1 -- 172.21.15.105:0/2892596489 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03181a51b0 con 0x7f0318106f20 2024-09-16T02:05:45.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.751+0000 7f030ffff640 1 -- 172.21.15.105:0/2892596489 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f031001c070 con 0x7f0318106f20 2024-09-16T02:05:45.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.751+0000 7f031f841740 1 Processor -- start 2024-09-16T02:05:45.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.752+0000 7f031f841740 1 -- start start 2024-09-16T02:05:45.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.752+0000 7f031f841740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56060bde7b40 0x56060bde6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:45.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.752+0000 7f031f841740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56060bde65d0 0x56060bde6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:45.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.752+0000 7f031f841740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56060bdf1370 0x56060bdf3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:45.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.752+0000 7f031f841740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56060bd4c2f0 con 0x56060bdf1370 2024-09-16T02:05:45.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.752+0000 7f031f841740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56060bcf3cf0 con 0x56060bde65d0 2024-09-16T02:05:45.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.752+0000 7f031f841740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56060bd482b0 con 0x56060bde7b40 2024-09-16T02:05:45.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.752+0000 7f030ffff640 1 --2- 172.21.15.105:0/2892596489 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f03040a3240 0x7f03040a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:45.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.752+0000 7f03167fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56060bdf1370 0x56060bdf3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:45.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.752+0000 7f03167fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56060bdf1370 0x56060bdf3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49706/0 (socket says 172.21.15.105:49706) 2024-09-16T02:05:45.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.752+0000 7f0316ffd640 1 --2- 172.21.15.105:0/2676858417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56060bde65d0 0x56060bde6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:45.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.752+0000 7f03167fc640 1 -- 172.21.15.105:0/2676858417 learned_addr learned my addr 172.21.15.105:0/2676858417 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:45.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.752+0000 7f03177fe640 1 --2- 172.21.15.105:0/2892596489 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f03040a3240 0x7f03040a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:45.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.752+0000 7f03167fc640 1 --2- 172.21.15.105:0/2676858417 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56060bde7b40 0x56060bde6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:45.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.753+0000 7f030ffff640 1 -- 172.21.15.105:0/2892596489 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f03100cef90 con 0x7f0318106f20 2024-09-16T02:05:45.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.753+0000 7f03167fc640 1 -- 172.21.15.105:0/2676858417 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56060bde7b40 msgr2=0x56060bde6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:45.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.753+0000 7f03167fc640 1 --2- 172.21.15.105:0/2676858417 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56060bde7b40 0x56060bde6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:45.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.753+0000 7f03167fc640 1 -- 172.21.15.105:0/2676858417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56060bde65d0 msgr2=0x56060bde6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:45.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.753+0000 7f03167fc640 1 --2- 172.21.15.105:0/2676858417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56060bde65d0 0x56060bde6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:45.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.753+0000 7f03167fc640 1 -- 172.21.15.105:0/2676858417 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56060bcd83a0 con 0x56060bdf1370 2024-09-16T02:05:45.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.753+0000 7f03177fe640 1 --2- 172.21.15.105:0/2892596489 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f03040a3240 0x7f03040a5700 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f031009b480 tx=0x7f03100909d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:45.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.753+0000 7f03167fc640 1 --2- 172.21.15.105:0/2676858417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56060bdf1370 0x56060bdf3760 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f03000096c0 tx=0x7f0300031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70709a301f74dd74 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:45.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.753+0000 7f030d7fa640 1 -- 172.21.15.105:0/2676858417 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0300035040 con 0x56060bdf1370 2024-09-16T02:05:45.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.753+0000 7f030d7fa640 1 -- 172.21.15.105:0/2676858417 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0300031670 con 0x56060bdf1370 2024-09-16T02:05:45.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.753+0000 7f030d7fa640 1 -- 172.21.15.105:0/2676858417 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0300005560 con 0x56060bdf1370 2024-09-16T02:05:45.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.754+0000 7f031f841740 1 -- 172.21.15.105:0/2676858417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56060bdf1370 msgr2=0x56060bdf3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:45.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.754+0000 7f031f841740 1 --2- 172.21.15.105:0/2676858417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56060bdf1370 0x56060bdf3760 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f03000096c0 tx=0x7f0300031400 comp rx=0 tx=0).stop 2024-09-16T02:05:45.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.754+0000 7f031f841740 1 -- 172.21.15.105:0/2676858417 shutdown_connections 2024-09-16T02:05:45.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.754+0000 7f031f841740 1 --2- 172.21.15.105:0/2676858417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56060bdf1370 0x56060bdf3760 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:45.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.754+0000 7f031f841740 1 --2- 172.21.15.105:0/2676858417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56060bde65d0 0x56060bde6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:45.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.754+0000 7f031f841740 1 --2- 172.21.15.105:0/2676858417 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56060bde7b40 0x56060bde6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:45.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.754+0000 7f031f841740 1 -- 172.21.15.105:0/2676858417 >> 172.21.15.105:0/2676858417 conn(0x56060bdaf310 msgr2=0x56060bde2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:45.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.754+0000 7f031f841740 1 -- 172.21.15.105:0/2676858417 shutdown_connections 2024-09-16T02:05:45.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.754+0000 7f031f841740 1 -- 172.21.15.105:0/2676858417 wait complete. 2024-09-16T02:05:45.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.754+0000 7f031f841740 1 -- start start 2024-09-16T02:05:45.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.754+0000 7f031f841740 1 Processor -- start 2024-09-16T02:05:45.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.755+0000 7f031f841740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56060bde65d0 0x56060bdf0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:45.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.755+0000 7f031f841740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56060bde7b40 0x56060bdee020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:45.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.755+0000 7f031f841740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56060bdf1370 0x56060bdee610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:45.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.755+0000 7f031f841740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56060bdf4010 con 0x56060bde65d0 2024-09-16T02:05:45.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.755+0000 7f031f841740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56060bd07060 con 0x56060bde7b40 2024-09-16T02:05:45.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.755+0000 7f031f841740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56060bdf4190 con 0x56060bdf1370 2024-09-16T02:05:45.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.755+0000 7f031f841740 10 client.0 _ll_register_callbacks cb 0x56060ba5a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:45.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.755+0000 7f03167fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56060bde65d0 0x56060bdf0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:45.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.755+0000 7f03167fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56060bde65d0 0x56060bdf0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49722/0 (socket says 172.21.15.105:49722) 2024-09-16T02:05:45.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.755+0000 7f03167fc640 1 -- 172.21.15.105:0/673647968 learned_addr learned my addr 172.21.15.105:0/673647968 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:45.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.755+0000 7f0316ffd640 1 --2- 172.21.15.105:0/673647968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56060bde7b40 0x56060bdee020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:45.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.755+0000 7f03167fc640 1 -- 172.21.15.105:0/673647968 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56060bdf1370 msgr2=0x56060bdee610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:45.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.755+0000 7f03167fc640 1 --2- 172.21.15.105:0/673647968 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56060bdf1370 0x56060bdee610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:45.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.755+0000 7f03167fc640 1 -- 172.21.15.105:0/673647968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56060bde7b40 msgr2=0x56060bdee020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:45.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.755+0000 7f03167fc640 1 --2- 172.21.15.105:0/673647968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56060bde7b40 0x56060bdee020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:45.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.755+0000 7f03167fc640 1 -- 172.21.15.105:0/673647968 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56060babd3c0 con 0x56060bde65d0 2024-09-16T02:05:45.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.755+0000 7f03167fc640 1 --2- 172.21.15.105:0/673647968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56060bde65d0 0x56060bdf0d70 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f0300009900 tx=0x7f030002ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a61e6f17985577 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:45.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.755+0000 7f02fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:05:45.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.755+0000 7f02fe7fc640 1 -- 172.21.15.105:0/673647968 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0300035070 con 0x56060bde65d0 2024-09-16T02:05:45.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.755+0000 7f02fe7fc640 1 -- 172.21.15.105:0/673647968 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f030003a870 con 0x56060bde65d0 2024-09-16T02:05:45.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.755+0000 7f031f841740 1 -- 172.21.15.105:0/673647968 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0300031670 con 0x56060bde65d0 2024-09-16T02:05:45.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.756+0000 7f02fe7fc640 1 -- 172.21.15.105:0/673647968 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f030005b050 con 0x56060bde65d0 2024-09-16T02:05:45.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.756+0000 7f031f841740 10 client.15957 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:45.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.756+0000 7f031f841740 1 -- 172.21.15.105:0/673647968 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56060bdf4010 con 0x56060bde65d0 2024-09-16T02:05:45.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.756+0000 7f02fe7fc640 1 -- 172.21.15.105:0/673647968 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0300031670 con 0x56060bde65d0 2024-09-16T02:05:45.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.756+0000 7f02fe7fc640 1 -- 172.21.15.105:0/673647968 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f030004a440 con 0x56060bde65d0 2024-09-16T02:05:45.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.756+0000 7f031f841740 10 client.15957 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:45.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.756+0000 7f031f841740 10 client.15957 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T02:05:45.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.756+0000 7f031f841740 20 client.15957 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:45.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.756+0000 7f031f841740 1 --2- 172.21.15.105:0/673647968 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x56060be00c30 0x56060be03020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:45.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.756+0000 7f031f841740 4 client.15957 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:05:45.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.756+0000 7f031f841740 1 -- 172.21.15.105:0/673647968 --> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56060bd59860 con 0x56060be00c30 2024-09-16T02:05:45.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.756+0000 7f0316ffd640 1 --2- 172.21.15.105:0/673647968 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x56060be00c30 0x56060be03020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:45.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.757+0000 7f02fe7fc640 10 client.15957 ms_handle_connect on v2:172.21.15.190:6836/3181257549 2024-09-16T02:05:45.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.757+0000 7f0316ffd640 1 --2- 172.21.15.105:0/673647968 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x56060be00c30 0x56060be03020 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=455e7ec5e8310357 server_cookie=f1d50c186cc9d2c7 in_seq=0 out_seq=0 2024-09-16T02:05:45.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.757+0000 7f02fe7fc640 1 -- 172.21.15.105:0/673647968 <== mds.3 v2:172.21.15.190:6836/3181257549 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56060bd59860 con 0x56060be00c30 2024-09-16T02:05:45.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.757+0000 7f02fe7fc640 10 client.15957 handle_command_reply: tid=0 2024-09-16T02:05:45.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.757+0000 7f031f841740 1 client.15957 shutdown 2024-09-16T02:05:45.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.757+0000 7f031f841740 1 -- 172.21.15.105:0/673647968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56060bde65d0 msgr2=0x56060bdf0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:45.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.757+0000 7f031f841740 1 --2- 172.21.15.105:0/673647968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56060bde65d0 0x56060bdf0d70 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f0300009900 tx=0x7f030002ab90 comp rx=0 tx=0).stop 2024-09-16T02:05:45.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.757+0000 7f031f841740 1 -- 172.21.15.105:0/673647968 shutdown_connections 2024-09-16T02:05:45.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.757+0000 7f031f841740 1 --2- 172.21.15.105:0/673647968 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56060bdf1370 0x56060bdee610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:45.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.757+0000 7f031f841740 1 --2- 172.21.15.105:0/673647968 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x56060be00c30 0x56060be03020 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:45.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.758+0000 7f031f841740 1 --2- 172.21.15.105:0/673647968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56060bde7b40 0x56060bdee020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:45.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.758+0000 7f031f841740 1 --2- 172.21.15.105:0/673647968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56060bde65d0 0x56060bdf0d70 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:45.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.758+0000 7f02fe7fc640 0 client.15957 ms_handle_reset on v2:172.21.15.190:6836/3181257549 2024-09-16T02:05:45.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.758+0000 7f031f841740 1 -- 172.21.15.105:0/673647968 >> 172.21.15.105:0/673647968 conn(0x56060bdaf310 msgr2=0x56060bde8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:45.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.758+0000 7f031f841740 1 -- 172.21.15.105:0/673647968 shutdown_connections 2024-09-16T02:05:45.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.758+0000 7f031f841740 1 -- 172.21.15.105:0/673647968 wait complete. 2024-09-16T02:05:45.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.758+0000 7f031f841740 20 client.15957 trim_cache size 0 max 16384 2024-09-16T02:05:45.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.771+0000 7f031f841740 1 Processor -- start 2024-09-16T02:05:45.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.771+0000 7f031f841740 1 -- start start 2024-09-16T02:05:45.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.771+0000 7f031f841740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56060bde65d0 0x56060bde2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:45.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.771+0000 7f031f841740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56060bde7b40 0x56060bde7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:45.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.771+0000 7f031f841740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56060bdf1370 0x56060bdfd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:45.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.771+0000 7f031f841740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f03181ab550 con 0x56060bde7b40 2024-09-16T02:05:45.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.771+0000 7f031f841740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f03181ad090 con 0x56060bdf1370 2024-09-16T02:05:45.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.771+0000 7f031f841740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f03181abc20 con 0x56060bde65d0 2024-09-16T02:05:45.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.771+0000 7f0316ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56060bde7b40 0x56060bde7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:45.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.771+0000 7f0316ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56060bde7b40 0x56060bde7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49726/0 (socket says 172.21.15.105:49726) 2024-09-16T02:05:45.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.771+0000 7f0316ffd640 1 -- 172.21.15.105:0/365739478 learned_addr learned my addr 172.21.15.105:0/365739478 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:45.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.772+0000 7f03167fc640 1 --2- 172.21.15.105:0/365739478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56060bde65d0 0x56060bde2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:45.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.772+0000 7f0316ffd640 1 -- 172.21.15.105:0/365739478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56060bde65d0 msgr2=0x56060bde2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:45.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.772+0000 7f0316ffd640 1 --2- 172.21.15.105:0/365739478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56060bde65d0 0x56060bde2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:45.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.772+0000 7f0316ffd640 1 -- 172.21.15.105:0/365739478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56060bdf1370 msgr2=0x56060bdfd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:45.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.772+0000 7f0316ffd640 1 --2- 172.21.15.105:0/365739478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56060bdf1370 0x56060bdfd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:45.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.772+0000 7f0316ffd640 1 -- 172.21.15.105:0/365739478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56060bdec590 con 0x56060bde7b40 2024-09-16T02:05:45.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.772+0000 7f03167fc640 1 --2- 172.21.15.105:0/365739478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56060bde65d0 0x56060bde2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:05:45.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.772+0000 7f0316ffd640 1 --2- 172.21.15.105:0/365739478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56060bde7b40 0x56060bde7300 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f0308009b00 tx=0x7f0308008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a65190a9d3de72 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:45.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.772+0000 7f030cff9640 1 -- 172.21.15.105:0/365739478 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0308021070 con 0x56060bde7b40 2024-09-16T02:05:45.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.772+0000 7f030cff9640 1 -- 172.21.15.105:0/365739478 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0308016cf0 con 0x56060bde7b40 2024-09-16T02:05:45.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.772+0000 7f030cff9640 1 -- 172.21.15.105:0/365739478 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0308015690 con 0x56060bde7b40 2024-09-16T02:05:45.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.773+0000 7f031f841740 1 -- 172.21.15.105:0/365739478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56060bde7b40 msgr2=0x56060bde7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:45.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.773+0000 7f031f841740 1 --2- 172.21.15.105:0/365739478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56060bde7b40 0x56060bde7300 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f0308009b00 tx=0x7f0308008040 comp rx=0 tx=0).stop 2024-09-16T02:05:45.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.773+0000 7f031f841740 1 -- 172.21.15.105:0/365739478 shutdown_connections 2024-09-16T02:05:45.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.773+0000 7f031f841740 1 --2- 172.21.15.105:0/365739478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56060bdf1370 0x56060bdfd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:45.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.773+0000 7f031f841740 1 --2- 172.21.15.105:0/365739478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56060bde7b40 0x56060bde7300 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:45.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.773+0000 7f031f841740 1 --2- 172.21.15.105:0/365739478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56060bde65d0 0x56060bde2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:45.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.773+0000 7f031f841740 1 -- 172.21.15.105:0/365739478 >> 172.21.15.105:0/365739478 conn(0x56060bdaf310 msgr2=0x56060bde26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:45.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.773+0000 7f031f841740 1 -- 172.21.15.105:0/365739478 shutdown_connections 2024-09-16T02:05:45.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.773+0000 7f031f841740 1 -- 172.21.15.105:0/365739478 wait complete. 2024-09-16T02:05:45.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.773+0000 7f031f841740 1 -- start start 2024-09-16T02:05:45.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.773+0000 7f031f841740 1 Processor -- start 2024-09-16T02:05:45.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.773+0000 7f031f841740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56060bde65d0 0x56060bdf9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:45.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.773+0000 7f031f841740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56060bde7b40 0x56060bdeaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:45.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.773+0000 7f031f841740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56060bdf1370 0x56060bdeb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:45.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.773+0000 7f031f841740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f03181ac9c0 con 0x56060bde7b40 2024-09-16T02:05:45.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.773+0000 7f031f841740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f03181ac2f0 con 0x56060bdf1370 2024-09-16T02:05:45.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.773+0000 7f031f841740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f03181a9530 con 0x56060bde65d0 2024-09-16T02:05:45.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.773+0000 7f031f841740 10 client.0 _ll_register_callbacks cb 0x7f03080093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:45.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.773+0000 7f0316ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56060bde7b40 0x56060bdeaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:45.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.774+0000 7f0316ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56060bde7b40 0x56060bdeaae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49732/0 (socket says 172.21.15.105:49732) 2024-09-16T02:05:45.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.774+0000 7f0316ffd640 1 -- 172.21.15.105:0/1098918301 learned_addr learned my addr 172.21.15.105:0/1098918301 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:45.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.774+0000 7f03167fc640 1 --2- 172.21.15.105:0/1098918301 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56060bde65d0 0x56060bdf9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:45.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.774+0000 7f0316ffd640 1 -- 172.21.15.105:0/1098918301 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56060bde65d0 msgr2=0x56060bdf9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:45.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.774+0000 7f0316ffd640 1 --2- 172.21.15.105:0/1098918301 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56060bde65d0 0x56060bdf9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:45.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.774+0000 7f0316ffd640 1 -- 172.21.15.105:0/1098918301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56060bdf1370 msgr2=0x56060bdeb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:45.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.774+0000 7f0316ffd640 1 --2- 172.21.15.105:0/1098918301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56060bdf1370 0x56060bdeb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:45.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.774+0000 7f0316ffd640 1 -- 172.21.15.105:0/1098918301 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56060bdeb560 con 0x56060bde7b40 2024-09-16T02:05:45.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.774+0000 7f0316ffd640 1 --2- 172.21.15.105:0/1098918301 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56060bde7b40 0x56060bdeaae0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f0308006e10 tx=0x7f030801d480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb723375ff5c05c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:45.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.774+0000 7f02fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:05:45.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.774+0000 7f02fe7fc640 1 -- 172.21.15.105:0/1098918301 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030802c070 con 0x56060bde7b40 2024-09-16T02:05:45.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.774+0000 7f02fe7fc640 1 -- 172.21.15.105:0/1098918301 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f030801daf0 con 0x56060bde7b40 2024-09-16T02:05:45.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.774+0000 7f031f841740 1 -- 172.21.15.105:0/1098918301 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0308016cf0 con 0x56060bde7b40 2024-09-16T02:05:45.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.774+0000 7f02fe7fc640 1 -- 172.21.15.105:0/1098918301 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0308038050 con 0x56060bde7b40 2024-09-16T02:05:45.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.774+0000 7f031f841740 10 client.15963 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:45.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.774+0000 7f031f841740 1 -- 172.21.15.105:0/1098918301 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56060be6b330 con 0x56060bde7b40 2024-09-16T02:05:45.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.774+0000 7f02fe7fc640 1 -- 172.21.15.105:0/1098918301 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0308016cf0 con 0x56060bde7b40 2024-09-16T02:05:45.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.775+0000 7f02fe7fc640 1 -- 172.21.15.105:0/1098918301 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f03080155d0 con 0x56060bde7b40 2024-09-16T02:05:45.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.775+0000 7f031f841740 10 client.15963 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:45.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.775+0000 7f031f841740 10 client.15963 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T02:05:45.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.775+0000 7f031f841740 20 client.15963 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:45.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.775+0000 7f031f841740 1 --2- 172.21.15.105:0/1098918301 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x56060be6b9f0 0x56060be6de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:45.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.775+0000 7f031f841740 4 client.15963 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T02:05:45.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.775+0000 7f031f841740 1 -- 172.21.15.105:0/1098918301 --> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56060bd53d70 con 0x56060be6b9f0 2024-09-16T02:05:45.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.775+0000 7f03167fc640 1 --2- 172.21.15.105:0/1098918301 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x56060be6b9f0 0x56060be6de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:45.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.775+0000 7f02fe7fc640 10 client.15963 ms_handle_connect on v2:172.21.15.190:6836/3181257549 2024-09-16T02:05:45.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.775+0000 7f03167fc640 1 --2- 172.21.15.105:0/1098918301 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x56060be6b9f0 0x56060be6de10 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f333626418376881 server_cookie=41e83bab63ab0aa1 in_seq=0 out_seq=0 2024-09-16T02:05:45.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.776+0000 7f02fe7fc640 1 -- 172.21.15.105:0/1098918301 <== mds.3 v2:172.21.15.190:6836/3181257549 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56060bd53d70 con 0x56060be6b9f0 2024-09-16T02:05:45.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.776+0000 7f02fe7fc640 10 client.15963 handle_command_reply: tid=0 2024-09-16T02:05:45.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.776+0000 7f031f841740 1 client.15963 shutdown 2024-09-16T02:05:45.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.776+0000 7f031f841740 1 -- 172.21.15.105:0/1098918301 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56060bde7b40 msgr2=0x56060bdeaae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:45.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.776+0000 7f031f841740 1 --2- 172.21.15.105:0/1098918301 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56060bde7b40 0x56060bdeaae0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f0308006e10 tx=0x7f030801d480 comp rx=0 tx=0).stop 2024-09-16T02:05:45.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.776+0000 7f031f841740 1 -- 172.21.15.105:0/1098918301 shutdown_connections 2024-09-16T02:05:45.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.776+0000 7f031f841740 1 --2- 172.21.15.105:0/1098918301 >> [v2:172.21.15.190:6836/3181257549,v1:172.21.15.190:6837/3181257549] conn(0x56060be6b9f0 0x56060be6de10 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:45.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.776+0000 7f031f841740 1 --2- 172.21.15.105:0/1098918301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56060bdf1370 0x56060bdeb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:45.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.776+0000 7f031f841740 1 --2- 172.21.15.105:0/1098918301 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56060bde7b40 0x56060bdeaae0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:45.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.776+0000 7f031f841740 1 --2- 172.21.15.105:0/1098918301 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56060bde65d0 0x56060bdf9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:45.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.776+0000 7f031f841740 1 -- 172.21.15.105:0/1098918301 >> 172.21.15.105:0/1098918301 conn(0x56060bdaf310 msgr2=0x56060bde26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:45.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.776+0000 7f02fe7fc640 0 client.15963 ms_handle_reset on v2:172.21.15.190:6836/3181257549 2024-09-16T02:05:45.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.776+0000 7f031f841740 1 -- 172.21.15.105:0/1098918301 shutdown_connections 2024-09-16T02:05:45.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.776+0000 7f031f841740 1 -- 172.21.15.105:0/1098918301 wait complete. 2024-09-16T02:05:45.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.776+0000 7f031f841740 20 client.15963 trim_cache size 0 max 16384 2024-09-16T02:05:45.795 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-09-16T02:05:45.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.776+0000 7f031d468640 1 -- 172.21.15.105:0/2892596489 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f03040a3240 msgr2=0x7f03040a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:45.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.776+0000 7f031d468640 1 --2- 172.21.15.105:0/2892596489 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f03040a3240 0x7f03040a5700 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f031009b480 tx=0x7f03100909d0 comp rx=0 tx=0).stop 2024-09-16T02:05:45.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.776+0000 7f031d468640 1 -- 172.21.15.105:0/2892596489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0318106f20 msgr2=0x7f031819cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:45.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.776+0000 7f031d468640 1 --2- 172.21.15.105:0/2892596489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0318106f20 0x7f031819cf00 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f031000b970 tx=0x7f031000bd70 comp rx=0 tx=0).stop 2024-09-16T02:05:45.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.776+0000 7f031d468640 1 -- 172.21.15.105:0/2892596489 shutdown_connections 2024-09-16T02:05:45.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.776+0000 7f031d468640 1 --2- 172.21.15.105:0/2892596489 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f03040a3240 0x7f03040a5700 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:45.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.776+0000 7f031d468640 1 --2- 172.21.15.105:0/2892596489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0318106f20 0x7f031819cf00 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:45.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.776+0000 7f031d468640 1 --2- 172.21.15.105:0/2892596489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03181027a0 0x7f0318198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:45.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.776+0000 7f031d468640 1 --2- 172.21.15.105:0/2892596489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0318101df0 0x7f0318198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:45.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.776+0000 7f031d468640 1 -- 172.21.15.105:0/2892596489 >> 172.21.15.105:0/2892596489 conn(0x7f03180fd8f0 msgr2=0x7f03180fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:45.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.777+0000 7f031d468640 1 -- 172.21.15.105:0/2892596489 shutdown_connections 2024-09-16T02:05:45.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:45.777+0000 7f031d468640 1 -- 172.21.15.105:0/2892596489 wait complete. 2024-09-16T02:05:46.721 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T02:05:46.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:46 smithi190 ceph-mon[34762]: pgmap v436: 97 pgs: 97 active+clean; 133 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.3 KiB/s wr, 222 op/s 2024-09-16T02:05:46.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:46 smithi137 ceph-mon[34099]: pgmap v436: 97 pgs: 97 active+clean; 133 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.3 KiB/s wr, 222 op/s 2024-09-16T02:05:46.967 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:47.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:46 smithi105 ceph-mon[31197]: pgmap v436: 97 pgs: 97 active+clean; 133 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.3 KiB/s wr, 222 op/s 2024-09-16T02:05:47.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.472+0000 7ff6f72ac640 1 -- 172.21.15.105:0/2209838236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6f01082a0 msgr2=0x7ff6f010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:47.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.472+0000 7ff6f72ac640 1 --2- 172.21.15.105:0/2209838236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6f01082a0 0x7ff6f010ec40 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8009cd0 tx=0x7ff6e802a8e0 comp rx=0 tx=0).stop 2024-09-16T02:05:47.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.472+0000 7ff6f72ac640 1 -- 172.21.15.105:0/2209838236 shutdown_connections 2024-09-16T02:05:47.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.472+0000 7ff6f72ac640 1 --2- 172.21.15.105:0/2209838236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6f01036e0 0x7ff6f0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:47.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.472+0000 7ff6f72ac640 1 --2- 172.21.15.105:0/2209838236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6f01082a0 0x7ff6f010ec40 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:47.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.472+0000 7ff6f72ac640 1 --2- 172.21.15.105:0/2209838236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6f0102d30 0x7ff6f0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:47.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.472+0000 7ff6f72ac640 1 -- 172.21.15.105:0/2209838236 >> 172.21.15.105:0/2209838236 conn(0x7ff6f00fc7d0 msgr2=0x7ff6f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:47.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.472+0000 7ff6f72ac640 1 -- 172.21.15.105:0/2209838236 shutdown_connections 2024-09-16T02:05:47.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.472+0000 7ff6f72ac640 1 -- 172.21.15.105:0/2209838236 wait complete. 2024-09-16T02:05:47.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.473+0000 7ff6f72ac640 1 Processor -- start 2024-09-16T02:05:47.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.473+0000 7ff6f72ac640 1 -- start start 2024-09-16T02:05:47.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.473+0000 7ff6f72ac640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6f0102d30 0x7ff6f0100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:47.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.473+0000 7ff6f72ac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6f01036e0 0x7ff6f0100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:47.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.473+0000 7ff6f72ac640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6f01082a0 0x7ff6f0101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:47.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.473+0000 7ff6f72ac640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f0111f50 con 0x7ff6f01036e0 2024-09-16T02:05:47.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.473+0000 7ff6f72ac640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f0111dd0 con 0x7ff6f01082a0 2024-09-16T02:05:47.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.473+0000 7ff6f72ac640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f01120d0 con 0x7ff6f0102d30 2024-09-16T02:05:47.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.473+0000 7ff6f4847640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6f01036e0 0x7ff6f0100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:47.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.474+0000 7ff6f4847640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6f01036e0 0x7ff6f0100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49760/0 (socket says 172.21.15.105:49760) 2024-09-16T02:05:47.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.474+0000 7ff6f4847640 1 -- 172.21.15.105:0/2243836761 learned_addr learned my addr 172.21.15.105:0/2243836761 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:47.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.474+0000 7ff6f5048640 1 --2- 172.21.15.105:0/2243836761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6f0102d30 0x7ff6f0100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:47.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.474+0000 7ff6f5849640 1 --2- 172.21.15.105:0/2243836761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6f01082a0 0x7ff6f0101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:47.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.474+0000 7ff6f4847640 1 -- 172.21.15.105:0/2243836761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6f0102d30 msgr2=0x7ff6f0100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:47.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.474+0000 7ff6f4847640 1 --2- 172.21.15.105:0/2243836761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6f0102d30 0x7ff6f0100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:47.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.474+0000 7ff6f4847640 1 -- 172.21.15.105:0/2243836761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6f01082a0 msgr2=0x7ff6f0101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:47.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.474+0000 7ff6f4847640 1 --2- 172.21.15.105:0/2243836761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6f01082a0 0x7ff6f0101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:47.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.474+0000 7ff6f4847640 1 -- 172.21.15.105:0/2243836761 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6f01a6960 con 0x7ff6f01036e0 2024-09-16T02:05:47.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.474+0000 7ff6f4847640 1 --2- 172.21.15.105:0/2243836761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6f01036e0 0x7ff6f0100ed0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff6e000e9c0 tx=0x7ff6e000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:47.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.474+0000 7ff6e67fc640 1 -- 172.21.15.105:0/2243836761 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e000ccc0 con 0x7ff6f01036e0 2024-09-16T02:05:47.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.474+0000 7ff6f72ac640 1 -- 172.21.15.105:0/2243836761 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6f01a6c50 con 0x7ff6f01036e0 2024-09-16T02:05:47.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.474+0000 7ff6e67fc640 1 -- 172.21.15.105:0/2243836761 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6e00054d0 con 0x7ff6f01036e0 2024-09-16T02:05:47.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.474+0000 7ff6e67fc640 1 -- 172.21.15.105:0/2243836761 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e0010430 con 0x7ff6f01036e0 2024-09-16T02:05:47.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.474+0000 7ff6f72ac640 1 -- 172.21.15.105:0/2243836761 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6f01a7190 con 0x7ff6f01036e0 2024-09-16T02:05:47.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.477+0000 7ff6f9685740 1 Processor -- start 2024-09-16T02:05:47.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.477+0000 7ff6f9685740 1 -- start start 2024-09-16T02:05:47.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.477+0000 7ff6e67fc640 1 -- 172.21.15.105:0/2243836761 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7ff6e0005000 con 0x7ff6f01036e0 2024-09-16T02:05:47.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.478+0000 7ff6f9685740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5569df828b40 0x5569df827090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:47.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.478+0000 7ff6f9685740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5569df8275d0 0x5569df827a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:47.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.478+0000 7ff6f9685740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569df832370 0x5569df834760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:47.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.478+0000 7ff6f9685740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5569df78d2f0 con 0x5569df828b40 2024-09-16T02:05:47.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.478+0000 7ff6f9685740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5569df734cf0 con 0x5569df832370 2024-09-16T02:05:47.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.478+0000 7ff6f9685740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5569df7892b0 con 0x5569df8275d0 2024-09-16T02:05:47.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.478+0000 7ff6f5849640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5569df828b40 0x5569df827090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:47.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.478+0000 7ff6f5849640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5569df828b40 0x5569df827090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49774/0 (socket says 172.21.15.105:49774) 2024-09-16T02:05:47.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.478+0000 7ff6f5849640 1 -- 172.21.15.105:0/3504139524 learned_addr learned my addr 172.21.15.105:0/3504139524 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:47.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.478+0000 7ff6f4847640 1 --2- 172.21.15.105:0/3504139524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569df832370 0x5569df834760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:47.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.478+0000 7ff6f5048640 1 --2- 172.21.15.105:0/3504139524 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5569df8275d0 0x5569df827a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:47.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.478+0000 7ff6f5849640 1 -- 172.21.15.105:0/3504139524 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5569df8275d0 msgr2=0x5569df827a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:47.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.478+0000 7ff6f5849640 1 --2- 172.21.15.105:0/3504139524 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5569df8275d0 0x5569df827a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:47.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.478+0000 7ff6f5849640 1 -- 172.21.15.105:0/3504139524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569df832370 msgr2=0x5569df834760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:47.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.478+0000 7ff6f5849640 1 --2- 172.21.15.105:0/3504139524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569df832370 0x5569df834760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:47.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.478+0000 7ff6f5849640 1 -- 172.21.15.105:0/3504139524 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569df7193a0 con 0x5569df828b40 2024-09-16T02:05:47.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.478+0000 7ff6f5048640 1 --2- 172.21.15.105:0/3504139524 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5569df8275d0 0x5569df827a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:05:47.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.478+0000 7ff6f4847640 1 --2- 172.21.15.105:0/3504139524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569df832370 0x5569df834760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:05:47.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.478+0000 7ff6e67fc640 1 --2- 172.21.15.105:0/2243836761 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff6dc0a32c0 0x7ff6dc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:47.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.478+0000 7ff6f5849640 1 --2- 172.21.15.105:0/3504139524 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5569df828b40 0x5569df827090 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8009cd0 tx=0x7ff6e8009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea1694a5389b28f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:47.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.479+0000 7ff6f5048640 1 --2- 172.21.15.105:0/2243836761 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff6dc0a32c0 0x7ff6dc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:47.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.479+0000 7ff6e67fc640 1 -- 172.21.15.105:0/2243836761 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7ff6e00cfbf0 con 0x7ff6f01036e0 2024-09-16T02:05:47.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.479+0000 7ff6db7fe640 1 -- 172.21.15.105:0/3504139524 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8049070 con 0x5569df828b40 2024-09-16T02:05:47.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.479+0000 7ff6db7fe640 1 -- 172.21.15.105:0/3504139524 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6e8036040 con 0x5569df828b40 2024-09-16T02:05:47.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.479+0000 7ff6db7fe640 1 -- 172.21.15.105:0/3504139524 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e80042a0 con 0x5569df828b40 2024-09-16T02:05:47.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.480+0000 7ff6f9685740 1 -- 172.21.15.105:0/3504139524 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5569df828b40 msgr2=0x5569df827090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:47.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.480+0000 7ff6f9685740 1 --2- 172.21.15.105:0/3504139524 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5569df828b40 0x5569df827090 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8009cd0 tx=0x7ff6e8009a20 comp rx=0 tx=0).stop 2024-09-16T02:05:47.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.480+0000 7ff6f5048640 1 --2- 172.21.15.105:0/2243836761 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff6dc0a32c0 0x7ff6dc0a5780 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x5569df824ac0 tx=0x7ff6ec00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:47.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.480+0000 7ff6f9685740 1 -- 172.21.15.105:0/3504139524 shutdown_connections 2024-09-16T02:05:47.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.480+0000 7ff6f9685740 1 --2- 172.21.15.105:0/3504139524 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5569df8275d0 0x5569df827a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:47.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.480+0000 7ff6f9685740 1 --2- 172.21.15.105:0/3504139524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569df832370 0x5569df834760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:47.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.480+0000 7ff6f9685740 1 --2- 172.21.15.105:0/3504139524 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5569df828b40 0x5569df827090 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:47.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.480+0000 7ff6f9685740 1 -- 172.21.15.105:0/3504139524 >> 172.21.15.105:0/3504139524 conn(0x5569df7f0310 msgr2=0x5569df823c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:47.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.480+0000 7ff6f9685740 1 -- 172.21.15.105:0/3504139524 shutdown_connections 2024-09-16T02:05:47.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.480+0000 7ff6f9685740 1 -- 172.21.15.105:0/3504139524 wait complete. 2024-09-16T02:05:47.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.480+0000 7ff6f9685740 1 -- start start 2024-09-16T02:05:47.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.480+0000 7ff6f9685740 1 Processor -- start 2024-09-16T02:05:47.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.480+0000 7ff6f9685740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5569df8275d0 0x5569df831cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:47.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.481+0000 7ff6f9685740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569df828b40 0x5569df82efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:47.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.481+0000 7ff6f9685740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5569df832370 0x5569df82f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:47.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.481+0000 7ff6f9685740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5569df8350b0 con 0x5569df832370 2024-09-16T02:05:47.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.481+0000 7ff6f9685740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5569df748060 con 0x5569df828b40 2024-09-16T02:05:47.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.481+0000 7ff6f9685740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5569df835230 con 0x5569df8275d0 2024-09-16T02:05:47.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.481+0000 7ff6f5849640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5569df832370 0x5569df82f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:47.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.481+0000 7ff6f5048640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5569df8275d0 0x5569df831cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:47.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.481+0000 7ff6f9685740 10 client.0 _ll_register_callbacks cb 0x5569df49b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:47.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.481+0000 7ff6f5849640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5569df832370 0x5569df82f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49786/0 (socket says 172.21.15.105:49786) 2024-09-16T02:05:47.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.481+0000 7ff6f5048640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5569df8275d0 0x5569df831cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.105:55540/0 (socket says 172.21.15.105:55540) 2024-09-16T02:05:47.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.481+0000 7ff6f5849640 1 -- 172.21.15.105:0/3204726677 learned_addr learned my addr 172.21.15.105:0/3204726677 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:47.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.481+0000 7ff6f4847640 1 --2- 172.21.15.105:0/3204726677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569df828b40 0x5569df82efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:47.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.481+0000 7ff6f4847640 1 -- 172.21.15.105:0/3204726677 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5569df8275d0 msgr2=0x5569df831cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:47.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.481+0000 7ff6f4847640 1 --2- 172.21.15.105:0/3204726677 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5569df8275d0 0x5569df831cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:47.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.481+0000 7ff6f4847640 1 -- 172.21.15.105:0/3204726677 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5569df832370 msgr2=0x5569df82f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:47.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.481+0000 7ff6f4847640 1 --2- 172.21.15.105:0/3204726677 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5569df832370 0x5569df82f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:47.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.481+0000 7ff6f4847640 1 -- 172.21.15.105:0/3204726677 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569df4fe3c0 con 0x5569df828b40 2024-09-16T02:05:47.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.481+0000 7ff6f5849640 1 --2- 172.21.15.105:0/3204726677 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5569df832370 0x5569df82f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:05:47.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.481+0000 7ff6f4847640 1 --2- 172.21.15.105:0/3204726677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569df828b40 0x5569df82efc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff6e00cd6e0 tx=0x7ff6e009c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edcd70ebde2f2e48 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:47.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.481+0000 7ff6d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-16T02:05:47.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.481+0000 7ff6d8ff9640 1 -- 172.21.15.105:0/3204726677 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e0005680 con 0x5569df828b40 2024-09-16T02:05:47.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.482+0000 7ff6d8ff9640 1 -- 172.21.15.105:0/3204726677 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7ff6e009a4b0 con 0x5569df828b40 2024-09-16T02:05:47.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.482+0000 7ff6f9685740 1 -- 172.21.15.105:0/3204726677 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6e8036040 con 0x5569df828b40 2024-09-16T02:05:47.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.482+0000 7ff6d8ff9640 1 -- 172.21.15.105:0/3204726677 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff6e00052f0 con 0x5569df828b40 2024-09-16T02:05:47.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.482+0000 7ff6f9685740 10 client.25181 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:47.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.482+0000 7ff6f9685740 1 -- 172.21.15.105:0/3204726677 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5569df840be0 con 0x5569df828b40 2024-09-16T02:05:47.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.482+0000 7ff6d8ff9640 1 -- 172.21.15.105:0/3204726677 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6e8036040 con 0x5569df828b40 2024-09-16T02:05:47.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.482+0000 7ff6d8ff9640 1 -- 172.21.15.105:0/3204726677 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff6e00d6070 con 0x5569df828b40 2024-09-16T02:05:47.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.482+0000 7ff6f9685740 10 client.25181 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:47.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.482+0000 7ff6f9685740 10 client.25181 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T02:05:47.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.482+0000 7ff6f9685740 20 client.25181 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:47.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.482+0000 7ff6f9685740 1 --2- 172.21.15.105:0/3204726677 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x5569df842040 0x5569df844430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:47.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.482+0000 7ff6f9685740 4 client.25181 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:05:47.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.482+0000 7ff6f9685740 1 -- 172.21.15.105:0/3204726677 --> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569df79a860 con 0x5569df842040 2024-09-16T02:05:47.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.483+0000 7ff6f5048640 1 --2- 172.21.15.105:0/3204726677 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x5569df842040 0x5569df844430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:47.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.483+0000 7ff6f5048640 1 --2- 172.21.15.105:0/3204726677 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x5569df842040 0x5569df844430 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=79b36b83fde3653c server_cookie=e7c22e055bb19c4e in_seq=0 out_seq=0 2024-09-16T02:05:47.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.483+0000 7ff6d8ff9640 10 client.25181 ms_handle_connect on v2:172.21.15.137:6836/2788541030 2024-09-16T02:05:47.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.484+0000 7ff6d8ff9640 1 -- 172.21.15.105:0/3204726677 <== mds.2 v2:172.21.15.137:6836/2788541030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5569df79a860 con 0x5569df842040 2024-09-16T02:05:47.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.484+0000 7ff6d8ff9640 10 client.25181 handle_command_reply: tid=0 2024-09-16T02:05:47.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.484+0000 7ff6f9685740 1 client.25181 shutdown 2024-09-16T02:05:47.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.484+0000 7ff6f9685740 1 -- 172.21.15.105:0/3204726677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569df828b40 msgr2=0x5569df82efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:47.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.484+0000 7ff6f9685740 1 --2- 172.21.15.105:0/3204726677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569df828b40 0x5569df82efc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff6e00cd6e0 tx=0x7ff6e009c040 comp rx=0 tx=0).stop 2024-09-16T02:05:47.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.485+0000 7ff6f9685740 1 -- 172.21.15.105:0/3204726677 shutdown_connections 2024-09-16T02:05:47.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.485+0000 7ff6f9685740 1 --2- 172.21.15.105:0/3204726677 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x5569df842040 0x5569df844430 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:47.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.485+0000 7ff6f9685740 1 --2- 172.21.15.105:0/3204726677 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5569df832370 0x5569df82f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:47.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.485+0000 7ff6f9685740 1 --2- 172.21.15.105:0/3204726677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569df828b40 0x5569df82efc0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:47.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.485+0000 7ff6f9685740 1 --2- 172.21.15.105:0/3204726677 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5569df8275d0 0x5569df831cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:47.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.485+0000 7ff6d8ff9640 0 client.25181 ms_handle_reset on v2:172.21.15.137:6836/2788541030 2024-09-16T02:05:47.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.485+0000 7ff6f9685740 1 -- 172.21.15.105:0/3204726677 >> 172.21.15.105:0/3204726677 conn(0x5569df7f0310 msgr2=0x5569df829930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:47.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.485+0000 7ff6f9685740 1 -- 172.21.15.105:0/3204726677 shutdown_connections 2024-09-16T02:05:47.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.485+0000 7ff6f9685740 1 -- 172.21.15.105:0/3204726677 wait complete. 2024-09-16T02:05:47.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.485+0000 7ff6f9685740 20 client.25181 trim_cache size 0 max 16384 2024-09-16T02:05:47.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.498+0000 7ff6f9685740 1 Processor -- start 2024-09-16T02:05:47.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.498+0000 7ff6f9685740 1 -- start start 2024-09-16T02:05:47.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.498+0000 7ff6f9685740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569df8275d0 0x5569df823e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:47.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.498+0000 7ff6f9685740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5569df828b40 0x5569df83e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:47.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.498+0000 7ff6f9685740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5569df832370 0x5569df82e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:47.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.498+0000 7ff6f9685740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f01af750 con 0x5569df832370 2024-09-16T02:05:47.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.498+0000 7ff6f9685740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f01b1290 con 0x5569df8275d0 2024-09-16T02:05:47.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.498+0000 7ff6f9685740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f01afe20 con 0x5569df828b40 2024-09-16T02:05:47.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.498+0000 7ff6f5849640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5569df832370 0x5569df82e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:47.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.498+0000 7ff6f5849640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5569df832370 0x5569df82e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49800/0 (socket says 172.21.15.105:49800) 2024-09-16T02:05:47.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.498+0000 7ff6f5849640 1 -- 172.21.15.105:0/3791489671 learned_addr learned my addr 172.21.15.105:0/3791489671 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:47.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.498+0000 7ff6f4847640 1 --2- 172.21.15.105:0/3791489671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5569df828b40 0x5569df83e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:47.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.498+0000 7ff6f5048640 1 --2- 172.21.15.105:0/3791489671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569df8275d0 0x5569df823e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:47.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.499+0000 7ff6f5849640 1 -- 172.21.15.105:0/3791489671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5569df828b40 msgr2=0x5569df83e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:47.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.499+0000 7ff6f5849640 1 --2- 172.21.15.105:0/3791489671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5569df828b40 0x5569df83e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:47.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.499+0000 7ff6f5849640 1 -- 172.21.15.105:0/3791489671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569df8275d0 msgr2=0x5569df823e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:47.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.499+0000 7ff6f5849640 1 --2- 172.21.15.105:0/3791489671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569df8275d0 0x5569df823e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:47.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.499+0000 7ff6f5849640 1 -- 172.21.15.105:0/3791489671 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569df82a120 con 0x5569df832370 2024-09-16T02:05:47.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.499+0000 7ff6f5048640 1 --2- 172.21.15.105:0/3791489671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569df8275d0 0x5569df823e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:05:47.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.499+0000 7ff6f5849640 1 --2- 172.21.15.105:0/3791489671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5569df832370 0x5569df82e880 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8033040 tx=0x7ff6e8000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=879c6b15be054b5c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:47.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.499+0000 7ff6daffd640 1 -- 172.21.15.105:0/3791489671 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8054070 con 0x5569df832370 2024-09-16T02:05:47.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.499+0000 7ff6daffd640 1 -- 172.21.15.105:0/3791489671 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6e8036040 con 0x5569df832370 2024-09-16T02:05:47.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.499+0000 7ff6daffd640 1 -- 172.21.15.105:0/3791489671 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8049040 con 0x5569df832370 2024-09-16T02:05:47.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.500+0000 7ff6f9685740 1 -- 172.21.15.105:0/3791489671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5569df832370 msgr2=0x5569df82e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:47.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.500+0000 7ff6f9685740 1 --2- 172.21.15.105:0/3791489671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5569df832370 0x5569df82e880 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8033040 tx=0x7ff6e8000f40 comp rx=0 tx=0).stop 2024-09-16T02:05:47.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.500+0000 7ff6f9685740 1 -- 172.21.15.105:0/3791489671 shutdown_connections 2024-09-16T02:05:47.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.500+0000 7ff6f9685740 1 --2- 172.21.15.105:0/3791489671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5569df828b40 0x5569df83e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:47.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.500+0000 7ff6f9685740 1 --2- 172.21.15.105:0/3791489671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5569df832370 0x5569df82e880 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:47.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.500+0000 7ff6f9685740 1 --2- 172.21.15.105:0/3791489671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569df8275d0 0x5569df823e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:47.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.500+0000 7ff6f9685740 1 -- 172.21.15.105:0/3791489671 >> 172.21.15.105:0/3791489671 conn(0x5569df7f0310 msgr2=0x5569df823700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:47.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.500+0000 7ff6f9685740 1 -- 172.21.15.105:0/3791489671 shutdown_connections 2024-09-16T02:05:47.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.500+0000 7ff6f9685740 1 -- 172.21.15.105:0/3791489671 wait complete. 2024-09-16T02:05:47.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.501+0000 7ff6f9685740 1 -- start start 2024-09-16T02:05:47.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.501+0000 7ff6f9685740 1 Processor -- start 2024-09-16T02:05:47.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.501+0000 7ff6f9685740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5569df8275d0 0x5569df83a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:47.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.501+0000 7ff6f9685740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5569df828b40 0x5569df82b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:47.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.501+0000 7ff6f9685740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569df832370 0x5569df82bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:47.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.501+0000 7ff6f9685740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f01b0bc0 con 0x5569df828b40 2024-09-16T02:05:47.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.501+0000 7ff6f9685740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f01b04f0 con 0x5569df832370 2024-09-16T02:05:47.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.501+0000 7ff6f9685740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f01ad730 con 0x5569df8275d0 2024-09-16T02:05:47.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.501+0000 7ff6f9685740 10 client.0 _ll_register_callbacks cb 0x7ff6e00046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:47.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.501+0000 7ff6f4847640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5569df828b40 0x5569df82b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:47.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.501+0000 7ff6f4847640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5569df828b40 0x5569df82b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49812/0 (socket says 172.21.15.105:49812) 2024-09-16T02:05:47.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.501+0000 7ff6f4847640 1 -- 172.21.15.105:0/3028495157 learned_addr learned my addr 172.21.15.105:0/3028495157 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:47.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.502+0000 7ff6f5048640 1 --2- 172.21.15.105:0/3028495157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5569df8275d0 0x5569df83a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:47.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.502+0000 7ff6f5849640 1 --2- 172.21.15.105:0/3028495157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569df832370 0x5569df82bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:47.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.502+0000 7ff6f4847640 1 -- 172.21.15.105:0/3028495157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5569df8275d0 msgr2=0x5569df83a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:47.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.502+0000 7ff6f4847640 1 --2- 172.21.15.105:0/3028495157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5569df8275d0 0x5569df83a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:47.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.502+0000 7ff6f4847640 1 -- 172.21.15.105:0/3028495157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569df832370 msgr2=0x5569df82bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:47.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.502+0000 7ff6f4847640 1 --2- 172.21.15.105:0/3028495157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569df832370 0x5569df82bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:47.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.502+0000 7ff6f4847640 1 -- 172.21.15.105:0/3028495157 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569df82c420 con 0x5569df828b40 2024-09-16T02:05:47.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.502+0000 7ff6f4847640 1 --2- 172.21.15.105:0/3028495157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5569df828b40 0x5569df82b9a0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff6e009c410 tx=0x7ff6e009c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa14a521ba8c4da server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:47.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.502+0000 7ff6d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:05:47.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.502+0000 7ff6d8ff9640 1 -- 172.21.15.105:0/3028495157 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e009bb10 con 0x5569df828b40 2024-09-16T02:05:47.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.502+0000 7ff6d8ff9640 1 -- 172.21.15.105:0/3028495157 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7ff6e00a4990 con 0x5569df828b40 2024-09-16T02:05:47.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.502+0000 7ff6f9685740 1 -- 172.21.15.105:0/3028495157 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6e8036040 con 0x5569df828b40 2024-09-16T02:05:47.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.502+0000 7ff6d8ff9640 1 -- 172.21.15.105:0/3028495157 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff6e0005680 con 0x5569df828b40 2024-09-16T02:05:47.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.502+0000 7ff6f9685740 10 client.15981 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:47.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.502+0000 7ff6f9685740 1 -- 172.21.15.105:0/3028495157 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5569df8ac9e0 con 0x5569df828b40 2024-09-16T02:05:47.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.503+0000 7ff6d8ff9640 1 -- 172.21.15.105:0/3028495157 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6e8036040 con 0x5569df828b40 2024-09-16T02:05:47.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.503+0000 7ff6d8ff9640 1 -- 172.21.15.105:0/3028495157 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff6e0082020 con 0x5569df828b40 2024-09-16T02:05:47.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.504+0000 7ff6f9685740 10 client.15981 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:47.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.504+0000 7ff6f9685740 10 client.15981 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T02:05:47.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.504+0000 7ff6f9685740 20 client.15981 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:47.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.504+0000 7ff6f9685740 1 --2- 172.21.15.105:0/3028495157 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x5569df8ad0f0 0x5569df8af510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:47.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.504+0000 7ff6f9685740 4 client.15981 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T02:05:47.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.504+0000 7ff6f9685740 1 -- 172.21.15.105:0/3028495157 --> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5569df794d70 con 0x5569df8ad0f0 2024-09-16T02:05:47.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.504+0000 7ff6f5048640 1 --2- 172.21.15.105:0/3028495157 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x5569df8ad0f0 0x5569df8af510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:47.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.504+0000 7ff6f5048640 1 --2- 172.21.15.105:0/3028495157 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x5569df8ad0f0 0x5569df8af510 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b3894e01289dd4a3 server_cookie=f327c7fc24d98e65 in_seq=0 out_seq=0 2024-09-16T02:05:47.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.504+0000 7ff6d8ff9640 10 client.15981 ms_handle_connect on v2:172.21.15.137:6836/2788541030 2024-09-16T02:05:47.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.505+0000 7ff6d8ff9640 1 -- 172.21.15.105:0/3028495157 <== mds.2 v2:172.21.15.137:6836/2788541030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5569df794d70 con 0x5569df8ad0f0 2024-09-16T02:05:47.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.505+0000 7ff6d8ff9640 10 client.15981 handle_command_reply: tid=0 2024-09-16T02:05:47.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.505+0000 7ff6f9685740 1 client.15981 shutdown 2024-09-16T02:05:47.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.505+0000 7ff6f9685740 1 -- 172.21.15.105:0/3028495157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5569df828b40 msgr2=0x5569df82b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:47.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.505+0000 7ff6f9685740 1 --2- 172.21.15.105:0/3028495157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5569df828b40 0x5569df82b9a0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff6e009c410 tx=0x7ff6e009c7c0 comp rx=0 tx=0).stop 2024-09-16T02:05:47.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.505+0000 7ff6f9685740 1 -- 172.21.15.105:0/3028495157 shutdown_connections 2024-09-16T02:05:47.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.505+0000 7ff6f9685740 1 --2- 172.21.15.105:0/3028495157 >> [v2:172.21.15.137:6836/2788541030,v1:172.21.15.137:6837/2788541030] conn(0x5569df8ad0f0 0x5569df8af510 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:47.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.505+0000 7ff6f9685740 1 --2- 172.21.15.105:0/3028495157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569df832370 0x5569df82bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:47.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.505+0000 7ff6d8ff9640 0 client.15981 ms_handle_reset on v2:172.21.15.137:6836/2788541030 2024-09-16T02:05:47.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.505+0000 7ff6f9685740 1 --2- 172.21.15.105:0/3028495157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5569df828b40 0x5569df82b9a0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:47.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.505+0000 7ff6f9685740 1 --2- 172.21.15.105:0/3028495157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5569df8275d0 0x5569df83a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:47.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.505+0000 7ff6f9685740 1 -- 172.21.15.105:0/3028495157 >> 172.21.15.105:0/3028495157 conn(0x5569df7f0310 msgr2=0x5569df830820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:47.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.505+0000 7ff6f9685740 1 -- 172.21.15.105:0/3028495157 shutdown_connections 2024-09-16T02:05:47.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.505+0000 7ff6f9685740 1 -- 172.21.15.105:0/3028495157 wait complete. 2024-09-16T02:05:47.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.505+0000 7ff6f9685740 20 client.15981 trim_cache size 0 max 16384 2024-09-16T02:05:47.514 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-09-16T02:05:47.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.506+0000 7ff6f72ac640 1 -- 172.21.15.105:0/2243836761 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff6dc0a32c0 msgr2=0x7ff6dc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:47.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.506+0000 7ff6f72ac640 1 --2- 172.21.15.105:0/2243836761 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff6dc0a32c0 0x7ff6dc0a5780 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x5569df824ac0 tx=0x7ff6ec00b410 comp rx=0 tx=0).stop 2024-09-16T02:05:47.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.506+0000 7ff6f72ac640 1 -- 172.21.15.105:0/2243836761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6f01036e0 msgr2=0x7ff6f0100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:47.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.506+0000 7ff6f72ac640 1 --2- 172.21.15.105:0/2243836761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6f01036e0 0x7ff6f0100ed0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff6e000e9c0 tx=0x7ff6e000ee80 comp rx=0 tx=0).stop 2024-09-16T02:05:47.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.506+0000 7ff6f72ac640 1 -- 172.21.15.105:0/2243836761 shutdown_connections 2024-09-16T02:05:47.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.506+0000 7ff6f72ac640 1 --2- 172.21.15.105:0/2243836761 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7ff6dc0a32c0 0x7ff6dc0a5780 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:47.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.506+0000 7ff6f72ac640 1 --2- 172.21.15.105:0/2243836761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6f01082a0 0x7ff6f0101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:47.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.506+0000 7ff6f72ac640 1 --2- 172.21.15.105:0/2243836761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6f01036e0 0x7ff6f0100ed0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:47.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.506+0000 7ff6f72ac640 1 --2- 172.21.15.105:0/2243836761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6f0102d30 0x7ff6f0100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:47.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.506+0000 7ff6f72ac640 1 -- 172.21.15.105:0/2243836761 >> 172.21.15.105:0/2243836761 conn(0x7ff6f00fc7d0 msgr2=0x7ff6f0105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:47.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.506+0000 7ff6f72ac640 1 -- 172.21.15.105:0/2243836761 shutdown_connections 2024-09-16T02:05:47.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:47.506+0000 7ff6f72ac640 1 -- 172.21.15.105:0/2243836761 wait complete. 2024-09-16T02:05:47.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:47 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:05:47.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:47 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:05:47.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:47 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:05:48.350 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T02:05:48.596 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:48.716 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:48 smithi105 ceph-mon[31197]: pgmap v437: 97 pgs: 97 active+clean; 960 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.3 KiB/s wr, 224 op/s 2024-09-16T02:05:48.717 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:48 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:05:48.717 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:48 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:05:48.717 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:48 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:05:48.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:48 smithi190 ceph-mon[34762]: pgmap v437: 97 pgs: 97 active+clean; 960 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.3 KiB/s wr, 224 op/s 2024-09-16T02:05:48.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:48 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:05:48.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:48 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:05:48.843 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:48 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:05:48.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:48 smithi137 ceph-mon[34099]: pgmap v437: 97 pgs: 97 active+clean; 960 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.3 KiB/s wr, 224 op/s 2024-09-16T02:05:48.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:48 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:05:48.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:48 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:05:48.922 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:48 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' 2024-09-16T02:05:49.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.688+0000 7feccd774640 1 -- 172.21.15.105:0/3069528118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecc8106f10 msgr2=0x7fecc810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:49.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.688+0000 7feccd774640 1 --2- 172.21.15.105:0/3069528118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecc8106f10 0x7fecc810ea90 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fecc0009ca0 tx=0x7fecc002a8c0 comp rx=0 tx=0).stop 2024-09-16T02:05:49.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.688+0000 7feccd774640 1 -- 172.21.15.105:0/3069528118 shutdown_connections 2024-09-16T02:05:49.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.688+0000 7feccd774640 1 --2- 172.21.15.105:0/3069528118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fecc8102790 0x7fecc81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:49.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.688+0000 7feccd774640 1 --2- 172.21.15.105:0/3069528118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecc8106f10 0x7fecc810ea90 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:49.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.688+0000 7feccd774640 1 --2- 172.21.15.105:0/3069528118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecc8101de0 0x7fecc81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:49.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.688+0000 7feccd774640 1 -- 172.21.15.105:0/3069528118 >> 172.21.15.105:0/3069528118 conn(0x7fecc80fd8c0 msgr2=0x7fecc80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:49.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.688+0000 7feccd774640 1 -- 172.21.15.105:0/3069528118 shutdown_connections 2024-09-16T02:05:49.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.688+0000 7feccd774640 1 -- 172.21.15.105:0/3069528118 wait complete. 2024-09-16T02:05:49.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.689+0000 7feccd774640 1 Processor -- start 2024-09-16T02:05:49.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.689+0000 7feccd774640 1 -- start start 2024-09-16T02:05:49.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.689+0000 7feccd774640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecc8101de0 0x7fecc8198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:49.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.689+0000 7feccd774640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecc8102790 0x7fecc8198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:49.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.689+0000 7feccd774640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fecc8106f10 0x7fecc819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:49.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.689+0000 7feccd774640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fecc8111e20 con 0x7fecc8101de0 2024-09-16T02:05:49.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.689+0000 7feccd774640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fecc8111ca0 con 0x7fecc8102790 2024-09-16T02:05:49.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.689+0000 7feccd774640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fecc8111fa0 con 0x7fecc8106f10 2024-09-16T02:05:49.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.689+0000 7fecc6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecc8101de0 0x7fecc8198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:49.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.689+0000 7fecc6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecc8101de0 0x7fecc8198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42968/0 (socket says 172.21.15.105:42968) 2024-09-16T02:05:49.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.689+0000 7fecc6ffd640 1 -- 172.21.15.105:0/2264461471 learned_addr learned my addr 172.21.15.105:0/2264461471 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:49.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.690+0000 7fecbffff640 1 --2- 172.21.15.105:0/2264461471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecc8102790 0x7fecc8198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:49.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.690+0000 7fecc6ffd640 1 -- 172.21.15.105:0/2264461471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fecc8106f10 msgr2=0x7fecc819cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:49.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.690+0000 7fecc77fe640 1 --2- 172.21.15.105:0/2264461471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fecc8106f10 0x7fecc819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:49.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.690+0000 7fecc6ffd640 1 --2- 172.21.15.105:0/2264461471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fecc8106f10 0x7fecc819cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:49.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.690+0000 7fecc6ffd640 1 -- 172.21.15.105:0/2264461471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecc8102790 msgr2=0x7fecc8198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:49.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.690+0000 7fecc6ffd640 1 --2- 172.21.15.105:0/2264461471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecc8102790 0x7fecc8198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:49.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.690+0000 7fecc6ffd640 1 -- 172.21.15.105:0/2264461471 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc819d2d0 con 0x7fecc8101de0 2024-09-16T02:05:49.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.690+0000 7fecc6ffd640 1 --2- 172.21.15.105:0/2264461471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecc8101de0 0x7fecc8198320 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fecb800d950 tx=0x7fecb800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:49.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.690+0000 7fecc4ff9640 1 -- 172.21.15.105:0/2264461471 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8017070 con 0x7fecc8101de0 2024-09-16T02:05:49.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.690+0000 7fecc4ff9640 1 -- 172.21.15.105:0/2264461471 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecb8009c30 con 0x7fecc8101de0 2024-09-16T02:05:49.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.690+0000 7feccd774640 1 -- 172.21.15.105:0/2264461471 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecc819d5c0 con 0x7fecc8101de0 2024-09-16T02:05:49.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.690+0000 7fecc4ff9640 1 -- 172.21.15.105:0/2264461471 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb800f620 con 0x7fecc8101de0 2024-09-16T02:05:49.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.690+0000 7feccd774640 1 -- 172.21.15.105:0/2264461471 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecc81a4ea0 con 0x7fecc8101de0 2024-09-16T02:05:49.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.692+0000 7fecc4ff9640 1 -- 172.21.15.105:0/2264461471 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fecb80109a0 con 0x7fecc8101de0 2024-09-16T02:05:49.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.693+0000 7feccfb4d740 1 Processor -- start 2024-09-16T02:05:49.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.693+0000 7feccfb4d740 1 -- start start 2024-09-16T02:05:49.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.693+0000 7feccfb4d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc6d653b40 0x55cc6d652090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:49.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.693+0000 7feccfb4d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc6d6525d0 0x55cc6d652a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:49.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.693+0000 7feccfb4d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc6d661780 0x55cc6d663b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:49.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.693+0000 7feccfb4d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55cc6d5b82f0 con 0x55cc6d653b40 2024-09-16T02:05:49.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.693+0000 7feccfb4d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55cc6d55fcf0 con 0x55cc6d661780 2024-09-16T02:05:49.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.693+0000 7feccfb4d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55cc6d5b42b0 con 0x55cc6d6525d0 2024-09-16T02:05:49.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.693+0000 7fecc4ff9640 1 --2- 172.21.15.105:0/2264461471 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fecb40a32c0 0x7fecb40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:49.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.693+0000 7fecc77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc6d653b40 0x55cc6d652090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:49.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.693+0000 7fecc77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc6d653b40 0x55cc6d652090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42982/0 (socket says 172.21.15.105:42982) 2024-09-16T02:05:49.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.693+0000 7fecc77fe640 1 -- 172.21.15.105:0/1497197827 learned_addr learned my addr 172.21.15.105:0/1497197827 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:49.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.693+0000 7fecc77fe640 1 --2- 172.21.15.105:0/2264461471 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fecb40a32c0 0x7fecb40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:49.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.694+0000 7fecc4ff9640 1 -- 172.21.15.105:0/2264461471 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fecb80cf540 con 0x7fecc8101de0 2024-09-16T02:05:49.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.694+0000 7fecbffff640 1 --2- 172.21.15.105:0/1497197827 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc6d661780 0x55cc6d663b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:49.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.694+0000 7fecc6ffd640 1 --2- 172.21.15.105:0/1497197827 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc6d6525d0 0x55cc6d652a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:49.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.694+0000 7fecc77fe640 1 -- 172.21.15.105:0/1497197827 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc6d6525d0 msgr2=0x55cc6d652a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:49.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.694+0000 7fecc77fe640 1 --2- 172.21.15.105:0/1497197827 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc6d6525d0 0x55cc6d652a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:49.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.694+0000 7fecc77fe640 1 -- 172.21.15.105:0/1497197827 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc6d661780 msgr2=0x55cc6d663b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:49.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.694+0000 7fecc77fe640 1 --2- 172.21.15.105:0/1497197827 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc6d661780 0x55cc6d663b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:49.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.694+0000 7fecc77fe640 1 -- 172.21.15.105:0/1497197827 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc6d5443a0 con 0x55cc6d653b40 2024-09-16T02:05:49.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.694+0000 7fecc77fe640 1 --2- 172.21.15.105:0/1497197827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc6d653b40 0x55cc6d652090 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fecc0006010 tx=0x7fecc0031880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c11307fe8cbf0a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:49.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.694+0000 7fecbd7fa640 1 -- 172.21.15.105:0/1497197827 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc0003970 con 0x55cc6d653b40 2024-09-16T02:05:49.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.694+0000 7fecbd7fa640 1 -- 172.21.15.105:0/1497197827 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecc0036050 con 0x55cc6d653b40 2024-09-16T02:05:49.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.694+0000 7fecc77fe640 1 --2- 172.21.15.105:0/2264461471 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fecb40a32c0 0x7fecb40a5780 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fecc004c000 tx=0x7fecc004d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:49.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.694+0000 7fecbd7fa640 1 -- 172.21.15.105:0/1497197827 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc00305c0 con 0x55cc6d653b40 2024-09-16T02:05:49.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.695+0000 7feccfb4d740 1 -- 172.21.15.105:0/1497197827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc6d653b40 msgr2=0x55cc6d652090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:49.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.695+0000 7feccfb4d740 1 --2- 172.21.15.105:0/1497197827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc6d653b40 0x55cc6d652090 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fecc0006010 tx=0x7fecc0031880 comp rx=0 tx=0).stop 2024-09-16T02:05:49.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.695+0000 7feccfb4d740 1 -- 172.21.15.105:0/1497197827 shutdown_connections 2024-09-16T02:05:49.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.695+0000 7feccfb4d740 1 --2- 172.21.15.105:0/1497197827 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc6d6525d0 0x55cc6d652a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:49.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.695+0000 7feccfb4d740 1 --2- 172.21.15.105:0/1497197827 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc6d661780 0x55cc6d663b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:49.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.695+0000 7feccfb4d740 1 --2- 172.21.15.105:0/1497197827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc6d653b40 0x55cc6d652090 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:49.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.695+0000 7feccfb4d740 1 -- 172.21.15.105:0/1497197827 >> 172.21.15.105:0/1497197827 conn(0x55cc6d61b310 msgr2=0x55cc6d64ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:49.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.695+0000 7feccfb4d740 1 -- 172.21.15.105:0/1497197827 shutdown_connections 2024-09-16T02:05:49.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.695+0000 7feccfb4d740 1 -- 172.21.15.105:0/1497197827 wait complete. 2024-09-16T02:05:49.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.695+0000 7feccfb4d740 1 -- start start 2024-09-16T02:05:49.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.696+0000 7feccfb4d740 1 Processor -- start 2024-09-16T02:05:49.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.696+0000 7feccfb4d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc6d6525d0 0x55cc6d661180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:49.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.696+0000 7feccfb4d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc6d653b40 0x55cc6d65e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:49.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.696+0000 7feccfb4d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc6d661780 0x55cc6d65ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:49.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.696+0000 7feccfb4d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55cc6d664300 con 0x55cc6d6525d0 2024-09-16T02:05:49.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.696+0000 7feccfb4d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55cc6d573060 con 0x55cc6d661780 2024-09-16T02:05:49.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.696+0000 7feccfb4d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55cc6d664480 con 0x55cc6d653b40 2024-09-16T02:05:49.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.696+0000 7feccfb4d740 10 client.0 _ll_register_callbacks cb 0x55cc6d2c68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:49.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.696+0000 7fecc6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc6d6525d0 0x55cc6d661180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:49.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.696+0000 7fecc6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc6d6525d0 0x55cc6d661180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42986/0 (socket says 172.21.15.105:42986) 2024-09-16T02:05:49.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.696+0000 7fecc6ffd640 1 -- 172.21.15.105:0/1941293870 learned_addr learned my addr 172.21.15.105:0/1941293870 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:49.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.696+0000 7fecbffff640 1 --2- 172.21.15.105:0/1941293870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc6d653b40 0x55cc6d65e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:49.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.696+0000 7fecc6ffd640 1 -- 172.21.15.105:0/1941293870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc6d653b40 msgr2=0x55cc6d65e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:49.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.696+0000 7fecc6ffd640 1 --2- 172.21.15.105:0/1941293870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc6d653b40 0x55cc6d65e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:49.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.696+0000 7fecc6ffd640 1 -- 172.21.15.105:0/1941293870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc6d661780 msgr2=0x55cc6d65ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:49.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.696+0000 7fecc6ffd640 1 --2- 172.21.15.105:0/1941293870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc6d661780 0x55cc6d65ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:49.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.696+0000 7fecc6ffd640 1 -- 172.21.15.105:0/1941293870 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc6d3293c0 con 0x55cc6d6525d0 2024-09-16T02:05:49.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.696+0000 7fecc6ffd640 1 --2- 172.21.15.105:0/1941293870 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc6d6525d0 0x55cc6d661180 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fecb80cd350 tx=0x7fecb8009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52bcf8f11e04a421 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:49.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.696+0000 7fecae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:05:49.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.697+0000 7fecae7fc640 1 -- 172.21.15.105:0/1941293870 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb80b4060 con 0x55cc6d6525d0 2024-09-16T02:05:49.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.697+0000 7fecae7fc640 1 -- 172.21.15.105:0/1941293870 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fecb80860c0 con 0x55cc6d6525d0 2024-09-16T02:05:49.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.697+0000 7feccfb4d740 1 -- 172.21.15.105:0/1941293870 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecc0036050 con 0x55cc6d6525d0 2024-09-16T02:05:49.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.697+0000 7fecae7fc640 1 -- 172.21.15.105:0/1941293870 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fecb8020040 con 0x55cc6d6525d0 2024-09-16T02:05:49.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.697+0000 7feccfb4d740 10 client.15993 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:49.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.697+0000 7feccfb4d740 1 -- 172.21.15.105:0/1941293870 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cc6d66fda0 con 0x55cc6d6525d0 2024-09-16T02:05:49.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.697+0000 7fecae7fc640 1 -- 172.21.15.105:0/1941293870 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecc0036050 con 0x55cc6d6525d0 2024-09-16T02:05:49.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.697+0000 7fecae7fc640 1 -- 172.21.15.105:0/1941293870 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fecb80cf200 con 0x55cc6d6525d0 2024-09-16T02:05:49.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.697+0000 7feccfb4d740 10 client.15993 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:49.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.697+0000 7feccfb4d740 10 client.15993 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T02:05:49.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.697+0000 7feccfb4d740 20 client.15993 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:49.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.697+0000 7feccfb4d740 1 --2- 172.21.15.105:0/1941293870 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55cc6d6711b0 0x55cc6d673640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:49.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.697+0000 7feccfb4d740 4 client.15993 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:05:49.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.697+0000 7feccfb4d740 1 -- 172.21.15.105:0/1941293870 --> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc6d5c5860 con 0x55cc6d6711b0 2024-09-16T02:05:49.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.698+0000 7fecbffff640 1 --2- 172.21.15.105:0/1941293870 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55cc6d6711b0 0x55cc6d673640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:49.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.698+0000 7fecae7fc640 10 client.15993 ms_handle_connect on v2:172.21.15.190:6838/1222573537 2024-09-16T02:05:49.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.698+0000 7fecbffff640 1 --2- 172.21.15.105:0/1941293870 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55cc6d6711b0 0x55cc6d673640 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=69fc00d59747332c server_cookie=4afd1f7eb44a2995 in_seq=0 out_seq=0 2024-09-16T02:05:49.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.699+0000 7fecae7fc640 1 -- 172.21.15.105:0/1941293870 <== mds.1 v2:172.21.15.190:6838/1222573537 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cc6d5c5860 con 0x55cc6d6711b0 2024-09-16T02:05:49.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.699+0000 7fecae7fc640 10 client.15993 handle_command_reply: tid=0 2024-09-16T02:05:49.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.699+0000 7feccfb4d740 1 client.15993 shutdown 2024-09-16T02:05:49.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.699+0000 7feccfb4d740 1 -- 172.21.15.105:0/1941293870 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc6d6525d0 msgr2=0x55cc6d661180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:49.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.699+0000 7feccfb4d740 1 --2- 172.21.15.105:0/1941293870 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc6d6525d0 0x55cc6d661180 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fecb80cd350 tx=0x7fecb8009de0 comp rx=0 tx=0).stop 2024-09-16T02:05:49.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.699+0000 7feccfb4d740 1 -- 172.21.15.105:0/1941293870 shutdown_connections 2024-09-16T02:05:49.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.699+0000 7feccfb4d740 1 --2- 172.21.15.105:0/1941293870 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55cc6d6711b0 0x55cc6d673640 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:49.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.699+0000 7feccfb4d740 1 --2- 172.21.15.105:0/1941293870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc6d653b40 0x55cc6d65e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:49.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.699+0000 7fecae7fc640 0 client.15993 ms_handle_reset on v2:172.21.15.190:6838/1222573537 2024-09-16T02:05:49.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.699+0000 7feccfb4d740 1 --2- 172.21.15.105:0/1941293870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc6d661780 0x55cc6d65ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:49.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.699+0000 7feccfb4d740 1 --2- 172.21.15.105:0/1941293870 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc6d6525d0 0x55cc6d661180 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:49.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.699+0000 7feccfb4d740 1 -- 172.21.15.105:0/1941293870 >> 172.21.15.105:0/1941293870 conn(0x55cc6d61b310 msgr2=0x55cc6d654960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:49.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.699+0000 7feccfb4d740 1 -- 172.21.15.105:0/1941293870 shutdown_connections 2024-09-16T02:05:49.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.699+0000 7feccfb4d740 1 -- 172.21.15.105:0/1941293870 wait complete. 2024-09-16T02:05:49.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.699+0000 7feccfb4d740 20 client.15993 trim_cache size 0 max 16384 2024-09-16T02:05:49.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.712+0000 7feccfb4d740 1 Processor -- start 2024-09-16T02:05:49.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.712+0000 7feccfb4d740 1 -- start start 2024-09-16T02:05:49.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.713+0000 7feccfb4d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc6d6525d0 0x55cc6d653300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:49.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.713+0000 7feccfb4d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc6d653b40 0x55cc6d66d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:49.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.713+0000 7feccfb4d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc6d661780 0x55cc6d65cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:49.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.713+0000 7feccfb4d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fecc81ab480 con 0x55cc6d6525d0 2024-09-16T02:05:49.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.713+0000 7feccfb4d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fecc81acfc0 con 0x55cc6d661780 2024-09-16T02:05:49.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.713+0000 7feccfb4d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fecc81abb50 con 0x55cc6d653b40 2024-09-16T02:05:49.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.713+0000 7fecc6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc6d6525d0 0x55cc6d653300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:49.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.713+0000 7fecc6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc6d6525d0 0x55cc6d653300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42994/0 (socket says 172.21.15.105:42994) 2024-09-16T02:05:49.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.713+0000 7fecc6ffd640 1 -- 172.21.15.105:0/3827917769 learned_addr learned my addr 172.21.15.105:0/3827917769 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:49.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.713+0000 7fecc6ffd640 1 -- 172.21.15.105:0/3827917769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc6d653b40 msgr2=0x55cc6d66d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:49.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.713+0000 7fecc6ffd640 1 --2- 172.21.15.105:0/3827917769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc6d653b40 0x55cc6d66d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:49.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.713+0000 7fecc6ffd640 1 -- 172.21.15.105:0/3827917769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc6d661780 msgr2=0x55cc6d65cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:49.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.713+0000 7fecc77fe640 1 --2- 172.21.15.105:0/3827917769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc6d661780 0x55cc6d65cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:49.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.713+0000 7fecc6ffd640 1 --2- 172.21.15.105:0/3827917769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc6d661780 0x55cc6d65cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:49.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.713+0000 7fecc6ffd640 1 -- 172.21.15.105:0/3827917769 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc6d659f40 con 0x55cc6d6525d0 2024-09-16T02:05:49.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.713+0000 7fecc77fe640 1 --2- 172.21.15.105:0/3827917769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc6d661780 0x55cc6d65cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:05:49.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.713+0000 7fecc6ffd640 1 --2- 172.21.15.105:0/3827917769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc6d6525d0 0x55cc6d653300 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fecb8000c00 tx=0x7fecb809b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14bc616f64508fa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:49.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.713+0000 7fecbcff9640 1 -- 172.21.15.105:0/3827917769 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb80b4050 con 0x55cc6d6525d0 2024-09-16T02:05:49.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.713+0000 7fecbcff9640 1 -- 172.21.15.105:0/3827917769 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecb8084e50 con 0x55cc6d6525d0 2024-09-16T02:05:49.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.714+0000 7fecbcff9640 1 -- 172.21.15.105:0/3827917769 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8086310 con 0x55cc6d6525d0 2024-09-16T02:05:49.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.714+0000 7feccfb4d740 1 -- 172.21.15.105:0/3827917769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc6d6525d0 msgr2=0x55cc6d653300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:49.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.714+0000 7feccfb4d740 1 --2- 172.21.15.105:0/3827917769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc6d6525d0 0x55cc6d653300 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fecb8000c00 tx=0x7fecb809b640 comp rx=0 tx=0).stop 2024-09-16T02:05:49.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.714+0000 7feccfb4d740 1 -- 172.21.15.105:0/3827917769 shutdown_connections 2024-09-16T02:05:49.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.714+0000 7feccfb4d740 1 --2- 172.21.15.105:0/3827917769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc6d653b40 0x55cc6d66d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:49.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.714+0000 7feccfb4d740 1 --2- 172.21.15.105:0/3827917769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc6d661780 0x55cc6d65cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:49.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.714+0000 7feccfb4d740 1 --2- 172.21.15.105:0/3827917769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc6d6525d0 0x55cc6d653300 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:49.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.714+0000 7feccfb4d740 1 -- 172.21.15.105:0/3827917769 >> 172.21.15.105:0/3827917769 conn(0x55cc6d61b310 msgr2=0x55cc6d649a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:49.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.714+0000 7feccfb4d740 1 -- 172.21.15.105:0/3827917769 shutdown_connections 2024-09-16T02:05:49.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.714+0000 7feccfb4d740 1 -- 172.21.15.105:0/3827917769 wait complete. 2024-09-16T02:05:49.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.714+0000 7feccfb4d740 1 -- start start 2024-09-16T02:05:49.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.715+0000 7feccfb4d740 1 Processor -- start 2024-09-16T02:05:49.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.715+0000 7feccfb4d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc6d6525d0 0x55cc6d6697c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:49.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.715+0000 7feccfb4d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc6d653b40 0x55cc6d669d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:49.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.715+0000 7feccfb4d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc6d661780 0x55cc6d664c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:49.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.715+0000 7feccfb4d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fecc81ac8f0 con 0x55cc6d653b40 2024-09-16T02:05:49.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.715+0000 7feccfb4d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fecc81ac220 con 0x55cc6d6525d0 2024-09-16T02:05:49.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.715+0000 7feccfb4d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fecc81a9460 con 0x55cc6d661780 2024-09-16T02:05:49.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.715+0000 7feccfb4d740 10 client.0 _ll_register_callbacks cb 0x7fecb809b510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:49.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.715+0000 7fecbffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc6d653b40 0x55cc6d669d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:49.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.715+0000 7fecbffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc6d653b40 0x55cc6d669d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42996/0 (socket says 172.21.15.105:42996) 2024-09-16T02:05:49.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.715+0000 7fecc6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc6d6525d0 0x55cc6d6697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:49.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.715+0000 7fecbffff640 1 -- 172.21.15.105:0/1963920641 learned_addr learned my addr 172.21.15.105:0/1963920641 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:49.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.715+0000 7fecc77fe640 1 --2- 172.21.15.105:0/1963920641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc6d661780 0x55cc6d664c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:49.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.715+0000 7fecbffff640 1 -- 172.21.15.105:0/1963920641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc6d661780 msgr2=0x55cc6d664c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:49.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.715+0000 7fecbffff640 1 --2- 172.21.15.105:0/1963920641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc6d661780 0x55cc6d664c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:49.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.715+0000 7fecbffff640 1 -- 172.21.15.105:0/1963920641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc6d6525d0 msgr2=0x55cc6d6697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:49.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.715+0000 7fecbffff640 1 --2- 172.21.15.105:0/1963920641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc6d6525d0 0x55cc6d6697c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:49.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.715+0000 7fecbffff640 1 -- 172.21.15.105:0/1963920641 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc6d665490 con 0x55cc6d653b40 2024-09-16T02:05:49.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.715+0000 7fecc6ffd640 1 --2- 172.21.15.105:0/1963920641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc6d6525d0 0x55cc6d6697c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:05:49.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.716+0000 7fecbffff640 1 --2- 172.21.15.105:0/1963920641 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc6d653b40 0x55cc6d669d00 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fecb000b820 tx=0x7fecb000bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d3b63fe5757894c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:49.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.716+0000 7fecae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:05:49.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.716+0000 7fecae7fc640 1 -- 172.21.15.105:0/1963920641 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb0005220 con 0x55cc6d653b40 2024-09-16T02:05:49.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.716+0000 7fecae7fc640 1 -- 172.21.15.105:0/1963920641 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fecb00053c0 con 0x55cc6d653b40 2024-09-16T02:05:49.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.716+0000 7feccfb4d740 1 -- 172.21.15.105:0/1963920641 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecb8084e50 con 0x55cc6d653b40 2024-09-16T02:05:49.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.716+0000 7fecae7fc640 1 -- 172.21.15.105:0/1963920641 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fecb00055c0 con 0x55cc6d653b40 2024-09-16T02:05:49.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.716+0000 7feccfb4d740 10 client.15999 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:49.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.716+0000 7feccfb4d740 1 -- 172.21.15.105:0/1963920641 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cc6d6db950 con 0x55cc6d653b40 2024-09-16T02:05:49.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.716+0000 7fecae7fc640 1 -- 172.21.15.105:0/1963920641 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecb8084e50 con 0x55cc6d653b40 2024-09-16T02:05:49.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.716+0000 7fecae7fc640 1 -- 172.21.15.105:0/1963920641 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fecb002f050 con 0x55cc6d653b40 2024-09-16T02:05:49.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.716+0000 7feccfb4d740 10 client.15999 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:49.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.716+0000 7feccfb4d740 10 client.15999 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T02:05:49.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.716+0000 7feccfb4d740 20 client.15999 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:49.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.716+0000 7feccfb4d740 1 --2- 172.21.15.105:0/1963920641 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55cc6d6dc0b0 0x55cc6d6de4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:49.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.716+0000 7feccfb4d740 4 client.15999 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T02:05:49.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.716+0000 7feccfb4d740 1 -- 172.21.15.105:0/1963920641 --> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cc6d5bfd70 con 0x55cc6d6dc0b0 2024-09-16T02:05:49.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.717+0000 7fecc6ffd640 1 --2- 172.21.15.105:0/1963920641 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55cc6d6dc0b0 0x55cc6d6de4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:49.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.717+0000 7fecae7fc640 10 client.15999 ms_handle_connect on v2:172.21.15.190:6838/1222573537 2024-09-16T02:05:49.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.717+0000 7fecc6ffd640 1 --2- 172.21.15.105:0/1963920641 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55cc6d6dc0b0 0x55cc6d6de4d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ab4fd110dd10ded4 server_cookie=4f3efcb12384f60c in_seq=0 out_seq=0 2024-09-16T02:05:49.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.717+0000 7fecae7fc640 1 -- 172.21.15.105:0/1963920641 <== mds.1 v2:172.21.15.190:6838/1222573537 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cc6d5bfd70 con 0x55cc6d6dc0b0 2024-09-16T02:05:49.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.717+0000 7fecae7fc640 10 client.15999 handle_command_reply: tid=0 2024-09-16T02:05:49.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.717+0000 7feccfb4d740 1 client.15999 shutdown 2024-09-16T02:05:49.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.717+0000 7feccfb4d740 1 -- 172.21.15.105:0/1963920641 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc6d653b40 msgr2=0x55cc6d669d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:49.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.717+0000 7feccfb4d740 1 --2- 172.21.15.105:0/1963920641 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc6d653b40 0x55cc6d669d00 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fecb000b820 tx=0x7fecb000bf00 comp rx=0 tx=0).stop 2024-09-16T02:05:49.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.717+0000 7feccfb4d740 1 -- 172.21.15.105:0/1963920641 shutdown_connections 2024-09-16T02:05:49.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.717+0000 7feccfb4d740 1 --2- 172.21.15.105:0/1963920641 >> [v2:172.21.15.190:6838/1222573537,v1:172.21.15.190:6839/1222573537] conn(0x55cc6d6dc0b0 0x55cc6d6de4d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:49.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.717+0000 7feccfb4d740 1 --2- 172.21.15.105:0/1963920641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc6d661780 0x55cc6d664c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:49.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.717+0000 7fecae7fc640 0 client.15999 ms_handle_reset on v2:172.21.15.190:6838/1222573537 2024-09-16T02:05:49.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.717+0000 7feccfb4d740 1 --2- 172.21.15.105:0/1963920641 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc6d653b40 0x55cc6d669d00 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:49.745 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-09-16T02:05:49.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.717+0000 7feccfb4d740 1 --2- 172.21.15.105:0/1963920641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc6d6525d0 0x55cc6d6697c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:49.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.717+0000 7feccfb4d740 1 -- 172.21.15.105:0/1963920641 >> 172.21.15.105:0/1963920641 conn(0x55cc6d61b310 msgr2=0x55cc6d65cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:49.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.717+0000 7feccfb4d740 1 -- 172.21.15.105:0/1963920641 shutdown_connections 2024-09-16T02:05:49.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.717+0000 7feccfb4d740 1 -- 172.21.15.105:0/1963920641 wait complete. 2024-09-16T02:05:49.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.717+0000 7feccfb4d740 20 client.15999 trim_cache size 0 max 16384 2024-09-16T02:05:49.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.718+0000 7feccd774640 1 -- 172.21.15.105:0/2264461471 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fecb40a32c0 msgr2=0x7fecb40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:49.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.718+0000 7feccd774640 1 --2- 172.21.15.105:0/2264461471 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fecb40a32c0 0x7fecb40a5780 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fecc004c000 tx=0x7fecc004d000 comp rx=0 tx=0).stop 2024-09-16T02:05:49.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.718+0000 7feccd774640 1 -- 172.21.15.105:0/2264461471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecc8101de0 msgr2=0x7fecc8198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:49.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.718+0000 7feccd774640 1 --2- 172.21.15.105:0/2264461471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecc8101de0 0x7fecc8198320 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fecb800d950 tx=0x7fecb800de10 comp rx=0 tx=0).stop 2024-09-16T02:05:49.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.718+0000 7feccd774640 1 -- 172.21.15.105:0/2264461471 shutdown_connections 2024-09-16T02:05:49.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.718+0000 7feccd774640 1 --2- 172.21.15.105:0/2264461471 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fecb40a32c0 0x7fecb40a5780 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:49.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.718+0000 7feccd774640 1 --2- 172.21.15.105:0/2264461471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fecc8106f10 0x7fecc819cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:49.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.718+0000 7feccd774640 1 --2- 172.21.15.105:0/2264461471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecc8102790 0x7fecc8198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:49.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.718+0000 7feccd774640 1 --2- 172.21.15.105:0/2264461471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecc8101de0 0x7fecc8198320 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:49.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.718+0000 7feccd774640 1 -- 172.21.15.105:0/2264461471 >> 172.21.15.105:0/2264461471 conn(0x7fecc80fd8c0 msgr2=0x7fecc80fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:49.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.718+0000 7feccd774640 1 -- 172.21.15.105:0/2264461471 shutdown_connections 2024-09-16T02:05:49.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:49.718+0000 7feccd774640 1 -- 172.21.15.105:0/2264461471 wait complete. 2024-09-16T02:05:50.546 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T02:05:50.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:50 smithi105 ceph-mon[31197]: pgmap v438: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.0 KiB/s wr, 226 op/s 2024-09-16T02:05:50.795 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:50.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:50 smithi190 ceph-mon[34762]: pgmap v438: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.0 KiB/s wr, 226 op/s 2024-09-16T02:05:50.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:50 smithi137 ceph-mon[34099]: pgmap v438: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.0 KiB/s wr, 226 op/s 2024-09-16T02:05:51.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.288+0000 7f6b3bafe640 1 -- 172.21.15.105:0/986277439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b34074590 msgr2=0x7f6b34074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:51.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.288+0000 7f6b3bafe640 1 --2- 172.21.15.105:0/986277439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b34074590 0x7f6b34074970 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6b30009900 tx=0x7f6b3002a480 comp rx=0 tx=0).stop 2024-09-16T02:05:51.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.288+0000 7f6b3bafe640 1 -- 172.21.15.105:0/986277439 shutdown_connections 2024-09-16T02:05:51.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.288+0000 7f6b3bafe640 1 --2- 172.21.15.105:0/986277439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b34112b90 0x7f6b34114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:51.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.288+0000 7f6b3bafe640 1 --2- 172.21.15.105:0/986277439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b34074eb0 0x7f6b34112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:51.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.288+0000 7f6b3bafe640 1 --2- 172.21.15.105:0/986277439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b34074590 0x7f6b34074970 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:51.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.288+0000 7f6b3bafe640 1 -- 172.21.15.105:0/986277439 >> 172.21.15.105:0/986277439 conn(0x7f6b340fd8c0 msgr2=0x7f6b340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:51.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.289+0000 7f6b3bafe640 1 -- 172.21.15.105:0/986277439 shutdown_connections 2024-09-16T02:05:51.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.289+0000 7f6b3bafe640 1 -- 172.21.15.105:0/986277439 wait complete. 2024-09-16T02:05:51.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.289+0000 7f6b3bafe640 1 Processor -- start 2024-09-16T02:05:51.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.289+0000 7f6b3bafe640 1 -- start start 2024-09-16T02:05:51.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.289+0000 7f6b3bafe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b34074590 0x7f6b341a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:51.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.289+0000 7f6b3bafe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b34074eb0 0x7f6b341a4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:51.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.290+0000 7f6b3bafe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b34112b90 0x7f6b3419e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:51.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.290+0000 7f6b3bafe640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6b34118500 con 0x7f6b34074eb0 2024-09-16T02:05:51.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.290+0000 7f6b3bafe640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6b34118380 con 0x7f6b34112b90 2024-09-16T02:05:51.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.290+0000 7f6b3bafe640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6b34118680 con 0x7f6b34074590 2024-09-16T02:05:51.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.290+0000 7f6b39099640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b34074eb0 0x7f6b341a4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:51.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.290+0000 7f6b39099640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b34074eb0 0x7f6b341a4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43014/0 (socket says 172.21.15.105:43014) 2024-09-16T02:05:51.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.290+0000 7f6b39099640 1 -- 172.21.15.105:0/3774478876 learned_addr learned my addr 172.21.15.105:0/3774478876 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:51.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.290+0000 7f6b39099640 1 -- 172.21.15.105:0/3774478876 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b34074590 msgr2=0x7f6b341a47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:51.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.290+0000 7f6b3989a640 1 --2- 172.21.15.105:0/3774478876 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b34074590 0x7f6b341a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:51.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.290+0000 7f6b39099640 1 --2- 172.21.15.105:0/3774478876 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b34074590 0x7f6b341a47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:51.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.290+0000 7f6b39099640 1 -- 172.21.15.105:0/3774478876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b34112b90 msgr2=0x7f6b3419e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:51.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.290+0000 7f6b39099640 1 --2- 172.21.15.105:0/3774478876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b34112b90 0x7f6b3419e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:51.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.290+0000 7f6b39099640 1 -- 172.21.15.105:0/3774478876 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b3419f0a0 con 0x7f6b34074eb0 2024-09-16T02:05:51.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.290+0000 7f6b39099640 1 --2- 172.21.15.105:0/3774478876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b34074eb0 0x7f6b341a4cf0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6b2400b4a0 tx=0x7f6b2400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:51.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.290+0000 7f6b2affd640 1 -- 172.21.15.105:0/3774478876 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b24014070 con 0x7f6b34074eb0 2024-09-16T02:05:51.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.290+0000 7f6b2affd640 1 -- 172.21.15.105:0/3774478876 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b24010ac0 con 0x7f6b34074eb0 2024-09-16T02:05:51.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.290+0000 7f6b3bafe640 1 -- 172.21.15.105:0/3774478876 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b3419f390 con 0x7f6b34074eb0 2024-09-16T02:05:51.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.290+0000 7f6b2affd640 1 -- 172.21.15.105:0/3774478876 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b24017c30 con 0x7f6b34074eb0 2024-09-16T02:05:51.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.290+0000 7f6b3bafe640 1 -- 172.21.15.105:0/3774478876 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b341ab680 con 0x7f6b34074eb0 2024-09-16T02:05:51.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.293+0000 7f6b3ded7740 1 Processor -- start 2024-09-16T02:05:51.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.293+0000 7f6b3ded7740 1 -- start start 2024-09-16T02:05:51.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.294+0000 7f6b2affd640 1 -- 172.21.15.105:0/3774478876 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f6b24017070 con 0x7f6b34074eb0 2024-09-16T02:05:51.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.294+0000 7f6b3ded7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcc8c60b40 0x55fcc8c5f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:51.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.294+0000 7f6b3ded7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fcc8c5f5d0 0x55fcc8c5fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:51.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.294+0000 7f6b3ded7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc8c6a370 0x55fcc8c6c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:51.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.294+0000 7f6b3ded7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55fcc8bc52f0 con 0x55fcc8c5f5d0 2024-09-16T02:05:51.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.294+0000 7f6b3ded7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fcc8b6ccf0 con 0x55fcc8c6a370 2024-09-16T02:05:51.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.294+0000 7f6b3ded7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55fcc8bc12b0 con 0x55fcc8c60b40 2024-09-16T02:05:51.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.294+0000 7f6b3989a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fcc8c5f5d0 0x55fcc8c5fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:51.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.294+0000 7f6b3989a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fcc8c5f5d0 0x55fcc8c5fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43030/0 (socket says 172.21.15.105:43030) 2024-09-16T02:05:51.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.294+0000 7f6b3989a640 1 -- 172.21.15.105:0/958276120 learned_addr learned my addr 172.21.15.105:0/958276120 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:51.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.294+0000 7f6b39099640 1 --2- 172.21.15.105:0/958276120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc8c6a370 0x55fcc8c6c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:51.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.294+0000 7f6b3a09b640 1 --2- 172.21.15.105:0/958276120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcc8c60b40 0x55fcc8c5f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:51.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.294+0000 7f6b3989a640 1 -- 172.21.15.105:0/958276120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcc8c60b40 msgr2=0x55fcc8c5f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:51.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.294+0000 7f6b3989a640 1 --2- 172.21.15.105:0/958276120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcc8c60b40 0x55fcc8c5f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:51.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.294+0000 7f6b3989a640 1 -- 172.21.15.105:0/958276120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc8c6a370 msgr2=0x55fcc8c6c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:51.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.294+0000 7f6b3989a640 1 --2- 172.21.15.105:0/958276120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc8c6a370 0x55fcc8c6c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:51.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.294+0000 7f6b3989a640 1 -- 172.21.15.105:0/958276120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcc8b513a0 con 0x55fcc8c5f5d0 2024-09-16T02:05:51.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.294+0000 7f6b2affd640 1 --2- 172.21.15.105:0/3774478876 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f6b200a3240 0x7f6b200a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:51.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.294+0000 7f6b3989a640 1 --2- 172.21.15.105:0/958276120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fcc8c5f5d0 0x55fcc8c5fa10 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6b30006010 tx=0x7f6b30032cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c629d486b0916f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:51.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.295+0000 7f6b3a09b640 1 --2- 172.21.15.105:0/3774478876 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f6b200a3240 0x7f6b200a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:51.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.295+0000 7f6b2affd640 1 -- 172.21.15.105:0/3774478876 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f6b240cf050 con 0x7f6b34074eb0 2024-09-16T02:05:51.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.295+0000 7f6b1ffff640 1 -- 172.21.15.105:0/958276120 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3003f070 con 0x55fcc8c5f5d0 2024-09-16T02:05:51.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.295+0000 7f6b1ffff640 1 -- 172.21.15.105:0/958276120 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b300316a0 con 0x55fcc8c5f5d0 2024-09-16T02:05:51.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.295+0000 7f6b1ffff640 1 -- 172.21.15.105:0/958276120 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b30035070 con 0x55fcc8c5f5d0 2024-09-16T02:05:51.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.296+0000 7f6b3ded7740 1 -- 172.21.15.105:0/958276120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fcc8c5f5d0 msgr2=0x55fcc8c5fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:51.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.296+0000 7f6b3a09b640 1 --2- 172.21.15.105:0/3774478876 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f6b200a3240 0x7f6b200a5700 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6b34076600 tx=0x7f6b2c009410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:51.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.296+0000 7f6b3ded7740 1 --2- 172.21.15.105:0/958276120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fcc8c5f5d0 0x55fcc8c5fa10 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6b30006010 tx=0x7f6b30032cc0 comp rx=0 tx=0).stop 2024-09-16T02:05:51.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.296+0000 7f6b3ded7740 1 -- 172.21.15.105:0/958276120 shutdown_connections 2024-09-16T02:05:51.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.296+0000 7f6b3ded7740 1 --2- 172.21.15.105:0/958276120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc8c6a370 0x55fcc8c6c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:51.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.296+0000 7f6b3ded7740 1 --2- 172.21.15.105:0/958276120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fcc8c5f5d0 0x55fcc8c5fa10 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:51.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.296+0000 7f6b3ded7740 1 --2- 172.21.15.105:0/958276120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcc8c60b40 0x55fcc8c5f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:51.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.296+0000 7f6b3ded7740 1 -- 172.21.15.105:0/958276120 >> 172.21.15.105:0/958276120 conn(0x55fcc8c28310 msgr2=0x55fcc8c5bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:51.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.296+0000 7f6b3ded7740 1 -- 172.21.15.105:0/958276120 shutdown_connections 2024-09-16T02:05:51.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.296+0000 7f6b3ded7740 1 -- 172.21.15.105:0/958276120 wait complete. 2024-09-16T02:05:51.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.296+0000 7f6b3ded7740 1 -- start start 2024-09-16T02:05:51.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.296+0000 7f6b3ded7740 1 Processor -- start 2024-09-16T02:05:51.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.297+0000 7f6b3ded7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fcc8c5f5d0 0x55fcc8c69d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:51.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.297+0000 7f6b3ded7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcc8c60b40 0x55fcc8c67010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:51.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.297+0000 7f6b3ded7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc8c6a370 0x55fcc8c67600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:51.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.297+0000 7f6b3ded7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55fcc8c6d010 con 0x55fcc8c5f5d0 2024-09-16T02:05:51.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.297+0000 7f6b3ded7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fcc8b80060 con 0x55fcc8c6a370 2024-09-16T02:05:51.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.297+0000 7f6b3ded7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55fcc8c6d190 con 0x55fcc8c60b40 2024-09-16T02:05:51.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.297+0000 7f6b3ded7740 10 client.0 _ll_register_callbacks cb 0x55fcc88d38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:51.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.297+0000 7f6b3989a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fcc8c5f5d0 0x55fcc8c69d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:51.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.297+0000 7f6b3989a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fcc8c5f5d0 0x55fcc8c69d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43044/0 (socket says 172.21.15.105:43044) 2024-09-16T02:05:51.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.297+0000 7f6b3989a640 1 -- 172.21.15.105:0/2298508639 learned_addr learned my addr 172.21.15.105:0/2298508639 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:51.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.297+0000 7f6b39099640 1 --2- 172.21.15.105:0/2298508639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcc8c60b40 0x55fcc8c67010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:51.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.297+0000 7f6b3a09b640 1 --2- 172.21.15.105:0/2298508639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc8c6a370 0x55fcc8c67600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:51.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.297+0000 7f6b3989a640 1 -- 172.21.15.105:0/2298508639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcc8c60b40 msgr2=0x55fcc8c67010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:51.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.297+0000 7f6b3989a640 1 --2- 172.21.15.105:0/2298508639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcc8c60b40 0x55fcc8c67010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:51.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.297+0000 7f6b3989a640 1 -- 172.21.15.105:0/2298508639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc8c6a370 msgr2=0x55fcc8c67600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:51.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.297+0000 7f6b3989a640 1 --2- 172.21.15.105:0/2298508639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc8c6a370 0x55fcc8c67600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:51.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.297+0000 7f6b3989a640 1 -- 172.21.15.105:0/2298508639 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcc89363c0 con 0x55fcc8c5f5d0 2024-09-16T02:05:51.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.297+0000 7f6b3a09b640 1 --2- 172.21.15.105:0/2298508639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc8c6a370 0x55fcc8c67600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:05:51.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.297+0000 7f6b3989a640 1 --2- 172.21.15.105:0/2298508639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fcc8c5f5d0 0x55fcc8c69d60 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6b3002a4c0 tx=0x7f6b30031510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61fc3db5667f2212 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:51.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.297+0000 7f6b1d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:05:51.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.297+0000 7f6b1d7fa640 1 -- 172.21.15.105:0/2298508639 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3003f070 con 0x55fcc8c5f5d0 2024-09-16T02:05:51.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.297+0000 7f6b1d7fa640 1 -- 172.21.15.105:0/2298508639 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f6b30004030 con 0x55fcc8c5f5d0 2024-09-16T02:05:51.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.297+0000 7f6b3ded7740 1 -- 172.21.15.105:0/2298508639 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b300316a0 con 0x55fcc8c5f5d0 2024-09-16T02:05:51.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.298+0000 7f6b1d7fa640 1 -- 172.21.15.105:0/2298508639 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6b30053050 con 0x55fcc8c5f5d0 2024-09-16T02:05:51.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.298+0000 7f6b3ded7740 10 client.16011 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:51.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.298+0000 7f6b3ded7740 1 -- 172.21.15.105:0/2298508639 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fcc8c78ab0 con 0x55fcc8c5f5d0 2024-09-16T02:05:51.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.298+0000 7f6b1d7fa640 1 -- 172.21.15.105:0/2298508639 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b300316a0 con 0x55fcc8c5f5d0 2024-09-16T02:05:51.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.298+0000 7f6b1d7fa640 1 -- 172.21.15.105:0/2298508639 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6b3003b460 con 0x55fcc8c5f5d0 2024-09-16T02:05:51.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.298+0000 7f6b3ded7740 10 client.16011 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:51.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.298+0000 7f6b3ded7740 10 client.16011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:05:51.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.298+0000 7f6b3ded7740 20 client.16011 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:51.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.298+0000 7f6b3ded7740 1 --2- 172.21.15.105:0/2298508639 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55fcc8c79e00 0x55fcc8c7c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:51.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.298+0000 7f6b3ded7740 4 client.16011 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:05:51.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.298+0000 7f6b3ded7740 1 -- 172.21.15.105:0/2298508639 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fcc8bd2860 con 0x55fcc8c79e00 2024-09-16T02:05:51.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.298+0000 7f6b39099640 1 --2- 172.21.15.105:0/2298508639 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55fcc8c79e00 0x55fcc8c7c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:51.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.299+0000 7f6b1d7fa640 10 client.16011 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:05:51.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.299+0000 7f6b39099640 1 --2- 172.21.15.105:0/2298508639 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55fcc8c79e00 0x55fcc8c7c1f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f24289a093acb487 server_cookie=28ad45e0ecd90310 in_seq=0 out_seq=0 2024-09-16T02:05:51.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.299+0000 7f6b1d7fa640 1 -- 172.21.15.105:0/2298508639 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fcc8bd2860 con 0x55fcc8c79e00 2024-09-16T02:05:51.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.299+0000 7f6b1d7fa640 10 client.16011 handle_command_reply: tid=0 2024-09-16T02:05:51.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.299+0000 7f6b3ded7740 1 client.16011 shutdown 2024-09-16T02:05:51.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.300+0000 7f6b3ded7740 1 -- 172.21.15.105:0/2298508639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fcc8c5f5d0 msgr2=0x55fcc8c69d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:51.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.300+0000 7f6b3ded7740 1 --2- 172.21.15.105:0/2298508639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fcc8c5f5d0 0x55fcc8c69d60 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6b3002a4c0 tx=0x7f6b30031510 comp rx=0 tx=0).stop 2024-09-16T02:05:51.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.300+0000 7f6b3ded7740 1 -- 172.21.15.105:0/2298508639 shutdown_connections 2024-09-16T02:05:51.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.300+0000 7f6b3ded7740 1 --2- 172.21.15.105:0/2298508639 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55fcc8c79e00 0x55fcc8c7c1f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:51.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.300+0000 7f6b3ded7740 1 --2- 172.21.15.105:0/2298508639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcc8c60b40 0x55fcc8c67010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:51.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.300+0000 7f6b3ded7740 1 --2- 172.21.15.105:0/2298508639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc8c6a370 0x55fcc8c67600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:51.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.300+0000 7f6b1d7fa640 0 client.16011 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:05:51.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.300+0000 7f6b3ded7740 1 --2- 172.21.15.105:0/2298508639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fcc8c5f5d0 0x55fcc8c69d60 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:51.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.300+0000 7f6b3ded7740 1 -- 172.21.15.105:0/2298508639 >> 172.21.15.105:0/2298508639 conn(0x55fcc8c28310 msgr2=0x55fcc8c61960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:51.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.300+0000 7f6b3ded7740 1 -- 172.21.15.105:0/2298508639 shutdown_connections 2024-09-16T02:05:51.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.300+0000 7f6b3ded7740 1 -- 172.21.15.105:0/2298508639 wait complete. 2024-09-16T02:05:51.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.300+0000 7f6b3ded7740 20 client.16011 trim_cache size 0 max 16384 2024-09-16T02:05:51.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.313+0000 7f6b3ded7740 1 Processor -- start 2024-09-16T02:05:51.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.313+0000 7f6b3ded7740 1 -- start start 2024-09-16T02:05:51.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.313+0000 7f6b3ded7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc8c5f5d0 0x55fcc8c762b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:51.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.313+0000 7f6b3ded7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcc8c60b40 0x55fcc8c65ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:51.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.313+0000 7f6b3ded7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fcc8c6a370 0x55fcc8c81790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:51.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.313+0000 7f6b3ded7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6b341b3e20 con 0x55fcc8c6a370 2024-09-16T02:05:51.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.313+0000 7f6b3ded7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6b341b5960 con 0x55fcc8c5f5d0 2024-09-16T02:05:51.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.313+0000 7f6b3ded7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6b341b44f0 con 0x55fcc8c60b40 2024-09-16T02:05:51.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.313+0000 7f6b3a09b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fcc8c6a370 0x55fcc8c81790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:51.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.313+0000 7f6b3a09b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fcc8c6a370 0x55fcc8c81790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43050/0 (socket says 172.21.15.105:43050) 2024-09-16T02:05:51.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.313+0000 7f6b3a09b640 1 -- 172.21.15.105:0/2638079882 learned_addr learned my addr 172.21.15.105:0/2638079882 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:51.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.313+0000 7f6b3989a640 1 --2- 172.21.15.105:0/2638079882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc8c5f5d0 0x55fcc8c762b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:51.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.313+0000 7f6b39099640 1 --2- 172.21.15.105:0/2638079882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcc8c60b40 0x55fcc8c65ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:51.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.313+0000 7f6b3a09b640 1 -- 172.21.15.105:0/2638079882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcc8c60b40 msgr2=0x55fcc8c65ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:51.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.313+0000 7f6b3a09b640 1 --2- 172.21.15.105:0/2638079882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcc8c60b40 0x55fcc8c65ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:51.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.313+0000 7f6b3a09b640 1 -- 172.21.15.105:0/2638079882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc8c5f5d0 msgr2=0x55fcc8c762b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:51.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.313+0000 7f6b3a09b640 1 --2- 172.21.15.105:0/2638079882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc8c5f5d0 0x55fcc8c762b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:51.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.313+0000 7f6b3a09b640 1 -- 172.21.15.105:0/2638079882 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcc8c6a090 con 0x55fcc8c6a370 2024-09-16T02:05:51.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.313+0000 7f6b39099640 1 --2- 172.21.15.105:0/2638079882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcc8c60b40 0x55fcc8c65ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:05:51.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.314+0000 7f6b3a09b640 1 --2- 172.21.15.105:0/2638079882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fcc8c6a370 0x55fcc8c81790 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c017710 tx=0x7f6b2c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1931846f1f15b23a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:51.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.314+0000 7f6b1f7fe640 1 -- 172.21.15.105:0/2638079882 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c027070 con 0x55fcc8c6a370 2024-09-16T02:05:51.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.314+0000 7f6b1f7fe640 1 -- 172.21.15.105:0/2638079882 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b2c017e40 con 0x55fcc8c6a370 2024-09-16T02:05:51.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.314+0000 7f6b1f7fe640 1 -- 172.21.15.105:0/2638079882 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c0167c0 con 0x55fcc8c6a370 2024-09-16T02:05:51.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.314+0000 7f6b3ded7740 1 -- 172.21.15.105:0/2638079882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fcc8c6a370 msgr2=0x55fcc8c81790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:51.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.314+0000 7f6b3ded7740 1 --2- 172.21.15.105:0/2638079882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fcc8c6a370 0x55fcc8c81790 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c017710 tx=0x7f6b2c015000 comp rx=0 tx=0).stop 2024-09-16T02:05:51.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.314+0000 7f6b3ded7740 1 -- 172.21.15.105:0/2638079882 shutdown_connections 2024-09-16T02:05:51.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.314+0000 7f6b3ded7740 1 --2- 172.21.15.105:0/2638079882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcc8c60b40 0x55fcc8c65ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:51.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.314+0000 7f6b3ded7740 1 --2- 172.21.15.105:0/2638079882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fcc8c6a370 0x55fcc8c81790 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:51.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.314+0000 7f6b3ded7740 1 --2- 172.21.15.105:0/2638079882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc8c5f5d0 0x55fcc8c762b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:51.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.314+0000 7f6b3ded7740 1 -- 172.21.15.105:0/2638079882 >> 172.21.15.105:0/2638079882 conn(0x55fcc8c28310 msgr2=0x55fcc8c658e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:51.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.314+0000 7f6b3ded7740 1 -- 172.21.15.105:0/2638079882 shutdown_connections 2024-09-16T02:05:51.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.314+0000 7f6b3ded7740 1 -- 172.21.15.105:0/2638079882 wait complete. 2024-09-16T02:05:51.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.315+0000 7f6b3ded7740 1 -- start start 2024-09-16T02:05:51.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.315+0000 7f6b3ded7740 1 Processor -- start 2024-09-16T02:05:51.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.315+0000 7f6b3ded7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fcc8c5f5d0 0x55fcc8c63f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:51.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.315+0000 7f6b3ded7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc8c60b40 0x55fcc8c64450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:51.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.315+0000 7f6b3ded7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcc8c6a370 0x55fcc8c64990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:51.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.315+0000 7f6b3ded7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6b341b5290 con 0x55fcc8c5f5d0 2024-09-16T02:05:51.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.315+0000 7f6b3ded7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6b341b4bc0 con 0x55fcc8c60b40 2024-09-16T02:05:51.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.315+0000 7f6b3ded7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6b341b1e00 con 0x55fcc8c6a370 2024-09-16T02:05:51.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.315+0000 7f6b3ded7740 10 client.0 _ll_register_callbacks cb 0x7f6b240d2b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:51.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.315+0000 7f6b3989a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fcc8c5f5d0 0x55fcc8c63f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:51.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.315+0000 7f6b3989a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fcc8c5f5d0 0x55fcc8c63f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43058/0 (socket says 172.21.15.105:43058) 2024-09-16T02:05:51.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.315+0000 7f6b3989a640 1 -- 172.21.15.105:0/1987728015 learned_addr learned my addr 172.21.15.105:0/1987728015 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:51.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.316+0000 7f6b39099640 1 --2- 172.21.15.105:0/1987728015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc8c60b40 0x55fcc8c64450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:51.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.316+0000 7f6b3a09b640 1 --2- 172.21.15.105:0/1987728015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcc8c6a370 0x55fcc8c64990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:51.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.316+0000 7f6b3989a640 1 -- 172.21.15.105:0/1987728015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcc8c6a370 msgr2=0x55fcc8c64990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:51.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.316+0000 7f6b3989a640 1 --2- 172.21.15.105:0/1987728015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcc8c6a370 0x55fcc8c64990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:51.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.316+0000 7f6b3989a640 1 -- 172.21.15.105:0/1987728015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc8c60b40 msgr2=0x55fcc8c64450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:51.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.316+0000 7f6b3989a640 1 --2- 172.21.15.105:0/1987728015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc8c60b40 0x55fcc8c64450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:51.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.316+0000 7f6b3989a640 1 -- 172.21.15.105:0/1987728015 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcc8ce3760 con 0x55fcc8c5f5d0 2024-09-16T02:05:51.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.316+0000 7f6b39099640 1 --2- 172.21.15.105:0/1987728015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc8c60b40 0x55fcc8c64450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:05:51.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.316+0000 7f6b3989a640 1 --2- 172.21.15.105:0/1987728015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fcc8c5f5d0 0x55fcc8c63f10 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6b3003b830 tx=0x7f6b3002abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f116fa01f2971403 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:51.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.316+0000 7f6b1d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-16T02:05:51.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.316+0000 7f6b1d7fa640 1 -- 172.21.15.105:0/1987728015 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3003f070 con 0x55fcc8c5f5d0 2024-09-16T02:05:51.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.316+0000 7f6b1d7fa640 1 -- 172.21.15.105:0/1987728015 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f6b3003b460 con 0x55fcc8c5f5d0 2024-09-16T02:05:51.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.316+0000 7f6b3ded7740 1 -- 172.21.15.105:0/1987728015 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fcc8bccd70 con 0x55fcc8c5f5d0 2024-09-16T02:05:51.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.316+0000 7f6b1d7fa640 1 -- 172.21.15.105:0/1987728015 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6b30035040 con 0x55fcc8c5f5d0 2024-09-16T02:05:51.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.316+0000 7f6b3ded7740 10 client.16017 fetch_fsmap learned FSMap version 28 2024-09-16T02:05:51.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.316+0000 7f6b3ded7740 1 -- 172.21.15.105:0/1987728015 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fcc8ce48b0 con 0x55fcc8c5f5d0 2024-09-16T02:05:51.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.316+0000 7f6b1d7fa640 1 -- 172.21.15.105:0/1987728015 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55fcc8bccd70 con 0x55fcc8c5f5d0 2024-09-16T02:05:51.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.317+0000 7f6b1d7fa640 1 -- 172.21.15.105:0/1987728015 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6b30032020 con 0x55fcc8c5f5d0 2024-09-16T02:05:51.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.317+0000 7f6b3ded7740 10 client.16017 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T02:05:51.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.317+0000 7f6b3ded7740 10 client.16017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:05:51.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.317+0000 7f6b3ded7740 20 client.16017 populate_metadata read hostname 'smithi105' 2024-09-16T02:05:51.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.317+0000 7f6b3ded7740 1 --2- 172.21.15.105:0/1987728015 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55fcc8ce4f60 0x55fcc8ce7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:51.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.317+0000 7f6b3ded7740 4 client.16017 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T02:05:51.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.317+0000 7f6b3ded7740 1 -- 172.21.15.105:0/1987728015 --> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fcc8b49170 con 0x55fcc8ce4f60 2024-09-16T02:05:51.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.317+0000 7f6b39099640 1 --2- 172.21.15.105:0/1987728015 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55fcc8ce4f60 0x55fcc8ce7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:51.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.317+0000 7f6b39099640 1 --2- 172.21.15.105:0/1987728015 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55fcc8ce4f60 0x55fcc8ce7380 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3021db76bd17c86c server_cookie=b91e679e11d298bf in_seq=0 out_seq=0 2024-09-16T02:05:51.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.317+0000 7f6b1d7fa640 10 client.16017 ms_handle_connect on v2:172.21.15.137:6838/3454924009 2024-09-16T02:05:51.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.317+0000 7f6b1d7fa640 1 -- 172.21.15.105:0/1987728015 <== mds.0 v2:172.21.15.137:6838/3454924009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fcc8b49170 con 0x55fcc8ce4f60 2024-09-16T02:05:51.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.317+0000 7f6b1d7fa640 10 client.16017 handle_command_reply: tid=0 2024-09-16T02:05:51.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.317+0000 7f6b3ded7740 1 client.16017 shutdown 2024-09-16T02:05:51.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.318+0000 7f6b3ded7740 1 -- 172.21.15.105:0/1987728015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fcc8c5f5d0 msgr2=0x55fcc8c63f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:51.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.318+0000 7f6b3ded7740 1 --2- 172.21.15.105:0/1987728015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fcc8c5f5d0 0x55fcc8c63f10 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6b3003b830 tx=0x7f6b3002abb0 comp rx=0 tx=0).stop 2024-09-16T02:05:51.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.318+0000 7f6b3ded7740 1 -- 172.21.15.105:0/1987728015 shutdown_connections 2024-09-16T02:05:51.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.318+0000 7f6b3ded7740 1 --2- 172.21.15.105:0/1987728015 >> [v2:172.21.15.137:6838/3454924009,v1:172.21.15.137:6839/3454924009] conn(0x55fcc8ce4f60 0x55fcc8ce7380 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:51.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.318+0000 7f6b3ded7740 1 --2- 172.21.15.105:0/1987728015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fcc8c6a370 0x55fcc8c64990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:51.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.318+0000 7f6b3ded7740 1 --2- 172.21.15.105:0/1987728015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc8c60b40 0x55fcc8c64450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:51.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.318+0000 7f6b3ded7740 1 --2- 172.21.15.105:0/1987728015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fcc8c5f5d0 0x55fcc8c63f10 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:51.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.318+0000 7f6b1d7fa640 0 client.16017 ms_handle_reset on v2:172.21.15.137:6838/3454924009 2024-09-16T02:05:51.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.318+0000 7f6b3ded7740 1 -- 172.21.15.105:0/1987728015 >> 172.21.15.105:0/1987728015 conn(0x55fcc8c28310 msgr2=0x55fcc8c66100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:51.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.318+0000 7f6b3ded7740 1 -- 172.21.15.105:0/1987728015 shutdown_connections 2024-09-16T02:05:51.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.318+0000 7f6b3ded7740 1 -- 172.21.15.105:0/1987728015 wait complete. 2024-09-16T02:05:51.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.318+0000 7f6b3ded7740 20 client.16017 trim_cache size 0 max 16384 2024-09-16T02:05:51.333 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-09-16T02:05:51.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.318+0000 7f6b3bafe640 1 -- 172.21.15.105:0/3774478876 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f6b200a3240 msgr2=0x7f6b200a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:51.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.318+0000 7f6b3bafe640 1 --2- 172.21.15.105:0/3774478876 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f6b200a3240 0x7f6b200a5700 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6b34076600 tx=0x7f6b2c009410 comp rx=0 tx=0).stop 2024-09-16T02:05:51.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.318+0000 7f6b3bafe640 1 -- 172.21.15.105:0/3774478876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b34074eb0 msgr2=0x7f6b341a4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:51.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.318+0000 7f6b3bafe640 1 --2- 172.21.15.105:0/3774478876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b34074eb0 0x7f6b341a4cf0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6b2400b4a0 tx=0x7f6b2400b960 comp rx=0 tx=0).stop 2024-09-16T02:05:51.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.318+0000 7f6b3bafe640 1 -- 172.21.15.105:0/3774478876 shutdown_connections 2024-09-16T02:05:51.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.318+0000 7f6b3bafe640 1 --2- 172.21.15.105:0/3774478876 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f6b200a3240 0x7f6b200a5700 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:51.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.318+0000 7f6b3bafe640 1 --2- 172.21.15.105:0/3774478876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b34112b90 0x7f6b3419e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:51.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.318+0000 7f6b3bafe640 1 --2- 172.21.15.105:0/3774478876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b34074eb0 0x7f6b341a4cf0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:51.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.318+0000 7f6b3bafe640 1 --2- 172.21.15.105:0/3774478876 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b34074590 0x7f6b341a47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:51.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.318+0000 7f6b3bafe640 1 -- 172.21.15.105:0/3774478876 >> 172.21.15.105:0/3774478876 conn(0x7f6b340fd8c0 msgr2=0x7f6b340ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:51.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.319+0000 7f6b3bafe640 1 -- 172.21.15.105:0/3774478876 shutdown_connections 2024-09-16T02:05:51.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:51.319+0000 7f6b3bafe640 1 -- 172.21.15.105:0/3774478876 wait complete. 2024-09-16T02:05:52.064 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-16T02:05:52.065 INFO:tasks.fwd_scrub:done joining 2024-09-16T02:05:52.065 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-16T02:05:52.075 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T02:05:52.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.176+0000 7fb574398640 1 -- 172.21.15.105:0/671377377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb56c100a10 msgr2=0x7fb56c100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:52.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.176+0000 7fb574398640 1 --2- 172.21.15.105:0/671377377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb56c100a10 0x7fb56c100e10 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fb568009900 tx=0x7fb56802a410 comp rx=0 tx=0).stop 2024-09-16T02:05:52.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.176+0000 7fb574398640 1 -- 172.21.15.105:0/671377377 shutdown_connections 2024-09-16T02:05:52.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.176+0000 7fb574398640 1 --2- 172.21.15.105:0/671377377 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb56c1025c0 0x7fb56c108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:52.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.176+0000 7fb574398640 1 --2- 172.21.15.105:0/671377377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb56c101c00 0x7fb56c102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:52.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.176+0000 7fb574398640 1 --2- 172.21.15.105:0/671377377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb56c100a10 0x7fb56c100e10 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:52.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.176+0000 7fb574398640 1 -- 172.21.15.105:0/671377377 >> 172.21.15.105:0/671377377 conn(0x7fb56c0fc170 msgr2=0x7fb56c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:52.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.176+0000 7fb574398640 1 -- 172.21.15.105:0/671377377 shutdown_connections 2024-09-16T02:05:52.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.176+0000 7fb574398640 1 -- 172.21.15.105:0/671377377 wait complete. 2024-09-16T02:05:52.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.176+0000 7fb574398640 1 Processor -- start 2024-09-16T02:05:52.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.176+0000 7fb574398640 1 -- start start 2024-09-16T02:05:52.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.177+0000 7fb574398640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb56c100a10 0x7fb56c197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:52.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.177+0000 7fb574398640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb56c101c00 0x7fb56c198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:52.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.177+0000 7fb574398640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb56c1025c0 0x7fb56c19f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:52.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.177+0000 7fb574398640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb56c10d6f0 con 0x7fb56c101c00 2024-09-16T02:05:52.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.177+0000 7fb574398640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb56c10d570 con 0x7fb56c100a10 2024-09-16T02:05:52.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.177+0000 7fb574398640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb56c10d870 con 0x7fb56c1025c0 2024-09-16T02:05:52.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.177+0000 7fb571933640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb56c101c00 0x7fb56c198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:52.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.177+0000 7fb571933640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb56c101c00 0x7fb56c198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43076/0 (socket says 172.21.15.105:43076) 2024-09-16T02:05:52.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.177+0000 7fb571933640 1 -- 172.21.15.105:0/1170118385 learned_addr learned my addr 172.21.15.105:0/1170118385 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:52.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.177+0000 7fb572134640 1 --2- 172.21.15.105:0/1170118385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb56c100a10 0x7fb56c197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:52.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.177+0000 7fb571933640 1 -- 172.21.15.105:0/1170118385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb56c1025c0 msgr2=0x7fb56c19f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:52.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.177+0000 7fb571933640 1 --2- 172.21.15.105:0/1170118385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb56c1025c0 0x7fb56c19f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:52.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.177+0000 7fb571933640 1 -- 172.21.15.105:0/1170118385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb56c100a10 msgr2=0x7fb56c197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:52.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.177+0000 7fb571933640 1 --2- 172.21.15.105:0/1170118385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb56c100a10 0x7fb56c197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:52.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.177+0000 7fb571933640 1 -- 172.21.15.105:0/1170118385 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb56c19fa20 con 0x7fb56c101c00 2024-09-16T02:05:52.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.177+0000 7fb572134640 1 --2- 172.21.15.105:0/1170118385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb56c100a10 0x7fb56c197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:05:52.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.177+0000 7fb571933640 1 --2- 172.21.15.105:0/1170118385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb56c101c00 0x7fb56c198340 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb55c00d950 tx=0x7fb55c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:52.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.177+0000 7fb5637fe640 1 -- 172.21.15.105:0/1170118385 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c017070 con 0x7fb56c101c00 2024-09-16T02:05:52.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.177+0000 7fb5637fe640 1 -- 172.21.15.105:0/1170118385 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb55c009c30 con 0x7fb56c101c00 2024-09-16T02:05:52.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.177+0000 7fb574398640 1 -- 172.21.15.105:0/1170118385 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb56c19fd10 con 0x7fb56c101c00 2024-09-16T02:05:52.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.177+0000 7fb5637fe640 1 -- 172.21.15.105:0/1170118385 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c00f620 con 0x7fb56c101c00 2024-09-16T02:05:52.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.177+0000 7fb574398640 1 -- 172.21.15.105:0/1170118385 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb56c1a0280 con 0x7fb56c101c00 2024-09-16T02:05:52.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.178+0000 7fb574398640 1 -- 172.21.15.105:0/1170118385 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb56c107e50 con 0x7fb56c101c00 2024-09-16T02:05:52.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.179+0000 7fb5637fe640 1 -- 172.21.15.105:0/1170118385 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fb55c0109a0 con 0x7fb56c101c00 2024-09-16T02:05:52.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.180+0000 7fb5637fe640 1 --2- 172.21.15.105:0/1170118385 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb5580a3250 0x7fb5580a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:52.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.180+0000 7fb572134640 1 --2- 172.21.15.105:0/1170118385 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb5580a3250 0x7fb5580a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:52.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.180+0000 7fb5637fe640 1 -- 172.21.15.105:0/1170118385 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fb55c0cf650 con 0x7fb56c101c00 2024-09-16T02:05:52.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.181+0000 7fb572134640 1 --2- 172.21.15.105:0/1170118385 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb5580a3250 0x7fb5580a5710 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb56802a930 tx=0x7fb568005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:52.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.184+0000 7fb5637fe640 1 -- 172.21.15.105:0/1170118385 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb55c009de0 con 0x7fb56c101c00 2024-09-16T02:05:52.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.350+0000 7fb574398640 1 -- 172.21.15.105:0/1170118385 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb56c062f30 con 0x7fb56c101c00 2024-09-16T02:05:52.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.352+0000 7fb5637fe640 1 -- 172.21.15.105:0/1170118385 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22962 (secure 0 0 0) 0x7fb55c09c310 con 0x7fb56c101c00 2024-09-16T02:05:52.354 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:05:52.356 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":83,"fsid":"423b7a2a-73ce-11ef-bceb-c7b262605968","created":"2024-09-16T01:52:44.388033+0000","modified":"2024-09-16T02:02:41.553985+0000","last_up_change":"2024-09-16T01:59:42.218034+0000","last_in_change":"2024-09-16T01:59:18.785252+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T01:55:53.680808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T02:00:31.286117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T02:00:33.310045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a60d6c72-da8b-4b02-bb33-672be795e7f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6803","nonce":1226660290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6805","nonce":1226660290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6809","nonce":1226660290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6807","nonce":1226660290}]},"public_addr":"172.21.15.105:6803/1226660290","cluster_addr":"172.21.15.105:6805/1226660290","heartbeat_back_addr":"172.21.15.105:6809/1226660290","heartbeat_front_addr":"172.21.15.105:6807/1226660290","state":["exists","up"]},{"osd":1,"uuid":"b1d9ae84-b9a0-4a7e-b0fe-c402d46809fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6801","nonce":34753956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6803","nonce":34753956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6807","nonce":34753956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6805","nonce":34753956}]},"public_addr":"172.21.15.137:6801/34753956","cluster_addr":"172.21.15.137:6803/34753956","heartbeat_back_addr":"172.21.15.137:6807/34753956","heartbeat_front_addr":"172.21.15.137:6805/34753956","state":["exists","up"]},{"osd":2,"uuid":"137faae6-2e8b-44b5-8976-3d95a6eef1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6801","nonce":1159531918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6803","nonce":1159531918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6807","nonce":1159531918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6805","nonce":1159531918}]},"public_addr":"172.21.15.190:6801/1159531918","cluster_addr":"172.21.15.190:6803/1159531918","heartbeat_back_addr":"172.21.15.190:6807/1159531918","heartbeat_front_addr":"172.21.15.190:6805/1159531918","state":["exists","up"]},{"osd":3,"uuid":"5743c37b-5cd8-42b6-81c8-5cf5afc71e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6811","nonce":604905679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6813","nonce":604905679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6817","nonce":604905679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6815","nonce":604905679}]},"public_addr":"172.21.15.105:6811/604905679","cluster_addr":"172.21.15.105:6813/604905679","heartbeat_back_addr":"172.21.15.105:6817/604905679","heartbeat_front_addr":"172.21.15.105:6815/604905679","state":["exists","up"]},{"osd":4,"uuid":"e439d4a2-b54c-4de9-97f6-3510afe57f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6809","nonce":3758109169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6811","nonce":3758109169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6815","nonce":3758109169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6813","nonce":3758109169}]},"public_addr":"172.21.15.137:6809/3758109169","cluster_addr":"172.21.15.137:6811/3758109169","heartbeat_back_addr":"172.21.15.137:6815/3758109169","heartbeat_front_addr":"172.21.15.137:6813/3758109169","state":["exists","up"]},{"osd":5,"uuid":"d1e2a7d5-99f8-40fa-af6e-0305cc0af677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6809","nonce":2086773754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6811","nonce":2086773754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6815","nonce":2086773754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6813","nonce":2086773754}]},"public_addr":"172.21.15.190:6809/2086773754","cluster_addr":"172.21.15.190:6811/2086773754","heartbeat_back_addr":"172.21.15.190:6815/2086773754","heartbeat_front_addr":"172.21.15.190:6813/2086773754","state":["exists","up"]},{"osd":6,"uuid":"cd418159-a0f6-44c7-9213-c010b132fb56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6819","nonce":4012018725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6821","nonce":4012018725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6825","nonce":4012018725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6823","nonce":4012018725}]},"public_addr":"172.21.15.105:6819/4012018725","cluster_addr":"172.21.15.105:6821/4012018725","heartbeat_back_addr":"172.21.15.105:6825/4012018725","heartbeat_front_addr":"172.21.15.105:6823/4012018725","state":["exists","up"]},{"osd":7,"uuid":"76c15863-801e-4a72-be73-ba9df46c15b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6817","nonce":888678288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6819","nonce":888678288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6823","nonce":888678288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6821","nonce":888678288}]},"public_addr":"172.21.15.137:6817/888678288","cluster_addr":"172.21.15.137:6819/888678288","heartbeat_back_addr":"172.21.15.137:6823/888678288","heartbeat_front_addr":"172.21.15.137:6821/888678288","state":["exists","up"]},{"osd":8,"uuid":"6718597c-6f42-48ee-9493-edb2396fe707","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6817","nonce":2132638341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6819","nonce":2132638341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6823","nonce":2132638341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6821","nonce":2132638341}]},"public_addr":"172.21.15.190:6817/2132638341","cluster_addr":"172.21.15.190:6819/2132638341","heartbeat_back_addr":"172.21.15.190:6823/2132638341","heartbeat_front_addr":"172.21.15.190:6821/2132638341","state":["exists","up"]},{"osd":9,"uuid":"5178edc0-a2e8-491c-821c-6f41ec5aeb2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6827","nonce":3375034979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6829","nonce":3375034979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6833","nonce":3375034979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6831","nonce":3375034979}]},"public_addr":"172.21.15.105:6827/3375034979","cluster_addr":"172.21.15.105:6829/3375034979","heartbeat_back_addr":"172.21.15.105:6833/3375034979","heartbeat_front_addr":"172.21.15.105:6831/3375034979","state":["exists","up"]},{"osd":10,"uuid":"ad145d3e-a6e2-4dd7-bea3-bcd01eb3413e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6825","nonce":318562234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6827","nonce":318562234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6831","nonce":318562234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6829","nonce":318562234}]},"public_addr":"172.21.15.137:6825/318562234","cluster_addr":"172.21.15.137:6827/318562234","heartbeat_back_addr":"172.21.15.137:6831/318562234","heartbeat_front_addr":"172.21.15.137:6829/318562234","state":["exists","up"]},{"osd":11,"uuid":"6d44bced-3a8b-4763-a93e-0ea34acb4cb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6825","nonce":3820460689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6827","nonce":3820460689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6831","nonce":3820460689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6829","nonce":3820460689}]},"public_addr":"172.21.15.190:6825/3820460689","cluster_addr":"172.21.15.190:6827/3820460689","heartbeat_back_addr":"172.21.15.190:6831/3820460689","heartbeat_front_addr":"172.21.15.190:6829/3820460689","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:55:02.759033+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:55:26.829142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:55:51.120200+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:56:17.277870+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:56:42.313701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:07.013416+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:32.477916+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:58.643764+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:23.454908+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:48.869082+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:14.921573+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:40.908993+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":7,"to":6}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.17","mappings":[{"from":5,"to":1}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":7}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:0/2453739657":"2024-09-17T01:53:39.351314+0000","172.21.15.105:6801/1469189484":"2024-09-17T01:53:39.351314+0000","172.21.15.105:0/3466166879":"2024-09-17T01:53:39.351314+0000","172.21.15.105:6801/1268348742":"2024-09-17T01:53:15.522227+0000","172.21.15.105:0/2897248478":"2024-09-17T01:53:39.351314+0000","172.21.15.105:0/3709363371":"2024-09-17T01:53:15.522227+0000","172.21.15.105:6800/1268348742":"2024-09-17T01:53:15.522227+0000","172.21.15.105:6800/1469189484":"2024-09-17T01:53:39.351314+0000","172.21.15.105:0/1198176299":"2024-09-17T01:53:15.522227+0000","172.21.15.105:0/3031917848":"2024-09-17T01:53:15.522227+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T02:05:52.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.354+0000 7fb574398640 1 -- 172.21.15.105:0/1170118385 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb5580a3250 msgr2=0x7fb5580a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:52.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.354+0000 7fb574398640 1 --2- 172.21.15.105:0/1170118385 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb5580a3250 0x7fb5580a5710 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb56802a930 tx=0x7fb568005fd0 comp rx=0 tx=0).stop 2024-09-16T02:05:52.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.354+0000 7fb574398640 1 -- 172.21.15.105:0/1170118385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb56c101c00 msgr2=0x7fb56c198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:52.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.354+0000 7fb574398640 1 --2- 172.21.15.105:0/1170118385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb56c101c00 0x7fb56c198340 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb55c00d950 tx=0x7fb55c00de10 comp rx=0 tx=0).stop 2024-09-16T02:05:52.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.354+0000 7fb574398640 1 -- 172.21.15.105:0/1170118385 shutdown_connections 2024-09-16T02:05:52.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.354+0000 7fb574398640 1 --2- 172.21.15.105:0/1170118385 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb5580a3250 0x7fb5580a5710 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:52.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.354+0000 7fb574398640 1 --2- 172.21.15.105:0/1170118385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb56c1025c0 0x7fb56c19f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:52.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.354+0000 7fb574398640 1 --2- 172.21.15.105:0/1170118385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb56c101c00 0x7fb56c198340 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:52.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.354+0000 7fb574398640 1 --2- 172.21.15.105:0/1170118385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb56c100a10 0x7fb56c197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:52.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.354+0000 7fb574398640 1 -- 172.21.15.105:0/1170118385 >> 172.21.15.105:0/1170118385 conn(0x7fb56c0fc170 msgr2=0x7fb56c105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:52.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.354+0000 7fb574398640 1 -- 172.21.15.105:0/1170118385 shutdown_connections 2024-09-16T02:05:52.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.354+0000 7fb574398640 1 -- 172.21.15.105:0/1170118385 wait complete. 2024-09-16T02:05:52.368 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T01:55:53.680808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-16T02:00:31.286117+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-16T02:00:33.310045+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T02:05:52.368 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T02:05:52.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.475+0000 7fc087665640 1 -- 172.21.15.105:0/2589210051 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc080102620 msgr2=0x7fc080104a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:52.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.475+0000 7fc087665640 1 --2- 172.21.15.105:0/2589210051 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc080102620 0x7fc080104a10 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fc07c009900 tx=0x7fc07c02a410 comp rx=0 tx=0).stop 2024-09-16T02:05:52.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.475+0000 7fc087665640 1 -- 172.21.15.105:0/2589210051 shutdown_connections 2024-09-16T02:05:52.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.475+0000 7fc087665640 1 --2- 172.21.15.105:0/2589210051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc080104f50 0x7fc080107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:52.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.475+0000 7fc087665640 1 --2- 172.21.15.105:0/2589210051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc080107880 0x7fc080109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:52.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.475+0000 7fc087665640 1 --2- 172.21.15.105:0/2589210051 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc080102620 0x7fc080104a10 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:52.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.475+0000 7fc087665640 1 -- 172.21.15.105:0/2589210051 >> 172.21.15.105:0/2589210051 conn(0x7fc0800fc140 msgr2=0x7fc0800fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:52.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.475+0000 7fc087665640 1 -- 172.21.15.105:0/2589210051 shutdown_connections 2024-09-16T02:05:52.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.475+0000 7fc087665640 1 -- 172.21.15.105:0/2589210051 wait complete. 2024-09-16T02:05:52.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.476+0000 7fc087665640 1 Processor -- start 2024-09-16T02:05:52.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.476+0000 7fc087665640 1 -- start start 2024-09-16T02:05:52.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.476+0000 7fc087665640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc080102620 0x7fc080197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:52.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.476+0000 7fc087665640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc080104f50 0x7fc080198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:52.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.476+0000 7fc087665640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc080107880 0x7fc08019f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:52.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.476+0000 7fc087665640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc08010d670 con 0x7fc080107880 2024-09-16T02:05:52.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.476+0000 7fc087665640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc08010d4f0 con 0x7fc080104f50 2024-09-16T02:05:52.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.476+0000 7fc087665640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc08010d7f0 con 0x7fc080102620 2024-09-16T02:05:52.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.476+0000 7fc085c02640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc080107880 0x7fc08019f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:52.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.476+0000 7fc085c02640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc080107880 0x7fc08019f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43106/0 (socket says 172.21.15.105:43106) 2024-09-16T02:05:52.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.476+0000 7fc085c02640 1 -- 172.21.15.105:0/1057314558 learned_addr learned my addr 172.21.15.105:0/1057314558 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:52.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.476+0000 7fc085401640 1 --2- 172.21.15.105:0/1057314558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc080102620 0x7fc080197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:52.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.476+0000 7fc084c00640 1 --2- 172.21.15.105:0/1057314558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc080104f50 0x7fc080198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:52.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.477+0000 7fc085c02640 1 -- 172.21.15.105:0/1057314558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc080102620 msgr2=0x7fc080197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:52.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.477+0000 7fc085c02640 1 --2- 172.21.15.105:0/1057314558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc080102620 0x7fc080197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:52.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.477+0000 7fc085c02640 1 -- 172.21.15.105:0/1057314558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc080104f50 msgr2=0x7fc080198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:52.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.477+0000 7fc085c02640 1 --2- 172.21.15.105:0/1057314558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc080104f50 0x7fc080198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:52.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.477+0000 7fc085c02640 1 -- 172.21.15.105:0/1057314558 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc08019fa70 con 0x7fc080107880 2024-09-16T02:05:52.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.477+0000 7fc084c00640 1 --2- 172.21.15.105:0/1057314558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc080104f50 0x7fc080198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:05:52.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.477+0000 7fc085c02640 1 --2- 172.21.15.105:0/1057314558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc080107880 0x7fc08019f400 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fc07800eb40 tx=0x7fc07800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:52.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.477+0000 7fc0727fc640 1 -- 172.21.15.105:0/1057314558 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc078004b20 con 0x7fc080107880 2024-09-16T02:05:52.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.477+0000 7fc0727fc640 1 -- 172.21.15.105:0/1057314558 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc078004cc0 con 0x7fc080107880 2024-09-16T02:05:52.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.477+0000 7fc087665640 1 -- 172.21.15.105:0/1057314558 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc08019fd60 con 0x7fc080107880 2024-09-16T02:05:52.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.477+0000 7fc0727fc640 1 -- 172.21.15.105:0/1057314558 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0780107b0 con 0x7fc080107880 2024-09-16T02:05:52.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.477+0000 7fc087665640 1 -- 172.21.15.105:0/1057314558 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0801a0200 con 0x7fc080107880 2024-09-16T02:05:52.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.478+0000 7fc087665640 1 -- 172.21.15.105:0/1057314558 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0800fdc30 con 0x7fc080107880 2024-09-16T02:05:52.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.479+0000 7fc0727fc640 1 -- 172.21.15.105:0/1057314558 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fc078005020 con 0x7fc080107880 2024-09-16T02:05:52.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.480+0000 7fc0727fc640 1 --2- 172.21.15.105:0/1057314558 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc06c0a3250 0x7fc06c0a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:52.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.480+0000 7fc085401640 1 --2- 172.21.15.105:0/1057314558 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc06c0a3250 0x7fc06c0a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:52.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.480+0000 7fc0727fc640 1 -- 172.21.15.105:0/1057314558 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fc0780d01a0 con 0x7fc080107880 2024-09-16T02:05:52.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.480+0000 7fc085401640 1 --2- 172.21.15.105:0/1057314558 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc06c0a3250 0x7fc06c0a5710 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc07c006010 tx=0x7fc07c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:52.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.484+0000 7fc0727fc640 1 -- 172.21.15.105:0/1057314558 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc07809ce60 con 0x7fc080107880 2024-09-16T02:05:52.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.650+0000 7fc087665640 1 -- 172.21.15.105:0/1057314558 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc080062e90 con 0x7fc080107880 2024-09-16T02:05:52.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.651+0000 7fc0727fc640 1 -- 172.21.15.105:0/1057314558 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7fc07809c760 con 0x7fc080107880 2024-09-16T02:05:52.653 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-09-16T02:05:52.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.653+0000 7fc087665640 1 -- 172.21.15.105:0/1057314558 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc06c0a3250 msgr2=0x7fc06c0a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:52.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.653+0000 7fc087665640 1 --2- 172.21.15.105:0/1057314558 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc06c0a3250 0x7fc06c0a5710 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc07c006010 tx=0x7fc07c000bc0 comp rx=0 tx=0).stop 2024-09-16T02:05:52.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.653+0000 7fc087665640 1 -- 172.21.15.105:0/1057314558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc080107880 msgr2=0x7fc08019f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:52.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.653+0000 7fc087665640 1 --2- 172.21.15.105:0/1057314558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc080107880 0x7fc08019f400 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fc07800eb40 tx=0x7fc07800c2d0 comp rx=0 tx=0).stop 2024-09-16T02:05:52.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.653+0000 7fc087665640 1 -- 172.21.15.105:0/1057314558 shutdown_connections 2024-09-16T02:05:52.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.653+0000 7fc087665640 1 --2- 172.21.15.105:0/1057314558 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fc06c0a3250 0x7fc06c0a5710 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:52.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.653+0000 7fc087665640 1 --2- 172.21.15.105:0/1057314558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc080107880 0x7fc08019f400 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:52.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.653+0000 7fc087665640 1 --2- 172.21.15.105:0/1057314558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc080104f50 0x7fc080198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:52.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.653+0000 7fc087665640 1 --2- 172.21.15.105:0/1057314558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc080102620 0x7fc080197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:52.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.653+0000 7fc087665640 1 -- 172.21.15.105:0/1057314558 >> 172.21.15.105:0/1057314558 conn(0x7fc0800fc140 msgr2=0x7fc080102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:52.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.653+0000 7fc087665640 1 -- 172.21.15.105:0/1057314558 shutdown_connections 2024-09-16T02:05:52.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.653+0000 7fc087665640 1 -- 172.21.15.105:0/1057314558 wait complete. 2024-09-16T02:05:52.665 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:52 smithi105 ceph-mon[31197]: pgmap v439: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.0 KiB/s wr, 225 op/s 2024-09-16T02:05:52.665 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:52 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/1170118385' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:05:52.666 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-16T02:05:52.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.767+0000 7fb94f12c640 1 -- 172.21.15.105:0/2001455762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb948102030 msgr2=0x7fb948102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:52.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.767+0000 7fb94f12c640 1 --2- 172.21.15.105:0/2001455762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb948102030 0x7fb948102430 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb93c009900 tx=0x7fb93c02a410 comp rx=0 tx=0).stop 2024-09-16T02:05:52.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.767+0000 7fb94f12c640 1 -- 172.21.15.105:0/2001455762 shutdown_connections 2024-09-16T02:05:52.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.767+0000 7fb94f12c640 1 --2- 172.21.15.105:0/2001455762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb948101640 0x7fb948108ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:52.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.767+0000 7fb94f12c640 1 --2- 172.21.15.105:0/2001455762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb948100c80 0x7fb948101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:52.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.767+0000 7fb94f12c640 1 --2- 172.21.15.105:0/2001455762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb948102030 0x7fb948102430 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:52.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.767+0000 7fb94f12c640 1 -- 172.21.15.105:0/2001455762 >> 172.21.15.105:0/2001455762 conn(0x7fb9480fb900 msgr2=0x7fb9480fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:52.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.767+0000 7fb94f12c640 1 -- 172.21.15.105:0/2001455762 shutdown_connections 2024-09-16T02:05:52.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.767+0000 7fb94f12c640 1 -- 172.21.15.105:0/2001455762 wait complete. 2024-09-16T02:05:52.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.767+0000 7fb94f12c640 1 Processor -- start 2024-09-16T02:05:52.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.767+0000 7fb94f12c640 1 -- start start 2024-09-16T02:05:52.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.767+0000 7fb94f12c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb948100c80 0x7fb948197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:52.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.767+0000 7fb94f12c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb948101640 0x7fb948198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:52.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.767+0000 7fb94f12c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb948102030 0x7fb94819f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:52.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.767+0000 7fb94f12c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb94810c5c0 con 0x7fb948101640 2024-09-16T02:05:52.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.767+0000 7fb94f12c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb94810c440 con 0x7fb948100c80 2024-09-16T02:05:52.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.767+0000 7fb94f12c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb94810c740 con 0x7fb948102030 2024-09-16T02:05:52.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.768+0000 7fb947fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb948101640 0x7fb948198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:52.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.768+0000 7fb947fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb948101640 0x7fb948198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43114/0 (socket says 172.21.15.105:43114) 2024-09-16T02:05:52.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.768+0000 7fb947fff640 1 -- 172.21.15.105:0/2897089704 learned_addr learned my addr 172.21.15.105:0/2897089704 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:52.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.768+0000 7fb94cec8640 1 --2- 172.21.15.105:0/2897089704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb948100c80 0x7fb948197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:52.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.768+0000 7fb94d6c9640 1 --2- 172.21.15.105:0/2897089704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb948102030 0x7fb94819f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:52.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.768+0000 7fb947fff640 1 -- 172.21.15.105:0/2897089704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb948102030 msgr2=0x7fb94819f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:52.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.768+0000 7fb947fff640 1 --2- 172.21.15.105:0/2897089704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb948102030 0x7fb94819f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:52.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.768+0000 7fb947fff640 1 -- 172.21.15.105:0/2897089704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb948100c80 msgr2=0x7fb948197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:52.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.768+0000 7fb947fff640 1 --2- 172.21.15.105:0/2897089704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb948100c80 0x7fb948197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:52.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.768+0000 7fb947fff640 1 -- 172.21.15.105:0/2897089704 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb94819fa50 con 0x7fb948101640 2024-09-16T02:05:52.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.768+0000 7fb947fff640 1 --2- 172.21.15.105:0/2897089704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb948101640 0x7fb948198370 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb93400d770 tx=0x7fb93400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:52.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.768+0000 7fb945ffb640 1 -- 172.21.15.105:0/2897089704 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb934017070 con 0x7fb948101640 2024-09-16T02:05:52.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.768+0000 7fb945ffb640 1 -- 172.21.15.105:0/2897089704 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb934009c30 con 0x7fb948101640 2024-09-16T02:05:52.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.768+0000 7fb94f12c640 1 -- 172.21.15.105:0/2897089704 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb94819fd40 con 0x7fb948101640 2024-09-16T02:05:52.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.768+0000 7fb945ffb640 1 -- 172.21.15.105:0/2897089704 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb934003d00 con 0x7fb948101640 2024-09-16T02:05:52.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.768+0000 7fb94f12c640 1 -- 172.21.15.105:0/2897089704 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9481a02b0 con 0x7fb948101640 2024-09-16T02:05:52.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.769+0000 7fb94f12c640 1 -- 172.21.15.105:0/2897089704 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9481a04e0 con 0x7fb948101640 2024-09-16T02:05:52.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.770+0000 7fb945ffb640 1 -- 172.21.15.105:0/2897089704 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fb934003ea0 con 0x7fb948101640 2024-09-16T02:05:52.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.771+0000 7fb945ffb640 1 --2- 172.21.15.105:0/2897089704 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb9380a3250 0x7fb9380a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:52.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.771+0000 7fb94cec8640 1 --2- 172.21.15.105:0/2897089704 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb9380a3250 0x7fb9380a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:52.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.771+0000 7fb945ffb640 1 -- 172.21.15.105:0/2897089704 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fb9340cf670 con 0x7fb948101640 2024-09-16T02:05:52.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.772+0000 7fb94cec8640 1 --2- 172.21.15.105:0/2897089704 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb9380a3250 0x7fb9380a5710 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb93c004770 tx=0x7fb93c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:52.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.774+0000 7fb945ffb640 1 -- 172.21.15.105:0/2897089704 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb93409c330 con 0x7fb948101640 2024-09-16T02:05:52.863 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:52 smithi190 ceph-mon[34762]: pgmap v439: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.0 KiB/s wr, 225 op/s 2024-09-16T02:05:52.863 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:52 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/1170118385' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:05:52.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:52 smithi137 ceph-mon[34099]: pgmap v439: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.0 KiB/s wr, 225 op/s 2024-09-16T02:05:52.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:52 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/1170118385' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:05:52.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.941+0000 7fb94f12c640 1 -- 172.21.15.105:0/2897089704 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb9481a06b0 con 0x7fb948101640 2024-09-16T02:05:52.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.941+0000 7fb945ffb640 1 -- 172.21.15.105:0/2897089704 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) ==== 104+0+11 (secure 0 0 0) 0x7fb934009de0 con 0x7fb948101640 2024-09-16T02:05:52.943 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-09-16T02:05:52.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.943+0000 7fb94f12c640 1 -- 172.21.15.105:0/2897089704 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb9380a3250 msgr2=0x7fb9380a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:52.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.943+0000 7fb94f12c640 1 --2- 172.21.15.105:0/2897089704 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb9380a3250 0x7fb9380a5710 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb93c004770 tx=0x7fb93c000bc0 comp rx=0 tx=0).stop 2024-09-16T02:05:52.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.943+0000 7fb94f12c640 1 -- 172.21.15.105:0/2897089704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb948101640 msgr2=0x7fb948198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:52.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.943+0000 7fb94f12c640 1 --2- 172.21.15.105:0/2897089704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb948101640 0x7fb948198370 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb93400d770 tx=0x7fb93400dc30 comp rx=0 tx=0).stop 2024-09-16T02:05:52.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.943+0000 7fb94f12c640 1 -- 172.21.15.105:0/2897089704 shutdown_connections 2024-09-16T02:05:52.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.943+0000 7fb94f12c640 1 --2- 172.21.15.105:0/2897089704 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb9380a3250 0x7fb9380a5710 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:52.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.943+0000 7fb94f12c640 1 --2- 172.21.15.105:0/2897089704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb948102030 0x7fb94819f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:52.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.943+0000 7fb94f12c640 1 --2- 172.21.15.105:0/2897089704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb948101640 0x7fb948198370 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:52.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.943+0000 7fb94f12c640 1 --2- 172.21.15.105:0/2897089704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb948100c80 0x7fb948197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:52.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.943+0000 7fb94f12c640 1 -- 172.21.15.105:0/2897089704 >> 172.21.15.105:0/2897089704 conn(0x7fb9480fb900 msgr2=0x7fb9480ffb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:52.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.943+0000 7fb94f12c640 1 -- 172.21.15.105:0/2897089704 shutdown_connections 2024-09-16T02:05:52.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:52.943+0000 7fb94f12c640 1 -- 172.21.15.105:0/2897089704 wait complete. 2024-09-16T02:05:52.956 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-16T02:05:53.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.057+0000 7fa09147b640 1 -- 172.21.15.105:0/2949836933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa08c102600 msgr2=0x7fa08c1049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:53.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.057+0000 7fa09147b640 1 --2- 172.21.15.105:0/2949836933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa08c102600 0x7fa08c1049f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa07c009900 tx=0x7fa07c02a410 comp rx=0 tx=0).stop 2024-09-16T02:05:53.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.057+0000 7fa09147b640 1 -- 172.21.15.105:0/2949836933 shutdown_connections 2024-09-16T02:05:53.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.057+0000 7fa09147b640 1 --2- 172.21.15.105:0/2949836933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa08c107860 0x7fa08c109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:53.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.057+0000 7fa09147b640 1 --2- 172.21.15.105:0/2949836933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa08c104f30 0x7fa08c107320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:53.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.057+0000 7fa09147b640 1 --2- 172.21.15.105:0/2949836933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa08c102600 0x7fa08c1049f0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:53.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.057+0000 7fa09147b640 1 -- 172.21.15.105:0/2949836933 >> 172.21.15.105:0/2949836933 conn(0x7fa08c0fc180 msgr2=0x7fa08c0fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:53.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.057+0000 7fa09147b640 1 -- 172.21.15.105:0/2949836933 shutdown_connections 2024-09-16T02:05:53.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.057+0000 7fa09147b640 1 -- 172.21.15.105:0/2949836933 wait complete. 2024-09-16T02:05:53.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.058+0000 7fa09147b640 1 Processor -- start 2024-09-16T02:05:53.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.058+0000 7fa09147b640 1 -- start start 2024-09-16T02:05:53.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.058+0000 7fa09147b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa08c102600 0x7fa08c197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:53.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.058+0000 7fa09147b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa08c104f30 0x7fa08c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:53.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.058+0000 7fa09147b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa08c107860 0x7fa08c19f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:53.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.058+0000 7fa09147b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa08c10d6f0 con 0x7fa08c104f30 2024-09-16T02:05:53.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.058+0000 7fa09147b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa08c10d570 con 0x7fa08c107860 2024-09-16T02:05:53.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.058+0000 7fa09147b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa08c10d870 con 0x7fa08c102600 2024-09-16T02:05:53.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.058+0000 7fa083fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa08c104f30 0x7fa08c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:53.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.058+0000 7fa083fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa08c104f30 0x7fa08c1983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43134/0 (socket says 172.21.15.105:43134) 2024-09-16T02:05:53.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.058+0000 7fa083fff640 1 -- 172.21.15.105:0/4017115349 learned_addr learned my addr 172.21.15.105:0/4017115349 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:53.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.058+0000 7fa08b7fe640 1 --2- 172.21.15.105:0/4017115349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa08c107860 0x7fa08c19f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:53.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.058+0000 7fa083fff640 1 -- 172.21.15.105:0/4017115349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa08c102600 msgr2=0x7fa08c197e60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T02:05:53.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.058+0000 7fa083fff640 1 --2- 172.21.15.105:0/4017115349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa08c102600 0x7fa08c197e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:53.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.058+0000 7fa083fff640 1 -- 172.21.15.105:0/4017115349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa08c107860 msgr2=0x7fa08c19f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:53.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.058+0000 7fa083fff640 1 --2- 172.21.15.105:0/4017115349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa08c107860 0x7fa08c19f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:53.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.058+0000 7fa083fff640 1 -- 172.21.15.105:0/4017115349 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa08c19fa80 con 0x7fa08c104f30 2024-09-16T02:05:53.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.058+0000 7fa08b7fe640 1 --2- 172.21.15.105:0/4017115349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa08c107860 0x7fa08c19f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:05:53.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.058+0000 7fa083fff640 1 --2- 172.21.15.105:0/4017115349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa08c104f30 0x7fa08c1983a0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fa074005fd0 tx=0x7fa07400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:53.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.059+0000 7fa088ff9640 1 -- 172.21.15.105:0/4017115349 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa074017070 con 0x7fa08c104f30 2024-09-16T02:05:53.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.059+0000 7fa088ff9640 1 -- 172.21.15.105:0/4017115349 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa074009c30 con 0x7fa08c104f30 2024-09-16T02:05:53.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.059+0000 7fa09147b640 1 -- 172.21.15.105:0/4017115349 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa08c19fd70 con 0x7fa08c104f30 2024-09-16T02:05:53.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.059+0000 7fa088ff9640 1 -- 172.21.15.105:0/4017115349 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa074003d40 con 0x7fa08c104f30 2024-09-16T02:05:53.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.059+0000 7fa09147b640 1 -- 172.21.15.105:0/4017115349 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa08c1a0260 con 0x7fa08c104f30 2024-09-16T02:05:53.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.060+0000 7fa09147b640 1 -- 172.21.15.105:0/4017115349 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa08c0fdc70 con 0x7fa08c104f30 2024-09-16T02:05:53.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.062+0000 7fa088ff9640 1 -- 172.21.15.105:0/4017115349 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fa074003ee0 con 0x7fa08c104f30 2024-09-16T02:05:53.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.062+0000 7fa088ff9640 1 --2- 172.21.15.105:0/4017115349 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa0780a3250 0x7fa0780a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:53.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.063+0000 7fa08affd640 1 --2- 172.21.15.105:0/4017115349 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa0780a3250 0x7fa0780a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:53.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.063+0000 7fa088ff9640 1 -- 172.21.15.105:0/4017115349 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fa0740cff30 con 0x7fa08c104f30 2024-09-16T02:05:53.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.063+0000 7fa08affd640 1 --2- 172.21.15.105:0/4017115349 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa0780a3250 0x7fa0780a5710 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa07c0096c0 tx=0x7fa07c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:53.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.065+0000 7fa088ff9640 1 -- 172.21.15.105:0/4017115349 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa0740ff1b0 con 0x7fa08c104f30 2024-09-16T02:05:53.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.233+0000 7fa09147b640 1 -- 172.21.15.105:0/4017115349 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa08c02d950 con 0x7fa08c104f30 2024-09-16T02:05:53.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.233+0000 7fa088ff9640 1 -- 172.21.15.105:0/4017115349 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) ==== 100+0+11 (secure 0 0 0) 0x7fa074020030 con 0x7fa08c104f30 2024-09-16T02:05:53.236 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-09-16T02:05:53.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.235+0000 7fa09147b640 1 -- 172.21.15.105:0/4017115349 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa0780a3250 msgr2=0x7fa0780a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:53.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.235+0000 7fa09147b640 1 --2- 172.21.15.105:0/4017115349 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa0780a3250 0x7fa0780a5710 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa07c0096c0 tx=0x7fa07c005fd0 comp rx=0 tx=0).stop 2024-09-16T02:05:53.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.235+0000 7fa09147b640 1 -- 172.21.15.105:0/4017115349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa08c104f30 msgr2=0x7fa08c1983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:53.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.235+0000 7fa09147b640 1 --2- 172.21.15.105:0/4017115349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa08c104f30 0x7fa08c1983a0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fa074005fd0 tx=0x7fa07400dde0 comp rx=0 tx=0).stop 2024-09-16T02:05:53.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.236+0000 7fa09147b640 1 -- 172.21.15.105:0/4017115349 shutdown_connections 2024-09-16T02:05:53.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.236+0000 7fa09147b640 1 --2- 172.21.15.105:0/4017115349 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fa0780a3250 0x7fa0780a5710 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:53.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.236+0000 7fa09147b640 1 --2- 172.21.15.105:0/4017115349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa08c107860 0x7fa08c19f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:53.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.236+0000 7fa09147b640 1 --2- 172.21.15.105:0/4017115349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa08c104f30 0x7fa08c1983a0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:53.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.236+0000 7fa09147b640 1 --2- 172.21.15.105:0/4017115349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa08c102600 0x7fa08c197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:53.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.236+0000 7fa09147b640 1 -- 172.21.15.105:0/4017115349 >> 172.21.15.105:0/4017115349 conn(0x7fa08c0fc180 msgr2=0x7fa08c103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:53.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.236+0000 7fa09147b640 1 -- 172.21.15.105:0/4017115349 shutdown_connections 2024-09-16T02:05:53.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.236+0000 7fa09147b640 1 -- 172.21.15.105:0/4017115349 wait complete. 2024-09-16T02:05:53.249 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-16T02:05:53.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.346+0000 7f8d6c749640 1 -- 172.21.15.105:0/253510433 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d64100a10 msgr2=0x7f8d64100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:53.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.346+0000 7f8d6c749640 1 --2- 172.21.15.105:0/253510433 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d64100a10 0x7f8d64100e10 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009900 tx=0x7f8d6002a410 comp rx=0 tx=0).stop 2024-09-16T02:05:53.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.347+0000 7f8d6c749640 1 -- 172.21.15.105:0/253510433 shutdown_connections 2024-09-16T02:05:53.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.347+0000 7f8d6c749640 1 --2- 172.21.15.105:0/253510433 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d64101c00 0x7f8d64102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:53.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.347+0000 7f8d6c749640 1 --2- 172.21.15.105:0/253510433 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d641025c0 0x7f8d64108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:53.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.347+0000 7f8d6c749640 1 --2- 172.21.15.105:0/253510433 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d64100a10 0x7f8d64100e10 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:53.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.347+0000 7f8d6c749640 1 -- 172.21.15.105:0/253510433 >> 172.21.15.105:0/253510433 conn(0x7f8d640fc170 msgr2=0x7f8d640fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:53.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.347+0000 7f8d6c749640 1 -- 172.21.15.105:0/253510433 shutdown_connections 2024-09-16T02:05:53.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.347+0000 7f8d6c749640 1 -- 172.21.15.105:0/253510433 wait complete. 2024-09-16T02:05:53.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.347+0000 7f8d6c749640 1 Processor -- start 2024-09-16T02:05:53.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.347+0000 7f8d6c749640 1 -- start start 2024-09-16T02:05:53.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.348+0000 7f8d6c749640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d64100a10 0x7f8d641049a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:53.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.348+0000 7f8d6c749640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d64101c00 0x7f8d64102fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:53.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.348+0000 7f8d6c749640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d641025c0 0x7f8d64103520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:53.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.348+0000 7f8d6c749640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6410c520 con 0x7f8d641025c0 2024-09-16T02:05:53.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.348+0000 7f8d6c749640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6410c3a0 con 0x7f8d64101c00 2024-09-16T02:05:53.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.348+0000 7f8d6c749640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6410c6a0 con 0x7f8d64100a10 2024-09-16T02:05:53.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.348+0000 7f8d6ace6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d641025c0 0x7f8d64103520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:53.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.348+0000 7f8d6ace6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d641025c0 0x7f8d64103520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43150/0 (socket says 172.21.15.105:43150) 2024-09-16T02:05:53.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.348+0000 7f8d6ace6640 1 -- 172.21.15.105:0/4022527764 learned_addr learned my addr 172.21.15.105:0/4022527764 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:05:53.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.348+0000 7f8d6a4e5640 1 --2- 172.21.15.105:0/4022527764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d64100a10 0x7f8d641049a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:53.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.348+0000 7f8d69ce4640 1 --2- 172.21.15.105:0/4022527764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d64101c00 0x7f8d64102fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:53.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.348+0000 7f8d6ace6640 1 -- 172.21.15.105:0/4022527764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d64100a10 msgr2=0x7f8d641049a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:53.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.348+0000 7f8d6ace6640 1 --2- 172.21.15.105:0/4022527764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d64100a10 0x7f8d641049a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:53.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.348+0000 7f8d6ace6640 1 -- 172.21.15.105:0/4022527764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d64101c00 msgr2=0x7f8d64102fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:53.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.348+0000 7f8d6ace6640 1 --2- 172.21.15.105:0/4022527764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d64101c00 0x7f8d64102fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:53.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.348+0000 7f8d6ace6640 1 -- 172.21.15.105:0/4022527764 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d64103db0 con 0x7f8d641025c0 2024-09-16T02:05:53.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.348+0000 7f8d6ace6640 1 --2- 172.21.15.105:0/4022527764 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d641025c0 0x7f8d64103520 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c00eaa0 tx=0x7f8d5c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:53.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.348+0000 7f8d5b7fe640 1 -- 172.21.15.105:0/4022527764 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c00cdb0 con 0x7f8d641025c0 2024-09-16T02:05:53.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.348+0000 7f8d5b7fe640 1 -- 172.21.15.105:0/4022527764 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d5c005020 con 0x7f8d641025c0 2024-09-16T02:05:53.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.348+0000 7f8d5b7fe640 1 -- 172.21.15.105:0/4022527764 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c0116d0 con 0x7f8d641025c0 2024-09-16T02:05:53.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.348+0000 7f8d6c749640 1 -- 172.21.15.105:0/4022527764 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d641a1e80 con 0x7f8d641025c0 2024-09-16T02:05:53.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.348+0000 7f8d6c749640 1 -- 172.21.15.105:0/4022527764 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d641a2290 con 0x7f8d641025c0 2024-09-16T02:05:53.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.349+0000 7f8d6c749640 1 -- 172.21.15.105:0/4022527764 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d641a25b0 con 0x7f8d641025c0 2024-09-16T02:05:53.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.351+0000 7f8d5b7fe640 1 -- 172.21.15.105:0/4022527764 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f8d5c0051d0 con 0x7f8d641025c0 2024-09-16T02:05:53.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.352+0000 7f8d5b7fe640 1 --2- 172.21.15.105:0/4022527764 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f8d500a3250 0x7f8d500a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:53.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.352+0000 7f8d6a4e5640 1 --2- 172.21.15.105:0/4022527764 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f8d500a3250 0x7f8d500a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:53.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.352+0000 7f8d5b7fe640 1 -- 172.21.15.105:0/4022527764 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f8d5c0cff10 con 0x7f8d641025c0 2024-09-16T02:05:53.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.352+0000 7f8d6a4e5640 1 --2- 172.21.15.105:0/4022527764 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f8d500a3250 0x7f8d500a5710 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8d60004770 tx=0x7f8d60000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:53.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.355+0000 7f8d5b7fe640 1 -- 172.21.15.105:0/4022527764 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8d5c016030 con 0x7f8d641025c0 2024-09-16T02:05:53.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.568+0000 7f8d6c749640 1 -- 172.21.15.105:0/4022527764 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f8d641a2940 con 0x7f8d641025c0 2024-09-16T02:05:53.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.575+0000 7f8d5b7fe640 1 -- 172.21.15.105:0/4022527764 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+742346 (secure 0 0 0) 0x7f8d5c09cbd0 con 0x7f8d641025c0 2024-09-16T02:05:53.578 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:05:53.578 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-16T02:05:53.578 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 18, 2024-09-16T02:05:53.579 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.579 INFO:teuthology.orchestra.run.smithi105.stdout: "active_gid": 14150, 2024-09-16T02:05:53.579 INFO:teuthology.orchestra.run.smithi105.stdout: "active_name": "x", 2024-09-16T02:05:53.579 INFO:teuthology.orchestra.run.smithi105.stdout: "active_addrs": { 2024-09-16T02:05:53.579 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-16T02:05:53.579 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:53.579 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-16T02:05:53.579 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6800", 2024-09-16T02:05:53.579 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 275626334 2024-09-16T02:05:53.579 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.579 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:53.579 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-16T02:05:53.579 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6801", 2024-09-16T02:05:53.579 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 275626334 2024-09-16T02:05:53.580 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.580 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-16T02:05:53.580 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.580 INFO:teuthology.orchestra.run.smithi105.stdout: "active_addr": "172.21.15.105:6801/275626334", 2024-09-16T02:05:53.580 INFO:teuthology.orchestra.run.smithi105.stdout: "active_change": "2024-09-16T01:53:39.351418+0000", 2024-09-16T02:05:53.580 INFO:teuthology.orchestra.run.smithi105.stdout: "active_mgr_features": 4540701547738038271, 2024-09-16T02:05:53.580 INFO:teuthology.orchestra.run.smithi105.stdout: "available": true, 2024-09-16T02:05:53.580 INFO:teuthology.orchestra.run.smithi105.stdout: "standbys": [ 2024-09-16T02:05:53.580 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:53.580 INFO:teuthology.orchestra.run.smithi105.stdout: "gid": 24109, 2024-09-16T02:05:53.580 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "z", 2024-09-16T02:05:53.580 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr_features": 4540701547738038271, 2024-09-16T02:05:53.580 INFO:teuthology.orchestra.run.smithi105.stdout: "available_modules": [ 2024-09-16T02:05:53.580 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:53.581 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "alerts", 2024-09-16T02:05:53.581 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:53.581 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:53.581 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:53.581 INFO:teuthology.orchestra.run.smithi105.stdout: "interval": { 2024-09-16T02:05:53.581 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "interval", 2024-09-16T02:05:53.581 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:53.581 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.581 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.582 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "60", 2024-09-16T02:05:53.582 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.582 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.582 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.582 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "How frequently to reexamine health status", 2024-09-16T02:05:53.582 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.582 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.583 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.583 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.583 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:53.583 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:53.583 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.583 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.583 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.583 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.583 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.583 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.583 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.583 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.583 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.583 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.584 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.584 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.584 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.584 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.584 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.584 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.584 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.584 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.584 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.584 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:53.584 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:53.584 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.585 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.585 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.585 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.585 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.585 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.585 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.585 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.585 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.585 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.585 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.585 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.585 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:53.585 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:53.585 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.586 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.586 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.586 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:53.586 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.586 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.586 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.586 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.586 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.586 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.586 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.586 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.586 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.586 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.586 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.587 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.587 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.587 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.587 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.587 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:53.587 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:53.588 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.588 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.588 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.588 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.588 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.588 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.588 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.588 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.588 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.588 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.588 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.589 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.589 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_destination": { 2024-09-16T02:05:53.589 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_destination", 2024-09-16T02:05:53.589 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.589 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.589 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.589 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.589 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.589 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.589 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.589 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Email address to send alerts to", 2024-09-16T02:05:53.589 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.589 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.590 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.590 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.590 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_from_name": { 2024-09-16T02:05:53.590 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_from_name", 2024-09-16T02:05:53.590 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.590 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.590 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.590 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "Ceph", 2024-09-16T02:05:53.590 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.590 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.590 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.590 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Email From: name", 2024-09-16T02:05:53.590 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.590 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.590 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.591 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.591 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_host": { 2024-09-16T02:05:53.591 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_host", 2024-09-16T02:05:53.591 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.591 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.591 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.591 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.591 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.591 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.591 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.591 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "SMTP server", 2024-09-16T02:05:53.591 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.591 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.591 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.591 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.592 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_password": { 2024-09-16T02:05:53.592 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_password", 2024-09-16T02:05:53.592 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.592 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.592 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.592 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.592 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.592 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.592 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.592 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Password to authenticate with", 2024-09-16T02:05:53.592 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.592 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.592 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.592 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.593 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_port": { 2024-09-16T02:05:53.593 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_port", 2024-09-16T02:05:53.593 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:53.593 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.593 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.593 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "465", 2024-09-16T02:05:53.593 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.593 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.593 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.593 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "SMTP port", 2024-09-16T02:05:53.593 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.593 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.593 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.594 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.594 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_sender": { 2024-09-16T02:05:53.594 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_sender", 2024-09-16T02:05:53.594 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.594 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.594 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.594 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.594 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.594 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.595 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.595 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "SMTP envelope sender", 2024-09-16T02:05:53.595 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.595 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.595 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.595 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.595 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_ssl": { 2024-09-16T02:05:53.595 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_ssl", 2024-09-16T02:05:53.595 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.595 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.595 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.595 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:53.595 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.596 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.596 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.596 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-16T02:05:53.596 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.596 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.596 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.596 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.596 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_user": { 2024-09-16T02:05:53.596 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_user", 2024-09-16T02:05:53.596 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.596 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.596 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.597 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.597 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.597 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.597 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.597 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "User to authenticate as", 2024-09-16T02:05:53.597 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.597 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.597 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.597 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.597 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.597 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.598 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:53.598 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "balancer", 2024-09-16T02:05:53.598 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:53.598 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:53.598 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:53.598 INFO:teuthology.orchestra.run.smithi105.stdout: "active": { 2024-09-16T02:05:53.598 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "active", 2024-09-16T02:05:53.598 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.598 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.599 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.599 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:53.599 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.599 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.599 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.599 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "automatically balance PGs across cluster", 2024-09-16T02:05:53.599 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.599 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.599 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.599 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.599 INFO:teuthology.orchestra.run.smithi105.stdout: "begin_time": { 2024-09-16T02:05:53.599 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "begin_time", 2024-09-16T02:05:53.599 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.599 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.600 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.600 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0000", 2024-09-16T02:05:53.600 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.600 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.600 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.600 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "beginning time of day to automatically balance", 2024-09-16T02:05:53.600 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T02:05:53.600 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.600 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.600 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.600 INFO:teuthology.orchestra.run.smithi105.stdout: "begin_weekday": { 2024-09-16T02:05:53.600 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "begin_weekday", 2024-09-16T02:05:53.600 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "uint", 2024-09-16T02:05:53.600 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.601 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.601 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0", 2024-09-16T02:05:53.601 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "0", 2024-09-16T02:05:53.601 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "6", 2024-09-16T02:05:53.601 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.601 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-16T02:05:53.601 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T02:05:53.601 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.602 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.602 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.602 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_compat_max_iterations": { 2024-09-16T02:05:53.602 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "crush_compat_max_iterations", 2024-09-16T02:05:53.602 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "uint", 2024-09-16T02:05:53.602 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.602 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.602 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "25", 2024-09-16T02:05:53.602 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "1", 2024-09-16T02:05:53.602 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "250", 2024-09-16T02:05:53.602 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.602 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-16T02:05:53.602 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.602 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.603 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.603 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.603 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_compat_metrics": { 2024-09-16T02:05:53.603 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "crush_compat_metrics", 2024-09-16T02:05:53.603 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.603 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.603 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.603 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "pgs,objects,bytes", 2024-09-16T02:05:53.603 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.603 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.603 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.603 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-16T02:05:53.603 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-16T02:05:53.604 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.604 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.604 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.604 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_compat_step": { 2024-09-16T02:05:53.604 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "crush_compat_step", 2024-09-16T02:05:53.604 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-09-16T02:05:53.604 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.604 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.604 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0.5", 2024-09-16T02:05:53.604 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "0.001", 2024-09-16T02:05:53.604 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "0.999", 2024-09-16T02:05:53.604 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.604 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "aggressiveness of optimization", 2024-09-16T02:05:53.604 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-16T02:05:53.605 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.605 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.605 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.605 INFO:teuthology.orchestra.run.smithi105.stdout: "end_time": { 2024-09-16T02:05:53.605 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "end_time", 2024-09-16T02:05:53.605 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.605 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.605 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.605 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "2359", 2024-09-16T02:05:53.605 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.605 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.606 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.606 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "ending time of day to automatically balance", 2024-09-16T02:05:53.606 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T02:05:53.606 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.606 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.606 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.606 INFO:teuthology.orchestra.run.smithi105.stdout: "end_weekday": { 2024-09-16T02:05:53.606 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "end_weekday", 2024-09-16T02:05:53.606 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "uint", 2024-09-16T02:05:53.606 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.606 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.606 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0", 2024-09-16T02:05:53.606 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "0", 2024-09-16T02:05:53.606 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "6", 2024-09-16T02:05:53.607 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.607 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-16T02:05:53.607 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T02:05:53.607 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.607 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.607 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.607 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:53.607 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:53.607 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.607 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.607 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.607 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.607 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.607 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.607 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.608 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.608 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.608 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.608 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.608 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.608 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.608 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.609 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.609 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.609 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.609 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.609 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.609 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:53.609 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:53.609 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.609 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.609 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.616 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.616 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.616 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.616 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.617 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.617 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.617 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.617 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.617 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.617 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:53.617 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:53.617 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.617 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.617 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.617 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:53.617 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.618 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.618 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.618 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.618 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.618 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.618 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.618 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.618 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.619 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.619 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.619 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.619 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.619 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.619 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.619 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:53.619 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:53.619 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.619 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.619 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.620 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.620 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.620 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.620 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.620 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.620 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.620 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.620 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.620 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.620 INFO:teuthology.orchestra.run.smithi105.stdout: "min_score": { 2024-09-16T02:05:53.620 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "min_score", 2024-09-16T02:05:53.620 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-09-16T02:05:53.620 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.620 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.621 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0", 2024-09-16T02:05:53.621 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.621 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.621 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.621 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-16T02:05:53.621 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.621 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.621 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.621 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.621 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": { 2024-09-16T02:05:53.621 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "mode", 2024-09-16T02:05:53.621 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.621 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.621 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.622 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "upmap", 2024-09-16T02:05:53.622 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.622 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.622 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.622 INFO:teuthology.orchestra.run.smithi105.stdout: "crush-compat", 2024-09-16T02:05:53.622 INFO:teuthology.orchestra.run.smithi105.stdout: "none", 2024-09-16T02:05:53.622 INFO:teuthology.orchestra.run.smithi105.stdout: "read", 2024-09-16T02:05:53.623 INFO:teuthology.orchestra.run.smithi105.stdout: "upmap", 2024-09-16T02:05:53.623 INFO:teuthology.orchestra.run.smithi105.stdout: "upmap-read" 2024-09-16T02:05:53.623 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.623 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Balancer mode", 2024-09-16T02:05:53.623 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.623 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.623 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.623 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.623 INFO:teuthology.orchestra.run.smithi105.stdout: "pool_ids": { 2024-09-16T02:05:53.623 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "pool_ids", 2024-09-16T02:05:53.623 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.623 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.624 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.624 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.624 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.624 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.624 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.625 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-16T02:05:53.625 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.625 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.626 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.626 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.626 INFO:teuthology.orchestra.run.smithi105.stdout: "sleep_interval": { 2024-09-16T02:05:53.626 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "sleep_interval", 2024-09-16T02:05:53.626 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:53.626 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.626 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.626 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "60", 2024-09-16T02:05:53.626 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.626 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.626 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.626 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-16T02:05:53.626 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.627 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.627 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.627 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.627 INFO:teuthology.orchestra.run.smithi105.stdout: "upmap_max_deviation": { 2024-09-16T02:05:53.627 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "upmap_max_deviation", 2024-09-16T02:05:53.627 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:53.627 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.628 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.628 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "5", 2024-09-16T02:05:53.628 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "1", 2024-09-16T02:05:53.628 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.628 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.628 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-16T02:05:53.628 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-16T02:05:53.628 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.628 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.628 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.628 INFO:teuthology.orchestra.run.smithi105.stdout: "upmap_max_optimizations": { 2024-09-16T02:05:53.628 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "upmap_max_optimizations", 2024-09-16T02:05:53.629 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "uint", 2024-09-16T02:05:53.629 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.629 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.629 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-09-16T02:05:53.629 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.629 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.629 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.629 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-16T02:05:53.629 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.629 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.629 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.629 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.630 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.630 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.630 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:53.630 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "cephadm", 2024-09-16T02:05:53.630 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:53.630 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:53.630 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:53.630 INFO:teuthology.orchestra.run.smithi105.stdout: "agent_down_multiplier": { 2024-09-16T02:05:53.630 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "agent_down_multiplier", 2024-09-16T02:05:53.630 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-09-16T02:05:53.630 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.630 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.630 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "3.0", 2024-09-16T02:05:53.631 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.631 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.631 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.631 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-16T02:05:53.631 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.631 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.631 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.631 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.631 INFO:teuthology.orchestra.run.smithi105.stdout: "agent_refresh_rate": { 2024-09-16T02:05:53.631 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "agent_refresh_rate", 2024-09-16T02:05:53.631 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:53.631 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.631 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.631 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "20", 2024-09-16T02:05:53.632 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.632 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.632 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.632 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-16T02:05:53.632 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.632 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.632 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.632 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.632 INFO:teuthology.orchestra.run.smithi105.stdout: "agent_starting_port": { 2024-09-16T02:05:53.632 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "agent_starting_port", 2024-09-16T02:05:53.632 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:53.632 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.632 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.633 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "4721", 2024-09-16T02:05:53.633 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.633 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.633 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.633 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-16T02:05:53.633 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.633 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.633 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.633 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.633 INFO:teuthology.orchestra.run.smithi105.stdout: "allow_ptrace": { 2024-09-16T02:05:53.633 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "allow_ptrace", 2024-09-16T02:05:53.633 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.634 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.634 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.634 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.634 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.634 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.634 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.634 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-16T02:05:53.634 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-16T02:05:53.634 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.634 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.635 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.635 INFO:teuthology.orchestra.run.smithi105.stdout: "autotune_interval": { 2024-09-16T02:05:53.635 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "autotune_interval", 2024-09-16T02:05:53.635 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:53.635 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.635 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.635 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "600", 2024-09-16T02:05:53.635 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.635 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.635 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.636 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-16T02:05:53.636 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.636 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.636 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.636 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.637 INFO:teuthology.orchestra.run.smithi105.stdout: "autotune_memory_target_ratio": { 2024-09-16T02:05:53.637 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "autotune_memory_target_ratio", 2024-09-16T02:05:53.637 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-09-16T02:05:53.637 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.637 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.637 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0.7", 2024-09-16T02:05:53.637 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.637 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.637 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.637 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-16T02:05:53.637 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.638 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.638 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.638 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.638 INFO:teuthology.orchestra.run.smithi105.stdout: "cephadm_log_destination": { 2024-09-16T02:05:53.639 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "cephadm_log_destination", 2024-09-16T02:05:53.639 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.639 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.639 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.639 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.639 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.640 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.640 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.640 INFO:teuthology.orchestra.run.smithi105.stdout: "file", 2024-09-16T02:05:53.640 INFO:teuthology.orchestra.run.smithi105.stdout: "file,syslog", 2024-09-16T02:05:53.640 INFO:teuthology.orchestra.run.smithi105.stdout: "syslog" 2024-09-16T02:05:53.640 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.640 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-16T02:05:53.640 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.640 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.641 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.641 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.641 INFO:teuthology.orchestra.run.smithi105.stdout: "cgroups_split": { 2024-09-16T02:05:53.641 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "cgroups_split", 2024-09-16T02:05:53.641 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.641 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.641 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.641 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:53.641 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.641 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.641 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.641 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-16T02:05:53.641 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.642 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.642 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.642 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.642 INFO:teuthology.orchestra.run.smithi105.stdout: "config_checks_enabled": { 2024-09-16T02:05:53.642 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "config_checks_enabled", 2024-09-16T02:05:53.642 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.642 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.642 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.642 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.642 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.642 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.642 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.642 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-16T02:05:53.643 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.643 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.643 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.643 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.643 INFO:teuthology.orchestra.run.smithi105.stdout: "config_dashboard": { 2024-09-16T02:05:53.643 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "config_dashboard", 2024-09-16T02:05:53.643 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.643 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.644 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.644 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:53.644 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.644 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.644 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.644 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-16T02:05:53.644 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.644 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.644 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.644 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.644 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_alertmanager": { 2024-09-16T02:05:53.645 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_alertmanager", 2024-09-16T02:05:53.645 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.645 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.645 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.645 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-16T02:05:53.645 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.645 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.645 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.645 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Prometheus container image", 2024-09-16T02:05:53.645 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.645 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.645 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.645 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.646 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_base": { 2024-09-16T02:05:53.646 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_base", 2024-09-16T02:05:53.646 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.646 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.646 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.646 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-16T02:05:53.646 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.646 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.646 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.646 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Container image name, without the tag", 2024-09-16T02:05:53.646 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.646 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.646 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.646 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.647 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_elasticsearch": { 2024-09-16T02:05:53.647 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_elasticsearch", 2024-09-16T02:05:53.647 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.647 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.647 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.647 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-16T02:05:53.647 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.647 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.647 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.647 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "elasticsearch container image", 2024-09-16T02:05:53.647 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.647 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.647 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.647 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.648 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_grafana": { 2024-09-16T02:05:53.648 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_grafana", 2024-09-16T02:05:53.648 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.648 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.648 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.648 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-16T02:05:53.648 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.648 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.648 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.648 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Prometheus container image", 2024-09-16T02:05:53.648 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.648 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.648 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.648 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.649 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_haproxy": { 2024-09-16T02:05:53.649 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_haproxy", 2024-09-16T02:05:53.649 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.649 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.649 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.649 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-16T02:05:53.649 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.649 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.649 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.649 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "HAproxy container image", 2024-09-16T02:05:53.649 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.649 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.650 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.650 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.650 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_jaeger_agent": { 2024-09-16T02:05:53.650 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_jaeger_agent", 2024-09-16T02:05:53.650 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.650 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.650 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.650 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-16T02:05:53.650 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.650 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.650 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.650 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Jaeger agent container image", 2024-09-16T02:05:53.650 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.650 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.651 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.651 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.651 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_jaeger_collector": { 2024-09-16T02:05:53.651 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_jaeger_collector", 2024-09-16T02:05:53.651 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.651 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.651 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.651 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-16T02:05:53.651 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.651 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.651 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.651 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Jaeger collector container image", 2024-09-16T02:05:53.651 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.651 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.652 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.652 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.652 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_jaeger_query": { 2024-09-16T02:05:53.652 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_jaeger_query", 2024-09-16T02:05:53.652 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.652 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.652 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.652 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-16T02:05:53.652 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.652 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.652 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.652 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Jaeger query container image", 2024-09-16T02:05:53.652 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.653 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.653 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.653 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.653 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_keepalived": { 2024-09-16T02:05:53.653 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_keepalived", 2024-09-16T02:05:53.653 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.653 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.653 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.653 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-16T02:05:53.653 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.653 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.653 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.653 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Keepalived container image", 2024-09-16T02:05:53.654 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.654 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.654 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.654 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.654 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_loki": { 2024-09-16T02:05:53.654 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_loki", 2024-09-16T02:05:53.654 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.654 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.654 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.654 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-16T02:05:53.654 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.654 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.654 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.654 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Loki container image", 2024-09-16T02:05:53.655 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.655 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.655 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.655 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.655 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_node_exporter": { 2024-09-16T02:05:53.655 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_node_exporter", 2024-09-16T02:05:53.655 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.655 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.655 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.655 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-16T02:05:53.655 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.655 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.655 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.656 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Prometheus container image", 2024-09-16T02:05:53.656 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.656 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.656 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.656 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.656 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_nvmeof": { 2024-09-16T02:05:53.656 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_nvmeof", 2024-09-16T02:05:53.656 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.656 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.656 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.656 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-16T02:05:53.656 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.656 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.656 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.657 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Nvme-of container image", 2024-09-16T02:05:53.657 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.657 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.657 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.657 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.657 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_prometheus": { 2024-09-16T02:05:53.657 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_prometheus", 2024-09-16T02:05:53.657 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.657 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.657 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.657 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-16T02:05:53.657 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.657 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.657 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.658 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Prometheus container image", 2024-09-16T02:05:53.658 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.658 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.658 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.658 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.658 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_promtail": { 2024-09-16T02:05:53.658 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_promtail", 2024-09-16T02:05:53.658 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.658 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.658 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.658 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-16T02:05:53.659 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.659 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.659 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.659 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Promtail container image", 2024-09-16T02:05:53.659 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.659 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.659 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.659 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.659 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_samba": { 2024-09-16T02:05:53.659 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_samba", 2024-09-16T02:05:53.659 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.659 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.659 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.659 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-16T02:05:53.660 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.660 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.660 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.660 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Samba/SMB container image", 2024-09-16T02:05:53.660 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.660 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.660 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.660 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.660 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_snmp_gateway": { 2024-09-16T02:05:53.660 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_snmp_gateway", 2024-09-16T02:05:53.660 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.660 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.660 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.661 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-16T02:05:53.661 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.661 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.661 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.661 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "SNMP Gateway container image", 2024-09-16T02:05:53.661 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.661 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.661 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.661 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.661 INFO:teuthology.orchestra.run.smithi105.stdout: "container_init": { 2024-09-16T02:05:53.661 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_init", 2024-09-16T02:05:53.661 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.661 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.661 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.661 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:53.662 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.662 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.662 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.662 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Run podman/docker with `--init`", 2024-09-16T02:05:53.662 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.662 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.662 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.662 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.662 INFO:teuthology.orchestra.run.smithi105.stdout: "daemon_cache_timeout": { 2024-09-16T02:05:53.662 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "daemon_cache_timeout", 2024-09-16T02:05:53.662 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:53.662 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.662 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.663 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "600", 2024-09-16T02:05:53.663 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.663 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.663 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.663 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-16T02:05:53.663 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.663 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.663 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.663 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.663 INFO:teuthology.orchestra.run.smithi105.stdout: "default_cephadm_command_timeout": { 2024-09-16T02:05:53.663 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "default_cephadm_command_timeout", 2024-09-16T02:05:53.663 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:53.663 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.663 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.664 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "900", 2024-09-16T02:05:53.664 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.664 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.664 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.664 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-16T02:05:53.664 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.664 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.664 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.664 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.664 INFO:teuthology.orchestra.run.smithi105.stdout: "default_registry": { 2024-09-16T02:05:53.664 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "default_registry", 2024-09-16T02:05:53.664 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.664 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.665 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.665 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "docker.io", 2024-09-16T02:05:53.665 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.665 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.665 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.665 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-16T02:05:53.665 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.665 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.665 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.665 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.665 INFO:teuthology.orchestra.run.smithi105.stdout: "device_cache_timeout": { 2024-09-16T02:05:53.665 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "device_cache_timeout", 2024-09-16T02:05:53.665 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:53.665 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.666 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.666 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "1800", 2024-09-16T02:05:53.666 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.666 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.666 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.666 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "seconds to cache device inventory", 2024-09-16T02:05:53.666 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.666 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.666 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.666 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.666 INFO:teuthology.orchestra.run.smithi105.stdout: "device_enhanced_scan": { 2024-09-16T02:05:53.666 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "device_enhanced_scan", 2024-09-16T02:05:53.666 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.666 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.667 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.667 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.667 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.667 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.668 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.668 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-16T02:05:53.668 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.668 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.668 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.669 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.669 INFO:teuthology.orchestra.run.smithi105.stdout: "facts_cache_timeout": { 2024-09-16T02:05:53.669 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "facts_cache_timeout", 2024-09-16T02:05:53.669 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:53.669 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.669 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.669 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "60", 2024-09-16T02:05:53.669 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.669 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.670 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.670 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "seconds to cache host facts data", 2024-09-16T02:05:53.670 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.670 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.670 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.670 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.670 INFO:teuthology.orchestra.run.smithi105.stdout: "grafana_dashboards_path": { 2024-09-16T02:05:53.670 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "grafana_dashboards_path", 2024-09-16T02:05:53.670 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.670 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.670 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.670 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-16T02:05:53.671 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.671 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.671 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.671 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-16T02:05:53.671 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.671 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.671 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.671 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.671 INFO:teuthology.orchestra.run.smithi105.stdout: "host_check_interval": { 2024-09-16T02:05:53.671 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "host_check_interval", 2024-09-16T02:05:53.671 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:53.671 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.672 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.672 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "600", 2024-09-16T02:05:53.672 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.672 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.672 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.672 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how frequently to perform a host check", 2024-09-16T02:05:53.672 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.672 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.672 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.672 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.672 INFO:teuthology.orchestra.run.smithi105.stdout: "hw_monitoring": { 2024-09-16T02:05:53.672 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "hw_monitoring", 2024-09-16T02:05:53.672 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.673 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.673 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.673 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.673 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.673 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.673 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.673 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-16T02:05:53.673 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.673 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.673 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.673 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.673 INFO:teuthology.orchestra.run.smithi105.stdout: "inventory_list_all": { 2024-09-16T02:05:53.673 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "inventory_list_all", 2024-09-16T02:05:53.673 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.673 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.674 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.674 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.674 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.674 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.674 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.674 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-16T02:05:53.674 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.674 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.674 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.674 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.674 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:53.674 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:53.674 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.674 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.675 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.675 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.675 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.675 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.675 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.675 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.675 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.675 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.675 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.675 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.675 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.675 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.675 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.675 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.676 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.676 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.676 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.676 INFO:teuthology.orchestra.run.smithi105.stdout: "log_refresh_metadata": { 2024-09-16T02:05:53.676 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_refresh_metadata", 2024-09-16T02:05:53.676 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.676 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.676 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.676 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.676 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.676 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.676 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.676 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-16T02:05:53.677 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.677 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.677 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.677 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.677 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:53.677 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:53.677 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.677 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.677 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.677 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:53.677 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.677 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.677 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.678 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-16T02:05:53.678 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.678 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.678 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.678 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.678 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:53.678 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:53.678 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.678 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.678 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.678 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:53.678 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.678 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.678 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.679 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.679 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.679 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.679 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.679 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.679 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.679 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.679 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.679 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.679 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.679 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.679 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.679 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:53.680 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:53.680 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.680 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.680 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.680 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.680 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.680 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.680 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.680 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.680 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.680 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.680 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.680 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.681 INFO:teuthology.orchestra.run.smithi105.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-16T02:05:53.681 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-16T02:05:53.681 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.681 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.681 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.681 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.681 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.681 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.681 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.681 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-16T02:05:53.681 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.681 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.681 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.681 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.682 INFO:teuthology.orchestra.run.smithi105.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-16T02:05:53.682 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-16T02:05:53.682 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.682 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.682 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.682 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "*", 2024-09-16T02:05:53.682 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.682 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.682 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.682 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-16T02:05:53.682 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.682 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.682 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.683 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.683 INFO:teuthology.orchestra.run.smithi105.stdout: "max_count_per_host": { 2024-09-16T02:05:53.683 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "max_count_per_host", 2024-09-16T02:05:53.683 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:53.683 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.683 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.683 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-09-16T02:05:53.683 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.683 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.683 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.683 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "max number of daemons per service per host", 2024-09-16T02:05:53.683 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.683 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.683 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.684 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.684 INFO:teuthology.orchestra.run.smithi105.stdout: "max_osd_draining_count": { 2024-09-16T02:05:53.684 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "max_osd_draining_count", 2024-09-16T02:05:53.684 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:53.684 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.684 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.684 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-09-16T02:05:53.684 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.684 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.684 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.684 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-16T02:05:53.684 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.684 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.684 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.685 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.685 INFO:teuthology.orchestra.run.smithi105.stdout: "migration_current": { 2024-09-16T02:05:53.685 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "migration_current", 2024-09-16T02:05:53.685 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:53.685 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.685 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.685 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.685 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.685 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.685 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.685 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "internal - do not modify", 2024-09-16T02:05:53.685 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.685 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.685 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.686 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.686 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": { 2024-09-16T02:05:53.686 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "mode", 2024-09-16T02:05:53.686 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.686 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.686 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.686 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "root", 2024-09-16T02:05:53.686 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.686 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.686 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.686 INFO:teuthology.orchestra.run.smithi105.stdout: "cephadm-package", 2024-09-16T02:05:53.686 INFO:teuthology.orchestra.run.smithi105.stdout: "root" 2024-09-16T02:05:53.686 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.686 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "mode for remote execution of cephadm", 2024-09-16T02:05:53.686 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.687 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.687 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.687 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.687 INFO:teuthology.orchestra.run.smithi105.stdout: "oob_default_addr": { 2024-09-16T02:05:53.687 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "oob_default_addr", 2024-09-16T02:05:53.687 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.687 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.687 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.687 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "169.254.1.1", 2024-09-16T02:05:53.687 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.687 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.687 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.687 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-16T02:05:53.687 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.688 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.688 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.688 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.688 INFO:teuthology.orchestra.run.smithi105.stdout: "prometheus_alerts_path": { 2024-09-16T02:05:53.688 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "prometheus_alerts_path", 2024-09-16T02:05:53.688 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.688 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.688 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.688 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-16T02:05:53.688 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.688 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.688 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.688 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-16T02:05:53.688 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.689 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.689 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.689 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.689 INFO:teuthology.orchestra.run.smithi105.stdout: "registry_insecure": { 2024-09-16T02:05:53.689 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "registry_insecure", 2024-09-16T02:05:53.689 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.689 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.689 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.689 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.689 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.689 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.689 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.690 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-16T02:05:53.690 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.690 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.690 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.690 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.690 INFO:teuthology.orchestra.run.smithi105.stdout: "registry_password": { 2024-09-16T02:05:53.690 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "registry_password", 2024-09-16T02:05:53.690 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.690 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.690 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.690 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.690 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.690 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.690 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.691 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-16T02:05:53.691 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.691 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.691 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.691 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.691 INFO:teuthology.orchestra.run.smithi105.stdout: "registry_url": { 2024-09-16T02:05:53.691 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "registry_url", 2024-09-16T02:05:53.691 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.691 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.691 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.691 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.691 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.691 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.691 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.692 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-16T02:05:53.692 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.692 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.692 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.692 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.692 INFO:teuthology.orchestra.run.smithi105.stdout: "registry_username": { 2024-09-16T02:05:53.692 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "registry_username", 2024-09-16T02:05:53.692 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.692 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.692 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.692 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.692 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.692 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.692 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.693 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-16T02:05:53.693 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.693 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.693 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.693 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.693 INFO:teuthology.orchestra.run.smithi105.stdout: "secure_monitoring_stack": { 2024-09-16T02:05:53.693 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "secure_monitoring_stack", 2024-09-16T02:05:53.693 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.693 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.693 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.693 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.693 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.693 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.694 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.694 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-16T02:05:53.694 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.694 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.694 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.694 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.694 INFO:teuthology.orchestra.run.smithi105.stdout: "service_discovery_port": { 2024-09-16T02:05:53.694 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "service_discovery_port", 2024-09-16T02:05:53.694 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:53.694 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.695 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.695 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "8765", 2024-09-16T02:05:53.695 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.695 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.695 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.695 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "cephadm service discovery port", 2024-09-16T02:05:53.695 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.695 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.695 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.695 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.695 INFO:teuthology.orchestra.run.smithi105.stdout: "ssh_config_file": { 2024-09-16T02:05:53.695 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ssh_config_file", 2024-09-16T02:05:53.695 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.696 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.696 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.696 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.696 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.696 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.696 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.696 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-16T02:05:53.696 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.696 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.696 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.696 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.696 INFO:teuthology.orchestra.run.smithi105.stdout: "use_agent": { 2024-09-16T02:05:53.696 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "use_agent", 2024-09-16T02:05:53.696 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.697 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.697 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.697 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.697 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.697 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.697 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.697 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-16T02:05:53.697 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.697 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.697 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.697 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.697 INFO:teuthology.orchestra.run.smithi105.stdout: "use_repo_digest": { 2024-09-16T02:05:53.697 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "use_repo_digest", 2024-09-16T02:05:53.698 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.698 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.698 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.698 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:53.698 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.698 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.698 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.698 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-16T02:05:53.698 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.698 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.698 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.698 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.698 INFO:teuthology.orchestra.run.smithi105.stdout: "warn_on_failed_host_check": { 2024-09-16T02:05:53.698 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "warn_on_failed_host_check", 2024-09-16T02:05:53.699 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.699 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.699 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.699 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:53.699 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.699 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.699 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.699 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "raise a health warning if the host check fails", 2024-09-16T02:05:53.699 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.699 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.699 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.699 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.699 INFO:teuthology.orchestra.run.smithi105.stdout: "warn_on_stray_daemons": { 2024-09-16T02:05:53.700 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "warn_on_stray_daemons", 2024-09-16T02:05:53.700 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.700 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.700 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.700 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:53.700 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.700 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.700 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.700 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-16T02:05:53.700 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.700 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.700 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.700 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.700 INFO:teuthology.orchestra.run.smithi105.stdout: "warn_on_stray_hosts": { 2024-09-16T02:05:53.701 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "warn_on_stray_hosts", 2024-09-16T02:05:53.701 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.701 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.701 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.701 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:53.701 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.701 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.701 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.701 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-16T02:05:53.701 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.701 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.701 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.701 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.702 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.702 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.702 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:53.702 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "crash", 2024-09-16T02:05:53.702 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:53.702 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:53.702 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:53.702 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:53.702 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:53.702 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.702 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.702 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.702 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.703 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.703 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.703 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.703 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.703 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.703 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.703 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.703 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.703 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.703 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.703 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.703 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.703 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.703 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.704 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.704 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:53.704 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:53.704 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.704 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.704 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.704 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.704 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.704 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.704 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.704 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.704 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.704 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.705 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.705 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.705 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:53.705 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:53.705 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.705 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.705 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.705 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:53.705 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.705 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.705 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.705 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.705 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.705 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.706 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.706 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.706 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.706 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.706 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.706 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.706 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.706 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.706 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.706 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:53.706 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:53.706 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.706 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.707 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.707 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.707 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.707 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.707 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.707 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.707 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.707 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.707 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.707 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.707 INFO:teuthology.orchestra.run.smithi105.stdout: "retain_interval": { 2024-09-16T02:05:53.707 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "retain_interval", 2024-09-16T02:05:53.707 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:53.707 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.708 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.708 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "31536000", 2024-09-16T02:05:53.708 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.708 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.708 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.708 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-16T02:05:53.708 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.708 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.708 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.708 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.708 INFO:teuthology.orchestra.run.smithi105.stdout: "warn_recent_interval": { 2024-09-16T02:05:53.708 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "warn_recent_interval", 2024-09-16T02:05:53.708 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:53.708 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.709 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.709 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "1209600", 2024-09-16T02:05:53.709 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.709 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.709 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.709 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-16T02:05:53.709 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.709 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.709 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.709 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.709 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.709 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.709 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:53.709 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "dashboard", 2024-09-16T02:05:53.710 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:53.710 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:53.710 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:53.710 INFO:teuthology.orchestra.run.smithi105.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-16T02:05:53.710 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-16T02:05:53.710 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:53.710 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.710 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.710 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-09-16T02:05:53.710 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.710 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.710 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.710 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.711 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.711 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.711 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.711 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.711 INFO:teuthology.orchestra.run.smithi105.stdout: "ALERTMANAGER_API_HOST": { 2024-09-16T02:05:53.711 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-16T02:05:53.711 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.711 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.711 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.711 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.711 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.711 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.711 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.711 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.712 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.712 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.712 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.712 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.712 INFO:teuthology.orchestra.run.smithi105.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-16T02:05:53.712 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-16T02:05:53.712 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.712 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.712 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.712 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:53.712 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.712 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.712 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.713 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.713 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.713 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.713 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.713 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.713 INFO:teuthology.orchestra.run.smithi105.stdout: "AUDIT_API_ENABLED": { 2024-09-16T02:05:53.713 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "AUDIT_API_ENABLED", 2024-09-16T02:05:53.713 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.713 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.713 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.713 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.713 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.713 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.714 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.714 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.714 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.714 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.714 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.714 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.714 INFO:teuthology.orchestra.run.smithi105.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-16T02:05:53.714 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-16T02:05:53.714 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.714 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.714 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.714 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:53.714 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.714 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.715 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.715 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.715 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.715 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.715 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.715 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.715 INFO:teuthology.orchestra.run.smithi105.stdout: "ENABLE_BROWSABLE_API": { 2024-09-16T02:05:53.715 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-16T02:05:53.715 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.715 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.715 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.715 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:53.715 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.715 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.715 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.716 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.716 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.716 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.716 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.716 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.716 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-16T02:05:53.716 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-16T02:05:53.716 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.716 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.716 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.716 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:53.717 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.717 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.717 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.717 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.717 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.717 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.717 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.717 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.717 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-16T02:05:53.717 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-16T02:05:53.717 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.717 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.717 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.718 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:53.718 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.718 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.718 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.718 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.718 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.718 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.718 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.718 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.718 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-16T02:05:53.718 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-16T02:05:53.718 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.718 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.718 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.719 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:53.719 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.719 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.719 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.719 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.719 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.719 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.719 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.719 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.719 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-16T02:05:53.719 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-16T02:05:53.719 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.719 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.719 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.720 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:53.720 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.720 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.720 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.720 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.720 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.720 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.720 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.720 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.720 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-16T02:05:53.720 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-16T02:05:53.720 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.720 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.720 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.721 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:53.721 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.721 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.721 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.721 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.721 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.721 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.721 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.721 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.721 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-16T02:05:53.722 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-16T02:05:53.722 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.722 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.722 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.722 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:53.722 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.722 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.722 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.722 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.722 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.722 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.722 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.722 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.723 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-16T02:05:53.723 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-16T02:05:53.723 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.723 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.723 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.723 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:53.723 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.723 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.723 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.723 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.723 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.723 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.723 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.723 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.724 INFO:teuthology.orchestra.run.smithi105.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-16T02:05:53.724 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-16T02:05:53.724 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.724 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.724 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.724 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.724 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.724 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.724 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.724 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.724 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.724 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.724 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.725 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.725 INFO:teuthology.orchestra.run.smithi105.stdout: "GRAFANA_API_PASSWORD": { 2024-09-16T02:05:53.725 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-16T02:05:53.725 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.725 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.725 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.725 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "admin", 2024-09-16T02:05:53.725 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.725 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.725 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.725 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.725 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.725 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.725 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.726 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.726 INFO:teuthology.orchestra.run.smithi105.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-16T02:05:53.726 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-16T02:05:53.726 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.726 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.726 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.726 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:53.726 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.726 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.726 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.726 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.726 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.726 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.726 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.726 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.727 INFO:teuthology.orchestra.run.smithi105.stdout: "GRAFANA_API_URL": { 2024-09-16T02:05:53.727 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GRAFANA_API_URL", 2024-09-16T02:05:53.727 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.727 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.727 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.727 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.727 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.727 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.727 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.727 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.727 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.727 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.727 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.727 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.728 INFO:teuthology.orchestra.run.smithi105.stdout: "GRAFANA_API_USERNAME": { 2024-09-16T02:05:53.728 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-16T02:05:53.728 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.728 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.728 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.728 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "admin", 2024-09-16T02:05:53.728 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.728 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.728 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.728 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.728 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.728 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.728 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.728 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.729 INFO:teuthology.orchestra.run.smithi105.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-16T02:05:53.729 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-16T02:05:53.729 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.729 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.729 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.729 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.729 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.729 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.729 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.729 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.729 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.729 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.729 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.730 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.730 INFO:teuthology.orchestra.run.smithi105.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-16T02:05:53.730 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-16T02:05:53.730 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.730 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.730 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.730 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.730 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.730 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.730 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.730 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.730 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.731 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.731 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.731 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.731 INFO:teuthology.orchestra.run.smithi105.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-16T02:05:53.731 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-16T02:05:53.731 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.731 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.731 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.731 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:53.731 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.731 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.731 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.731 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.732 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.732 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.732 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.732 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.732 INFO:teuthology.orchestra.run.smithi105.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-16T02:05:53.732 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-16T02:05:53.732 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.732 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.732 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.732 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.732 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.732 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.732 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.732 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.733 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.733 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.733 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.733 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.733 INFO:teuthology.orchestra.run.smithi105.stdout: "PROMETHEUS_API_HOST": { 2024-09-16T02:05:53.733 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-16T02:05:53.733 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.733 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.733 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.733 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.733 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.733 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.733 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.733 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.734 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.734 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.734 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.734 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.734 INFO:teuthology.orchestra.run.smithi105.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-16T02:05:53.734 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-16T02:05:53.734 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.734 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.734 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.734 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:53.734 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.734 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.734 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.734 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.735 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.735 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.735 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.735 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.735 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-16T02:05:53.735 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-16T02:05:53.735 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.735 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.735 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.735 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.735 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.735 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.735 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.735 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.735 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.736 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.736 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.736 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.736 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-16T02:05:53.736 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-16T02:05:53.736 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.736 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.736 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.736 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.736 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.736 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.736 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.736 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.737 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.737 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.737 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.737 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.737 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-16T02:05:53.737 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-16T02:05:53.737 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.737 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.737 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.737 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:53.737 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.737 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.737 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.737 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.738 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.738 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.738 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.738 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.738 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-16T02:05:53.738 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-16T02:05:53.738 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.738 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.738 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.738 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:53.738 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.738 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.738 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.738 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.738 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.739 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.739 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.739 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.739 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-16T02:05:53.739 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-16T02:05:53.739 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.739 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.739 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.739 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.739 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.739 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.739 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.739 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.739 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.740 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.740 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.740 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.740 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-16T02:05:53.740 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-16T02:05:53.740 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.740 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.740 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.740 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.740 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.740 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.740 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.740 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.740 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.741 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.741 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.741 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.741 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-16T02:05:53.741 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-16T02:05:53.741 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.741 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.741 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.741 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.741 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.741 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.741 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.741 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.741 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.742 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.742 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.742 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.742 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_ENABLED": { 2024-09-16T02:05:53.742 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-16T02:05:53.742 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.742 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.742 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.742 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:53.742 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.742 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.742 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.742 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.742 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.743 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.743 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.743 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.743 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-16T02:05:53.743 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-16T02:05:53.743 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.743 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.743 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.743 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-16T02:05:53.743 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.743 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.743 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.743 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.744 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.744 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.744 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.744 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.744 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-16T02:05:53.744 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-16T02:05:53.744 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:53.744 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.744 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.744 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-09-16T02:05:53.744 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.744 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.744 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.744 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.745 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.745 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.745 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.745 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.745 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-16T02:05:53.745 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-16T02:05:53.745 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:53.745 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.745 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.745 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "8", 2024-09-16T02:05:53.745 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.745 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.745 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.746 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.746 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.746 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.746 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.746 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.746 INFO:teuthology.orchestra.run.smithi105.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-16T02:05:53.746 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-16T02:05:53.746 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:53.746 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.746 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.746 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "45", 2024-09-16T02:05:53.746 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.746 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.746 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.747 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.747 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.747 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.747 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.747 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.747 INFO:teuthology.orchestra.run.smithi105.stdout: "RGW_API_ACCESS_KEY": { 2024-09-16T02:05:53.747 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-16T02:05:53.747 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.747 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.747 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.747 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.747 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.747 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.747 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.747 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.748 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.748 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.748 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.748 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.748 INFO:teuthology.orchestra.run.smithi105.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-16T02:05:53.748 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-16T02:05:53.748 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.748 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.748 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.748 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "admin", 2024-09-16T02:05:53.748 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.748 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.748 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.749 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.749 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.749 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.749 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.749 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.749 INFO:teuthology.orchestra.run.smithi105.stdout: "RGW_API_SECRET_KEY": { 2024-09-16T02:05:53.749 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-16T02:05:53.749 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.749 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.749 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.749 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.749 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.749 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.750 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.750 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.750 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.750 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.750 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.750 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.750 INFO:teuthology.orchestra.run.smithi105.stdout: "RGW_API_SSL_VERIFY": { 2024-09-16T02:05:53.750 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-16T02:05:53.750 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.750 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.750 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.750 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:53.750 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.750 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.751 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.751 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.751 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.751 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.751 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.751 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.751 INFO:teuthology.orchestra.run.smithi105.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-16T02:05:53.751 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-16T02:05:53.751 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:53.751 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.751 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.751 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0", 2024-09-16T02:05:53.751 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.751 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.751 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.752 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.752 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.752 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.752 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.752 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.752 INFO:teuthology.orchestra.run.smithi105.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-16T02:05:53.752 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-16T02:05:53.752 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:53.752 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.752 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.752 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-09-16T02:05:53.752 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.752 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.752 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.753 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.753 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.753 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.753 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.753 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.753 INFO:teuthology.orchestra.run.smithi105.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-16T02:05:53.753 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-16T02:05:53.753 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:53.753 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.753 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.753 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "5", 2024-09-16T02:05:53.753 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.753 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.753 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.754 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.754 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.754 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.754 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.754 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.754 INFO:teuthology.orchestra.run.smithi105.stdout: "cross_origin_url": { 2024-09-16T02:05:53.754 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "cross_origin_url", 2024-09-16T02:05:53.754 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.754 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.754 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.754 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.754 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.754 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.754 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.755 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.755 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.755 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.755 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.755 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.755 INFO:teuthology.orchestra.run.smithi105.stdout: "crt_file": { 2024-09-16T02:05:53.755 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "crt_file", 2024-09-16T02:05:53.755 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.755 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.755 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.755 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.755 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.755 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.755 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.755 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.756 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.756 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.756 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.756 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.756 INFO:teuthology.orchestra.run.smithi105.stdout: "debug": { 2024-09-16T02:05:53.756 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "debug", 2024-09-16T02:05:53.756 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.756 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.756 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.756 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.756 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.756 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.757 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.757 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Enable/disable debug options", 2024-09-16T02:05:53.757 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.757 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.757 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.757 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.757 INFO:teuthology.orchestra.run.smithi105.stdout: "jwt_token_ttl": { 2024-09-16T02:05:53.757 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "jwt_token_ttl", 2024-09-16T02:05:53.757 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:53.757 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.757 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.757 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "28800", 2024-09-16T02:05:53.757 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.757 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.758 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.758 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.758 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.758 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.758 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.758 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.758 INFO:teuthology.orchestra.run.smithi105.stdout: "key_file": { 2024-09-16T02:05:53.758 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "key_file", 2024-09-16T02:05:53.758 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.758 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.759 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.759 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.759 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.759 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.759 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.759 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.759 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.759 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.759 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.759 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.759 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:53.759 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:53.759 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.759 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.760 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.760 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.760 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.760 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.760 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.760 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.760 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.760 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.760 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.760 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.760 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.760 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.760 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.761 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.761 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.761 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.761 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.761 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:53.761 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:53.761 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.761 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.761 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.761 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.761 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.761 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.761 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.761 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.762 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.762 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.762 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.762 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.762 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:53.762 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:53.762 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.762 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.762 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.762 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:53.762 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.762 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.762 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.763 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.763 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.763 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.763 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.763 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.763 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.763 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.763 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.763 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.763 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.763 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.763 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.763 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:53.763 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:53.764 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.764 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.764 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.764 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.764 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.764 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.764 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.764 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.764 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.764 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.764 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.764 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.764 INFO:teuthology.orchestra.run.smithi105.stdout: "motd": { 2024-09-16T02:05:53.764 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "motd", 2024-09-16T02:05:53.765 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.765 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.765 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.765 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.765 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.765 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.765 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.765 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "The message of the day", 2024-09-16T02:05:53.765 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.765 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.765 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.765 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.765 INFO:teuthology.orchestra.run.smithi105.stdout: "redirect_resolve_ip_addr": { 2024-09-16T02:05:53.765 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "redirect_resolve_ip_addr", 2024-09-16T02:05:53.766 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.766 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.766 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.766 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.766 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.766 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.766 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.766 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.766 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.766 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.766 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.766 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.766 INFO:teuthology.orchestra.run.smithi105.stdout: "server_addr": { 2024-09-16T02:05:53.766 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "server_addr", 2024-09-16T02:05:53.767 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.767 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.767 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.767 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "::", 2024-09-16T02:05:53.767 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.767 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.767 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.767 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.768 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.768 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.768 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.768 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.768 INFO:teuthology.orchestra.run.smithi105.stdout: "server_port": { 2024-09-16T02:05:53.769 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "server_port", 2024-09-16T02:05:53.769 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:53.769 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.769 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.769 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "8080", 2024-09-16T02:05:53.769 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.769 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.769 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.769 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.769 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.769 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.769 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.769 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.770 INFO:teuthology.orchestra.run.smithi105.stdout: "ssl": { 2024-09-16T02:05:53.770 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ssl", 2024-09-16T02:05:53.770 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.770 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.770 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.770 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:53.770 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.770 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.770 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.771 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.771 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.771 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.771 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.771 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.771 INFO:teuthology.orchestra.run.smithi105.stdout: "ssl_server_port": { 2024-09-16T02:05:53.771 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ssl_server_port", 2024-09-16T02:05:53.771 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:53.771 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.771 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.771 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "8443", 2024-09-16T02:05:53.771 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.771 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.771 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.772 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.772 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.772 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.772 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.772 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.772 INFO:teuthology.orchestra.run.smithi105.stdout: "standby_behaviour": { 2024-09-16T02:05:53.772 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "standby_behaviour", 2024-09-16T02:05:53.772 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.772 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.772 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.772 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "redirect", 2024-09-16T02:05:53.772 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.773 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.773 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.773 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.773 INFO:teuthology.orchestra.run.smithi105.stdout: "redirect" 2024-09-16T02:05:53.773 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.773 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.773 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.773 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.773 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.773 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.773 INFO:teuthology.orchestra.run.smithi105.stdout: "standby_error_status_code": { 2024-09-16T02:05:53.773 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "standby_error_status_code", 2024-09-16T02:05:53.773 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:53.773 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.774 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.774 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "500", 2024-09-16T02:05:53.774 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "400", 2024-09-16T02:05:53.774 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "599", 2024-09-16T02:05:53.774 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.774 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.774 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.774 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.774 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.774 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.774 INFO:teuthology.orchestra.run.smithi105.stdout: "url_prefix": { 2024-09-16T02:05:53.774 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "url_prefix", 2024-09-16T02:05:53.774 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.774 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.775 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.775 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.775 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.775 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.775 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.775 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.775 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.775 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.775 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.775 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.775 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.775 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.775 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:53.775 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "devicehealth", 2024-09-16T02:05:53.776 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:53.776 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:53.776 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:53.776 INFO:teuthology.orchestra.run.smithi105.stdout: "enable_monitoring": { 2024-09-16T02:05:53.776 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "enable_monitoring", 2024-09-16T02:05:53.776 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.776 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.776 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.776 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:53.776 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.776 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.776 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.776 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "monitor device health metrics", 2024-09-16T02:05:53.776 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.777 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.777 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.777 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.777 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:53.777 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:53.777 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.777 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.777 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.777 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.777 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.777 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.777 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.777 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.777 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.778 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.778 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.778 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.778 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.778 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.778 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.778 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.778 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.778 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.778 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.778 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:53.778 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:53.778 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.778 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.779 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.779 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.779 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.779 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.779 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.779 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.779 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.779 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.779 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.779 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.779 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:53.779 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:53.779 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.779 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.779 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.780 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:53.780 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.780 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.780 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.780 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.780 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.780 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.780 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.780 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.780 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.780 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.780 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.780 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.780 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.781 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.781 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.781 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:53.781 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:53.781 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.781 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.781 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.781 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.781 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.781 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.781 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.781 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.781 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.781 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.782 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.782 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.782 INFO:teuthology.orchestra.run.smithi105.stdout: "mark_out_threshold": { 2024-09-16T02:05:53.782 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "mark_out_threshold", 2024-09-16T02:05:53.782 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:53.782 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.782 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.782 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "2419200", 2024-09-16T02:05:53.782 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.782 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.782 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.782 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-16T02:05:53.782 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.782 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.783 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.783 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.783 INFO:teuthology.orchestra.run.smithi105.stdout: "pool_name": { 2024-09-16T02:05:53.783 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "pool_name", 2024-09-16T02:05:53.783 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.783 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.783 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.783 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "device_health_metrics", 2024-09-16T02:05:53.783 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.783 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.783 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.783 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-16T02:05:53.783 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.783 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.784 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.784 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.784 INFO:teuthology.orchestra.run.smithi105.stdout: "retention_period": { 2024-09-16T02:05:53.784 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "retention_period", 2024-09-16T02:05:53.784 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:53.784 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.784 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.784 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "15552000", 2024-09-16T02:05:53.784 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.784 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.784 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.784 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how long to retain device health metrics", 2024-09-16T02:05:53.784 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.784 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.785 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.785 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.785 INFO:teuthology.orchestra.run.smithi105.stdout: "scrape_frequency": { 2024-09-16T02:05:53.785 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "scrape_frequency", 2024-09-16T02:05:53.785 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:53.785 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.785 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.785 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "86400", 2024-09-16T02:05:53.785 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.785 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.785 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.785 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-16T02:05:53.785 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.786 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.786 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.786 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.786 INFO:teuthology.orchestra.run.smithi105.stdout: "self_heal": { 2024-09-16T02:05:53.786 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "self_heal", 2024-09-16T02:05:53.786 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.786 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.786 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.786 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:53.786 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.786 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.786 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.786 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-16T02:05:53.786 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.787 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.787 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.787 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.787 INFO:teuthology.orchestra.run.smithi105.stdout: "sleep_interval": { 2024-09-16T02:05:53.787 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "sleep_interval", 2024-09-16T02:05:53.787 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:53.787 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.787 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.787 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "600", 2024-09-16T02:05:53.787 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.787 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.787 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.787 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how frequently to wake up and check device health", 2024-09-16T02:05:53.787 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.788 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.788 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.788 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.788 INFO:teuthology.orchestra.run.smithi105.stdout: "warn_threshold": { 2024-09-16T02:05:53.788 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "warn_threshold", 2024-09-16T02:05:53.788 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:53.788 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.788 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.788 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "7257600", 2024-09-16T02:05:53.788 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.788 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.788 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.788 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-16T02:05:53.789 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.789 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.789 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.789 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.789 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.789 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.789 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:53.789 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "diskprediction_local", 2024-09-16T02:05:53.789 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:53.789 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:53.789 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:53.789 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:53.789 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:53.789 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.790 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.790 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.790 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.790 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.790 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.790 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.790 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.790 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.790 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.790 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.790 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.790 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.790 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.790 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.791 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.791 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.791 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.791 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.791 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:53.791 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:53.791 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.791 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.791 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.791 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.791 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.791 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.791 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.792 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.792 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.792 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.792 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.792 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.792 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:53.792 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:53.792 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.792 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.792 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.792 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:53.792 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.792 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.792 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.793 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.793 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.793 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.793 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.793 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.793 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.793 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.793 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.793 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.793 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.793 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.793 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.793 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:53.793 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:53.794 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.794 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.794 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.794 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.794 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.794 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.794 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.794 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.794 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.794 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.794 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.794 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.794 INFO:teuthology.orchestra.run.smithi105.stdout: "predict_interval": { 2024-09-16T02:05:53.794 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "predict_interval", 2024-09-16T02:05:53.795 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.795 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.795 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.795 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "86400", 2024-09-16T02:05:53.795 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.795 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.795 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.795 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.795 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.795 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.795 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.795 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.795 INFO:teuthology.orchestra.run.smithi105.stdout: "predictor_model": { 2024-09-16T02:05:53.795 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "predictor_model", 2024-09-16T02:05:53.796 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.796 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.796 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.796 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "prophetstor", 2024-09-16T02:05:53.796 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.796 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.796 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.796 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.796 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.796 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.796 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.796 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.796 INFO:teuthology.orchestra.run.smithi105.stdout: "sleep_interval": { 2024-09-16T02:05:53.797 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "sleep_interval", 2024-09-16T02:05:53.797 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.797 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.797 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.797 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "600", 2024-09-16T02:05:53.797 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.797 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.797 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.797 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.797 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.797 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.797 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.797 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.797 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.798 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.798 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:53.798 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "influx", 2024-09-16T02:05:53.798 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": false, 2024-09-16T02:05:53.798 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "influxdb python module not found", 2024-09-16T02:05:53.798 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:53.798 INFO:teuthology.orchestra.run.smithi105.stdout: "batch_size": { 2024-09-16T02:05:53.798 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "batch_size", 2024-09-16T02:05:53.798 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:53.798 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.798 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.798 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "5000", 2024-09-16T02:05:53.798 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.798 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.799 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.799 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-16T02:05:53.799 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.799 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.799 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.799 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.799 INFO:teuthology.orchestra.run.smithi105.stdout: "database": { 2024-09-16T02:05:53.799 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "database", 2024-09-16T02:05:53.799 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.799 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.799 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.799 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "ceph", 2024-09-16T02:05:53.799 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.800 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.800 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.800 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-16T02:05:53.800 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.800 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.800 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.800 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.800 INFO:teuthology.orchestra.run.smithi105.stdout: "hostname": { 2024-09-16T02:05:53.800 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "hostname", 2024-09-16T02:05:53.800 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.800 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.800 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.800 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.801 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.801 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.801 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.801 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "InfluxDB server hostname", 2024-09-16T02:05:53.801 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.801 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.801 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.801 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.801 INFO:teuthology.orchestra.run.smithi105.stdout: "interval": { 2024-09-16T02:05:53.801 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "interval", 2024-09-16T02:05:53.801 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:53.801 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.801 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.802 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "30", 2024-09-16T02:05:53.802 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "5", 2024-09-16T02:05:53.802 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.802 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.802 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-16T02:05:53.802 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.802 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.802 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.802 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.802 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:53.802 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:53.802 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.802 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.802 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.803 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.803 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.803 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.803 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.803 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.803 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.803 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.803 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.803 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.803 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.803 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.803 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.803 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.803 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.804 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.804 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.804 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:53.804 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:53.804 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.804 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.804 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.804 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.804 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.804 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.804 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.804 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.804 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.805 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.805 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.805 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.805 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:53.805 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:53.805 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.805 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.805 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.805 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:53.805 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.805 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.805 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.805 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.805 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.806 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.806 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.806 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.806 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.806 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.806 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.806 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.806 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.806 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.806 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.806 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:53.806 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:53.806 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.806 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.807 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.807 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.807 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.807 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.807 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.807 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.807 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.807 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.807 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.807 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.807 INFO:teuthology.orchestra.run.smithi105.stdout: "password": { 2024-09-16T02:05:53.807 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "password", 2024-09-16T02:05:53.807 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.807 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.808 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.808 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.808 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.808 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.808 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.808 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "password of InfluxDB server user", 2024-09-16T02:05:53.808 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.808 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.808 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.808 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.808 INFO:teuthology.orchestra.run.smithi105.stdout: "port": { 2024-09-16T02:05:53.808 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "port", 2024-09-16T02:05:53.808 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:53.809 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.809 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.809 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "8086", 2024-09-16T02:05:53.809 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.809 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.809 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.809 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "InfluxDB server port", 2024-09-16T02:05:53.809 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.809 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.809 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.809 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.809 INFO:teuthology.orchestra.run.smithi105.stdout: "ssl": { 2024-09-16T02:05:53.809 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ssl", 2024-09-16T02:05:53.809 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.810 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.810 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.810 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "false", 2024-09-16T02:05:53.810 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.810 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.810 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.810 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T02:05:53.810 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.810 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.810 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.810 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.810 INFO:teuthology.orchestra.run.smithi105.stdout: "threads": { 2024-09-16T02:05:53.810 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "threads", 2024-09-16T02:05:53.810 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:53.811 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.811 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.811 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "5", 2024-09-16T02:05:53.811 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "1", 2024-09-16T02:05:53.811 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "32", 2024-09-16T02:05:53.811 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.811 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-16T02:05:53.811 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.811 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.811 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.811 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.811 INFO:teuthology.orchestra.run.smithi105.stdout: "username": { 2024-09-16T02:05:53.811 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "username", 2024-09-16T02:05:53.811 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.811 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.812 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.812 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.812 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.812 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.812 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.812 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "username of InfluxDB server user", 2024-09-16T02:05:53.812 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.812 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.812 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.812 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.812 INFO:teuthology.orchestra.run.smithi105.stdout: "verify_ssl": { 2024-09-16T02:05:53.812 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "verify_ssl", 2024-09-16T02:05:53.812 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.813 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.813 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.813 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "true", 2024-09-16T02:05:53.813 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.813 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.813 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.813 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T02:05:53.813 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.813 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.813 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.813 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.813 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.813 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.813 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:53.814 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "insights", 2024-09-16T02:05:53.814 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:53.814 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:53.814 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:53.814 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:53.814 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:53.814 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.814 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.814 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.814 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.814 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.814 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.814 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.814 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.815 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.815 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.815 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.815 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.815 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.815 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.815 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.815 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.815 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.815 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.815 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.815 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:53.815 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:53.815 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.816 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.816 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.816 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.816 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.816 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.816 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.816 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.816 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.816 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.816 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.816 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.816 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:53.816 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:53.817 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.817 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.817 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.817 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:53.817 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.817 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.817 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.817 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.817 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.817 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.817 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.817 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.817 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.817 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.817 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.818 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.818 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.818 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.818 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.818 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:53.818 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:53.818 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.818 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.818 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.818 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.818 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.818 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.818 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.818 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.819 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.819 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.819 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.819 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.819 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.819 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.819 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:53.819 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "iostat", 2024-09-16T02:05:53.819 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:53.819 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:53.819 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:53.819 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:53.819 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:53.819 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.820 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.820 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.820 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.820 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.820 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.820 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.820 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.820 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.820 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.820 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.820 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.820 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.820 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.820 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.821 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.821 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.821 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.821 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.821 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:53.821 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:53.821 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.821 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.821 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.821 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.821 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.822 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.822 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.822 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.822 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.822 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.822 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.822 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.822 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:53.822 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:53.822 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.822 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.822 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.822 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:53.822 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.823 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.823 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.823 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.823 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.823 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.823 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.823 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.823 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.823 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.823 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.823 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.823 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.823 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.823 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.824 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:53.824 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:53.824 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.824 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.824 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.824 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.824 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.824 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.824 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.824 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.824 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.824 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.824 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.824 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.825 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.825 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.825 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:53.825 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "k8sevents", 2024-09-16T02:05:53.825 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:53.825 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:53.825 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:53.825 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_event_retention_days": { 2024-09-16T02:05:53.825 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ceph_event_retention_days", 2024-09-16T02:05:53.825 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:53.825 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.825 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.825 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "7", 2024-09-16T02:05:53.825 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.825 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.826 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.826 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-16T02:05:53.826 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.826 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.826 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.826 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.826 INFO:teuthology.orchestra.run.smithi105.stdout: "config_check_secs": { 2024-09-16T02:05:53.826 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "config_check_secs", 2024-09-16T02:05:53.826 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:53.826 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.826 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.826 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-09-16T02:05:53.826 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "10", 2024-09-16T02:05:53.827 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.827 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.827 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-16T02:05:53.827 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.827 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.827 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.827 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.827 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:53.827 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:53.827 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.827 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.827 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.827 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.827 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.828 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.828 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.828 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.828 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.828 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.828 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.828 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.828 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.828 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.828 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.828 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.828 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.828 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.829 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.829 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:53.829 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:53.829 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.829 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.829 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.829 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.829 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.829 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.829 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.829 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.829 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.829 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.829 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.830 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.830 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:53.830 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:53.830 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.830 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.830 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.830 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:53.830 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.830 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.830 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.830 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.830 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.831 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.831 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.831 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.831 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.831 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.831 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.831 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.831 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.831 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.831 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.831 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:53.831 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:53.831 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.831 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.832 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.832 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.832 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.832 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.832 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.832 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.832 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.832 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.832 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.832 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.832 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.832 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.832 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:53.833 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "localpool", 2024-09-16T02:05:53.833 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:53.833 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:53.833 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:53.833 INFO:teuthology.orchestra.run.smithi105.stdout: "failure_domain": { 2024-09-16T02:05:53.833 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "failure_domain", 2024-09-16T02:05:53.833 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.833 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.833 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.833 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "host", 2024-09-16T02:05:53.833 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.833 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.833 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.833 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "failure domain for any created local pool", 2024-09-16T02:05:53.834 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-16T02:05:53.834 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.834 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.834 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.834 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:53.834 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:53.834 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.834 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.834 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.834 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.834 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.834 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.834 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.834 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.835 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.835 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.835 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.835 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.835 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.835 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.835 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.835 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.835 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.835 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.835 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.835 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:53.835 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:53.835 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.836 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.836 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.836 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.836 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.836 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.836 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.836 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.836 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.836 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.836 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.836 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.836 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:53.836 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:53.836 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.837 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.837 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.837 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:53.837 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.837 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.837 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.837 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.837 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.837 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.837 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.837 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.837 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.837 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.837 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.838 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.838 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.838 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.838 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.838 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:53.838 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:53.838 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.838 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.838 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.838 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.838 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.838 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.838 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.838 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.838 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.839 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.839 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.839 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.839 INFO:teuthology.orchestra.run.smithi105.stdout: "min_size": { 2024-09-16T02:05:53.839 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "min_size", 2024-09-16T02:05:53.839 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:53.839 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.839 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.839 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.839 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.839 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.839 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.839 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "default min_size for any created local pool", 2024-09-16T02:05:53.839 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-16T02:05:53.840 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.840 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.840 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.840 INFO:teuthology.orchestra.run.smithi105.stdout: "num_rep": { 2024-09-16T02:05:53.840 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "num_rep", 2024-09-16T02:05:53.840 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:53.840 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.840 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.840 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "3", 2024-09-16T02:05:53.840 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.840 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.840 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.840 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "default replica count for any created local pool", 2024-09-16T02:05:53.841 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.841 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.841 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.841 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.841 INFO:teuthology.orchestra.run.smithi105.stdout: "pg_num": { 2024-09-16T02:05:53.841 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "pg_num", 2024-09-16T02:05:53.841 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:53.841 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.841 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.841 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "128", 2024-09-16T02:05:53.841 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.841 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.841 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.841 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "default pg_num for any created local pool", 2024-09-16T02:05:53.842 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.842 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.842 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.842 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.842 INFO:teuthology.orchestra.run.smithi105.stdout: "prefix": { 2024-09-16T02:05:53.842 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "prefix", 2024-09-16T02:05:53.842 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.842 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.842 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.842 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.842 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.842 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.842 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.842 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "name prefix for any created local pool", 2024-09-16T02:05:53.843 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.843 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.843 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.843 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.843 INFO:teuthology.orchestra.run.smithi105.stdout: "subtree": { 2024-09-16T02:05:53.843 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "subtree", 2024-09-16T02:05:53.843 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.843 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.843 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.843 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "rack", 2024-09-16T02:05:53.843 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.843 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.843 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.843 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-16T02:05:53.844 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-16T02:05:53.844 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.844 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.844 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.844 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.844 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.844 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:53.844 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "mds_autoscaler", 2024-09-16T02:05:53.844 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:53.844 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:53.844 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:53.844 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:53.844 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:53.844 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.845 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.845 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.845 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.845 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.845 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.845 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.845 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.845 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.845 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.845 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.845 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.845 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.845 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.845 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.846 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.846 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.846 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.846 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.846 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:53.846 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:53.846 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.846 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.846 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.846 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.846 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.846 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.846 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.846 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.847 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.847 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.847 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.847 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.847 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:53.847 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:53.847 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.847 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.847 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.847 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:53.847 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.847 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.847 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.848 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.848 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.848 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.848 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.848 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.848 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.848 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.848 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.848 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.848 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.848 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.848 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.848 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:53.848 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:53.849 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.849 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.849 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.849 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.849 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.849 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.849 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.849 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.849 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.849 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.849 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.849 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.849 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.849 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.849 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:53.850 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "mirroring", 2024-09-16T02:05:53.850 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:53.850 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:53.850 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:53.850 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:53.850 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:53.850 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.850 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.850 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.850 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.850 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.850 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.850 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.850 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.851 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.851 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.851 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.851 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.851 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.851 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.851 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.851 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.851 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.851 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.851 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.851 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:53.851 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:53.851 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.852 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.852 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.852 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.852 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.852 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.852 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.852 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.852 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.852 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.852 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.852 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.852 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:53.852 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:53.853 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.853 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.853 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.853 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:53.853 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.853 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.853 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.853 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.853 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.853 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.853 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.853 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.853 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.853 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.854 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.854 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.854 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.854 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.854 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.854 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:53.854 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:53.854 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.854 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.854 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.854 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.854 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.854 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.855 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.855 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.855 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.855 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.855 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.855 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.855 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.855 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.855 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:53.855 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "nfs", 2024-09-16T02:05:53.855 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:53.855 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:53.855 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:53.856 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:53.856 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:53.856 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.856 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.856 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.856 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.856 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.856 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.856 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.856 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.856 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.856 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.856 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.856 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.857 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.857 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.857 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.857 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.857 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.857 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.857 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.857 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:53.857 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:53.857 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.857 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.857 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.857 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.857 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.858 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.858 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.858 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.858 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.858 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.858 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.858 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.858 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:53.858 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:53.858 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.858 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.858 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.858 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:53.859 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.859 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.859 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.859 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.859 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.859 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.859 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.859 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.859 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.859 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.859 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.859 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.859 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.859 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.860 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.860 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:53.860 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:53.860 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.860 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.860 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.860 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.860 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.860 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.860 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.860 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.860 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.860 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.860 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.861 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.861 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.861 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.861 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:53.861 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "orchestrator", 2024-09-16T02:05:53.861 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:53.861 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:53.861 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:53.861 INFO:teuthology.orchestra.run.smithi105.stdout: "fail_fs": { 2024-09-16T02:05:53.861 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "fail_fs", 2024-09-16T02:05:53.861 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.861 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.861 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.861 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.862 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.862 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.862 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.862 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-16T02:05:53.862 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.862 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.862 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.862 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.862 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:53.862 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:53.862 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.862 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.862 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.862 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.862 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.863 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.863 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.863 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.863 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.863 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.863 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.863 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.863 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.863 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.863 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.863 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.863 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.863 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.863 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.864 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:53.864 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:53.864 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.864 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.864 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.864 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.864 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.864 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.864 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.864 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.864 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.864 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.864 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.864 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.865 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:53.865 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:53.865 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.865 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.865 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.865 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:53.865 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.865 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.865 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.865 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.865 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.865 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.865 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.865 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.866 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.866 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.866 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.866 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.866 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.866 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.866 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.866 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:53.866 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:53.866 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.866 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.866 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.866 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.867 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.867 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.867 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.867 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.867 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.867 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.867 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.867 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.867 INFO:teuthology.orchestra.run.smithi105.stdout: "orchestrator": { 2024-09-16T02:05:53.867 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "orchestrator", 2024-09-16T02:05:53.867 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.868 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.868 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.868 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.868 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.868 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.869 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.869 INFO:teuthology.orchestra.run.smithi105.stdout: "cephadm", 2024-09-16T02:05:53.869 INFO:teuthology.orchestra.run.smithi105.stdout: "rook", 2024-09-16T02:05:53.869 INFO:teuthology.orchestra.run.smithi105.stdout: "test_orchestrator" 2024-09-16T02:05:53.869 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.869 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Orchestrator backend", 2024-09-16T02:05:53.869 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.869 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.869 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.869 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.869 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.870 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.870 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:53.870 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "osd_perf_query", 2024-09-16T02:05:53.870 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:53.870 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:53.870 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:53.870 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:53.871 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:53.871 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.871 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.871 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.871 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.871 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.871 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.871 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.871 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.871 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.871 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.871 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.871 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.871 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.872 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.872 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.872 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.872 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.872 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.872 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.872 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:53.872 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:53.872 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.872 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.872 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.872 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.872 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.872 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.873 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.873 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.873 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.873 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.873 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.873 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.873 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:53.873 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:53.873 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.873 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.873 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.873 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:53.873 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.873 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.874 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.874 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.874 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.874 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.874 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.874 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.874 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.874 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.874 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.874 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.874 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.874 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.874 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.874 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:53.875 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:53.875 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.875 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.875 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.875 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.875 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.875 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.875 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.875 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.875 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.875 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.875 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.875 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.875 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.876 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.876 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:53.876 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "osd_support", 2024-09-16T02:05:53.876 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:53.876 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:53.876 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:53.876 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:53.876 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:53.876 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.876 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.876 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.876 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.876 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.876 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.877 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.877 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.877 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.877 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.877 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.877 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.877 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.877 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.877 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.877 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.877 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.877 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.877 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.877 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:53.878 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:53.878 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.878 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.878 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.878 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.878 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.878 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.878 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.878 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.878 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.878 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.878 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.878 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.878 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:53.879 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:53.879 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.879 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.879 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.879 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:53.879 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.879 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.879 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.879 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.879 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.879 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.879 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.879 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.879 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.879 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.880 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.880 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.880 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.880 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.880 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.880 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:53.880 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:53.880 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.880 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.880 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.880 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.880 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.880 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.880 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.881 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.881 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.881 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.881 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.881 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.881 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.881 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.881 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:53.881 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "pg_autoscaler", 2024-09-16T02:05:53.881 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:53.881 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:53.881 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:53.881 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:53.881 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:53.882 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.882 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.882 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.882 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.882 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.882 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.882 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.882 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.882 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.882 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.882 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.882 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.882 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.882 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.883 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.883 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.883 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.883 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.883 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.883 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:53.883 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:53.883 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.883 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.883 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.883 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.883 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.883 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.883 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.883 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.884 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.884 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.884 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.884 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.884 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:53.884 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:53.884 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.884 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.884 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.884 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:53.884 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.884 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.885 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.885 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.885 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.885 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.885 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.885 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.885 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.885 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.885 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.885 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.885 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.885 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.885 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.885 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:53.886 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:53.886 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.886 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.886 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.886 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.886 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.886 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.886 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.886 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.886 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.886 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.886 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.886 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.886 INFO:teuthology.orchestra.run.smithi105.stdout: "sleep_interval": { 2024-09-16T02:05:53.887 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "sleep_interval", 2024-09-16T02:05:53.887 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:53.887 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.887 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.887 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "60", 2024-09-16T02:05:53.887 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.887 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.887 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.887 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.887 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.887 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.887 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.887 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.887 INFO:teuthology.orchestra.run.smithi105.stdout: "threshold": { 2024-09-16T02:05:53.888 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "threshold", 2024-09-16T02:05:53.888 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-09-16T02:05:53.888 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.888 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.888 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "3.0", 2024-09-16T02:05:53.888 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "1.0", 2024-09-16T02:05:53.888 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.888 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.888 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "scaling threshold", 2024-09-16T02:05:53.888 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-16T02:05:53.888 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.888 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.888 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.889 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.889 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.889 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:53.889 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "progress", 2024-09-16T02:05:53.889 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:53.889 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:53.889 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:53.889 INFO:teuthology.orchestra.run.smithi105.stdout: "allow_pg_recovery_event": { 2024-09-16T02:05:53.889 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "allow_pg_recovery_event", 2024-09-16T02:05:53.889 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.889 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.889 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.889 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.890 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.890 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.890 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.890 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-16T02:05:53.890 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.890 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.890 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.890 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.890 INFO:teuthology.orchestra.run.smithi105.stdout: "enabled": { 2024-09-16T02:05:53.890 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "enabled", 2024-09-16T02:05:53.890 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.890 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.890 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.890 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:53.891 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.891 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.891 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.891 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.891 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.891 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.891 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.891 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.891 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:53.891 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:53.891 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.891 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.891 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.892 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.892 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.892 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.892 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.892 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.892 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.892 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.892 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.892 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.892 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.892 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.892 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.892 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.892 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.893 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.893 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.893 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:53.893 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:53.893 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.893 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.893 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.893 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.893 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.893 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.893 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.893 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.893 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.893 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.894 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.894 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.894 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:53.894 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:53.894 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.894 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.894 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.894 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:53.894 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.894 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.894 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.894 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.894 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.895 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.895 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.895 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.895 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.895 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.895 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.895 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.895 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.895 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.895 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.895 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:53.895 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:53.895 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.895 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.895 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.896 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.896 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.896 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.896 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.896 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.896 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.896 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.896 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.896 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.896 INFO:teuthology.orchestra.run.smithi105.stdout: "max_completed_events": { 2024-09-16T02:05:53.896 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "max_completed_events", 2024-09-16T02:05:53.896 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:53.896 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.897 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.897 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "50", 2024-09-16T02:05:53.897 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.897 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.897 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.897 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "number of past completed events to remember", 2024-09-16T02:05:53.897 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.897 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.897 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.897 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.897 INFO:teuthology.orchestra.run.smithi105.stdout: "sleep_interval": { 2024-09-16T02:05:53.897 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "sleep_interval", 2024-09-16T02:05:53.897 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:53.897 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.897 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.898 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "5", 2024-09-16T02:05:53.898 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.898 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.898 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.898 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how long the module is going to sleep", 2024-09-16T02:05:53.898 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.898 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.898 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.898 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.898 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.898 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.898 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:53.898 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "prometheus", 2024-09-16T02:05:53.899 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:53.899 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:53.899 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:53.899 INFO:teuthology.orchestra.run.smithi105.stdout: "cache": { 2024-09-16T02:05:53.899 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "cache", 2024-09-16T02:05:53.899 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.899 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.899 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.899 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:53.899 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.899 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.899 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.899 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.899 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.900 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.900 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.900 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.900 INFO:teuthology.orchestra.run.smithi105.stdout: "exclude_perf_counters": { 2024-09-16T02:05:53.900 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "exclude_perf_counters", 2024-09-16T02:05:53.900 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.900 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.900 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.900 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:53.900 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.900 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.900 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.900 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-16T02:05:53.901 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-16T02:05:53.901 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.901 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.901 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.901 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:53.901 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:53.901 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.901 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.901 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.901 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.901 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.901 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.901 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.901 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.902 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.902 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.902 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.902 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.902 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.902 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.902 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.902 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.902 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.902 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.902 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.902 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:53.902 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:53.902 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.903 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.903 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.903 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.903 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.903 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.903 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.903 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.903 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.903 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.903 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.903 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.903 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:53.903 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:53.904 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.904 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.904 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.904 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:53.904 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.904 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.904 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.904 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.904 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.904 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.904 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.904 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.904 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.905 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.905 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.905 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.905 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.905 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.905 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.905 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:53.905 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:53.905 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.905 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.905 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.905 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.905 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.905 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.906 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.906 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.906 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.906 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.906 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.906 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.906 INFO:teuthology.orchestra.run.smithi105.stdout: "rbd_stats_pools": { 2024-09-16T02:05:53.906 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rbd_stats_pools", 2024-09-16T02:05:53.906 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.906 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.906 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.906 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.906 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.906 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.907 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.907 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.907 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.907 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.907 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.907 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.907 INFO:teuthology.orchestra.run.smithi105.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-16T02:05:53.907 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-16T02:05:53.907 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:53.907 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.907 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.907 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "300", 2024-09-16T02:05:53.907 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.907 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.908 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.908 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.908 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.908 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.908 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.908 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.908 INFO:teuthology.orchestra.run.smithi105.stdout: "scrape_interval": { 2024-09-16T02:05:53.908 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "scrape_interval", 2024-09-16T02:05:53.908 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-09-16T02:05:53.908 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.908 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.908 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "15.0", 2024-09-16T02:05:53.908 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.909 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.909 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.909 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.909 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.909 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.909 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.909 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.909 INFO:teuthology.orchestra.run.smithi105.stdout: "server_addr": { 2024-09-16T02:05:53.909 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "server_addr", 2024-09-16T02:05:53.909 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.909 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.909 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.909 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "::", 2024-09-16T02:05:53.909 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.910 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.910 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.910 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-16T02:05:53.910 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.910 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.910 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.910 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.910 INFO:teuthology.orchestra.run.smithi105.stdout: "server_port": { 2024-09-16T02:05:53.910 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "server_port", 2024-09-16T02:05:53.910 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:53.910 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.910 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.910 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "9283", 2024-09-16T02:05:53.910 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.911 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.911 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.911 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-16T02:05:53.911 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.911 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.911 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.911 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.911 INFO:teuthology.orchestra.run.smithi105.stdout: "stale_cache_strategy": { 2024-09-16T02:05:53.911 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "stale_cache_strategy", 2024-09-16T02:05:53.911 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.911 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.911 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.911 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "log", 2024-09-16T02:05:53.912 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.912 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.912 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.912 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.912 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.912 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.912 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.912 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.912 INFO:teuthology.orchestra.run.smithi105.stdout: "standby_behaviour": { 2024-09-16T02:05:53.912 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "standby_behaviour", 2024-09-16T02:05:53.912 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.912 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.912 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.913 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "default", 2024-09-16T02:05:53.913 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.913 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.913 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.913 INFO:teuthology.orchestra.run.smithi105.stdout: "default", 2024-09-16T02:05:53.913 INFO:teuthology.orchestra.run.smithi105.stdout: "error" 2024-09-16T02:05:53.913 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.913 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.913 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.913 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.913 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.913 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.913 INFO:teuthology.orchestra.run.smithi105.stdout: "standby_error_status_code": { 2024-09-16T02:05:53.913 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "standby_error_status_code", 2024-09-16T02:05:53.914 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:53.914 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.914 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.914 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "500", 2024-09-16T02:05:53.914 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "400", 2024-09-16T02:05:53.914 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "599", 2024-09-16T02:05:53.914 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.914 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.914 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.914 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.914 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.914 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.914 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.915 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.915 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:53.915 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rbd_support", 2024-09-16T02:05:53.915 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:53.915 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:53.915 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:53.915 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:53.915 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:53.915 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.915 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.915 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.915 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.915 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.915 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.916 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.916 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.916 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.916 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.916 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.916 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.916 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.916 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.916 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.916 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.916 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.916 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.916 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.917 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:53.917 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:53.917 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.917 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.917 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.917 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.917 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.917 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.917 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.917 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.917 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.917 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.917 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.917 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.918 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:53.918 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:53.918 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.918 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.918 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.918 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:53.918 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.918 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.918 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.918 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.918 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.918 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.918 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.918 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.919 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.919 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.919 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.919 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.919 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.919 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.919 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.919 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:53.919 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:53.919 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.919 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.919 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.919 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.919 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.920 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.920 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.920 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.920 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.920 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.920 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.920 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.920 INFO:teuthology.orchestra.run.smithi105.stdout: "max_concurrent_snap_create": { 2024-09-16T02:05:53.920 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "max_concurrent_snap_create", 2024-09-16T02:05:53.920 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:53.920 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.920 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.920 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-09-16T02:05:53.920 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.921 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.921 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.921 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.921 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.921 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.921 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.921 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.921 INFO:teuthology.orchestra.run.smithi105.stdout: "mirror_snapshot_schedule": { 2024-09-16T02:05:53.921 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "mirror_snapshot_schedule", 2024-09-16T02:05:53.921 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.921 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.921 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.922 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.922 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.922 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.922 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.922 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.922 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.922 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.922 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.922 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.922 INFO:teuthology.orchestra.run.smithi105.stdout: "trash_purge_schedule": { 2024-09-16T02:05:53.922 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "trash_purge_schedule", 2024-09-16T02:05:53.922 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.923 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.923 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.923 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.923 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.923 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.923 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.923 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.923 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.923 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.923 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.923 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.923 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.923 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.923 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:53.924 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "restful", 2024-09-16T02:05:53.924 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:53.924 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:53.924 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:53.924 INFO:teuthology.orchestra.run.smithi105.stdout: "enable_auth": { 2024-09-16T02:05:53.924 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "enable_auth", 2024-09-16T02:05:53.924 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.924 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.924 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.924 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:53.924 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.924 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.924 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.925 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.925 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.925 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.925 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.925 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.925 INFO:teuthology.orchestra.run.smithi105.stdout: "key_file": { 2024-09-16T02:05:53.925 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "key_file", 2024-09-16T02:05:53.925 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.925 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.925 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.925 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.925 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.925 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.925 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.926 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.926 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.926 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.926 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.926 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.926 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:53.926 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:53.926 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.926 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.926 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.927 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.927 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.927 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.927 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.927 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.927 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.927 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.927 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.927 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.927 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.927 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.927 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.927 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.927 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.928 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.928 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.928 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:53.928 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:53.928 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.928 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.928 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.928 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.928 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.928 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.928 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.928 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.928 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.928 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.929 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.929 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.929 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:53.929 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:53.929 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.929 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.929 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.929 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:53.929 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.929 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.929 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.929 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.929 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.929 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.930 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.930 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.930 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.930 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.930 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.930 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.930 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.930 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.930 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.930 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:53.930 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:53.930 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.931 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.931 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.931 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.931 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.931 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.931 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.931 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.931 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.931 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.931 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.931 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.931 INFO:teuthology.orchestra.run.smithi105.stdout: "server_addr": { 2024-09-16T02:05:53.931 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "server_addr", 2024-09-16T02:05:53.932 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.932 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.932 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.932 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.932 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.932 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.932 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.932 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.932 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.932 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.932 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.932 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.932 INFO:teuthology.orchestra.run.smithi105.stdout: "server_port": { 2024-09-16T02:05:53.932 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "server_port", 2024-09-16T02:05:53.933 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.933 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.933 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.933 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.933 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.933 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.933 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.933 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.933 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.933 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.933 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.933 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.933 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.934 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.934 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:53.934 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rgw", 2024-09-16T02:05:53.934 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:53.934 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:53.934 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:53.934 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:53.934 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:53.934 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.934 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.934 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.934 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.934 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.934 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.935 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.935 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.935 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.935 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.935 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.935 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.935 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.935 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.935 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.935 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.935 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.935 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.935 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.936 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:53.936 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:53.936 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.936 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.936 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.936 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.936 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.936 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.936 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.936 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.936 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.936 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.936 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.936 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.937 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:53.937 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:53.937 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.937 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.937 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.937 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:53.937 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.937 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.937 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.937 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.937 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.937 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.937 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.937 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.938 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.938 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.938 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.938 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.938 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.938 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.938 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.938 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:53.938 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:53.938 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.938 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.938 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.938 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.938 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.939 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.939 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.939 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.939 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.939 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.939 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.939 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.939 INFO:teuthology.orchestra.run.smithi105.stdout: "secondary_zone_period_retry_limit": { 2024-09-16T02:05:53.939 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-16T02:05:53.939 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:53.939 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.939 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.939 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "5", 2024-09-16T02:05:53.939 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.940 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.940 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.940 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-16T02:05:53.940 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.940 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.940 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.940 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.940 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.940 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.940 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:53.940 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rook", 2024-09-16T02:05:53.940 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:53.941 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:53.941 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:53.941 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:53.941 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:53.941 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.941 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.941 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.941 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.941 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.941 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.941 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.941 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.941 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.942 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.942 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.942 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.942 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.942 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.942 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.942 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.942 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.942 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.942 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.942 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:53.942 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:53.942 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.943 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.943 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.943 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.943 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.943 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.943 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.943 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.943 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.943 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.943 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.943 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.943 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:53.943 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:53.944 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.944 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.944 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.944 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:53.944 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.944 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.944 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.944 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.944 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.944 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.944 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.944 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.944 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.944 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.945 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.945 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.945 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.945 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.945 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.945 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:53.945 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:53.945 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.945 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.945 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.947 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.948 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.948 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.948 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.948 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.948 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.948 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.948 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.948 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.948 INFO:teuthology.orchestra.run.smithi105.stdout: "storage_class": { 2024-09-16T02:05:53.948 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "storage_class", 2024-09-16T02:05:53.948 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.948 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.948 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.949 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "local", 2024-09-16T02:05:53.949 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.949 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.949 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.949 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-16T02:05:53.949 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.949 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.949 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.949 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.949 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.949 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.949 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:53.949 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "selftest", 2024-09-16T02:05:53.949 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:53.950 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:53.950 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:53.950 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:53.950 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:53.950 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.950 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.950 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.950 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.950 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.950 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.950 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.950 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.950 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.950 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.951 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.951 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.951 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.951 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.951 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.951 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.951 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.951 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.951 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.951 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:53.951 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:53.951 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.951 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.951 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.952 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.952 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.952 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.952 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.952 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.952 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.952 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.952 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.952 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.952 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:53.952 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:53.952 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.952 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.953 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.953 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:53.953 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.953 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.953 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.953 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.953 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.953 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.953 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.953 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.953 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.953 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.953 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.953 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.954 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.954 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.954 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.954 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:53.954 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:53.954 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.954 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.954 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.954 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.954 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.954 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.954 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.954 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.954 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.954 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.955 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.955 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.955 INFO:teuthology.orchestra.run.smithi105.stdout: "roption1": { 2024-09-16T02:05:53.955 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "roption1", 2024-09-16T02:05:53.955 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.955 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.955 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.955 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.955 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.955 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.955 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.955 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.955 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.955 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.956 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.956 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.956 INFO:teuthology.orchestra.run.smithi105.stdout: "roption2": { 2024-09-16T02:05:53.956 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "roption2", 2024-09-16T02:05:53.956 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.956 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.956 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.956 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "xyz", 2024-09-16T02:05:53.956 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.956 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.956 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.956 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.956 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.956 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.957 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.957 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.957 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption1": { 2024-09-16T02:05:53.957 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption1", 2024-09-16T02:05:53.957 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.957 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.957 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.957 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.957 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.957 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.957 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.957 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.957 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.958 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.958 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.958 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.958 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption2": { 2024-09-16T02:05:53.958 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption2", 2024-09-16T02:05:53.958 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:53.958 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.958 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.958 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.958 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.958 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.958 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.958 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.958 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.958 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.959 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.959 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.959 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption3": { 2024-09-16T02:05:53.959 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption3", 2024-09-16T02:05:53.959 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-09-16T02:05:53.959 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.959 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.959 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.959 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.959 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.959 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.959 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.959 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.959 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.960 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.960 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.960 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption4": { 2024-09-16T02:05:53.960 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption4", 2024-09-16T02:05:53.960 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.960 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.960 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.960 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.960 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.960 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.960 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.960 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.960 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.960 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.961 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.961 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.961 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption5": { 2024-09-16T02:05:53.961 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption5", 2024-09-16T02:05:53.961 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.961 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.961 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.961 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.961 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.961 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.961 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.961 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.961 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.961 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.962 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.962 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.962 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption6": { 2024-09-16T02:05:53.962 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption6", 2024-09-16T02:05:53.962 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.962 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.962 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.962 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:53.962 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.962 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.962 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.962 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.962 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.962 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.963 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.963 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.963 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption7": { 2024-09-16T02:05:53.963 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption7", 2024-09-16T02:05:53.963 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:53.963 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.963 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.963 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.963 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "1", 2024-09-16T02:05:53.963 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "42", 2024-09-16T02:05:53.963 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.963 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.963 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.964 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.964 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.964 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.964 INFO:teuthology.orchestra.run.smithi105.stdout: "testkey": { 2024-09-16T02:05:53.964 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "testkey", 2024-09-16T02:05:53.964 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.964 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.964 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.964 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.964 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.964 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.964 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.964 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.964 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.965 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.965 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.965 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.965 INFO:teuthology.orchestra.run.smithi105.stdout: "testlkey": { 2024-09-16T02:05:53.965 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "testlkey", 2024-09-16T02:05:53.965 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.965 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.965 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.965 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.965 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.965 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.965 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.965 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.965 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.965 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.966 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.966 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.966 INFO:teuthology.orchestra.run.smithi105.stdout: "testnewline": { 2024-09-16T02:05:53.966 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "testnewline", 2024-09-16T02:05:53.966 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.966 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.966 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.966 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.966 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.966 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.966 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.966 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.966 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.966 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.967 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.967 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.967 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.967 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.967 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:53.967 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "snap_schedule", 2024-09-16T02:05:53.967 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:53.967 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:53.967 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:53.967 INFO:teuthology.orchestra.run.smithi105.stdout: "allow_m_granularity": { 2024-09-16T02:05:53.967 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "allow_m_granularity", 2024-09-16T02:05:53.967 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.967 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.967 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.968 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.968 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.968 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.968 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.968 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "allow minute scheduled snapshots", 2024-09-16T02:05:53.968 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.968 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.968 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.968 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.968 INFO:teuthology.orchestra.run.smithi105.stdout: "dump_on_update": { 2024-09-16T02:05:53.968 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "dump_on_update", 2024-09-16T02:05:53.968 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.968 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.969 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.969 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.969 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.969 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.969 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.969 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "dump database to debug log on update", 2024-09-16T02:05:53.969 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.969 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.969 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.969 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.970 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:53.970 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:53.970 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.970 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.970 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.970 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.970 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.970 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.970 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.970 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.970 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.970 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.970 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.970 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.971 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.971 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.971 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.971 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.971 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.971 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.971 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.971 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:53.971 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:53.971 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.971 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.971 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.971 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.972 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.972 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.972 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.972 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.972 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.972 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.972 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.972 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.972 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:53.972 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:53.972 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.972 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.972 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.972 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:53.973 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.973 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.973 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.973 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.973 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.973 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.973 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.973 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.973 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.973 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.973 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.973 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.973 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.974 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.974 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.974 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:53.974 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:53.974 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.974 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.974 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.974 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.974 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.974 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.974 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.974 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.974 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.974 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.974 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.975 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.975 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.975 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.975 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:53.975 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "stats", 2024-09-16T02:05:53.975 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:53.975 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:53.975 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:53.975 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:53.975 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:53.975 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.975 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.975 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.975 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.976 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.976 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.976 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.976 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.976 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.976 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.976 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.976 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.976 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.976 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.976 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.976 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.976 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.976 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.977 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.977 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:53.977 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:53.977 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.977 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.977 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.977 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.977 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.977 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.977 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.977 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.977 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.977 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.978 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.978 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.978 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:53.978 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:53.978 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.978 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.978 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.978 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:53.978 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.978 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.978 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.978 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.978 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.978 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.979 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.979 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.979 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.979 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.979 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.979 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.979 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.979 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.979 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.979 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:53.979 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:53.979 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.979 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.979 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.979 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.980 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.980 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.980 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.980 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.980 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.980 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.980 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.980 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.980 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.980 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.980 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:53.980 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "status", 2024-09-16T02:05:53.980 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:53.981 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:53.981 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:53.981 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:53.981 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:53.981 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.981 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.981 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.981 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.981 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.981 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.981 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.981 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.981 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.981 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.981 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.982 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.982 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.982 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.982 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.982 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.982 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.982 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.982 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.982 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:53.982 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:53.982 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.982 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.982 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.982 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.983 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.983 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.983 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.983 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.983 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.983 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.983 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.983 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.983 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:53.983 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:53.983 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.983 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.983 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.983 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:53.984 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.984 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.984 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.984 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.984 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.984 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.984 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.984 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.984 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.984 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.984 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.984 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.984 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.984 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.985 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.985 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:53.985 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:53.985 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.985 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.985 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.985 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.985 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.985 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.985 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.985 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.985 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.985 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.986 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.986 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.986 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.986 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.986 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:53.986 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "telegraf", 2024-09-16T02:05:53.986 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:53.986 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:53.986 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:53.986 INFO:teuthology.orchestra.run.smithi105.stdout: "address": { 2024-09-16T02:05:53.986 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "address", 2024-09-16T02:05:53.986 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.986 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.987 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.987 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-16T02:05:53.987 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.987 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.987 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.987 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.987 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.987 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.987 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.987 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.987 INFO:teuthology.orchestra.run.smithi105.stdout: "interval": { 2024-09-16T02:05:53.987 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "interval", 2024-09-16T02:05:53.987 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:53.987 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.988 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.988 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "15", 2024-09-16T02:05:53.988 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.988 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.988 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.988 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.988 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.988 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.988 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.988 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.988 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:53.988 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:53.988 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.988 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.989 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.989 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.989 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.989 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.989 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.989 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.989 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.989 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.989 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.989 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.989 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.989 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.989 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.989 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.990 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.990 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.990 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.990 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:53.990 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:53.990 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.990 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.990 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.990 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.990 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.990 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.990 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.990 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.991 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.991 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.991 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.991 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.991 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:53.991 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:53.991 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.991 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.991 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.991 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:53.991 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.991 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.991 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:53.991 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:53.992 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:53.992 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:53.992 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:53.992 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:53.992 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:53.992 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:53.992 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.992 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.992 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.992 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.992 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.992 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:53.992 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:53.993 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.993 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.993 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:53.993 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.993 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.993 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.993 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.993 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:53.993 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.993 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.993 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.993 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.993 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:53.993 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.994 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:53.994 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "telemetry", 2024-09-16T02:05:53.994 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:53.994 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:53.994 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:53.994 INFO:teuthology.orchestra.run.smithi105.stdout: "channel_basic": { 2024-09-16T02:05:53.994 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "channel_basic", 2024-09-16T02:05:53.994 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.994 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.994 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.994 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:53.994 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.994 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.994 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.994 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-16T02:05:53.995 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.995 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.995 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.995 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.995 INFO:teuthology.orchestra.run.smithi105.stdout: "channel_crash": { 2024-09-16T02:05:53.995 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "channel_crash", 2024-09-16T02:05:53.995 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.995 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.995 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.995 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:53.995 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.995 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.995 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.995 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-16T02:05:53.996 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.996 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.996 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.996 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.996 INFO:teuthology.orchestra.run.smithi105.stdout: "channel_device": { 2024-09-16T02:05:53.996 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "channel_device", 2024-09-16T02:05:53.996 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.996 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.996 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.996 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:53.996 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.996 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.996 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.996 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-16T02:05:53.997 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.997 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.997 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.997 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.997 INFO:teuthology.orchestra.run.smithi105.stdout: "channel_ident": { 2024-09-16T02:05:53.997 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "channel_ident", 2024-09-16T02:05:53.997 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.997 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.997 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.997 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.997 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.997 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.997 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.997 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-16T02:05:53.998 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.998 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.998 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.998 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.998 INFO:teuthology.orchestra.run.smithi105.stdout: "channel_perf": { 2024-09-16T02:05:53.998 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "channel_perf", 2024-09-16T02:05:53.998 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:53.998 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.998 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.998 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:53.998 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.998 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.998 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:53.998 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-16T02:05:53.999 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:53.999 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:53.999 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:53.999 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:53.999 INFO:teuthology.orchestra.run.smithi105.stdout: "contact": { 2024-09-16T02:05:53.999 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "contact", 2024-09-16T02:05:53.999 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:53.999 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:53.999 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:53.999 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:53.999 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:53.999 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:53.999 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.000 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.000 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.000 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.000 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.000 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.000 INFO:teuthology.orchestra.run.smithi105.stdout: "description": { 2024-09-16T02:05:54.000 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "description", 2024-09-16T02:05:54.000 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.000 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.000 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.000 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.000 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.000 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.001 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.001 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.001 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.001 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.001 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.001 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.001 INFO:teuthology.orchestra.run.smithi105.stdout: "device_url": { 2024-09-16T02:05:54.001 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "device_url", 2024-09-16T02:05:54.001 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.001 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.001 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.001 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-16T02:05:54.001 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.001 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.002 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.002 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.002 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.002 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.002 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.002 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.002 INFO:teuthology.orchestra.run.smithi105.stdout: "enabled": { 2024-09-16T02:05:54.002 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "enabled", 2024-09-16T02:05:54.002 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.002 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.002 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.002 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.002 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.002 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.003 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.003 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.003 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.003 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.003 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.003 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.003 INFO:teuthology.orchestra.run.smithi105.stdout: "interval": { 2024-09-16T02:05:54.003 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "interval", 2024-09-16T02:05:54.003 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.003 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.003 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.003 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "24", 2024-09-16T02:05:54.003 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "8", 2024-09-16T02:05:54.003 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.004 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.004 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.004 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.004 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.004 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.004 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.004 INFO:teuthology.orchestra.run.smithi105.stdout: "last_opt_revision": { 2024-09-16T02:05:54.004 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "last_opt_revision", 2024-09-16T02:05:54.004 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.004 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.004 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.004 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "1", 2024-09-16T02:05:54.004 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.004 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.005 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.005 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.005 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.005 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.005 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.005 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.005 INFO:teuthology.orchestra.run.smithi105.stdout: "leaderboard": { 2024-09-16T02:05:54.005 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "leaderboard", 2024-09-16T02:05:54.005 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.005 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.005 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.005 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.005 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.005 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.006 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.006 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.006 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.006 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.006 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.006 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.006 INFO:teuthology.orchestra.run.smithi105.stdout: "leaderboard_description": { 2024-09-16T02:05:54.006 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "leaderboard_description", 2024-09-16T02:05:54.006 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.006 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.006 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.006 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.006 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.006 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.007 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.007 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.007 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.007 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.007 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.007 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.007 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.007 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.007 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.007 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.007 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.007 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.007 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.007 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.008 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.008 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.008 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.008 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.008 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.008 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.008 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.008 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.008 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.008 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.008 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.008 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.008 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.008 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.009 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.009 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.009 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.009 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.009 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.009 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.009 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.009 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.009 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.009 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.009 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.009 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.009 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.010 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.010 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.010 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.010 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.010 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.010 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.010 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.010 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.010 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.010 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.010 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.010 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.010 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.010 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.010 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.011 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.011 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.011 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.011 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.011 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.011 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.011 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.011 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.011 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.011 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.011 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.011 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.011 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.011 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.012 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.012 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.012 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.012 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.012 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.012 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.012 INFO:teuthology.orchestra.run.smithi105.stdout: "organization": { 2024-09-16T02:05:54.012 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "organization", 2024-09-16T02:05:54.012 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.012 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.012 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.012 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.012 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.012 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.013 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.013 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.013 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.013 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.013 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.013 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.013 INFO:teuthology.orchestra.run.smithi105.stdout: "proxy": { 2024-09-16T02:05:54.013 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "proxy", 2024-09-16T02:05:54.013 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.013 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.013 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.013 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.013 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.014 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.014 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.014 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.014 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.014 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.014 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.014 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.014 INFO:teuthology.orchestra.run.smithi105.stdout: "url": { 2024-09-16T02:05:54.014 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "url", 2024-09-16T02:05:54.014 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.014 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.014 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.014 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-16T02:05:54.014 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.015 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.015 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.015 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.015 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.015 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.015 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.015 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.015 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.015 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.015 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.015 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "test_orchestrator", 2024-09-16T02:05:54.015 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.015 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.016 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.016 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.016 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.016 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.016 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.016 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.016 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.016 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.016 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.016 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.016 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.016 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.016 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.016 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.017 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.017 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.017 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.017 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.017 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.017 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.017 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.017 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.017 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.017 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.017 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.017 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.017 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.017 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.018 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.018 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.018 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.018 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.018 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.018 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.018 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.018 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.018 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.018 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.018 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.018 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.018 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.018 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.019 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.019 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.019 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.019 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.019 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.019 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.019 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.019 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.019 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.019 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.019 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.019 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.019 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.019 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.020 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.020 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.020 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.020 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.020 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.020 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.020 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.020 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.020 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.020 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.020 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.020 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.020 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.021 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.021 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.021 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.021 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.021 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.021 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "volumes", 2024-09-16T02:05:54.021 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.021 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.021 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.021 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.021 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.021 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.021 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.021 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.022 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.022 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.022 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.022 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.022 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.022 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.022 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.022 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.022 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.022 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.022 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.022 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.022 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.022 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.023 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.023 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.023 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.023 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.023 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.023 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.023 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.023 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.023 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.023 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.023 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.023 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.023 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.023 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.024 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.024 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.024 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.024 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.024 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.024 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.024 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.024 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.024 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.024 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.024 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.024 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.024 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.025 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.025 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.025 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.025 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.025 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.025 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.025 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.025 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.025 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.025 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.025 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.025 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.025 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.025 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.026 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.026 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.026 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.026 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.026 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.026 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.026 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.026 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.026 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.026 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.026 INFO:teuthology.orchestra.run.smithi105.stdout: "max_concurrent_clones": { 2024-09-16T02:05:54.026 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "max_concurrent_clones", 2024-09-16T02:05:54.026 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.026 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.027 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.027 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "4", 2024-09-16T02:05:54.027 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.027 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.027 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.027 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-16T02:05:54.027 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.027 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.027 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.027 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.027 INFO:teuthology.orchestra.run.smithi105.stdout: "periodic_async_work": { 2024-09-16T02:05:54.027 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "periodic_async_work", 2024-09-16T02:05:54.027 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.028 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.028 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.028 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.028 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.028 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.028 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.028 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Periodically check for async work", 2024-09-16T02:05:54.028 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.028 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.028 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.028 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.028 INFO:teuthology.orchestra.run.smithi105.stdout: "snapshot_clone_delay": { 2024-09-16T02:05:54.028 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "snapshot_clone_delay", 2024-09-16T02:05:54.028 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.029 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.029 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.029 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0", 2024-09-16T02:05:54.029 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.029 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.029 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.029 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-16T02:05:54.029 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.029 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.029 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.029 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.029 INFO:teuthology.orchestra.run.smithi105.stdout: "snapshot_clone_no_wait": { 2024-09-16T02:05:54.029 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "snapshot_clone_no_wait", 2024-09-16T02:05:54.030 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.030 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.030 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.030 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.030 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.030 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.030 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.030 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-16T02:05:54.030 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.030 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.030 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.030 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.030 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.031 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.031 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.031 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "zabbix", 2024-09-16T02:05:54.031 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.031 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.031 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.031 INFO:teuthology.orchestra.run.smithi105.stdout: "discovery_interval": { 2024-09-16T02:05:54.031 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "discovery_interval", 2024-09-16T02:05:54.031 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "uint", 2024-09-16T02:05:54.031 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.031 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.031 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "100", 2024-09-16T02:05:54.031 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.032 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.032 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.032 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.032 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.032 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.032 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.032 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.032 INFO:teuthology.orchestra.run.smithi105.stdout: "identifier": { 2024-09-16T02:05:54.032 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "identifier", 2024-09-16T02:05:54.032 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.032 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.032 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.032 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.033 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.033 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.033 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.033 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.033 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.033 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.033 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.033 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.033 INFO:teuthology.orchestra.run.smithi105.stdout: "interval": { 2024-09-16T02:05:54.033 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "interval", 2024-09-16T02:05:54.033 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.033 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.033 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.033 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "60", 2024-09-16T02:05:54.034 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.034 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.034 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.034 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.034 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.034 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.034 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.034 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.034 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.034 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.034 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.034 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.034 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.035 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.035 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.035 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.035 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.035 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.035 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.035 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.035 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.035 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.035 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.035 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.035 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.035 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.035 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.036 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.036 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.036 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.036 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.036 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.036 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.036 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.036 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.036 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.036 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.036 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.036 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.036 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.037 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.037 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.037 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.037 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.037 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.037 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.037 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.037 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.037 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.037 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.037 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.037 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.037 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.037 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.038 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.038 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.038 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.038 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.038 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.038 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.038 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.038 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.038 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.038 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.038 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.038 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.038 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.038 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.039 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.039 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.039 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.039 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.039 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.039 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.039 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.039 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.039 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.039 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.039 INFO:teuthology.orchestra.run.smithi105.stdout: "zabbix_host": { 2024-09-16T02:05:54.039 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "zabbix_host", 2024-09-16T02:05:54.039 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.039 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.040 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.040 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.040 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.040 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.040 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.040 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.040 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.040 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.040 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.040 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.040 INFO:teuthology.orchestra.run.smithi105.stdout: "zabbix_port": { 2024-09-16T02:05:54.040 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "zabbix_port", 2024-09-16T02:05:54.040 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.040 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.041 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.041 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10051", 2024-09-16T02:05:54.041 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.041 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.041 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.041 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.041 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.041 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.041 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.041 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.041 INFO:teuthology.orchestra.run.smithi105.stdout: "zabbix_sender": { 2024-09-16T02:05:54.041 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "zabbix_sender", 2024-09-16T02:05:54.041 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.042 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.042 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.042 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-16T02:05:54.042 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.042 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.042 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.042 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.042 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.042 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.042 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.042 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.042 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.042 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.043 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-16T02:05:54.043 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.043 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.043 INFO:teuthology.orchestra.run.smithi105.stdout: "gid": 24113, 2024-09-16T02:05:54.043 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "y", 2024-09-16T02:05:54.043 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr_features": 4540701547738038271, 2024-09-16T02:05:54.043 INFO:teuthology.orchestra.run.smithi105.stdout: "available_modules": [ 2024-09-16T02:05:54.043 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.043 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "alerts", 2024-09-16T02:05:54.043 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.043 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.043 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.043 INFO:teuthology.orchestra.run.smithi105.stdout: "interval": { 2024-09-16T02:05:54.043 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "interval", 2024-09-16T02:05:54.044 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.044 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.044 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.044 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "60", 2024-09-16T02:05:54.044 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.044 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.044 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.044 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "How frequently to reexamine health status", 2024-09-16T02:05:54.044 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.044 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.044 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.044 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.044 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.045 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.045 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.045 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.045 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.045 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.045 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.045 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.045 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.045 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.045 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.045 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.045 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.045 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.045 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.046 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.046 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.046 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.046 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.046 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.046 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.046 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.046 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.046 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.046 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.046 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.046 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.046 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.047 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.047 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.047 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.047 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.047 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.047 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.047 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.047 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.047 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.047 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.047 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.047 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.047 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.047 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.048 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.048 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.048 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.048 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.048 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.048 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.048 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.048 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.048 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.048 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.048 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.048 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.048 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.049 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.049 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.049 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.049 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.049 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.049 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.049 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.049 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.049 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.049 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.049 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.049 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.049 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.049 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.050 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.050 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_destination": { 2024-09-16T02:05:54.050 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_destination", 2024-09-16T02:05:54.050 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.050 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.050 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.050 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.050 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.050 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.050 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.050 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Email address to send alerts to", 2024-09-16T02:05:54.050 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.050 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.050 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.050 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.051 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_from_name": { 2024-09-16T02:05:54.051 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_from_name", 2024-09-16T02:05:54.051 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.051 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.051 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.051 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "Ceph", 2024-09-16T02:05:54.051 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.051 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.051 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.051 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Email From: name", 2024-09-16T02:05:54.051 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.051 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.051 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.051 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.052 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_host": { 2024-09-16T02:05:54.052 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_host", 2024-09-16T02:05:54.052 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.052 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.052 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.052 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.052 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.052 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.052 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.052 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "SMTP server", 2024-09-16T02:05:54.052 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.052 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.052 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.052 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.053 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_password": { 2024-09-16T02:05:54.053 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_password", 2024-09-16T02:05:54.053 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.053 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.053 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.053 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.053 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.053 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.053 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.053 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Password to authenticate with", 2024-09-16T02:05:54.053 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.053 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.053 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.053 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.054 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_port": { 2024-09-16T02:05:54.054 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_port", 2024-09-16T02:05:54.054 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.054 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.054 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.054 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "465", 2024-09-16T02:05:54.054 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.054 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.054 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.054 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "SMTP port", 2024-09-16T02:05:54.054 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.054 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.054 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.054 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.055 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_sender": { 2024-09-16T02:05:54.055 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_sender", 2024-09-16T02:05:54.055 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.055 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.055 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.055 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.055 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.055 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.055 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.055 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "SMTP envelope sender", 2024-09-16T02:05:54.055 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.055 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.055 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.056 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.056 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_ssl": { 2024-09-16T02:05:54.056 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_ssl", 2024-09-16T02:05:54.056 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.056 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.056 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.056 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.056 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.056 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.056 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.056 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-16T02:05:54.056 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.056 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.056 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.057 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.057 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_user": { 2024-09-16T02:05:54.057 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_user", 2024-09-16T02:05:54.057 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.057 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.057 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.057 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.057 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.057 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.057 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.057 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "User to authenticate as", 2024-09-16T02:05:54.057 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.057 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.057 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.058 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.058 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.058 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.058 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.058 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "balancer", 2024-09-16T02:05:54.058 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.058 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.058 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.058 INFO:teuthology.orchestra.run.smithi105.stdout: "active": { 2024-09-16T02:05:54.058 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "active", 2024-09-16T02:05:54.058 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.058 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.058 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.058 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.059 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.059 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.059 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.059 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "automatically balance PGs across cluster", 2024-09-16T02:05:54.059 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.059 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.059 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.059 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.059 INFO:teuthology.orchestra.run.smithi105.stdout: "begin_time": { 2024-09-16T02:05:54.059 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "begin_time", 2024-09-16T02:05:54.059 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.059 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.059 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.059 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0000", 2024-09-16T02:05:54.060 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.060 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.060 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.060 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "beginning time of day to automatically balance", 2024-09-16T02:05:54.060 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T02:05:54.060 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.060 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.060 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.060 INFO:teuthology.orchestra.run.smithi105.stdout: "begin_weekday": { 2024-09-16T02:05:54.060 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "begin_weekday", 2024-09-16T02:05:54.060 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "uint", 2024-09-16T02:05:54.060 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.060 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.061 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0", 2024-09-16T02:05:54.061 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "0", 2024-09-16T02:05:54.061 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "6", 2024-09-16T02:05:54.061 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.061 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-16T02:05:54.061 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T02:05:54.061 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.061 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.061 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.061 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_compat_max_iterations": { 2024-09-16T02:05:54.061 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "crush_compat_max_iterations", 2024-09-16T02:05:54.061 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "uint", 2024-09-16T02:05:54.061 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.061 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.062 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "25", 2024-09-16T02:05:54.062 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "1", 2024-09-16T02:05:54.062 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "250", 2024-09-16T02:05:54.062 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.062 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-16T02:05:54.062 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.062 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.062 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.062 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.062 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_compat_metrics": { 2024-09-16T02:05:54.062 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "crush_compat_metrics", 2024-09-16T02:05:54.062 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.062 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.062 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.063 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "pgs,objects,bytes", 2024-09-16T02:05:54.063 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.063 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.063 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.063 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-16T02:05:54.063 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-16T02:05:54.063 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.063 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.063 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.063 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_compat_step": { 2024-09-16T02:05:54.063 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "crush_compat_step", 2024-09-16T02:05:54.063 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-09-16T02:05:54.063 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.063 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.064 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0.5", 2024-09-16T02:05:54.064 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "0.001", 2024-09-16T02:05:54.064 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "0.999", 2024-09-16T02:05:54.064 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.064 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "aggressiveness of optimization", 2024-09-16T02:05:54.064 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-16T02:05:54.064 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.064 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.064 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.064 INFO:teuthology.orchestra.run.smithi105.stdout: "end_time": { 2024-09-16T02:05:54.064 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "end_time", 2024-09-16T02:05:54.064 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.064 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.064 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.065 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "2359", 2024-09-16T02:05:54.065 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.065 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.065 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.065 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "ending time of day to automatically balance", 2024-09-16T02:05:54.065 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T02:05:54.065 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.065 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.065 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.065 INFO:teuthology.orchestra.run.smithi105.stdout: "end_weekday": { 2024-09-16T02:05:54.065 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "end_weekday", 2024-09-16T02:05:54.065 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "uint", 2024-09-16T02:05:54.065 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.065 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.066 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0", 2024-09-16T02:05:54.066 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "0", 2024-09-16T02:05:54.066 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "6", 2024-09-16T02:05:54.066 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.066 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-16T02:05:54.066 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T02:05:54.066 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.066 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.066 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.066 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.066 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.066 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.066 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.066 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.067 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.067 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.067 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.067 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.067 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.067 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.067 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.067 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.067 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.067 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.067 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.067 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.067 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.067 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.068 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.068 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.068 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.068 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.068 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.068 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.068 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.068 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.068 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.068 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.068 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.068 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.068 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.069 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.069 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.069 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.069 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.069 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.069 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.069 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.069 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.069 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.069 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.069 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.069 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.069 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.070 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.070 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.070 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.070 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.070 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.070 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.070 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.070 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.070 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.070 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.070 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.070 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.070 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.070 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.071 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.071 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.071 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.071 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.071 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.071 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.071 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.071 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.071 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.071 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.071 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.071 INFO:teuthology.orchestra.run.smithi105.stdout: "min_score": { 2024-09-16T02:05:54.071 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "min_score", 2024-09-16T02:05:54.071 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-09-16T02:05:54.072 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.072 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.072 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0", 2024-09-16T02:05:54.072 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.072 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.072 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.072 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-16T02:05:54.072 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.072 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.072 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.072 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.072 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": { 2024-09-16T02:05:54.072 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "mode", 2024-09-16T02:05:54.072 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.073 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.073 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.073 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "upmap", 2024-09-16T02:05:54.073 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.073 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.073 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.073 INFO:teuthology.orchestra.run.smithi105.stdout: "crush-compat", 2024-09-16T02:05:54.073 INFO:teuthology.orchestra.run.smithi105.stdout: "none", 2024-09-16T02:05:54.073 INFO:teuthology.orchestra.run.smithi105.stdout: "read", 2024-09-16T02:05:54.073 INFO:teuthology.orchestra.run.smithi105.stdout: "upmap", 2024-09-16T02:05:54.073 INFO:teuthology.orchestra.run.smithi105.stdout: "upmap-read" 2024-09-16T02:05:54.073 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.073 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Balancer mode", 2024-09-16T02:05:54.073 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.074 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.074 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.074 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.074 INFO:teuthology.orchestra.run.smithi105.stdout: "pool_ids": { 2024-09-16T02:05:54.074 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "pool_ids", 2024-09-16T02:05:54.074 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.074 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.074 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.074 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.074 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.074 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.074 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.074 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-16T02:05:54.074 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.075 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.075 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.075 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.075 INFO:teuthology.orchestra.run.smithi105.stdout: "sleep_interval": { 2024-09-16T02:05:54.075 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "sleep_interval", 2024-09-16T02:05:54.075 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.075 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.075 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.075 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "60", 2024-09-16T02:05:54.075 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.075 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.075 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.075 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-16T02:05:54.075 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.076 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.076 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.076 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.076 INFO:teuthology.orchestra.run.smithi105.stdout: "upmap_max_deviation": { 2024-09-16T02:05:54.076 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "upmap_max_deviation", 2024-09-16T02:05:54.076 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.076 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.076 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.076 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "5", 2024-09-16T02:05:54.076 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "1", 2024-09-16T02:05:54.076 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.076 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.076 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-16T02:05:54.076 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-16T02:05:54.077 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.077 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.077 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.077 INFO:teuthology.orchestra.run.smithi105.stdout: "upmap_max_optimizations": { 2024-09-16T02:05:54.077 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "upmap_max_optimizations", 2024-09-16T02:05:54.077 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "uint", 2024-09-16T02:05:54.077 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.077 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.077 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-09-16T02:05:54.077 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.077 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.077 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.077 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-16T02:05:54.077 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.078 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.078 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.078 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.078 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.078 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.078 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.078 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "cephadm", 2024-09-16T02:05:54.078 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.078 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.078 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.078 INFO:teuthology.orchestra.run.smithi105.stdout: "agent_down_multiplier": { 2024-09-16T02:05:54.078 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "agent_down_multiplier", 2024-09-16T02:05:54.078 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-09-16T02:05:54.078 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.079 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.079 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "3.0", 2024-09-16T02:05:54.079 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.079 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.079 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.079 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-16T02:05:54.079 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.079 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.079 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.079 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.079 INFO:teuthology.orchestra.run.smithi105.stdout: "agent_refresh_rate": { 2024-09-16T02:05:54.079 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "agent_refresh_rate", 2024-09-16T02:05:54.079 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.079 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.080 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.080 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "20", 2024-09-16T02:05:54.080 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.080 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.080 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.080 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-16T02:05:54.080 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.080 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.080 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.080 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.080 INFO:teuthology.orchestra.run.smithi105.stdout: "agent_starting_port": { 2024-09-16T02:05:54.080 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "agent_starting_port", 2024-09-16T02:05:54.080 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.080 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.081 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.081 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "4721", 2024-09-16T02:05:54.081 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.081 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.081 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.081 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-16T02:05:54.081 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.081 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.081 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.081 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.081 INFO:teuthology.orchestra.run.smithi105.stdout: "allow_ptrace": { 2024-09-16T02:05:54.081 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "allow_ptrace", 2024-09-16T02:05:54.081 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.082 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.082 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.082 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.082 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.082 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.082 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.082 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-16T02:05:54.082 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-16T02:05:54.082 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.082 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.082 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.082 INFO:teuthology.orchestra.run.smithi105.stdout: "autotune_interval": { 2024-09-16T02:05:54.082 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "autotune_interval", 2024-09-16T02:05:54.082 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.083 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.083 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.083 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "600", 2024-09-16T02:05:54.083 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.083 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.083 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.083 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-16T02:05:54.083 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.083 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.083 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.083 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.083 INFO:teuthology.orchestra.run.smithi105.stdout: "autotune_memory_target_ratio": { 2024-09-16T02:05:54.083 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "autotune_memory_target_ratio", 2024-09-16T02:05:54.083 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-09-16T02:05:54.084 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.084 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.084 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0.7", 2024-09-16T02:05:54.084 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.084 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.084 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.084 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-16T02:05:54.084 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.084 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.084 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.084 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.084 INFO:teuthology.orchestra.run.smithi105.stdout: "cephadm_log_destination": { 2024-09-16T02:05:54.084 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "cephadm_log_destination", 2024-09-16T02:05:54.085 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.085 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.085 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.085 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.085 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.085 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.085 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.085 INFO:teuthology.orchestra.run.smithi105.stdout: "file", 2024-09-16T02:05:54.085 INFO:teuthology.orchestra.run.smithi105.stdout: "file,syslog", 2024-09-16T02:05:54.085 INFO:teuthology.orchestra.run.smithi105.stdout: "syslog" 2024-09-16T02:05:54.085 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.085 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-16T02:05:54.085 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.085 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.086 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.086 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.086 INFO:teuthology.orchestra.run.smithi105.stdout: "cgroups_split": { 2024-09-16T02:05:54.086 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "cgroups_split", 2024-09-16T02:05:54.086 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.086 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.086 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.086 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.086 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.086 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.086 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.086 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-16T02:05:54.086 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.086 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.087 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.087 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.087 INFO:teuthology.orchestra.run.smithi105.stdout: "config_checks_enabled": { 2024-09-16T02:05:54.087 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "config_checks_enabled", 2024-09-16T02:05:54.087 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.087 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.087 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.087 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.087 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.087 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.087 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.087 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-16T02:05:54.087 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.087 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.088 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.088 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.088 INFO:teuthology.orchestra.run.smithi105.stdout: "config_dashboard": { 2024-09-16T02:05:54.088 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "config_dashboard", 2024-09-16T02:05:54.088 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.088 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.088 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.088 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.088 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.088 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.088 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.088 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-16T02:05:54.088 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.089 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.089 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.089 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.089 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_alertmanager": { 2024-09-16T02:05:54.089 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_alertmanager", 2024-09-16T02:05:54.089 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.089 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.089 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.089 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-16T02:05:54.089 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.089 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.089 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.089 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Prometheus container image", 2024-09-16T02:05:54.089 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.089 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.090 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.090 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.090 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_base": { 2024-09-16T02:05:54.090 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_base", 2024-09-16T02:05:54.090 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.090 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.090 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.090 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-16T02:05:54.090 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.090 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.090 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.090 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Container image name, without the tag", 2024-09-16T02:05:54.090 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.090 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.091 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.091 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.091 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_elasticsearch": { 2024-09-16T02:05:54.091 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_elasticsearch", 2024-09-16T02:05:54.091 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.091 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.091 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.091 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-16T02:05:54.091 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.091 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.091 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.091 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "elasticsearch container image", 2024-09-16T02:05:54.091 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.091 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.092 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.092 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.092 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_grafana": { 2024-09-16T02:05:54.092 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_grafana", 2024-09-16T02:05:54.092 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.092 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.092 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.092 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-16T02:05:54.092 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.092 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.092 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.092 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Prometheus container image", 2024-09-16T02:05:54.092 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.092 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.093 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.093 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.093 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_haproxy": { 2024-09-16T02:05:54.093 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_haproxy", 2024-09-16T02:05:54.093 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.093 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.093 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.093 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-16T02:05:54.093 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.093 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.093 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.093 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "HAproxy container image", 2024-09-16T02:05:54.093 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.094 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.094 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.094 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.094 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_jaeger_agent": { 2024-09-16T02:05:54.094 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_jaeger_agent", 2024-09-16T02:05:54.094 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.094 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.094 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.094 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-16T02:05:54.094 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.094 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.094 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.094 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Jaeger agent container image", 2024-09-16T02:05:54.094 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.095 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.095 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.095 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.095 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_jaeger_collector": { 2024-09-16T02:05:54.095 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_jaeger_collector", 2024-09-16T02:05:54.095 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.095 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.095 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.095 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-16T02:05:54.095 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.095 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.095 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.095 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Jaeger collector container image", 2024-09-16T02:05:54.095 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.096 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.096 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.096 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.096 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_jaeger_query": { 2024-09-16T02:05:54.096 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_jaeger_query", 2024-09-16T02:05:54.096 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.096 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.096 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.096 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-16T02:05:54.096 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.096 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.096 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.096 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Jaeger query container image", 2024-09-16T02:05:54.096 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.097 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.097 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.097 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.097 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_keepalived": { 2024-09-16T02:05:54.097 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_keepalived", 2024-09-16T02:05:54.097 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.097 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.097 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.097 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-16T02:05:54.097 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.097 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.097 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.097 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Keepalived container image", 2024-09-16T02:05:54.097 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.098 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.098 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.098 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.098 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_loki": { 2024-09-16T02:05:54.098 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_loki", 2024-09-16T02:05:54.098 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.098 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.098 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.098 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-16T02:05:54.098 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.098 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.098 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.098 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Loki container image", 2024-09-16T02:05:54.098 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.099 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.099 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.099 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.099 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_node_exporter": { 2024-09-16T02:05:54.099 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_node_exporter", 2024-09-16T02:05:54.099 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.099 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.099 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.099 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-16T02:05:54.099 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.099 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.099 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.099 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Prometheus container image", 2024-09-16T02:05:54.099 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.100 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.100 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.100 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.100 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_nvmeof": { 2024-09-16T02:05:54.100 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_nvmeof", 2024-09-16T02:05:54.100 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.100 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.100 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.100 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-16T02:05:54.100 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.100 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.100 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.100 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Nvme-of container image", 2024-09-16T02:05:54.100 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.101 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.101 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.101 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.101 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_prometheus": { 2024-09-16T02:05:54.101 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_prometheus", 2024-09-16T02:05:54.101 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.101 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.101 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.101 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-16T02:05:54.101 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.101 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.101 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.101 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Prometheus container image", 2024-09-16T02:05:54.101 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.101 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.102 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.102 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.102 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_promtail": { 2024-09-16T02:05:54.102 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_promtail", 2024-09-16T02:05:54.102 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.102 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.102 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.102 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-16T02:05:54.102 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.102 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.102 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.102 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Promtail container image", 2024-09-16T02:05:54.102 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.102 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.103 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.103 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.103 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_samba": { 2024-09-16T02:05:54.103 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_samba", 2024-09-16T02:05:54.103 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.103 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.103 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.103 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-16T02:05:54.103 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.103 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.103 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.103 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Samba/SMB container image", 2024-09-16T02:05:54.103 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.103 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.104 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.104 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.104 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_snmp_gateway": { 2024-09-16T02:05:54.104 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_snmp_gateway", 2024-09-16T02:05:54.104 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.104 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.104 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.104 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-16T02:05:54.104 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.104 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.104 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.104 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "SNMP Gateway container image", 2024-09-16T02:05:54.104 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.104 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.105 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.105 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.105 INFO:teuthology.orchestra.run.smithi105.stdout: "container_init": { 2024-09-16T02:05:54.105 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_init", 2024-09-16T02:05:54.105 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.105 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.105 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.105 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.105 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.105 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.105 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.105 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Run podman/docker with `--init`", 2024-09-16T02:05:54.105 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.106 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.106 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.106 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.106 INFO:teuthology.orchestra.run.smithi105.stdout: "daemon_cache_timeout": { 2024-09-16T02:05:54.106 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "daemon_cache_timeout", 2024-09-16T02:05:54.106 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.106 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.106 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.106 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "600", 2024-09-16T02:05:54.106 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.106 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.106 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.106 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-16T02:05:54.106 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.107 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.107 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.107 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.107 INFO:teuthology.orchestra.run.smithi105.stdout: "default_cephadm_command_timeout": { 2024-09-16T02:05:54.107 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "default_cephadm_command_timeout", 2024-09-16T02:05:54.107 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.107 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.107 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.107 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "900", 2024-09-16T02:05:54.107 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.107 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.107 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.107 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-16T02:05:54.107 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.108 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.108 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.108 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.108 INFO:teuthology.orchestra.run.smithi105.stdout: "default_registry": { 2024-09-16T02:05:54.108 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "default_registry", 2024-09-16T02:05:54.108 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.108 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.108 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.108 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "docker.io", 2024-09-16T02:05:54.108 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.108 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.108 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.108 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-16T02:05:54.108 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.109 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.109 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.109 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.109 INFO:teuthology.orchestra.run.smithi105.stdout: "device_cache_timeout": { 2024-09-16T02:05:54.109 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "device_cache_timeout", 2024-09-16T02:05:54.109 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.109 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.109 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.109 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "1800", 2024-09-16T02:05:54.109 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.109 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.109 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.109 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "seconds to cache device inventory", 2024-09-16T02:05:54.110 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.110 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.110 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.110 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.110 INFO:teuthology.orchestra.run.smithi105.stdout: "device_enhanced_scan": { 2024-09-16T02:05:54.110 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "device_enhanced_scan", 2024-09-16T02:05:54.110 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.110 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.110 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.110 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.110 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.110 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.110 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.111 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-16T02:05:54.111 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.111 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.111 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.111 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.111 INFO:teuthology.orchestra.run.smithi105.stdout: "facts_cache_timeout": { 2024-09-16T02:05:54.111 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "facts_cache_timeout", 2024-09-16T02:05:54.111 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.111 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.111 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.111 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "60", 2024-09-16T02:05:54.111 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.111 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.111 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.112 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "seconds to cache host facts data", 2024-09-16T02:05:54.112 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.112 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.112 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.112 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.112 INFO:teuthology.orchestra.run.smithi105.stdout: "grafana_dashboards_path": { 2024-09-16T02:05:54.112 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "grafana_dashboards_path", 2024-09-16T02:05:54.112 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.112 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.112 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.112 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-16T02:05:54.112 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.112 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.112 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.113 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-16T02:05:54.113 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.113 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.113 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.113 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.113 INFO:teuthology.orchestra.run.smithi105.stdout: "host_check_interval": { 2024-09-16T02:05:54.113 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "host_check_interval", 2024-09-16T02:05:54.113 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.113 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.113 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.113 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "600", 2024-09-16T02:05:54.113 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.113 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.113 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.114 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how frequently to perform a host check", 2024-09-16T02:05:54.114 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.114 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.114 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.114 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.114 INFO:teuthology.orchestra.run.smithi105.stdout: "hw_monitoring": { 2024-09-16T02:05:54.114 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "hw_monitoring", 2024-09-16T02:05:54.114 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.114 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.114 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.114 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.114 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.114 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.115 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.115 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-16T02:05:54.115 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.115 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.115 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.115 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.115 INFO:teuthology.orchestra.run.smithi105.stdout: "inventory_list_all": { 2024-09-16T02:05:54.115 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "inventory_list_all", 2024-09-16T02:05:54.115 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.115 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.115 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.115 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.115 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.115 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.116 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.116 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-16T02:05:54.116 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.116 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.116 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.116 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.116 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.116 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.116 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.116 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.116 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.116 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.116 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.116 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.117 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.117 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.117 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.117 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.117 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.117 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.117 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.117 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.117 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.117 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.117 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.117 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.117 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.117 INFO:teuthology.orchestra.run.smithi105.stdout: "log_refresh_metadata": { 2024-09-16T02:05:54.118 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_refresh_metadata", 2024-09-16T02:05:54.118 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.118 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.118 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.118 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.118 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.118 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.118 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.118 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-16T02:05:54.118 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.118 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.118 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.118 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.119 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.119 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.119 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.119 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.119 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.119 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.119 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.119 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.119 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.119 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-16T02:05:54.119 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.119 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.119 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.119 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.120 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.120 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.120 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.120 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.120 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.120 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.120 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.120 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.120 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.120 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.120 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.120 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.120 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.120 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.121 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.121 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.121 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.121 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.121 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.121 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.121 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.121 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.121 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.121 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.121 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.121 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.121 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.121 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.122 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.122 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.122 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.122 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.122 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.122 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.122 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.122 INFO:teuthology.orchestra.run.smithi105.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-16T02:05:54.122 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-16T02:05:54.122 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.122 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.122 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.122 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.122 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.123 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.123 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.123 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-16T02:05:54.123 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.123 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.123 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.123 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.123 INFO:teuthology.orchestra.run.smithi105.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-16T02:05:54.123 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-16T02:05:54.123 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.123 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.123 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.123 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "*", 2024-09-16T02:05:54.124 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.124 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.124 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.124 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-16T02:05:54.124 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.124 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.124 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.124 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.124 INFO:teuthology.orchestra.run.smithi105.stdout: "max_count_per_host": { 2024-09-16T02:05:54.124 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "max_count_per_host", 2024-09-16T02:05:54.124 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.124 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.124 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.124 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-09-16T02:05:54.125 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.125 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.125 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.125 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "max number of daemons per service per host", 2024-09-16T02:05:54.125 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.125 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.125 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.125 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.125 INFO:teuthology.orchestra.run.smithi105.stdout: "max_osd_draining_count": { 2024-09-16T02:05:54.125 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "max_osd_draining_count", 2024-09-16T02:05:54.125 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.125 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.125 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.125 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-09-16T02:05:54.126 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.126 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.126 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.126 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-16T02:05:54.126 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.126 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.126 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.126 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.126 INFO:teuthology.orchestra.run.smithi105.stdout: "migration_current": { 2024-09-16T02:05:54.126 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "migration_current", 2024-09-16T02:05:54.126 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.126 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.126 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.126 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.126 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.127 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.127 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.127 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "internal - do not modify", 2024-09-16T02:05:54.127 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.127 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.127 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.127 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.127 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": { 2024-09-16T02:05:54.127 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "mode", 2024-09-16T02:05:54.127 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.127 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.127 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.127 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "root", 2024-09-16T02:05:54.128 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.128 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.128 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.128 INFO:teuthology.orchestra.run.smithi105.stdout: "cephadm-package", 2024-09-16T02:05:54.128 INFO:teuthology.orchestra.run.smithi105.stdout: "root" 2024-09-16T02:05:54.128 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.128 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "mode for remote execution of cephadm", 2024-09-16T02:05:54.128 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.128 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.128 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.128 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.128 INFO:teuthology.orchestra.run.smithi105.stdout: "oob_default_addr": { 2024-09-16T02:05:54.128 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "oob_default_addr", 2024-09-16T02:05:54.128 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.128 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.129 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.129 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "169.254.1.1", 2024-09-16T02:05:54.129 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.129 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.129 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.129 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-16T02:05:54.129 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.129 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.129 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.129 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.129 INFO:teuthology.orchestra.run.smithi105.stdout: "prometheus_alerts_path": { 2024-09-16T02:05:54.129 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "prometheus_alerts_path", 2024-09-16T02:05:54.129 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.129 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.130 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.130 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-16T02:05:54.130 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.130 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.130 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.130 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-16T02:05:54.130 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.130 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.130 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.130 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.130 INFO:teuthology.orchestra.run.smithi105.stdout: "registry_insecure": { 2024-09-16T02:05:54.130 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "registry_insecure", 2024-09-16T02:05:54.130 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.130 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.131 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.131 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.131 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.131 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.131 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.131 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-16T02:05:54.131 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.131 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.131 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.131 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.131 INFO:teuthology.orchestra.run.smithi105.stdout: "registry_password": { 2024-09-16T02:05:54.131 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "registry_password", 2024-09-16T02:05:54.131 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.131 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.132 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.132 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.132 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.132 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.132 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.132 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-16T02:05:54.132 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.132 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.132 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.132 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.132 INFO:teuthology.orchestra.run.smithi105.stdout: "registry_url": { 2024-09-16T02:05:54.132 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "registry_url", 2024-09-16T02:05:54.132 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.133 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.133 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.133 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.133 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.133 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.133 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.133 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-16T02:05:54.133 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.133 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.133 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.133 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.133 INFO:teuthology.orchestra.run.smithi105.stdout: "registry_username": { 2024-09-16T02:05:54.133 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "registry_username", 2024-09-16T02:05:54.133 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.133 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.134 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.134 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.134 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.134 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.134 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.134 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-16T02:05:54.134 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.134 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.134 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.134 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.134 INFO:teuthology.orchestra.run.smithi105.stdout: "secure_monitoring_stack": { 2024-09-16T02:05:54.134 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "secure_monitoring_stack", 2024-09-16T02:05:54.134 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.134 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.135 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.135 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.135 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.135 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.135 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.135 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-16T02:05:54.135 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.135 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.135 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.135 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.135 INFO:teuthology.orchestra.run.smithi105.stdout: "service_discovery_port": { 2024-09-16T02:05:54.135 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "service_discovery_port", 2024-09-16T02:05:54.135 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.135 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.136 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.136 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "8765", 2024-09-16T02:05:54.136 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.136 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.136 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.136 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "cephadm service discovery port", 2024-09-16T02:05:54.136 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.136 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.136 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.136 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.136 INFO:teuthology.orchestra.run.smithi105.stdout: "ssh_config_file": { 2024-09-16T02:05:54.136 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ssh_config_file", 2024-09-16T02:05:54.136 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.137 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.137 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.137 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.137 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.137 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.137 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.137 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-16T02:05:54.137 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.137 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.137 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.137 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.137 INFO:teuthology.orchestra.run.smithi105.stdout: "use_agent": { 2024-09-16T02:05:54.137 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "use_agent", 2024-09-16T02:05:54.137 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.138 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.138 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.138 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.138 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.138 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.138 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.138 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-16T02:05:54.138 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.138 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.138 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.138 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.138 INFO:teuthology.orchestra.run.smithi105.stdout: "use_repo_digest": { 2024-09-16T02:05:54.138 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "use_repo_digest", 2024-09-16T02:05:54.138 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.139 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.139 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.139 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.139 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.139 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.139 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.139 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-16T02:05:54.139 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.139 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.139 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.139 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.139 INFO:teuthology.orchestra.run.smithi105.stdout: "warn_on_failed_host_check": { 2024-09-16T02:05:54.139 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "warn_on_failed_host_check", 2024-09-16T02:05:54.139 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.140 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.140 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.140 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.140 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.140 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.140 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.140 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "raise a health warning if the host check fails", 2024-09-16T02:05:54.140 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.140 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.140 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.140 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.140 INFO:teuthology.orchestra.run.smithi105.stdout: "warn_on_stray_daemons": { 2024-09-16T02:05:54.140 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "warn_on_stray_daemons", 2024-09-16T02:05:54.141 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.141 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.141 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.141 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.141 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.141 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.141 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.141 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-16T02:05:54.141 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.141 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.141 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.141 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.141 INFO:teuthology.orchestra.run.smithi105.stdout: "warn_on_stray_hosts": { 2024-09-16T02:05:54.141 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "warn_on_stray_hosts", 2024-09-16T02:05:54.142 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.142 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.142 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.142 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.142 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.142 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.142 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.142 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-16T02:05:54.142 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.142 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.142 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.142 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.142 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.142 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.143 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.143 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "crash", 2024-09-16T02:05:54.143 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.143 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.143 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.143 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.143 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.143 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.143 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.143 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.143 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.143 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.143 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.143 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.144 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.144 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.144 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.144 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.144 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.144 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.144 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.144 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.144 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.144 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.144 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.144 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.144 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.145 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.145 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.145 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.145 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.145 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.145 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.145 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.145 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.145 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.145 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.145 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.145 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.145 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.145 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.146 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.146 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.146 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.146 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.146 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.146 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.146 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.146 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.146 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.146 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.146 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.146 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.146 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.146 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.147 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.147 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.147 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.147 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.147 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.147 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.147 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.147 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.147 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.147 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.147 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.147 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.147 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.148 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.148 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.148 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.148 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.148 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.148 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.148 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.148 INFO:teuthology.orchestra.run.smithi105.stdout: "retain_interval": { 2024-09-16T02:05:54.148 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "retain_interval", 2024-09-16T02:05:54.148 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.148 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.148 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.148 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "31536000", 2024-09-16T02:05:54.148 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.149 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.149 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.149 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-16T02:05:54.149 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.149 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.149 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.149 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.149 INFO:teuthology.orchestra.run.smithi105.stdout: "warn_recent_interval": { 2024-09-16T02:05:54.149 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "warn_recent_interval", 2024-09-16T02:05:54.149 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.149 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.149 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.149 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "1209600", 2024-09-16T02:05:54.149 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.150 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.150 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.150 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-16T02:05:54.150 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.150 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.150 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.150 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.150 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.150 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.152 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.152 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "dashboard", 2024-09-16T02:05:54.152 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.152 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.152 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.152 INFO:teuthology.orchestra.run.smithi105.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-16T02:05:54.152 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-16T02:05:54.152 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.152 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.152 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.152 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-09-16T02:05:54.152 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.152 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.153 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.153 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.153 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.153 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.153 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.153 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.153 INFO:teuthology.orchestra.run.smithi105.stdout: "ALERTMANAGER_API_HOST": { 2024-09-16T02:05:54.153 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-16T02:05:54.153 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.153 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.153 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.153 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.153 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.153 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.154 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.154 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.154 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.154 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.154 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.154 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.154 INFO:teuthology.orchestra.run.smithi105.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-16T02:05:54.154 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-16T02:05:54.154 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.154 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.154 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.154 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.154 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.154 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.155 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.155 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.155 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.155 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.155 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.155 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.155 INFO:teuthology.orchestra.run.smithi105.stdout: "AUDIT_API_ENABLED": { 2024-09-16T02:05:54.155 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "AUDIT_API_ENABLED", 2024-09-16T02:05:54.155 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.155 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.155 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.155 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.155 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.155 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.156 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.156 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.156 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.156 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.156 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.156 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.156 INFO:teuthology.orchestra.run.smithi105.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-16T02:05:54.156 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-16T02:05:54.156 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.156 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.156 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.156 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.156 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.156 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.157 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.157 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.157 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.157 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.157 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.157 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.157 INFO:teuthology.orchestra.run.smithi105.stdout: "ENABLE_BROWSABLE_API": { 2024-09-16T02:05:54.157 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-16T02:05:54.157 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.157 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.157 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.157 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.157 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.158 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.158 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.158 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.158 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.158 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.158 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.158 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.158 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-16T02:05:54.158 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-16T02:05:54.158 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.158 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.158 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.158 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.158 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.159 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.159 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.159 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.159 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.159 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.159 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.159 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.159 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-16T02:05:54.159 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-16T02:05:54.159 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.159 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.159 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.159 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.159 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.160 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.160 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.160 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.160 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.160 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.160 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.160 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.160 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-16T02:05:54.160 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-16T02:05:54.160 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.160 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.160 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.160 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.161 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.161 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.161 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.161 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.161 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.161 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.161 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.161 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.161 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-16T02:05:54.161 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-16T02:05:54.161 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.161 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.161 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.161 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.162 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.162 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.162 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.162 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.162 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.162 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.162 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.162 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.162 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-16T02:05:54.162 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-16T02:05:54.162 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.162 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.162 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.162 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.163 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.163 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.163 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.163 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.163 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.163 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.163 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.163 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.163 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-16T02:05:54.163 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-16T02:05:54.163 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.163 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.163 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.163 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.164 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.164 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.164 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.164 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.164 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.164 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.164 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.164 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.164 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-16T02:05:54.164 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-16T02:05:54.164 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.164 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.164 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.164 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.165 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.165 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.165 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.165 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.165 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.165 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.165 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.165 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.165 INFO:teuthology.orchestra.run.smithi105.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-16T02:05:54.165 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-16T02:05:54.165 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.165 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.165 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.165 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.166 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.166 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.166 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.166 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.166 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.166 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.166 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.166 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.166 INFO:teuthology.orchestra.run.smithi105.stdout: "GRAFANA_API_PASSWORD": { 2024-09-16T02:05:54.166 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-16T02:05:54.166 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.166 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.166 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.166 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "admin", 2024-09-16T02:05:54.167 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.167 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.167 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.167 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.167 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.167 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.167 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.167 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.167 INFO:teuthology.orchestra.run.smithi105.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-16T02:05:54.167 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-16T02:05:54.167 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.167 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.167 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.167 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.167 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.168 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.168 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.168 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.168 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.168 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.168 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.168 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.168 INFO:teuthology.orchestra.run.smithi105.stdout: "GRAFANA_API_URL": { 2024-09-16T02:05:54.168 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GRAFANA_API_URL", 2024-09-16T02:05:54.168 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.168 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.168 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.168 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.168 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.169 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.169 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.169 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.169 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.169 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.169 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.169 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.169 INFO:teuthology.orchestra.run.smithi105.stdout: "GRAFANA_API_USERNAME": { 2024-09-16T02:05:54.169 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-16T02:05:54.169 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.169 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.169 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.169 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "admin", 2024-09-16T02:05:54.169 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.170 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.170 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.170 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.170 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.170 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.170 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.170 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.170 INFO:teuthology.orchestra.run.smithi105.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-16T02:05:54.170 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-16T02:05:54.170 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.170 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.170 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.170 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.170 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.171 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.171 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.171 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.171 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.171 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.171 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.171 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.171 INFO:teuthology.orchestra.run.smithi105.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-16T02:05:54.171 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-16T02:05:54.171 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.171 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.171 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.171 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.171 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.172 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.172 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.172 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.172 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.172 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.172 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.172 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.172 INFO:teuthology.orchestra.run.smithi105.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-16T02:05:54.172 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-16T02:05:54.172 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.172 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.172 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.172 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.173 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.173 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.173 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.173 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.173 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.173 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.173 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.173 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.173 INFO:teuthology.orchestra.run.smithi105.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-16T02:05:54.173 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-16T02:05:54.173 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.173 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.173 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.173 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.174 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.174 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.174 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.174 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.174 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.174 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.174 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.174 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.174 INFO:teuthology.orchestra.run.smithi105.stdout: "PROMETHEUS_API_HOST": { 2024-09-16T02:05:54.174 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-16T02:05:54.174 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.174 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.174 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.174 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.175 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.175 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.175 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.175 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.175 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.175 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.175 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.175 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.175 INFO:teuthology.orchestra.run.smithi105.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-16T02:05:54.175 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-16T02:05:54.175 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.175 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.175 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.175 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.175 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.176 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.176 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.176 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.176 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.176 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.176 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.176 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.176 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-16T02:05:54.176 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-16T02:05:54.176 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.176 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.176 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.176 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.177 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.177 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.177 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.177 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.177 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.177 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.177 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.177 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.177 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-16T02:05:54.177 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-16T02:05:54.177 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.177 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.177 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.177 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.178 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.178 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.178 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.178 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.178 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.178 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.178 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.178 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.178 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-16T02:05:54.178 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-16T02:05:54.178 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.178 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.178 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.178 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.178 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.179 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.179 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.179 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.179 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.179 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.179 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.179 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.179 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-16T02:05:54.179 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-16T02:05:54.179 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.179 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.179 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.179 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.179 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.180 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.180 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.180 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.180 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.180 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.180 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.180 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.180 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-16T02:05:54.180 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-16T02:05:54.180 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.180 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.180 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.180 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.180 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.181 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.181 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.181 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.181 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.181 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.181 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.181 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.181 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-16T02:05:54.181 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-16T02:05:54.181 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.181 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.181 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.181 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.181 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.182 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.182 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.182 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.182 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.182 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.182 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.182 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.182 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-16T02:05:54.182 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-16T02:05:54.182 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.182 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.182 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.182 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.182 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.183 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.183 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.183 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.183 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.183 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.183 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.183 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.183 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_ENABLED": { 2024-09-16T02:05:54.183 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-16T02:05:54.183 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.183 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.183 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.183 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.183 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.184 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.184 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.184 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.184 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.184 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.184 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.184 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.184 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-16T02:05:54.184 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-16T02:05:54.184 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.184 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.184 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.184 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-16T02:05:54.185 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.185 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.185 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.185 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.185 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.185 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.185 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.185 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.185 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-16T02:05:54.185 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-16T02:05:54.185 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.185 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.185 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.185 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-09-16T02:05:54.186 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.186 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.186 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.186 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.186 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.186 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.186 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.186 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.186 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-16T02:05:54.186 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-16T02:05:54.186 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.186 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.186 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.186 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "8", 2024-09-16T02:05:54.187 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.187 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.187 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.187 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.187 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.187 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.187 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.187 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.187 INFO:teuthology.orchestra.run.smithi105.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-16T02:05:54.187 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-16T02:05:54.187 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.187 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.187 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.187 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "45", 2024-09-16T02:05:54.188 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.188 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.188 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.188 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.188 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.188 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.188 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.188 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.188 INFO:teuthology.orchestra.run.smithi105.stdout: "RGW_API_ACCESS_KEY": { 2024-09-16T02:05:54.188 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-16T02:05:54.188 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.188 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.188 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.188 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.189 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.189 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.189 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.189 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.189 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.189 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.189 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.189 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.189 INFO:teuthology.orchestra.run.smithi105.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-16T02:05:54.189 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-16T02:05:54.189 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.189 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.189 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.189 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "admin", 2024-09-16T02:05:54.190 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.190 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.190 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.190 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.190 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.190 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.190 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.190 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.190 INFO:teuthology.orchestra.run.smithi105.stdout: "RGW_API_SECRET_KEY": { 2024-09-16T02:05:54.190 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-16T02:05:54.190 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.190 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.190 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.190 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.191 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.191 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.191 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.191 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.191 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.191 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.191 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.191 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.191 INFO:teuthology.orchestra.run.smithi105.stdout: "RGW_API_SSL_VERIFY": { 2024-09-16T02:05:54.191 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-16T02:05:54.191 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.191 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.191 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.191 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.192 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.192 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.192 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.192 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.192 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.192 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.192 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.192 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.192 INFO:teuthology.orchestra.run.smithi105.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-16T02:05:54.192 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-16T02:05:54.192 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.192 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.192 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.192 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0", 2024-09-16T02:05:54.193 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.193 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.193 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.193 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.193 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.193 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.193 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.193 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.193 INFO:teuthology.orchestra.run.smithi105.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-16T02:05:54.193 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-16T02:05:54.193 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.193 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.193 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.193 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-09-16T02:05:54.194 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.194 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.194 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.194 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.194 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.194 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.194 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.194 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.194 INFO:teuthology.orchestra.run.smithi105.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-16T02:05:54.194 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-16T02:05:54.194 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.194 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.194 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.194 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "5", 2024-09-16T02:05:54.195 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.195 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.195 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.195 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.195 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.195 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.195 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.195 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.195 INFO:teuthology.orchestra.run.smithi105.stdout: "cross_origin_url": { 2024-09-16T02:05:54.195 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "cross_origin_url", 2024-09-16T02:05:54.195 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.195 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.195 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.195 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.196 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.196 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.196 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.196 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.196 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.196 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.196 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.196 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.196 INFO:teuthology.orchestra.run.smithi105.stdout: "crt_file": { 2024-09-16T02:05:54.196 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "crt_file", 2024-09-16T02:05:54.196 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.196 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.196 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.196 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.197 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.197 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.197 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.197 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.197 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.197 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.197 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.197 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.197 INFO:teuthology.orchestra.run.smithi105.stdout: "debug": { 2024-09-16T02:05:54.197 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "debug", 2024-09-16T02:05:54.197 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.197 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.197 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.197 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.198 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.198 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.198 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.198 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Enable/disable debug options", 2024-09-16T02:05:54.198 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.198 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.198 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.198 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.198 INFO:teuthology.orchestra.run.smithi105.stdout: "jwt_token_ttl": { 2024-09-16T02:05:54.198 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "jwt_token_ttl", 2024-09-16T02:05:54.198 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.198 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.198 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.198 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "28800", 2024-09-16T02:05:54.199 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.199 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.199 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.199 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.199 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.199 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.199 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.199 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.199 INFO:teuthology.orchestra.run.smithi105.stdout: "key_file": { 2024-09-16T02:05:54.199 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "key_file", 2024-09-16T02:05:54.199 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.199 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.199 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.200 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.200 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.200 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.200 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.200 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.200 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.200 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.200 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.200 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.200 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.200 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.200 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.200 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.200 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.201 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.201 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.201 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.201 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.201 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.201 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.201 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.201 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.201 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.201 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.201 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.201 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.201 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.201 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.202 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.202 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.202 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.202 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.202 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.202 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.202 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.202 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.202 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.202 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.202 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.202 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.202 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.202 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.203 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.203 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.203 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.203 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.203 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.203 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.203 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.203 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.203 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.203 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.203 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.203 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.203 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.203 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.204 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.204 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.204 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.204 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.204 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.204 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.204 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.204 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.204 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.204 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.204 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.204 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.204 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.204 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.205 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.205 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.205 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.205 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.205 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.205 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.205 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.205 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.205 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.205 INFO:teuthology.orchestra.run.smithi105.stdout: "motd": { 2024-09-16T02:05:54.205 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "motd", 2024-09-16T02:05:54.205 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.205 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.205 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.206 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.206 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.206 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.206 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.206 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "The message of the day", 2024-09-16T02:05:54.206 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.206 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.206 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.206 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.206 INFO:teuthology.orchestra.run.smithi105.stdout: "redirect_resolve_ip_addr": { 2024-09-16T02:05:54.206 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "redirect_resolve_ip_addr", 2024-09-16T02:05:54.206 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.206 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.206 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.207 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.207 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.207 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.207 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.207 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.207 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.207 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.207 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.207 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.207 INFO:teuthology.orchestra.run.smithi105.stdout: "server_addr": { 2024-09-16T02:05:54.207 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "server_addr", 2024-09-16T02:05:54.207 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.207 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.207 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.208 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "::", 2024-09-16T02:05:54.208 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.208 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.208 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.208 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.208 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.208 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.208 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.208 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.208 INFO:teuthology.orchestra.run.smithi105.stdout: "server_port": { 2024-09-16T02:05:54.208 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "server_port", 2024-09-16T02:05:54.208 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.208 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.208 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.209 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "8080", 2024-09-16T02:05:54.209 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.209 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.209 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.209 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.209 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.209 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.209 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.209 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.209 INFO:teuthology.orchestra.run.smithi105.stdout: "ssl": { 2024-09-16T02:05:54.209 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ssl", 2024-09-16T02:05:54.209 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.209 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.209 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.210 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.210 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.210 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.210 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.210 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.210 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.210 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.210 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.210 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.210 INFO:teuthology.orchestra.run.smithi105.stdout: "ssl_server_port": { 2024-09-16T02:05:54.210 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ssl_server_port", 2024-09-16T02:05:54.210 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.210 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.210 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.211 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "8443", 2024-09-16T02:05:54.211 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.211 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.211 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.211 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.211 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.211 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.211 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.211 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.211 INFO:teuthology.orchestra.run.smithi105.stdout: "standby_behaviour": { 2024-09-16T02:05:54.211 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "standby_behaviour", 2024-09-16T02:05:54.211 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.211 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.211 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.212 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "redirect", 2024-09-16T02:05:54.212 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.212 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.212 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.212 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.212 INFO:teuthology.orchestra.run.smithi105.stdout: "redirect" 2024-09-16T02:05:54.212 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.212 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.212 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.212 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.212 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.212 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.212 INFO:teuthology.orchestra.run.smithi105.stdout: "standby_error_status_code": { 2024-09-16T02:05:54.212 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "standby_error_status_code", 2024-09-16T02:05:54.213 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.213 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.213 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.213 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "500", 2024-09-16T02:05:54.213 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "400", 2024-09-16T02:05:54.213 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "599", 2024-09-16T02:05:54.213 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.213 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.213 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.213 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.213 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.213 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.213 INFO:teuthology.orchestra.run.smithi105.stdout: "url_prefix": { 2024-09-16T02:05:54.213 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "url_prefix", 2024-09-16T02:05:54.214 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.214 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.214 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.214 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.214 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.214 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.214 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.214 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.214 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.214 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.214 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.214 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.214 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.214 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.215 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.215 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "devicehealth", 2024-09-16T02:05:54.215 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.215 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.215 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.215 INFO:teuthology.orchestra.run.smithi105.stdout: "enable_monitoring": { 2024-09-16T02:05:54.215 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "enable_monitoring", 2024-09-16T02:05:54.215 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.215 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.215 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.215 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.215 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.215 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.215 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.216 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "monitor device health metrics", 2024-09-16T02:05:54.216 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.216 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.216 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.216 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.216 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.216 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.216 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.216 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.216 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.216 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.216 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.216 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.217 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.217 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.217 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.217 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.217 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.217 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.217 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.217 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.217 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.217 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.217 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.217 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.217 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.217 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.218 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.218 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.218 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.218 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.218 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.218 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.218 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.218 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.218 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.218 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.218 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.218 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.218 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.218 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.218 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.219 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.219 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.219 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.219 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.219 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.219 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.219 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.219 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.219 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.219 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.219 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.219 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.219 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.219 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.220 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.220 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.220 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.220 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.220 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.220 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.220 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.220 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.220 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.220 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.220 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.220 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.220 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.220 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.221 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.221 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.221 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.221 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.221 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.221 INFO:teuthology.orchestra.run.smithi105.stdout: "mark_out_threshold": { 2024-09-16T02:05:54.221 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "mark_out_threshold", 2024-09-16T02:05:54.221 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.221 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.221 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.221 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "2419200", 2024-09-16T02:05:54.221 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.221 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.221 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.222 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-16T02:05:54.222 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.222 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.222 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.222 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.222 INFO:teuthology.orchestra.run.smithi105.stdout: "pool_name": { 2024-09-16T02:05:54.222 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "pool_name", 2024-09-16T02:05:54.222 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.222 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.222 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.222 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "device_health_metrics", 2024-09-16T02:05:54.222 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.222 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.222 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.223 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-16T02:05:54.223 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.223 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.223 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.223 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.223 INFO:teuthology.orchestra.run.smithi105.stdout: "retention_period": { 2024-09-16T02:05:54.223 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "retention_period", 2024-09-16T02:05:54.223 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.223 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.223 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.223 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "15552000", 2024-09-16T02:05:54.223 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.223 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.223 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.224 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how long to retain device health metrics", 2024-09-16T02:05:54.224 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.224 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.224 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.224 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.224 INFO:teuthology.orchestra.run.smithi105.stdout: "scrape_frequency": { 2024-09-16T02:05:54.224 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "scrape_frequency", 2024-09-16T02:05:54.224 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.224 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.224 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.224 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "86400", 2024-09-16T02:05:54.224 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.224 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.225 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.225 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-16T02:05:54.225 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.225 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.225 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.225 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.225 INFO:teuthology.orchestra.run.smithi105.stdout: "self_heal": { 2024-09-16T02:05:54.225 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "self_heal", 2024-09-16T02:05:54.225 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.225 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.225 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.225 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.225 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.226 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.226 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.226 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-16T02:05:54.226 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.226 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.226 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.226 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.226 INFO:teuthology.orchestra.run.smithi105.stdout: "sleep_interval": { 2024-09-16T02:05:54.226 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "sleep_interval", 2024-09-16T02:05:54.226 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.226 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.226 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.226 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "600", 2024-09-16T02:05:54.226 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.227 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.227 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.227 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how frequently to wake up and check device health", 2024-09-16T02:05:54.227 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.227 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.227 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.227 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.227 INFO:teuthology.orchestra.run.smithi105.stdout: "warn_threshold": { 2024-09-16T02:05:54.227 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "warn_threshold", 2024-09-16T02:05:54.227 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.227 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.227 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.227 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "7257600", 2024-09-16T02:05:54.227 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.228 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.228 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.228 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-16T02:05:54.228 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.228 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.228 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.228 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.228 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.228 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.228 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.228 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "diskprediction_local", 2024-09-16T02:05:54.228 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.228 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.229 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.229 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.229 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.229 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.229 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.229 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.229 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.229 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.229 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.229 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.229 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.229 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.229 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.229 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.230 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.230 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.230 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.230 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.230 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.230 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.230 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.230 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.230 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.230 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.230 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.230 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.230 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.231 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.231 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.231 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.231 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.231 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.231 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.231 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.231 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.231 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.231 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.231 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.231 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.231 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.231 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.232 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.232 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.232 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.232 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.232 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.232 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.232 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.232 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.232 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.232 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.232 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.232 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.232 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.232 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.233 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.233 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.233 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.233 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.233 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.233 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.233 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.233 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.233 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.233 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.233 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.233 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.233 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.233 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.234 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.234 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.234 INFO:teuthology.orchestra.run.smithi105.stdout: "predict_interval": { 2024-09-16T02:05:54.234 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "predict_interval", 2024-09-16T02:05:54.234 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.234 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.234 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.234 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "86400", 2024-09-16T02:05:54.234 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.234 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.234 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.234 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.234 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.234 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.234 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.235 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.235 INFO:teuthology.orchestra.run.smithi105.stdout: "predictor_model": { 2024-09-16T02:05:54.235 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "predictor_model", 2024-09-16T02:05:54.235 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.235 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.235 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.235 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "prophetstor", 2024-09-16T02:05:54.235 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.235 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.235 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.235 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.235 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.235 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.235 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.236 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.236 INFO:teuthology.orchestra.run.smithi105.stdout: "sleep_interval": { 2024-09-16T02:05:54.236 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "sleep_interval", 2024-09-16T02:05:54.236 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.236 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.236 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.236 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "600", 2024-09-16T02:05:54.236 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.236 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.236 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.236 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.236 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.236 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.236 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.237 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.237 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.237 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.237 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.237 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "influx", 2024-09-16T02:05:54.237 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": false, 2024-09-16T02:05:54.237 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "influxdb python module not found", 2024-09-16T02:05:54.237 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.237 INFO:teuthology.orchestra.run.smithi105.stdout: "batch_size": { 2024-09-16T02:05:54.237 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "batch_size", 2024-09-16T02:05:54.237 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.237 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.237 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.237 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "5000", 2024-09-16T02:05:54.238 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.238 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.238 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.238 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-16T02:05:54.238 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.238 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.238 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.238 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.238 INFO:teuthology.orchestra.run.smithi105.stdout: "database": { 2024-09-16T02:05:54.238 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "database", 2024-09-16T02:05:54.238 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.238 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.238 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.239 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "ceph", 2024-09-16T02:05:54.239 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.239 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.239 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.239 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-16T02:05:54.239 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.239 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.239 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.239 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.239 INFO:teuthology.orchestra.run.smithi105.stdout: "hostname": { 2024-09-16T02:05:54.239 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "hostname", 2024-09-16T02:05:54.239 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.239 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.240 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.240 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.240 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.240 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.240 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.240 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "InfluxDB server hostname", 2024-09-16T02:05:54.240 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.240 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.240 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.240 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.240 INFO:teuthology.orchestra.run.smithi105.stdout: "interval": { 2024-09-16T02:05:54.240 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "interval", 2024-09-16T02:05:54.240 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.241 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.241 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.241 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "30", 2024-09-16T02:05:54.241 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "5", 2024-09-16T02:05:54.241 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.241 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.241 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-16T02:05:54.241 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.241 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.241 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.241 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.241 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.241 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.241 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.242 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.242 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.242 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.242 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.242 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.242 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.242 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.242 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.242 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.242 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.242 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.242 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.242 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.242 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.243 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.243 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.243 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.243 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.243 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.243 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.243 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.243 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.243 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.243 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.243 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.243 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.243 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.243 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.244 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.244 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.244 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.244 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.244 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.244 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.244 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.244 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.244 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.244 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.244 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.244 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.244 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.244 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.245 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.245 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.245 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.245 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.245 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.245 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.245 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.245 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.245 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.245 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.245 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.245 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.245 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.245 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.246 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.246 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.246 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.246 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.246 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.246 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.246 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.246 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.246 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.246 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.246 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.246 INFO:teuthology.orchestra.run.smithi105.stdout: "password": { 2024-09-16T02:05:54.246 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "password", 2024-09-16T02:05:54.246 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.247 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.247 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.247 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.247 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.247 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.247 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.247 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "password of InfluxDB server user", 2024-09-16T02:05:54.247 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.247 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.247 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.247 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.247 INFO:teuthology.orchestra.run.smithi105.stdout: "port": { 2024-09-16T02:05:54.247 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "port", 2024-09-16T02:05:54.247 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.248 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.248 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.248 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "8086", 2024-09-16T02:05:54.248 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.248 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.248 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.248 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "InfluxDB server port", 2024-09-16T02:05:54.248 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.248 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.248 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.248 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.248 INFO:teuthology.orchestra.run.smithi105.stdout: "ssl": { 2024-09-16T02:05:54.248 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ssl", 2024-09-16T02:05:54.248 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.249 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.249 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.249 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "false", 2024-09-16T02:05:54.249 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.249 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.249 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.249 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T02:05:54.249 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.249 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.249 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.249 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.249 INFO:teuthology.orchestra.run.smithi105.stdout: "threads": { 2024-09-16T02:05:54.249 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "threads", 2024-09-16T02:05:54.249 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.250 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.250 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.250 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "5", 2024-09-16T02:05:54.250 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "1", 2024-09-16T02:05:54.250 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "32", 2024-09-16T02:05:54.250 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.250 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-16T02:05:54.250 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.250 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.250 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.250 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.250 INFO:teuthology.orchestra.run.smithi105.stdout: "username": { 2024-09-16T02:05:54.250 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "username", 2024-09-16T02:05:54.250 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.250 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.251 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.251 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.251 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.251 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.251 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.251 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "username of InfluxDB server user", 2024-09-16T02:05:54.251 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.251 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.251 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.251 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.251 INFO:teuthology.orchestra.run.smithi105.stdout: "verify_ssl": { 2024-09-16T02:05:54.251 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "verify_ssl", 2024-09-16T02:05:54.251 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.251 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.252 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.252 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "true", 2024-09-16T02:05:54.252 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.252 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.252 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.252 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T02:05:54.252 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.252 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.252 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.252 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.252 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.252 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.252 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.253 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "insights", 2024-09-16T02:05:54.253 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.253 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.253 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.253 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.253 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.253 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.253 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.253 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.253 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.253 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.253 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.253 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.253 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.254 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.254 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.254 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.254 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.254 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.254 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.254 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.254 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.254 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.254 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.254 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.254 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.254 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.254 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.255 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.255 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.255 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.255 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.255 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.255 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.255 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.255 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.255 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.255 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.255 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.255 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.255 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.255 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.256 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.256 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.256 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.256 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.256 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.256 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.256 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.256 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.256 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.256 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.256 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.256 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.256 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.256 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.257 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.257 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.257 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.257 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.257 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.257 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.257 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.257 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.257 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.257 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.257 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.257 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.257 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.257 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.258 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.258 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.258 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.258 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.258 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.258 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.258 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.258 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "iostat", 2024-09-16T02:05:54.258 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.258 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.258 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.258 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.258 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.258 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.259 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.259 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.259 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.259 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.259 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.259 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.259 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.259 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.259 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.259 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.259 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.259 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.259 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.259 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.260 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.260 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.260 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.260 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.260 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.260 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.260 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.260 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.260 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.260 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.260 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.260 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.260 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.260 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.261 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.261 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.261 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.261 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.261 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.261 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.261 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.261 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.261 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.261 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.261 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.261 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.261 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.261 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.262 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.262 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.262 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.262 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.262 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.262 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.262 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.262 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.262 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.262 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.262 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.262 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.262 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.262 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.263 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.263 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.263 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.263 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.263 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.263 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.263 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.263 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.263 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.263 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.263 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.263 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.263 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.263 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.263 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "k8sevents", 2024-09-16T02:05:54.264 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.264 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.264 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.264 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_event_retention_days": { 2024-09-16T02:05:54.264 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ceph_event_retention_days", 2024-09-16T02:05:54.264 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.264 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.264 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.264 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "7", 2024-09-16T02:05:54.264 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.264 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.264 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.264 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-16T02:05:54.264 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.265 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.265 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.265 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.265 INFO:teuthology.orchestra.run.smithi105.stdout: "config_check_secs": { 2024-09-16T02:05:54.265 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "config_check_secs", 2024-09-16T02:05:54.265 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.265 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.265 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.265 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-09-16T02:05:54.265 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "10", 2024-09-16T02:05:54.265 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.265 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.265 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-16T02:05:54.266 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.266 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.266 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.266 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.266 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.266 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.266 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.266 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.266 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.266 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.266 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.266 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.266 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.266 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.267 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.267 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.267 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.267 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.267 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.267 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.267 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.267 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.267 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.267 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.267 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.267 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.267 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.267 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.268 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.268 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.268 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.268 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.268 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.268 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.268 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.268 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.268 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.268 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.268 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.268 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.268 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.269 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.269 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.269 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.269 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.269 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.269 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.269 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.269 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.269 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.269 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.269 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.269 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.269 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.269 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.270 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.270 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.270 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.270 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.270 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.270 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.270 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.270 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.270 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.270 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.270 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.270 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.270 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.270 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.271 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.271 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.271 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.271 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.271 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.271 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.271 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.271 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.271 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "localpool", 2024-09-16T02:05:54.271 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.271 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.271 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.271 INFO:teuthology.orchestra.run.smithi105.stdout: "failure_domain": { 2024-09-16T02:05:54.271 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "failure_domain", 2024-09-16T02:05:54.272 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.272 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.272 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.272 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "host", 2024-09-16T02:05:54.272 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.272 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.272 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.272 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "failure domain for any created local pool", 2024-09-16T02:05:54.272 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-16T02:05:54.272 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.272 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.272 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.272 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.272 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.273 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.273 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.273 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.273 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.273 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.273 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.273 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.273 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.273 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.273 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.273 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.273 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.273 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.273 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.274 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.274 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.274 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.274 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.274 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.274 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.274 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.274 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.274 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.274 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.274 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.274 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.274 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.274 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.275 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.275 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.275 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.275 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.275 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.275 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.275 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.275 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.275 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.275 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.275 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.275 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.275 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.275 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.275 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.276 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.276 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.276 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.276 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.276 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.276 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.276 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.276 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.276 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.276 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.276 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.276 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.276 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.276 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.277 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.277 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.277 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.277 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.277 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.277 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.277 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.277 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.277 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.277 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.277 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.277 INFO:teuthology.orchestra.run.smithi105.stdout: "min_size": { 2024-09-16T02:05:54.277 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "min_size", 2024-09-16T02:05:54.277 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.278 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.278 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.278 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.278 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.278 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.278 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.278 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "default min_size for any created local pool", 2024-09-16T02:05:54.278 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-16T02:05:54.278 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.278 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.278 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.278 INFO:teuthology.orchestra.run.smithi105.stdout: "num_rep": { 2024-09-16T02:05:54.278 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "num_rep", 2024-09-16T02:05:54.278 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.279 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.279 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.279 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "3", 2024-09-16T02:05:54.279 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.279 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.279 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.279 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "default replica count for any created local pool", 2024-09-16T02:05:54.279 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.279 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.279 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.280 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.280 INFO:teuthology.orchestra.run.smithi105.stdout: "pg_num": { 2024-09-16T02:05:54.280 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "pg_num", 2024-09-16T02:05:54.280 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.280 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.280 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.280 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "128", 2024-09-16T02:05:54.280 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.280 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.280 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.280 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "default pg_num for any created local pool", 2024-09-16T02:05:54.280 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.280 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.281 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.281 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.281 INFO:teuthology.orchestra.run.smithi105.stdout: "prefix": { 2024-09-16T02:05:54.281 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "prefix", 2024-09-16T02:05:54.281 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.281 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.281 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.281 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.281 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.281 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.281 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.281 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "name prefix for any created local pool", 2024-09-16T02:05:54.281 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.281 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.282 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.282 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.282 INFO:teuthology.orchestra.run.smithi105.stdout: "subtree": { 2024-09-16T02:05:54.282 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "subtree", 2024-09-16T02:05:54.282 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.282 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.282 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.282 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "rack", 2024-09-16T02:05:54.282 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.282 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.282 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.282 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-16T02:05:54.282 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-16T02:05:54.282 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.283 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.283 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.283 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.283 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.283 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.283 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "mds_autoscaler", 2024-09-16T02:05:54.283 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.283 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.283 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.283 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.283 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.283 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.283 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.283 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.284 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.284 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.284 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.284 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.284 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.284 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.284 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.284 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.284 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.284 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.284 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.284 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.284 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.285 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.285 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.285 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.285 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.285 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.285 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.285 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.285 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.285 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.285 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.285 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.285 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.285 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.286 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.286 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.286 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.286 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.286 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.286 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.286 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.286 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.286 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.286 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.286 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.286 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.286 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.286 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.286 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.287 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.287 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.287 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.287 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.287 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.287 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.287 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.287 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.287 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.287 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.287 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.287 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.287 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.288 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.288 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.288 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.288 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.288 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.288 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.288 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.288 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.288 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.288 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.288 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.288 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.288 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.288 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.289 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "mirroring", 2024-09-16T02:05:54.289 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.289 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.289 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.289 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.289 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.289 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.289 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.289 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.289 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.289 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.289 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.289 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.289 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.289 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.290 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.290 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.290 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.290 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.290 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.290 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.290 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.290 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.290 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.290 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.290 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.290 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.290 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.290 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.291 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.291 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.291 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.291 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.291 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.291 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.291 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.291 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.291 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.291 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.291 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.291 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.291 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.291 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.292 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.292 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.292 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.292 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.292 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.292 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.292 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.292 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.292 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.292 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.292 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.292 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.292 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.292 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.293 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.293 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.293 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.293 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.293 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.293 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.293 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.293 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.293 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.293 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.293 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.293 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.293 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.294 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.294 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.294 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.294 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.294 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.294 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.294 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.294 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "nfs", 2024-09-16T02:05:54.294 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.294 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.294 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.294 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.294 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.294 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.295 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.295 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.295 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.295 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.295 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.295 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.295 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.295 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.295 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.295 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.295 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.295 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.295 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.295 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.296 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.296 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.296 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.296 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.296 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.296 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.296 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.296 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.296 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.296 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.296 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.296 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.296 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.297 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.297 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.297 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.297 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.297 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.297 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.297 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.297 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.297 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.297 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.297 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.297 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.297 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.297 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.298 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.298 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.298 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.298 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.298 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.298 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.298 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.298 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.298 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.298 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.298 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.298 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.298 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.298 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.299 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.299 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.299 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.299 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.299 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.299 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.299 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.299 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.299 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.299 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.299 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.299 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.299 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.299 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.300 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.300 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "orchestrator", 2024-09-16T02:05:54.300 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.300 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.300 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.300 INFO:teuthology.orchestra.run.smithi105.stdout: "fail_fs": { 2024-09-16T02:05:54.300 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "fail_fs", 2024-09-16T02:05:54.300 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.300 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.300 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.300 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.300 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.300 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.300 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.301 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-16T02:05:54.301 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.301 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.301 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.301 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.301 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.301 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.301 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.301 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.301 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.301 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.301 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.301 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.301 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.302 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.302 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.302 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.302 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.302 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.302 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.302 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.302 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.302 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.302 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.302 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.302 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.302 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.302 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.302 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.303 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.303 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.303 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.303 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.303 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.303 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.303 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.303 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.303 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.303 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.303 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.303 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.303 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.303 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.304 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.304 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.304 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.304 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.304 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.304 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.304 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.304 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.304 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.304 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.304 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.304 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.304 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.304 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.305 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.305 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.305 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.305 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.305 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.305 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.305 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.305 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.305 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.305 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.305 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.305 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.305 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.306 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.306 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.306 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.306 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.306 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.306 INFO:teuthology.orchestra.run.smithi105.stdout: "orchestrator": { 2024-09-16T02:05:54.306 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "orchestrator", 2024-09-16T02:05:54.306 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.306 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.306 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.306 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.306 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.306 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.306 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.306 INFO:teuthology.orchestra.run.smithi105.stdout: "cephadm", 2024-09-16T02:05:54.307 INFO:teuthology.orchestra.run.smithi105.stdout: "rook", 2024-09-16T02:05:54.307 INFO:teuthology.orchestra.run.smithi105.stdout: "test_orchestrator" 2024-09-16T02:05:54.307 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.307 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Orchestrator backend", 2024-09-16T02:05:54.307 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.307 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.307 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.307 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.307 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.307 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.307 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.307 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "osd_perf_query", 2024-09-16T02:05:54.307 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.308 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.308 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.308 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.308 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.308 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.308 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.308 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.308 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.308 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.308 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.308 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.308 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.308 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.308 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.309 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.309 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.309 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.309 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.309 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.309 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.309 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.309 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.309 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.309 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.309 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.309 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.309 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.310 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.310 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.310 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.310 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.310 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.310 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.310 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.310 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.310 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.310 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.310 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.310 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.310 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.310 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.311 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.311 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.311 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.311 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.311 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.311 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.311 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.311 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.311 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.311 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.311 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.311 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.311 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.312 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.312 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.312 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.312 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.312 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.312 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.312 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.312 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.312 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.312 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.312 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.312 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.312 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.313 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.313 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.313 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.313 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.313 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.313 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.313 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.313 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.313 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "osd_support", 2024-09-16T02:05:54.313 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.313 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.313 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.313 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.313 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.314 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.314 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.314 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.314 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.314 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.314 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.314 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.314 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.314 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.314 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.314 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.314 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.315 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.315 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.315 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.315 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.315 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.315 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.315 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.315 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.315 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.315 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.315 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.315 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.315 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.316 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.316 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.316 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.316 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.316 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.316 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.316 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.316 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.316 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.316 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.316 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.316 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.316 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.317 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.317 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.317 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.317 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.317 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.317 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.317 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.317 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.317 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.317 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.317 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.317 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.317 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.317 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.318 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.318 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.318 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.318 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.318 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.318 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.318 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.318 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.318 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.318 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.318 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.318 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.318 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.318 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.319 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.319 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.319 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.319 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.319 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.319 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "pg_autoscaler", 2024-09-16T02:05:54.319 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.319 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.319 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.319 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.319 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.319 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.319 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.320 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.320 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.320 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.320 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.320 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.320 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.320 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.320 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.320 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.320 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.320 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.320 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.320 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.320 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.321 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.321 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.321 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.321 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.321 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.321 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.321 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.321 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.321 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.321 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.321 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.321 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.321 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.321 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.322 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.322 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.322 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.322 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.322 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.322 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.322 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.322 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.322 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.322 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.322 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.322 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.322 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.323 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.323 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.323 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.323 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.323 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.323 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.323 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.323 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.323 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.323 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.323 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.323 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.323 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.324 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.324 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.324 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.324 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.324 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.324 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.324 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.324 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.324 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.324 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.324 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.324 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.324 INFO:teuthology.orchestra.run.smithi105.stdout: "sleep_interval": { 2024-09-16T02:05:54.325 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "sleep_interval", 2024-09-16T02:05:54.325 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.325 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.325 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.325 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "60", 2024-09-16T02:05:54.325 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.325 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.325 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.325 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.325 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.325 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.325 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.325 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.325 INFO:teuthology.orchestra.run.smithi105.stdout: "threshold": { 2024-09-16T02:05:54.326 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "threshold", 2024-09-16T02:05:54.326 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-09-16T02:05:54.326 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.326 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.326 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "3.0", 2024-09-16T02:05:54.326 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "1.0", 2024-09-16T02:05:54.326 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.326 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.326 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "scaling threshold", 2024-09-16T02:05:54.326 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-16T02:05:54.326 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.326 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.326 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.327 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.327 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.327 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.327 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "progress", 2024-09-16T02:05:54.327 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.327 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.327 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.327 INFO:teuthology.orchestra.run.smithi105.stdout: "allow_pg_recovery_event": { 2024-09-16T02:05:54.327 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "allow_pg_recovery_event", 2024-09-16T02:05:54.327 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.327 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.327 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.327 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.327 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.328 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.328 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.328 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-16T02:05:54.328 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.328 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.328 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.328 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.328 INFO:teuthology.orchestra.run.smithi105.stdout: "enabled": { 2024-09-16T02:05:54.328 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "enabled", 2024-09-16T02:05:54.328 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.328 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.328 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.328 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.329 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.329 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.329 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.329 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.329 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.329 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.329 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.329 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.329 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.329 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.329 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.329 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.329 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.329 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.329 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.330 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.330 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.330 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.330 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.330 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.330 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.330 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.330 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.330 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.330 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.330 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.330 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.330 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.331 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.331 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.331 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.331 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.331 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.331 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.331 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.331 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.331 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.331 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.331 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.331 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.331 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.331 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.332 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.332 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.332 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.332 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.332 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.332 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.332 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.332 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.332 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.332 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.332 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.332 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.332 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.332 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.333 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.333 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.333 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.333 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.333 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.333 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.333 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.333 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.333 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.333 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.333 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.333 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.333 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.333 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.334 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.334 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.334 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.334 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.334 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.334 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.334 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.334 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.334 INFO:teuthology.orchestra.run.smithi105.stdout: "max_completed_events": { 2024-09-16T02:05:54.334 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "max_completed_events", 2024-09-16T02:05:54.334 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.334 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.334 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.334 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "50", 2024-09-16T02:05:54.335 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.335 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.335 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.335 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "number of past completed events to remember", 2024-09-16T02:05:54.335 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.335 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.335 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.335 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.335 INFO:teuthology.orchestra.run.smithi105.stdout: "sleep_interval": { 2024-09-16T02:05:54.335 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "sleep_interval", 2024-09-16T02:05:54.335 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.335 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.336 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.336 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "5", 2024-09-16T02:05:54.336 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.336 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.336 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.336 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how long the module is going to sleep", 2024-09-16T02:05:54.336 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.336 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.336 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.336 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.336 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.336 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.336 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.337 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "prometheus", 2024-09-16T02:05:54.337 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.337 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.337 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.337 INFO:teuthology.orchestra.run.smithi105.stdout: "cache": { 2024-09-16T02:05:54.337 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "cache", 2024-09-16T02:05:54.337 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.337 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.337 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.337 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.337 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.337 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.337 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.338 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.338 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.338 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.338 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.338 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.338 INFO:teuthology.orchestra.run.smithi105.stdout: "exclude_perf_counters": { 2024-09-16T02:05:54.338 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "exclude_perf_counters", 2024-09-16T02:05:54.338 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.338 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.338 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.338 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.338 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.338 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.338 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.339 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-16T02:05:54.339 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-16T02:05:54.339 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.339 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.339 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.339 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.339 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.339 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.339 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.339 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.339 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.339 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.339 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.340 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.340 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.340 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.340 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.340 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.340 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.340 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.340 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.340 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.340 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.340 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.340 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.340 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.340 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.341 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.341 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.341 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.341 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.341 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.341 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.341 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.341 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.341 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.341 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.341 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.341 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.341 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.342 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.342 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.342 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.342 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.342 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.342 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.342 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.342 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.342 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.342 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.342 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.342 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.342 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.343 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.343 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.343 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.343 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.343 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.343 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.343 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.343 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.343 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.343 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.343 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.343 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.343 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.343 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.344 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.344 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.344 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.344 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.344 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.344 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.344 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.344 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.344 INFO:teuthology.orchestra.run.smithi105.stdout: "rbd_stats_pools": { 2024-09-16T02:05:54.344 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rbd_stats_pools", 2024-09-16T02:05:54.344 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.344 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.344 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.345 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.345 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.345 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.345 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.345 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.345 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.345 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.345 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.345 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.345 INFO:teuthology.orchestra.run.smithi105.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-16T02:05:54.345 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-16T02:05:54.345 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.345 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.346 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.346 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "300", 2024-09-16T02:05:54.346 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.346 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.346 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.346 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.346 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.346 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.346 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.346 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.346 INFO:teuthology.orchestra.run.smithi105.stdout: "scrape_interval": { 2024-09-16T02:05:54.346 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "scrape_interval", 2024-09-16T02:05:54.346 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-09-16T02:05:54.346 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.347 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.347 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "15.0", 2024-09-16T02:05:54.347 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.347 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.347 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.347 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.347 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.347 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.347 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.347 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.347 INFO:teuthology.orchestra.run.smithi105.stdout: "server_addr": { 2024-09-16T02:05:54.347 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "server_addr", 2024-09-16T02:05:54.347 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.347 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.348 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.348 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "::", 2024-09-16T02:05:54.348 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.348 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.348 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.348 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-16T02:05:54.348 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.348 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.348 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.348 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.348 INFO:teuthology.orchestra.run.smithi105.stdout: "server_port": { 2024-09-16T02:05:54.348 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "server_port", 2024-09-16T02:05:54.348 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.349 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.349 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.349 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "9283", 2024-09-16T02:05:54.349 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.349 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.349 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.349 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-16T02:05:54.349 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.349 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.349 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.349 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.349 INFO:teuthology.orchestra.run.smithi105.stdout: "stale_cache_strategy": { 2024-09-16T02:05:54.349 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "stale_cache_strategy", 2024-09-16T02:05:54.350 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.350 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.350 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.350 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "log", 2024-09-16T02:05:54.350 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.350 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.350 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.350 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.350 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.350 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.350 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.350 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.350 INFO:teuthology.orchestra.run.smithi105.stdout: "standby_behaviour": { 2024-09-16T02:05:54.350 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "standby_behaviour", 2024-09-16T02:05:54.350 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.351 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.351 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.351 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "default", 2024-09-16T02:05:54.351 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.351 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.351 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.351 INFO:teuthology.orchestra.run.smithi105.stdout: "default", 2024-09-16T02:05:54.351 INFO:teuthology.orchestra.run.smithi105.stdout: "error" 2024-09-16T02:05:54.351 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.351 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.351 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.351 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.351 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.352 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.352 INFO:teuthology.orchestra.run.smithi105.stdout: "standby_error_status_code": { 2024-09-16T02:05:54.352 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "standby_error_status_code", 2024-09-16T02:05:54.352 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.352 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.352 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.352 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "500", 2024-09-16T02:05:54.352 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "400", 2024-09-16T02:05:54.352 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "599", 2024-09-16T02:05:54.352 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.352 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.352 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.352 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.352 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.353 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.353 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.353 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.353 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.353 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rbd_support", 2024-09-16T02:05:54.353 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.353 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.353 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.353 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.353 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.353 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.353 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.353 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.353 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.354 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.354 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.354 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.354 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.354 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.354 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.354 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.354 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.354 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.354 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.354 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.354 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.354 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.354 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.355 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.355 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.355 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.355 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.355 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.355 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.355 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.355 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.355 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.355 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.355 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.355 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.355 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.355 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.356 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.356 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.356 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.356 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.356 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.356 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.356 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.356 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.356 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.356 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.356 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.356 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.356 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.357 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.357 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.357 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.357 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.357 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.357 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.357 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.357 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.357 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.357 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.357 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.357 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.357 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.357 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.358 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.358 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.358 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.358 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.358 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.358 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.358 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.358 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.358 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.358 INFO:teuthology.orchestra.run.smithi105.stdout: "max_concurrent_snap_create": { 2024-09-16T02:05:54.358 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "max_concurrent_snap_create", 2024-09-16T02:05:54.358 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.358 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.358 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.359 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-09-16T02:05:54.359 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.359 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.359 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.359 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.359 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.359 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.359 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.359 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.359 INFO:teuthology.orchestra.run.smithi105.stdout: "mirror_snapshot_schedule": { 2024-09-16T02:05:54.359 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "mirror_snapshot_schedule", 2024-09-16T02:05:54.359 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.359 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.359 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.360 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.360 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.360 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.360 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.360 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.360 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.360 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.360 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.360 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.360 INFO:teuthology.orchestra.run.smithi105.stdout: "trash_purge_schedule": { 2024-09-16T02:05:54.360 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "trash_purge_schedule", 2024-09-16T02:05:54.360 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.360 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.360 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.361 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.361 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.361 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.361 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.361 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.361 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.361 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.361 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.361 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.361 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.361 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.361 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.361 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "restful", 2024-09-16T02:05:54.361 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.362 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.362 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.362 INFO:teuthology.orchestra.run.smithi105.stdout: "enable_auth": { 2024-09-16T02:05:54.362 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "enable_auth", 2024-09-16T02:05:54.362 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.362 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.362 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.362 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.362 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.362 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.362 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.362 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.362 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.362 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.362 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.363 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.363 INFO:teuthology.orchestra.run.smithi105.stdout: "key_file": { 2024-09-16T02:05:54.363 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "key_file", 2024-09-16T02:05:54.363 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.363 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.363 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.363 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.363 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.363 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.363 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.363 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.363 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.363 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.364 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.364 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.364 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.364 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.364 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.364 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.364 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.364 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.364 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.364 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.364 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.364 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.364 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.364 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.365 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.365 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.365 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.365 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.365 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.365 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.365 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.365 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.365 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.365 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.365 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.365 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.365 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.365 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.366 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.366 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.366 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.366 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.366 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.366 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.366 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.366 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.366 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.366 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.366 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.366 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.366 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.366 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.367 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.367 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.367 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.367 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.367 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.367 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.367 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.367 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.367 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.367 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.367 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.367 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.367 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.368 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.368 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.368 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.368 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.368 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.368 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.368 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.368 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.368 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.368 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.368 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.368 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.368 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.368 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.369 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.369 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.369 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.369 INFO:teuthology.orchestra.run.smithi105.stdout: "server_addr": { 2024-09-16T02:05:54.369 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "server_addr", 2024-09-16T02:05:54.369 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.369 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.369 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.369 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.369 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.369 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.369 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.369 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.369 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.370 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.370 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.370 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.370 INFO:teuthology.orchestra.run.smithi105.stdout: "server_port": { 2024-09-16T02:05:54.370 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "server_port", 2024-09-16T02:05:54.370 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.370 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.370 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.370 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.370 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.370 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.370 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.370 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.370 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.371 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.371 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.371 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.371 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.371 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.371 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.371 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rgw", 2024-09-16T02:05:54.371 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.371 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.371 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.371 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.371 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.371 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.371 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.372 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.372 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.372 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.372 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.372 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.372 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.372 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.372 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.372 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.372 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.372 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.372 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.372 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.372 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.373 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.373 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.373 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.373 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.373 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.373 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.373 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.373 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.373 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.373 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.373 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.373 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.373 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.373 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.373 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.374 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.374 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.374 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.374 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.374 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.374 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.374 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.374 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.374 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.374 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.374 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.374 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.374 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.374 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.375 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.375 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.375 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.375 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.375 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.375 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.375 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.375 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.375 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.375 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.375 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.375 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.375 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.375 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.376 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.376 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.376 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.376 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.376 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.376 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.376 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.376 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.376 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.376 INFO:teuthology.orchestra.run.smithi105.stdout: "secondary_zone_period_retry_limit": { 2024-09-16T02:05:54.376 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-16T02:05:54.376 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.376 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.376 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.377 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "5", 2024-09-16T02:05:54.377 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.377 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.377 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.377 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-16T02:05:54.377 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.377 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.377 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.377 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.377 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.377 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.377 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.378 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rook", 2024-09-16T02:05:54.378 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.378 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.378 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.378 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.378 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.378 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.378 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.378 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.378 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.378 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.378 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.378 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.378 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.379 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.379 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.379 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.379 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.379 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.379 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.379 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.379 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.379 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.379 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.379 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.379 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.379 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.379 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.380 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.380 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.380 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.380 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.380 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.380 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.380 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.380 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.380 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.380 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.380 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.380 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.380 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.380 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.381 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.381 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.381 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.381 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.381 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.381 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.381 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.381 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.381 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.381 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.381 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.381 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.381 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.382 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.382 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.382 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.382 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.382 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.382 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.382 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.382 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.382 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.382 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.382 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.382 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.382 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.382 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.383 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.383 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.383 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.383 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.383 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.383 INFO:teuthology.orchestra.run.smithi105.stdout: "storage_class": { 2024-09-16T02:05:54.383 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "storage_class", 2024-09-16T02:05:54.383 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.383 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.383 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.383 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "local", 2024-09-16T02:05:54.383 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.383 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.383 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.384 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-16T02:05:54.384 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.384 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.384 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.384 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.384 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.384 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.384 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.384 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "selftest", 2024-09-16T02:05:54.384 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.384 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.384 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.384 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.384 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.385 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.385 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.385 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.385 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.385 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.385 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.385 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.385 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.385 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.385 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.385 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.385 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.385 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.386 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.386 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.386 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.386 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.386 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.386 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.386 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.386 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.386 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.386 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.386 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.386 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.386 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.386 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.387 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.387 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.387 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.387 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.387 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.387 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.387 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.387 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.387 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.387 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.387 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.387 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.387 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.387 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.388 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.388 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.388 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.388 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.388 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.388 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.388 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.388 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.388 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.388 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.388 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.388 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.388 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.388 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.389 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.389 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.389 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.389 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.389 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.389 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.389 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.389 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.389 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.389 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.389 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.389 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.389 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.389 INFO:teuthology.orchestra.run.smithi105.stdout: "roption1": { 2024-09-16T02:05:54.390 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "roption1", 2024-09-16T02:05:54.390 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.390 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.390 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.390 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.390 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.390 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.390 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.390 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.390 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.390 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.390 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.390 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.390 INFO:teuthology.orchestra.run.smithi105.stdout: "roption2": { 2024-09-16T02:05:54.391 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "roption2", 2024-09-16T02:05:54.391 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.391 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.391 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.391 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "xyz", 2024-09-16T02:05:54.391 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.391 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.391 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.391 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.391 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.391 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.391 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.391 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.391 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption1": { 2024-09-16T02:05:54.392 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption1", 2024-09-16T02:05:54.392 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.392 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.392 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.392 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.392 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.392 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.392 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.392 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.392 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.392 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.392 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.392 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.392 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption2": { 2024-09-16T02:05:54.393 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption2", 2024-09-16T02:05:54.393 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.393 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.393 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.393 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.393 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.393 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.393 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.393 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.393 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.393 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.393 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.393 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.394 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption3": { 2024-09-16T02:05:54.394 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption3", 2024-09-16T02:05:54.394 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-09-16T02:05:54.394 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.394 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.394 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.394 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.394 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.394 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.394 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.394 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.394 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.394 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.394 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.395 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption4": { 2024-09-16T02:05:54.395 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption4", 2024-09-16T02:05:54.395 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.395 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.395 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.395 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.395 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.395 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.395 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.395 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.395 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.395 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.395 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.395 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.396 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption5": { 2024-09-16T02:05:54.396 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption5", 2024-09-16T02:05:54.396 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.396 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.396 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.396 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.396 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.396 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.396 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.396 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.396 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.396 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.396 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.397 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.397 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption6": { 2024-09-16T02:05:54.397 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption6", 2024-09-16T02:05:54.397 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.397 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.397 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.397 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.397 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.397 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.397 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.397 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.397 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.397 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.397 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.398 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.398 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption7": { 2024-09-16T02:05:54.398 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption7", 2024-09-16T02:05:54.398 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.398 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.398 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.398 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.398 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "1", 2024-09-16T02:05:54.398 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "42", 2024-09-16T02:05:54.398 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.398 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.398 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.398 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.398 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.399 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.399 INFO:teuthology.orchestra.run.smithi105.stdout: "testkey": { 2024-09-16T02:05:54.399 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "testkey", 2024-09-16T02:05:54.399 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.399 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.399 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.399 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.399 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.399 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.399 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.399 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.399 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.399 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.399 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.400 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.400 INFO:teuthology.orchestra.run.smithi105.stdout: "testlkey": { 2024-09-16T02:05:54.400 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "testlkey", 2024-09-16T02:05:54.400 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.400 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.400 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.400 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.400 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.400 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.400 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.400 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.400 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.400 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.400 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.401 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.401 INFO:teuthology.orchestra.run.smithi105.stdout: "testnewline": { 2024-09-16T02:05:54.401 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "testnewline", 2024-09-16T02:05:54.401 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.401 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.401 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.401 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.401 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.401 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.401 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.401 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.401 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.401 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.402 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.402 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.402 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.402 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.402 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.402 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "snap_schedule", 2024-09-16T02:05:54.402 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.402 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.402 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.402 INFO:teuthology.orchestra.run.smithi105.stdout: "allow_m_granularity": { 2024-09-16T02:05:54.402 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "allow_m_granularity", 2024-09-16T02:05:54.402 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.402 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.402 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.402 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.403 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.403 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.403 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.403 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "allow minute scheduled snapshots", 2024-09-16T02:05:54.403 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.403 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.403 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.403 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.403 INFO:teuthology.orchestra.run.smithi105.stdout: "dump_on_update": { 2024-09-16T02:05:54.403 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "dump_on_update", 2024-09-16T02:05:54.403 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.403 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.403 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.403 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.404 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.404 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.404 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.404 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "dump database to debug log on update", 2024-09-16T02:05:54.404 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.404 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.404 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.404 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.404 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.404 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.404 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.404 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.404 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.405 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.405 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.405 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.405 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.405 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.405 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.405 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.405 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.405 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.405 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.405 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.405 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.405 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.405 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.406 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.406 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.406 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.406 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.406 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.406 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.406 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.406 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.406 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.406 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.406 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.406 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.406 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.406 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.407 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.407 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.407 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.407 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.407 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.407 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.407 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.407 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.407 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.407 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.407 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.407 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.407 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.407 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.408 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.408 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.408 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.408 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.408 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.408 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.408 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.408 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.408 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.408 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.408 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.408 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.408 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.409 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.409 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.409 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.409 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.409 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.409 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.409 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.409 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.409 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.409 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.409 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.409 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.409 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.409 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "stats", 2024-09-16T02:05:54.410 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.410 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.410 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.410 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.410 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.410 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.410 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.410 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.410 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.410 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.410 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.410 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.410 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.410 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.411 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.411 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.411 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.411 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.411 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.411 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.411 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.411 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.411 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.411 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.411 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.411 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.411 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.412 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.412 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.412 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.412 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.412 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.412 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.412 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.412 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.412 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.412 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.412 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.412 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.412 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.412 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.413 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.413 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.413 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.413 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.413 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.413 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.413 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.413 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.413 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.413 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.413 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.413 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.413 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.413 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.414 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.414 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.414 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.414 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.414 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.414 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.414 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.414 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.414 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.414 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.414 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.414 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.414 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.414 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.415 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.415 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.415 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.415 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.415 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.415 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.415 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.415 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "status", 2024-09-16T02:05:54.415 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.415 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.415 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.415 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.415 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.416 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.416 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.416 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.416 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.416 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.416 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.416 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.416 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.416 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.416 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.416 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.416 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.416 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.416 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.417 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.417 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.417 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.417 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.417 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.417 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.417 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.417 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.417 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.417 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.417 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.417 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.417 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.417 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.418 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.418 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.418 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.418 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.418 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.418 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.418 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.418 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.418 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.418 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.418 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.418 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.418 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.418 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.419 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.419 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.419 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.419 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.419 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.419 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.419 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.419 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.419 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.419 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.419 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.419 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.419 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.419 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.420 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.420 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.420 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.420 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.420 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.420 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.420 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.420 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.420 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.420 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.420 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.420 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.420 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.421 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.421 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.421 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "telegraf", 2024-09-16T02:05:54.421 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.421 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.421 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.421 INFO:teuthology.orchestra.run.smithi105.stdout: "address": { 2024-09-16T02:05:54.421 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "address", 2024-09-16T02:05:54.421 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.421 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.421 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.421 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-16T02:05:54.421 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.421 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.422 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.422 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.422 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.422 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.422 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.422 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.422 INFO:teuthology.orchestra.run.smithi105.stdout: "interval": { 2024-09-16T02:05:54.422 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "interval", 2024-09-16T02:05:54.422 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.422 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.422 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.422 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "15", 2024-09-16T02:05:54.422 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.423 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.423 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.423 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.423 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.423 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.423 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.423 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.423 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.423 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.423 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.423 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.423 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.423 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.423 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.424 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.424 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.424 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.424 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.424 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.424 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.424 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.424 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.424 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.424 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.424 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.424 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.424 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.424 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.425 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.425 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.425 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.425 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.425 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.425 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.425 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.425 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.425 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.425 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.425 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.425 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.425 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.426 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.426 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.426 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.426 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.426 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.426 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.426 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.426 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.426 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.426 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.426 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.426 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.426 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.426 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.427 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.427 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.427 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.427 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.427 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.427 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.427 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.427 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.427 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.427 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.427 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.427 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.427 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.427 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.428 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.428 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.428 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.428 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.428 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.428 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.428 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.428 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.428 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.428 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.428 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.428 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "telemetry", 2024-09-16T02:05:54.428 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.429 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.429 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.429 INFO:teuthology.orchestra.run.smithi105.stdout: "channel_basic": { 2024-09-16T02:05:54.429 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "channel_basic", 2024-09-16T02:05:54.429 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.429 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.429 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.429 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.429 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.429 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.429 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.429 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-16T02:05:54.429 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.429 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.430 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.430 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.430 INFO:teuthology.orchestra.run.smithi105.stdout: "channel_crash": { 2024-09-16T02:05:54.430 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "channel_crash", 2024-09-16T02:05:54.430 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.430 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.430 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.430 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.430 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.430 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.430 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.430 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-16T02:05:54.430 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.430 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.431 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.431 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.431 INFO:teuthology.orchestra.run.smithi105.stdout: "channel_device": { 2024-09-16T02:05:54.431 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "channel_device", 2024-09-16T02:05:54.431 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.431 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.431 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.431 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.431 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.431 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.431 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.431 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-16T02:05:54.431 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.431 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.432 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.432 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.432 INFO:teuthology.orchestra.run.smithi105.stdout: "channel_ident": { 2024-09-16T02:05:54.432 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "channel_ident", 2024-09-16T02:05:54.432 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.432 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.432 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.432 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.432 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.432 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.432 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.432 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-16T02:05:54.432 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.432 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.433 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.433 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.433 INFO:teuthology.orchestra.run.smithi105.stdout: "channel_perf": { 2024-09-16T02:05:54.433 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "channel_perf", 2024-09-16T02:05:54.433 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.433 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.433 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.433 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.433 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.433 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.433 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.433 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-16T02:05:54.433 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.434 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.434 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.434 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.434 INFO:teuthology.orchestra.run.smithi105.stdout: "contact": { 2024-09-16T02:05:54.434 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "contact", 2024-09-16T02:05:54.434 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.434 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.434 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.434 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.434 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.434 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.434 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.434 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.434 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.435 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.435 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.435 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.435 INFO:teuthology.orchestra.run.smithi105.stdout: "description": { 2024-09-16T02:05:54.435 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "description", 2024-09-16T02:05:54.435 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.435 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.435 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.435 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.435 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.435 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.435 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.435 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.435 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.436 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.436 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.436 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.436 INFO:teuthology.orchestra.run.smithi105.stdout: "device_url": { 2024-09-16T02:05:54.436 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "device_url", 2024-09-16T02:05:54.436 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.436 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.436 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.436 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-16T02:05:54.436 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.436 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.436 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.436 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.436 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.437 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.437 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.437 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.437 INFO:teuthology.orchestra.run.smithi105.stdout: "enabled": { 2024-09-16T02:05:54.437 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "enabled", 2024-09-16T02:05:54.437 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.437 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.437 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.437 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.437 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.437 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.437 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.437 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.438 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.438 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.438 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.438 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.438 INFO:teuthology.orchestra.run.smithi105.stdout: "interval": { 2024-09-16T02:05:54.438 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "interval", 2024-09-16T02:05:54.438 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.438 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.438 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.438 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "24", 2024-09-16T02:05:54.438 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "8", 2024-09-16T02:05:54.438 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.438 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.438 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.439 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.439 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.439 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.439 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.439 INFO:teuthology.orchestra.run.smithi105.stdout: "last_opt_revision": { 2024-09-16T02:05:54.439 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "last_opt_revision", 2024-09-16T02:05:54.439 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.439 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.439 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.439 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "1", 2024-09-16T02:05:54.439 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.439 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.439 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.439 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.440 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.440 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.440 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.440 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.440 INFO:teuthology.orchestra.run.smithi105.stdout: "leaderboard": { 2024-09-16T02:05:54.440 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "leaderboard", 2024-09-16T02:05:54.440 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.440 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.440 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.440 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.440 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.440 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.440 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.440 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.441 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.441 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.441 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.441 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.441 INFO:teuthology.orchestra.run.smithi105.stdout: "leaderboard_description": { 2024-09-16T02:05:54.441 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "leaderboard_description", 2024-09-16T02:05:54.441 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.441 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.441 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.441 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.441 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.441 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.441 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.441 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.442 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.442 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.442 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.442 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.442 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.442 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.442 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.442 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.442 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.442 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.442 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.442 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.442 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.442 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.443 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.443 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.443 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.443 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.443 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.443 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.443 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.443 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.443 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.443 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.443 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.443 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.443 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.443 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.444 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.444 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.444 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.444 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.444 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.444 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.444 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.444 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.444 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.444 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.444 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.444 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.444 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.444 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.445 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.445 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.445 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.445 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.445 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.445 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.445 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.445 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.445 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.445 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.445 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.445 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.445 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.445 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.446 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.446 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.446 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.446 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.446 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.446 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.446 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.446 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.446 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.446 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.446 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.446 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.446 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.446 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.447 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.447 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.447 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.447 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.447 INFO:teuthology.orchestra.run.smithi105.stdout: "organization": { 2024-09-16T02:05:54.447 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "organization", 2024-09-16T02:05:54.447 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.447 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.447 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.447 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.581+0000 7f8d6c749640 1 -- 172.21.15.105:0/4022527764 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f8d500a3250 msgr2=0x7f8d500a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:54.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.581+0000 7f8d6c749640 1 --2- 172.21.15.105:0/4022527764 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f8d500a3250 0x7f8d500a5710 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8d60004770 tx=0x7f8d60000bc0 comp rx=0 tx=0).stop 2024-09-16T02:05:54.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.581+0000 7f8d6c749640 1 -- 172.21.15.105:0/4022527764 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d641025c0 msgr2=0x7f8d64103520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:54.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.581+0000 7f8d6c749640 1 --2- 172.21.15.105:0/4022527764 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d641025c0 0x7f8d64103520 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c00eaa0 tx=0x7f8d5c00ef60 comp rx=0 tx=0).stop 2024-09-16T02:05:54.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.581+0000 7f8d6c749640 1 -- 172.21.15.105:0/4022527764 shutdown_connections 2024-09-16T02:05:54.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.581+0000 7f8d6c749640 1 --2- 172.21.15.105:0/4022527764 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f8d500a3250 0x7f8d500a5710 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:54.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.581+0000 7f8d6c749640 1 --2- 172.21.15.105:0/4022527764 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d641025c0 0x7f8d64103520 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:54.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.581+0000 7f8d6c749640 1 --2- 172.21.15.105:0/4022527764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d64101c00 0x7f8d64102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:54.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.581+0000 7f8d6c749640 1 --2- 172.21.15.105:0/4022527764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d64100a10 0x7f8d641049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:54.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.581+0000 7f8d6c749640 1 -- 172.21.15.105:0/4022527764 >> 172.21.15.105:0/4022527764 conn(0x7f8d640fc170 msgr2=0x7f8d640693b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:54.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.581+0000 7f8d6c749640 1 -- 172.21.15.105:0/4022527764 shutdown_connections 2024-09-16T02:05:54.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:05:53.581+0000 7f8d6c749640 1 -- 172.21.15.105:0/4022527764 wait complete. 2024-09-16T02:05:54.451 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.451 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.451 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.451 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.451 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.451 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.451 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.451 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.452 INFO:teuthology.orchestra.run.smithi105.stdout: "proxy": { 2024-09-16T02:05:54.452 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "proxy", 2024-09-16T02:05:54.452 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.452 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.452 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.452 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.452 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.452 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.452 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.452 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.452 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.452 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.452 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.453 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.453 INFO:teuthology.orchestra.run.smithi105.stdout: "url": { 2024-09-16T02:05:54.453 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "url", 2024-09-16T02:05:54.453 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.453 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.453 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.453 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-16T02:05:54.453 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.453 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.453 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.453 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.453 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.453 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.454 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.454 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.454 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.454 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.454 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.454 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "test_orchestrator", 2024-09-16T02:05:54.454 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.454 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.454 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.454 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.454 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.454 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.454 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.455 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.455 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.455 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.455 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.455 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.455 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.455 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.455 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.455 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.455 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.455 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.455 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.455 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.455 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.456 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.456 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.456 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.456 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.456 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.456 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.456 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.456 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.456 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.456 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.456 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.456 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.456 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.456 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.457 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.457 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.457 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.457 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.457 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.457 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.457 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.457 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.457 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.457 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.457 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.457 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.457 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.457 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.458 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.458 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.458 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.458 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.458 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.458 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.458 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.458 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.458 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.458 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.458 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.458 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.458 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.458 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.459 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.459 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.459 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.459 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.459 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.459 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.459 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.459 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.459 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.459 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.459 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.459 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.459 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.459 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "volumes", 2024-09-16T02:05:54.460 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.460 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.460 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.460 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.460 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.460 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.460 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.460 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.460 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.460 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.460 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.460 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.460 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.460 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.461 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.461 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.461 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.461 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.461 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.461 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.461 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.461 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.461 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.461 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.461 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.461 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.461 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.461 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.461 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.462 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.462 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.462 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.462 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.462 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.462 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.463 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.463 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.464 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.464 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.464 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.464 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.464 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.464 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.464 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.464 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.464 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.464 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.464 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.464 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.464 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.464 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.465 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.465 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.465 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.465 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.465 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.465 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.465 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.465 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.465 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.465 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.465 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.465 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.465 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.465 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.466 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.466 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.466 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.466 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.466 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.466 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.466 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.466 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.466 INFO:teuthology.orchestra.run.smithi105.stdout: "max_concurrent_clones": { 2024-09-16T02:05:54.466 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "max_concurrent_clones", 2024-09-16T02:05:54.466 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.466 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.466 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.466 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "4", 2024-09-16T02:05:54.467 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.467 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.467 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.467 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-16T02:05:54.467 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.467 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.467 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.467 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.467 INFO:teuthology.orchestra.run.smithi105.stdout: "periodic_async_work": { 2024-09-16T02:05:54.467 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "periodic_async_work", 2024-09-16T02:05:54.467 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.467 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.467 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.468 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.468 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.468 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.468 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.468 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Periodically check for async work", 2024-09-16T02:05:54.468 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.468 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.468 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.468 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.468 INFO:teuthology.orchestra.run.smithi105.stdout: "snapshot_clone_delay": { 2024-09-16T02:05:54.468 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "snapshot_clone_delay", 2024-09-16T02:05:54.468 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.468 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.468 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.469 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0", 2024-09-16T02:05:54.469 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.469 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.469 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.469 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-16T02:05:54.469 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.469 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.469 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.469 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.469 INFO:teuthology.orchestra.run.smithi105.stdout: "snapshot_clone_no_wait": { 2024-09-16T02:05:54.469 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "snapshot_clone_no_wait", 2024-09-16T02:05:54.469 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.469 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.470 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.470 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.470 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.470 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.470 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.470 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-16T02:05:54.470 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.470 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.470 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.470 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.470 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.470 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.470 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.470 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "zabbix", 2024-09-16T02:05:54.471 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.471 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.471 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.471 INFO:teuthology.orchestra.run.smithi105.stdout: "discovery_interval": { 2024-09-16T02:05:54.471 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "discovery_interval", 2024-09-16T02:05:54.471 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "uint", 2024-09-16T02:05:54.471 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.471 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.471 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "100", 2024-09-16T02:05:54.471 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.471 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.471 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.471 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.471 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.472 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.472 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.472 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.472 INFO:teuthology.orchestra.run.smithi105.stdout: "identifier": { 2024-09-16T02:05:54.472 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "identifier", 2024-09-16T02:05:54.472 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.472 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.472 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.472 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.472 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.472 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.472 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.472 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.473 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.473 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.473 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.473 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.473 INFO:teuthology.orchestra.run.smithi105.stdout: "interval": { 2024-09-16T02:05:54.473 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "interval", 2024-09-16T02:05:54.473 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.473 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.473 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.473 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "60", 2024-09-16T02:05:54.473 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.473 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.473 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.473 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.474 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.474 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.474 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.474 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.474 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.474 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.474 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.474 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.474 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.474 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.474 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.474 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.474 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.474 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.474 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.475 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.475 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.475 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.475 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.475 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.475 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.475 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.475 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.475 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.475 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.475 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.475 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.475 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.475 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.476 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.476 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.476 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.476 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.476 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.476 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.476 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.476 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.476 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.476 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.476 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.476 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.476 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.476 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.477 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.477 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.477 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.477 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.477 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.477 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.477 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.477 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.477 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.477 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.477 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.477 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.477 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.477 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.478 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.478 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.478 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.478 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.478 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.478 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.478 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.478 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.478 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.478 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.478 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.478 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.478 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.478 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.479 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.479 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.479 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.479 INFO:teuthology.orchestra.run.smithi105.stdout: "zabbix_host": { 2024-09-16T02:05:54.479 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "zabbix_host", 2024-09-16T02:05:54.479 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.479 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.479 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.479 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.479 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.479 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.479 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.479 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.479 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.480 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.480 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.480 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.480 INFO:teuthology.orchestra.run.smithi105.stdout: "zabbix_port": { 2024-09-16T02:05:54.480 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "zabbix_port", 2024-09-16T02:05:54.480 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.480 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.480 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.480 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10051", 2024-09-16T02:05:54.480 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.480 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.480 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.480 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.481 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.481 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.481 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.481 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.481 INFO:teuthology.orchestra.run.smithi105.stdout: "zabbix_sender": { 2024-09-16T02:05:54.481 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "zabbix_sender", 2024-09-16T02:05:54.481 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.481 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.481 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.481 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-16T02:05:54.481 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.481 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.481 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.482 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.482 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.482 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.482 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.482 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.482 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.482 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.482 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-16T02:05:54.482 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.482 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.482 INFO:teuthology.orchestra.run.smithi105.stdout: "modules": [ 2024-09-16T02:05:54.482 INFO:teuthology.orchestra.run.smithi105.stdout: "cephadm", 2024-09-16T02:05:54.482 INFO:teuthology.orchestra.run.smithi105.stdout: "dashboard", 2024-09-16T02:05:54.483 INFO:teuthology.orchestra.run.smithi105.stdout: "iostat", 2024-09-16T02:05:54.483 INFO:teuthology.orchestra.run.smithi105.stdout: "nfs", 2024-09-16T02:05:54.483 INFO:teuthology.orchestra.run.smithi105.stdout: "restful" 2024-09-16T02:05:54.483 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.483 INFO:teuthology.orchestra.run.smithi105.stdout: "available_modules": [ 2024-09-16T02:05:54.483 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.483 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "alerts", 2024-09-16T02:05:54.483 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.483 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.483 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.483 INFO:teuthology.orchestra.run.smithi105.stdout: "interval": { 2024-09-16T02:05:54.483 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "interval", 2024-09-16T02:05:54.483 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.483 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.483 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.484 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "60", 2024-09-16T02:05:54.484 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.484 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.484 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.484 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "How frequently to reexamine health status", 2024-09-16T02:05:54.484 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.484 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.484 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.484 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.484 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.484 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.484 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.484 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.485 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.485 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.485 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.485 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.485 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.485 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.485 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.485 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.485 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.485 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.485 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.485 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.485 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.485 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.486 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.486 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.486 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.486 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.486 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.486 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.486 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.486 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.486 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.486 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.486 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.486 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.486 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.486 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.487 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.487 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.487 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.487 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.487 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.487 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.487 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.487 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.487 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.487 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.487 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.487 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.487 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.487 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.488 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.488 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.488 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.488 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.488 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.488 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.488 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.488 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.488 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.488 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.488 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.488 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.488 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.489 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.489 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.489 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.489 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.489 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.489 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.489 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.489 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.489 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.489 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.489 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.489 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_destination": { 2024-09-16T02:05:54.489 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_destination", 2024-09-16T02:05:54.489 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.490 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.490 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.490 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.490 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.490 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.490 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.490 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Email address to send alerts to", 2024-09-16T02:05:54.490 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.490 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.490 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.490 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.490 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_from_name": { 2024-09-16T02:05:54.490 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_from_name", 2024-09-16T02:05:54.490 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.491 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.491 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.491 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "Ceph", 2024-09-16T02:05:54.491 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.491 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.491 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.491 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Email From: name", 2024-09-16T02:05:54.491 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.491 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.491 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.491 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.491 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_host": { 2024-09-16T02:05:54.491 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_host", 2024-09-16T02:05:54.491 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.491 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.492 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.492 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.492 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.492 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.492 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.492 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "SMTP server", 2024-09-16T02:05:54.492 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.492 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.492 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.492 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.492 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_password": { 2024-09-16T02:05:54.492 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_password", 2024-09-16T02:05:54.492 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.493 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.493 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.493 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.493 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.493 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.493 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.493 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Password to authenticate with", 2024-09-16T02:05:54.493 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.493 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.493 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.493 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.493 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_port": { 2024-09-16T02:05:54.493 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_port", 2024-09-16T02:05:54.493 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.494 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.494 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.494 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "465", 2024-09-16T02:05:54.494 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.494 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.494 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.494 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "SMTP port", 2024-09-16T02:05:54.494 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.494 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.494 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.494 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.494 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_sender": { 2024-09-16T02:05:54.494 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_sender", 2024-09-16T02:05:54.494 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.494 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.495 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.495 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.495 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.495 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.495 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.495 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "SMTP envelope sender", 2024-09-16T02:05:54.495 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.495 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.495 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.495 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.495 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_ssl": { 2024-09-16T02:05:54.495 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_ssl", 2024-09-16T02:05:54.495 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.495 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.496 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.496 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.496 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.496 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.496 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.496 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-16T02:05:54.496 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.496 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.496 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.496 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.496 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_user": { 2024-09-16T02:05:54.496 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_user", 2024-09-16T02:05:54.496 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.496 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.497 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.497 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.497 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.497 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.497 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.497 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "User to authenticate as", 2024-09-16T02:05:54.497 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.497 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.497 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.497 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.497 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.497 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.497 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.498 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "balancer", 2024-09-16T02:05:54.498 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.498 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.498 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.498 INFO:teuthology.orchestra.run.smithi105.stdout: "active": { 2024-09-16T02:05:54.498 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "active", 2024-09-16T02:05:54.498 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.498 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.498 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.498 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.498 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.498 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.498 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.498 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "automatically balance PGs across cluster", 2024-09-16T02:05:54.499 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.499 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.499 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.499 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.499 INFO:teuthology.orchestra.run.smithi105.stdout: "begin_time": { 2024-09-16T02:05:54.499 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "begin_time", 2024-09-16T02:05:54.499 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.499 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.499 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.499 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0000", 2024-09-16T02:05:54.499 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.499 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.499 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.499 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "beginning time of day to automatically balance", 2024-09-16T02:05:54.500 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T02:05:54.500 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.500 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.500 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.500 INFO:teuthology.orchestra.run.smithi105.stdout: "begin_weekday": { 2024-09-16T02:05:54.500 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "begin_weekday", 2024-09-16T02:05:54.500 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "uint", 2024-09-16T02:05:54.500 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.500 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.500 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0", 2024-09-16T02:05:54.500 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "0", 2024-09-16T02:05:54.500 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "6", 2024-09-16T02:05:54.500 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.501 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-16T02:05:54.501 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T02:05:54.501 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.501 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.501 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.501 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_compat_max_iterations": { 2024-09-16T02:05:54.501 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "crush_compat_max_iterations", 2024-09-16T02:05:54.501 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "uint", 2024-09-16T02:05:54.501 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.501 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.501 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "25", 2024-09-16T02:05:54.501 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "1", 2024-09-16T02:05:54.501 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "250", 2024-09-16T02:05:54.501 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.502 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-16T02:05:54.502 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.502 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.502 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.502 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.502 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_compat_metrics": { 2024-09-16T02:05:54.502 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "crush_compat_metrics", 2024-09-16T02:05:54.502 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.502 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.502 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.502 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "pgs,objects,bytes", 2024-09-16T02:05:54.502 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.502 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.502 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.503 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-16T02:05:54.503 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-16T02:05:54.503 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.503 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.503 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.503 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_compat_step": { 2024-09-16T02:05:54.503 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "crush_compat_step", 2024-09-16T02:05:54.503 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-09-16T02:05:54.503 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.503 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.503 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0.5", 2024-09-16T02:05:54.503 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "0.001", 2024-09-16T02:05:54.503 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "0.999", 2024-09-16T02:05:54.503 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.504 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "aggressiveness of optimization", 2024-09-16T02:05:54.504 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-16T02:05:54.504 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.504 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.504 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.504 INFO:teuthology.orchestra.run.smithi105.stdout: "end_time": { 2024-09-16T02:05:54.504 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "end_time", 2024-09-16T02:05:54.504 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.504 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.504 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.504 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "2359", 2024-09-16T02:05:54.504 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.504 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.505 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.505 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "ending time of day to automatically balance", 2024-09-16T02:05:54.505 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T02:05:54.505 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.505 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.505 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.505 INFO:teuthology.orchestra.run.smithi105.stdout: "end_weekday": { 2024-09-16T02:05:54.505 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "end_weekday", 2024-09-16T02:05:54.505 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "uint", 2024-09-16T02:05:54.505 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.505 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.505 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0", 2024-09-16T02:05:54.505 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "0", 2024-09-16T02:05:54.505 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "6", 2024-09-16T02:05:54.506 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.506 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-16T02:05:54.506 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T02:05:54.506 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.506 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.506 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.506 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.506 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.506 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.506 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.506 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.506 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.506 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.506 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.507 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.507 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.507 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.507 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.507 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.507 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.507 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.507 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.507 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.507 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.507 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.507 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.507 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.507 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.507 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.508 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.508 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.508 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.508 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.508 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.508 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.508 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.508 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.508 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.508 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.508 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.508 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.508 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.508 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.509 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.509 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.509 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.509 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.509 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.509 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.509 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.509 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.509 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.509 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.509 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.509 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.509 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.509 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.510 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.510 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.510 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.510 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.510 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.510 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.510 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.510 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.510 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.510 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.510 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.510 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.510 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.510 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.511 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.511 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.511 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.511 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.511 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.511 INFO:teuthology.orchestra.run.smithi105.stdout: "min_score": { 2024-09-16T02:05:54.511 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "min_score", 2024-09-16T02:05:54.511 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-09-16T02:05:54.511 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.511 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.511 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0", 2024-09-16T02:05:54.511 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.511 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.511 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.512 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-16T02:05:54.512 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.512 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.512 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.512 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.512 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": { 2024-09-16T02:05:54.512 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "mode", 2024-09-16T02:05:54.512 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.512 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.512 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.512 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "upmap", 2024-09-16T02:05:54.512 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.512 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.512 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.513 INFO:teuthology.orchestra.run.smithi105.stdout: "crush-compat", 2024-09-16T02:05:54.513 INFO:teuthology.orchestra.run.smithi105.stdout: "none", 2024-09-16T02:05:54.513 INFO:teuthology.orchestra.run.smithi105.stdout: "read", 2024-09-16T02:05:54.513 INFO:teuthology.orchestra.run.smithi105.stdout: "upmap", 2024-09-16T02:05:54.513 INFO:teuthology.orchestra.run.smithi105.stdout: "upmap-read" 2024-09-16T02:05:54.513 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.513 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Balancer mode", 2024-09-16T02:05:54.513 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.513 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.513 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.513 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.513 INFO:teuthology.orchestra.run.smithi105.stdout: "pool_ids": { 2024-09-16T02:05:54.513 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "pool_ids", 2024-09-16T02:05:54.514 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.514 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.514 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.514 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.514 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.514 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.514 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.514 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-16T02:05:54.514 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.514 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.514 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.514 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.514 INFO:teuthology.orchestra.run.smithi105.stdout: "sleep_interval": { 2024-09-16T02:05:54.514 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "sleep_interval", 2024-09-16T02:05:54.515 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.515 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.515 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.515 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "60", 2024-09-16T02:05:54.515 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.515 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.515 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.515 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-16T02:05:54.515 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.515 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.515 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.515 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.515 INFO:teuthology.orchestra.run.smithi105.stdout: "upmap_max_deviation": { 2024-09-16T02:05:54.516 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "upmap_max_deviation", 2024-09-16T02:05:54.516 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.516 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.516 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.516 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "5", 2024-09-16T02:05:54.516 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "1", 2024-09-16T02:05:54.516 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.516 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.516 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-16T02:05:54.516 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-16T02:05:54.516 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.516 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.516 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.516 INFO:teuthology.orchestra.run.smithi105.stdout: "upmap_max_optimizations": { 2024-09-16T02:05:54.517 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "upmap_max_optimizations", 2024-09-16T02:05:54.517 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "uint", 2024-09-16T02:05:54.517 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.517 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.517 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-09-16T02:05:54.517 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.517 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.517 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.517 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-16T02:05:54.517 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.517 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.517 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.517 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.517 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.518 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.518 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.518 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "cephadm", 2024-09-16T02:05:54.518 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.518 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.518 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.518 INFO:teuthology.orchestra.run.smithi105.stdout: "agent_down_multiplier": { 2024-09-16T02:05:54.518 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "agent_down_multiplier", 2024-09-16T02:05:54.518 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-09-16T02:05:54.518 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.518 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.518 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "3.0", 2024-09-16T02:05:54.518 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.518 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.519 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.519 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-16T02:05:54.519 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.519 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.519 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.519 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.519 INFO:teuthology.orchestra.run.smithi105.stdout: "agent_refresh_rate": { 2024-09-16T02:05:54.519 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "agent_refresh_rate", 2024-09-16T02:05:54.519 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.519 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.519 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.519 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "20", 2024-09-16T02:05:54.519 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.519 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.520 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.520 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-16T02:05:54.520 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.520 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.520 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.520 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.520 INFO:teuthology.orchestra.run.smithi105.stdout: "agent_starting_port": { 2024-09-16T02:05:54.520 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "agent_starting_port", 2024-09-16T02:05:54.520 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.520 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.520 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.520 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "4721", 2024-09-16T02:05:54.520 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.521 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.521 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.521 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-16T02:05:54.521 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.521 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.521 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.521 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.521 INFO:teuthology.orchestra.run.smithi105.stdout: "allow_ptrace": { 2024-09-16T02:05:54.521 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "allow_ptrace", 2024-09-16T02:05:54.521 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.521 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.521 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.521 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.521 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.522 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.522 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.522 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-16T02:05:54.522 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-16T02:05:54.522 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.522 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.522 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.522 INFO:teuthology.orchestra.run.smithi105.stdout: "autotune_interval": { 2024-09-16T02:05:54.522 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "autotune_interval", 2024-09-16T02:05:54.522 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.522 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.522 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.522 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "600", 2024-09-16T02:05:54.522 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.522 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.523 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.523 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-16T02:05:54.523 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.523 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.523 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.523 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.523 INFO:teuthology.orchestra.run.smithi105.stdout: "autotune_memory_target_ratio": { 2024-09-16T02:05:54.523 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "autotune_memory_target_ratio", 2024-09-16T02:05:54.523 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-09-16T02:05:54.523 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.523 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.523 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0.7", 2024-09-16T02:05:54.523 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.523 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.524 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.524 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-16T02:05:54.524 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.524 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.524 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.524 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.524 INFO:teuthology.orchestra.run.smithi105.stdout: "cephadm_log_destination": { 2024-09-16T02:05:54.524 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "cephadm_log_destination", 2024-09-16T02:05:54.524 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.524 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.524 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.524 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.524 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.525 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.525 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.525 INFO:teuthology.orchestra.run.smithi105.stdout: "file", 2024-09-16T02:05:54.525 INFO:teuthology.orchestra.run.smithi105.stdout: "file,syslog", 2024-09-16T02:05:54.525 INFO:teuthology.orchestra.run.smithi105.stdout: "syslog" 2024-09-16T02:05:54.525 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.525 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-16T02:05:54.525 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.525 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.525 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.525 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.525 INFO:teuthology.orchestra.run.smithi105.stdout: "cgroups_split": { 2024-09-16T02:05:54.525 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "cgroups_split", 2024-09-16T02:05:54.525 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.525 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.526 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.526 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.526 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.526 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.526 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.526 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-16T02:05:54.526 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.526 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.526 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.526 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.526 INFO:teuthology.orchestra.run.smithi105.stdout: "config_checks_enabled": { 2024-09-16T02:05:54.526 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "config_checks_enabled", 2024-09-16T02:05:54.526 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.526 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.527 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.527 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.527 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.527 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.527 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.527 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-16T02:05:54.527 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.527 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.527 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.527 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.527 INFO:teuthology.orchestra.run.smithi105.stdout: "config_dashboard": { 2024-09-16T02:05:54.527 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "config_dashboard", 2024-09-16T02:05:54.527 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.528 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.528 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.528 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.528 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.528 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.528 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.528 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-16T02:05:54.528 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.528 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.528 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.528 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.528 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_alertmanager": { 2024-09-16T02:05:54.528 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_alertmanager", 2024-09-16T02:05:54.529 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.529 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.529 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.529 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-16T02:05:54.529 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.529 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.529 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.529 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Prometheus container image", 2024-09-16T02:05:54.529 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.529 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.529 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.529 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.529 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_base": { 2024-09-16T02:05:54.530 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_base", 2024-09-16T02:05:54.530 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.530 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.530 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.530 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-16T02:05:54.530 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.530 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.530 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.530 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Container image name, without the tag", 2024-09-16T02:05:54.530 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.530 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.530 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.530 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.530 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_elasticsearch": { 2024-09-16T02:05:54.531 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_elasticsearch", 2024-09-16T02:05:54.531 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.531 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.531 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.531 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-16T02:05:54.531 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.531 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.531 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.531 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "elasticsearch container image", 2024-09-16T02:05:54.531 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.531 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.531 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.531 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.531 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_grafana": { 2024-09-16T02:05:54.532 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_grafana", 2024-09-16T02:05:54.532 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.532 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.532 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.532 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-16T02:05:54.532 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.532 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.532 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.532 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Prometheus container image", 2024-09-16T02:05:54.532 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.532 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.532 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.532 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.533 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_haproxy": { 2024-09-16T02:05:54.533 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_haproxy", 2024-09-16T02:05:54.533 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.533 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.533 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.533 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-16T02:05:54.533 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.533 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.533 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.533 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "HAproxy container image", 2024-09-16T02:05:54.533 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.533 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.533 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.533 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.534 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_jaeger_agent": { 2024-09-16T02:05:54.534 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_jaeger_agent", 2024-09-16T02:05:54.534 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.534 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.534 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.534 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-16T02:05:54.534 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.534 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.534 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.534 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Jaeger agent container image", 2024-09-16T02:05:54.534 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.534 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.534 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.534 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.535 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_jaeger_collector": { 2024-09-16T02:05:54.535 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_jaeger_collector", 2024-09-16T02:05:54.535 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.535 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.535 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.535 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-16T02:05:54.535 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.535 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.535 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.535 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Jaeger collector container image", 2024-09-16T02:05:54.535 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.535 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.535 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.535 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.536 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_jaeger_query": { 2024-09-16T02:05:54.536 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_jaeger_query", 2024-09-16T02:05:54.536 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.536 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.536 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.536 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-16T02:05:54.536 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.536 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.536 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.536 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Jaeger query container image", 2024-09-16T02:05:54.536 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.536 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.536 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.536 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.537 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_keepalived": { 2024-09-16T02:05:54.537 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_keepalived", 2024-09-16T02:05:54.537 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.537 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.537 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.537 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-16T02:05:54.537 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.537 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.537 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.537 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Keepalived container image", 2024-09-16T02:05:54.537 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.537 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.537 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.537 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.538 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_loki": { 2024-09-16T02:05:54.538 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_loki", 2024-09-16T02:05:54.538 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.538 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.538 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.538 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-16T02:05:54.538 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.538 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.538 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.538 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Loki container image", 2024-09-16T02:05:54.538 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.538 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.538 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.538 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.539 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_node_exporter": { 2024-09-16T02:05:54.539 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_node_exporter", 2024-09-16T02:05:54.539 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.539 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.539 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.539 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-16T02:05:54.539 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.539 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.539 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.539 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Prometheus container image", 2024-09-16T02:05:54.539 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.539 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.539 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.539 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.540 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_nvmeof": { 2024-09-16T02:05:54.540 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_nvmeof", 2024-09-16T02:05:54.540 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.540 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.540 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.540 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-16T02:05:54.540 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.540 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.540 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.540 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Nvme-of container image", 2024-09-16T02:05:54.540 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.540 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.540 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.540 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.541 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_prometheus": { 2024-09-16T02:05:54.541 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_prometheus", 2024-09-16T02:05:54.541 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.541 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.541 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.541 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-16T02:05:54.541 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.541 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.541 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.541 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Prometheus container image", 2024-09-16T02:05:54.541 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.541 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.541 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.541 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.542 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_promtail": { 2024-09-16T02:05:54.542 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_promtail", 2024-09-16T02:05:54.542 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.542 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.542 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.542 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-16T02:05:54.542 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.542 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.542 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.542 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Promtail container image", 2024-09-16T02:05:54.542 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.542 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.542 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.542 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.543 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_samba": { 2024-09-16T02:05:54.543 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_samba", 2024-09-16T02:05:54.543 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.543 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.543 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.543 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-16T02:05:54.543 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.543 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.543 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.543 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Samba/SMB container image", 2024-09-16T02:05:54.543 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.543 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.543 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.544 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.544 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_snmp_gateway": { 2024-09-16T02:05:54.544 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_snmp_gateway", 2024-09-16T02:05:54.544 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.544 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.544 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.544 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-16T02:05:54.544 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.544 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.544 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.544 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "SNMP Gateway container image", 2024-09-16T02:05:54.544 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.544 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.544 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.545 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.545 INFO:teuthology.orchestra.run.smithi105.stdout: "container_init": { 2024-09-16T02:05:54.545 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_init", 2024-09-16T02:05:54.545 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.545 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.545 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.545 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.545 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.545 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.545 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.545 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Run podman/docker with `--init`", 2024-09-16T02:05:54.545 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.545 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.545 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.546 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.546 INFO:teuthology.orchestra.run.smithi105.stdout: "daemon_cache_timeout": { 2024-09-16T02:05:54.546 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "daemon_cache_timeout", 2024-09-16T02:05:54.546 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.546 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.546 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.546 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "600", 2024-09-16T02:05:54.546 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.546 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.546 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.546 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-16T02:05:54.546 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.546 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.546 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.547 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.547 INFO:teuthology.orchestra.run.smithi105.stdout: "default_cephadm_command_timeout": { 2024-09-16T02:05:54.547 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "default_cephadm_command_timeout", 2024-09-16T02:05:54.547 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.547 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.547 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.547 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "900", 2024-09-16T02:05:54.547 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.547 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.547 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.547 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-16T02:05:54.547 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.547 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.547 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.548 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.548 INFO:teuthology.orchestra.run.smithi105.stdout: "default_registry": { 2024-09-16T02:05:54.548 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "default_registry", 2024-09-16T02:05:54.548 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.548 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.548 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.548 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "docker.io", 2024-09-16T02:05:54.548 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.548 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.548 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.548 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-16T02:05:54.548 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.548 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.548 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.549 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.549 INFO:teuthology.orchestra.run.smithi105.stdout: "device_cache_timeout": { 2024-09-16T02:05:54.549 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "device_cache_timeout", 2024-09-16T02:05:54.549 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.549 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.549 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.549 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "1800", 2024-09-16T02:05:54.549 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.549 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.549 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.549 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "seconds to cache device inventory", 2024-09-16T02:05:54.549 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.549 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.549 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.550 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.550 INFO:teuthology.orchestra.run.smithi105.stdout: "device_enhanced_scan": { 2024-09-16T02:05:54.550 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "device_enhanced_scan", 2024-09-16T02:05:54.550 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.550 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.550 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.550 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.550 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.550 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.550 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.550 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-16T02:05:54.550 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.550 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.551 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.551 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.551 INFO:teuthology.orchestra.run.smithi105.stdout: "facts_cache_timeout": { 2024-09-16T02:05:54.551 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "facts_cache_timeout", 2024-09-16T02:05:54.551 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.551 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.551 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.551 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "60", 2024-09-16T02:05:54.551 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.551 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.551 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.551 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "seconds to cache host facts data", 2024-09-16T02:05:54.551 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.551 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.551 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.552 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.552 INFO:teuthology.orchestra.run.smithi105.stdout: "grafana_dashboards_path": { 2024-09-16T02:05:54.552 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "grafana_dashboards_path", 2024-09-16T02:05:54.552 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.552 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.552 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.552 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-16T02:05:54.552 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.552 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.552 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.552 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-16T02:05:54.552 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.552 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.552 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.553 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.553 INFO:teuthology.orchestra.run.smithi105.stdout: "host_check_interval": { 2024-09-16T02:05:54.553 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "host_check_interval", 2024-09-16T02:05:54.553 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.553 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.553 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.553 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "600", 2024-09-16T02:05:54.553 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.553 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.553 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.553 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how frequently to perform a host check", 2024-09-16T02:05:54.553 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.553 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.553 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.554 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.554 INFO:teuthology.orchestra.run.smithi105.stdout: "hw_monitoring": { 2024-09-16T02:05:54.554 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "hw_monitoring", 2024-09-16T02:05:54.554 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.554 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.554 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.554 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.554 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.554 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.554 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.554 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-16T02:05:54.554 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.554 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.554 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.555 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.555 INFO:teuthology.orchestra.run.smithi105.stdout: "inventory_list_all": { 2024-09-16T02:05:54.555 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "inventory_list_all", 2024-09-16T02:05:54.555 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.555 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.555 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.555 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.555 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.555 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.555 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.555 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-16T02:05:54.555 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.555 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.555 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.555 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.556 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.556 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.556 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.556 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.556 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.556 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.556 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.556 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.556 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.556 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.556 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.556 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.556 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.556 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.557 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.557 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.557 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.557 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.557 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.557 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.557 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.557 INFO:teuthology.orchestra.run.smithi105.stdout: "log_refresh_metadata": { 2024-09-16T02:05:54.557 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_refresh_metadata", 2024-09-16T02:05:54.557 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.557 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.557 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.557 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.558 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.558 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.558 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.558 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-16T02:05:54.558 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.558 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.558 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.558 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.558 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.558 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.558 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.558 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.558 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.558 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.559 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.559 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.559 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.559 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-16T02:05:54.559 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.559 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.559 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.559 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.559 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.559 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.559 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.559 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.559 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.559 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.560 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.560 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.560 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.560 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.560 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.560 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.560 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.560 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.560 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.560 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.560 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.560 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.560 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.560 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.561 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.561 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.561 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.561 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.561 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.561 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.561 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.561 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.561 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.561 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.561 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.561 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.561 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.563 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.563 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.564 INFO:teuthology.orchestra.run.smithi105.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-16T02:05:54.564 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-16T02:05:54.564 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.564 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.564 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.564 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.564 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.564 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.564 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.564 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-16T02:05:54.564 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.564 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.564 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.564 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.565 INFO:teuthology.orchestra.run.smithi105.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-16T02:05:54.565 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-16T02:05:54.565 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.565 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.565 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.565 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "*", 2024-09-16T02:05:54.565 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.565 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.565 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.565 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-16T02:05:54.565 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.565 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.565 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.565 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.566 INFO:teuthology.orchestra.run.smithi105.stdout: "max_count_per_host": { 2024-09-16T02:05:54.566 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "max_count_per_host", 2024-09-16T02:05:54.566 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.566 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.566 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.566 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-09-16T02:05:54.566 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.566 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.566 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.566 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "max number of daemons per service per host", 2024-09-16T02:05:54.566 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.566 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.566 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.566 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.567 INFO:teuthology.orchestra.run.smithi105.stdout: "max_osd_draining_count": { 2024-09-16T02:05:54.567 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "max_osd_draining_count", 2024-09-16T02:05:54.567 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.567 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.567 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.567 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-09-16T02:05:54.567 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.567 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.567 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.567 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-16T02:05:54.567 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.567 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.567 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.567 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.568 INFO:teuthology.orchestra.run.smithi105.stdout: "migration_current": { 2024-09-16T02:05:54.568 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "migration_current", 2024-09-16T02:05:54.568 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.568 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.568 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.568 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.568 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.568 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.568 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.568 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "internal - do not modify", 2024-09-16T02:05:54.568 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.568 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.568 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.568 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.569 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": { 2024-09-16T02:05:54.569 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "mode", 2024-09-16T02:05:54.569 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.569 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.569 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.569 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "root", 2024-09-16T02:05:54.569 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.569 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.569 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.569 INFO:teuthology.orchestra.run.smithi105.stdout: "cephadm-package", 2024-09-16T02:05:54.569 INFO:teuthology.orchestra.run.smithi105.stdout: "root" 2024-09-16T02:05:54.569 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.569 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "mode for remote execution of cephadm", 2024-09-16T02:05:54.569 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.570 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.570 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.570 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.570 INFO:teuthology.orchestra.run.smithi105.stdout: "oob_default_addr": { 2024-09-16T02:05:54.570 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "oob_default_addr", 2024-09-16T02:05:54.570 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.570 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.570 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.570 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "169.254.1.1", 2024-09-16T02:05:54.570 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.570 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.570 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.570 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-16T02:05:54.570 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.571 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.571 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.571 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.571 INFO:teuthology.orchestra.run.smithi105.stdout: "prometheus_alerts_path": { 2024-09-16T02:05:54.571 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "prometheus_alerts_path", 2024-09-16T02:05:54.571 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.571 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.571 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.571 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-16T02:05:54.571 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.571 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.571 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.571 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-16T02:05:54.571 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.572 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.572 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.572 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.572 INFO:teuthology.orchestra.run.smithi105.stdout: "registry_insecure": { 2024-09-16T02:05:54.572 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "registry_insecure", 2024-09-16T02:05:54.572 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.572 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.572 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.572 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.572 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.572 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.572 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.572 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-16T02:05:54.573 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.573 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.573 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.573 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.573 INFO:teuthology.orchestra.run.smithi105.stdout: "registry_password": { 2024-09-16T02:05:54.573 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "registry_password", 2024-09-16T02:05:54.573 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.573 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.573 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.573 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.573 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.573 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.573 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.574 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-16T02:05:54.574 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.574 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.574 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.574 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.574 INFO:teuthology.orchestra.run.smithi105.stdout: "registry_url": { 2024-09-16T02:05:54.574 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "registry_url", 2024-09-16T02:05:54.574 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.574 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.574 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.574 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.574 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.574 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.574 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.575 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-16T02:05:54.575 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.575 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.575 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.575 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.575 INFO:teuthology.orchestra.run.smithi105.stdout: "registry_username": { 2024-09-16T02:05:54.575 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "registry_username", 2024-09-16T02:05:54.575 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.575 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.575 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.575 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.575 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.575 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.576 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.576 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-16T02:05:54.576 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.576 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.576 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.576 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.576 INFO:teuthology.orchestra.run.smithi105.stdout: "secure_monitoring_stack": { 2024-09-16T02:05:54.576 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "secure_monitoring_stack", 2024-09-16T02:05:54.576 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.576 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.576 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.576 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.576 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.576 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.577 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.577 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-16T02:05:54.577 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.577 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.577 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.577 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.577 INFO:teuthology.orchestra.run.smithi105.stdout: "service_discovery_port": { 2024-09-16T02:05:54.577 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "service_discovery_port", 2024-09-16T02:05:54.577 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.577 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.577 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.577 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "8765", 2024-09-16T02:05:54.577 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.577 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.578 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.578 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "cephadm service discovery port", 2024-09-16T02:05:54.578 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.578 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.578 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.578 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.578 INFO:teuthology.orchestra.run.smithi105.stdout: "ssh_config_file": { 2024-09-16T02:05:54.578 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ssh_config_file", 2024-09-16T02:05:54.578 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.578 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.578 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.578 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.578 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.579 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.579 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.579 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-16T02:05:54.579 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.579 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.579 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.579 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.579 INFO:teuthology.orchestra.run.smithi105.stdout: "use_agent": { 2024-09-16T02:05:54.579 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "use_agent", 2024-09-16T02:05:54.579 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.579 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.579 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.579 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.579 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.580 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.580 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.580 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-16T02:05:54.580 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.580 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.580 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.580 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.580 INFO:teuthology.orchestra.run.smithi105.stdout: "use_repo_digest": { 2024-09-16T02:05:54.580 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "use_repo_digest", 2024-09-16T02:05:54.580 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.580 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.580 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.580 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.581 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.581 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.581 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.581 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-16T02:05:54.581 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.581 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.581 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.581 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.581 INFO:teuthology.orchestra.run.smithi105.stdout: "warn_on_failed_host_check": { 2024-09-16T02:05:54.581 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "warn_on_failed_host_check", 2024-09-16T02:05:54.581 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.581 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.581 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.581 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.582 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.582 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.582 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.582 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "raise a health warning if the host check fails", 2024-09-16T02:05:54.582 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.582 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.582 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.582 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.582 INFO:teuthology.orchestra.run.smithi105.stdout: "warn_on_stray_daemons": { 2024-09-16T02:05:54.582 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "warn_on_stray_daemons", 2024-09-16T02:05:54.582 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.582 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.582 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.582 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.583 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.583 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.583 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.583 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-16T02:05:54.583 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.583 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.583 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.583 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.583 INFO:teuthology.orchestra.run.smithi105.stdout: "warn_on_stray_hosts": { 2024-09-16T02:05:54.583 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "warn_on_stray_hosts", 2024-09-16T02:05:54.583 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.583 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.583 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.583 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.584 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.584 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.584 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.584 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-16T02:05:54.584 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.584 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.584 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.584 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.584 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.584 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.584 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.584 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "crash", 2024-09-16T02:05:54.584 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.584 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.585 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.585 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.585 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.585 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.585 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.585 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.585 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.585 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.585 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.585 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.585 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.585 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.585 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.585 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.586 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.586 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.586 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.586 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.586 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.586 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.586 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.586 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.586 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.586 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.586 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.586 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.586 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.586 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.586 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.587 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.587 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.587 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.587 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.587 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.587 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.587 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.587 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.587 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.587 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.587 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.587 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.587 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.587 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.588 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.588 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.588 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.588 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.588 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.588 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.588 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.588 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.588 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.588 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.588 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.588 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.588 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.588 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.589 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.589 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.589 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.589 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.589 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.589 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.589 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.589 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.589 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.589 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.589 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.589 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.589 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.589 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.590 INFO:teuthology.orchestra.run.smithi105.stdout: "retain_interval": { 2024-09-16T02:05:54.590 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "retain_interval", 2024-09-16T02:05:54.590 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.590 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.590 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.590 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "31536000", 2024-09-16T02:05:54.590 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.590 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.590 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.590 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-16T02:05:54.590 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.590 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.590 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.591 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.591 INFO:teuthology.orchestra.run.smithi105.stdout: "warn_recent_interval": { 2024-09-16T02:05:54.591 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "warn_recent_interval", 2024-09-16T02:05:54.591 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.591 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.591 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.591 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "1209600", 2024-09-16T02:05:54.591 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.591 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.591 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.591 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-16T02:05:54.591 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.591 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.591 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.592 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.592 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.592 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.592 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.592 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "dashboard", 2024-09-16T02:05:54.592 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.592 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.592 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.592 INFO:teuthology.orchestra.run.smithi105.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-16T02:05:54.592 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-16T02:05:54.592 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.592 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.593 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.593 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-09-16T02:05:54.593 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.593 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.593 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.593 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.593 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.593 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.593 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.593 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.593 INFO:teuthology.orchestra.run.smithi105.stdout: "ALERTMANAGER_API_HOST": { 2024-09-16T02:05:54.593 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-16T02:05:54.593 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.593 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.594 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.594 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.594 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.594 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.594 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.594 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.594 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.594 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.594 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.594 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.594 INFO:teuthology.orchestra.run.smithi105.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-16T02:05:54.594 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-16T02:05:54.594 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.594 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.595 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.595 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.595 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.595 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.595 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.595 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.595 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.595 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.595 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.595 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.595 INFO:teuthology.orchestra.run.smithi105.stdout: "AUDIT_API_ENABLED": { 2024-09-16T02:05:54.595 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "AUDIT_API_ENABLED", 2024-09-16T02:05:54.595 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.595 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.596 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.596 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.596 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.596 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.596 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.596 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.596 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.596 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.596 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.596 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.596 INFO:teuthology.orchestra.run.smithi105.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-16T02:05:54.596 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-16T02:05:54.596 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.597 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.597 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.597 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.597 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.597 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.597 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.597 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.597 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.597 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.597 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.597 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.597 INFO:teuthology.orchestra.run.smithi105.stdout: "ENABLE_BROWSABLE_API": { 2024-09-16T02:05:54.597 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-16T02:05:54.597 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.598 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.598 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.598 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.598 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.598 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.598 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.598 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.598 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.598 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.598 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.598 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.598 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-16T02:05:54.598 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-16T02:05:54.598 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.599 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.599 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.599 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.599 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.599 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.599 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.599 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.599 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.599 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.599 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.599 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.599 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-16T02:05:54.599 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-16T02:05:54.599 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.599 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.600 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.600 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.600 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.600 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.600 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.600 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.600 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.600 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.600 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.600 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.600 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-16T02:05:54.600 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-16T02:05:54.600 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.600 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.601 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.601 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.601 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.601 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.601 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.601 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.601 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.601 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.601 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.601 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.601 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-16T02:05:54.601 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-16T02:05:54.601 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.601 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.602 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.602 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.602 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.602 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.602 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.602 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.602 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.602 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.602 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.602 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.602 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-16T02:05:54.602 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-16T02:05:54.602 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.602 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.603 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.603 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.603 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.603 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.603 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.603 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.603 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.603 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.603 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.603 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.603 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-16T02:05:54.603 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-16T02:05:54.603 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.603 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.604 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.604 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.604 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.604 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.604 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.604 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.604 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.604 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.604 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.604 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.604 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-16T02:05:54.604 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-16T02:05:54.604 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.604 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.605 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.605 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.605 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.605 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.605 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.605 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.605 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.605 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.605 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.605 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.605 INFO:teuthology.orchestra.run.smithi105.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-16T02:05:54.605 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-16T02:05:54.605 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.606 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.606 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.606 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.606 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.606 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.606 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.606 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.606 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.606 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.606 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.606 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.606 INFO:teuthology.orchestra.run.smithi105.stdout: "GRAFANA_API_PASSWORD": { 2024-09-16T02:05:54.606 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-16T02:05:54.606 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.607 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.607 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.607 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "admin", 2024-09-16T02:05:54.607 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.607 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.607 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.607 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.607 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.607 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.607 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.607 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.607 INFO:teuthology.orchestra.run.smithi105.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-16T02:05:54.607 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-16T02:05:54.607 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.608 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.608 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.608 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.608 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.608 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.608 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.608 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.608 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.608 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.608 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.608 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.608 INFO:teuthology.orchestra.run.smithi105.stdout: "GRAFANA_API_URL": { 2024-09-16T02:05:54.608 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GRAFANA_API_URL", 2024-09-16T02:05:54.609 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.609 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.609 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.609 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.609 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.609 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.609 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.609 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.609 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.609 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.609 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.609 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.609 INFO:teuthology.orchestra.run.smithi105.stdout: "GRAFANA_API_USERNAME": { 2024-09-16T02:05:54.609 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-16T02:05:54.610 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.610 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.610 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.610 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "admin", 2024-09-16T02:05:54.610 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.610 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.610 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.610 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.610 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.610 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.610 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.610 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.610 INFO:teuthology.orchestra.run.smithi105.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-16T02:05:54.610 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-16T02:05:54.611 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.611 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.611 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.611 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.611 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.611 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.611 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.611 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.611 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.611 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.611 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.611 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.611 INFO:teuthology.orchestra.run.smithi105.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-16T02:05:54.611 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-16T02:05:54.612 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.612 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.612 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.612 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.612 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.612 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.612 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.612 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.612 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.612 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.612 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.612 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.612 INFO:teuthology.orchestra.run.smithi105.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-16T02:05:54.612 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-16T02:05:54.613 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.613 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.613 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.613 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.613 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.613 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.613 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.613 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.613 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.613 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.613 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.613 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.613 INFO:teuthology.orchestra.run.smithi105.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-16T02:05:54.614 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-16T02:05:54.614 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.614 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.614 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.614 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.614 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.614 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.614 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.614 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.614 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.614 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.614 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.614 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.614 INFO:teuthology.orchestra.run.smithi105.stdout: "PROMETHEUS_API_HOST": { 2024-09-16T02:05:54.615 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-16T02:05:54.615 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.615 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.615 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.615 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.615 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.615 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.615 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.615 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.615 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.615 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.615 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.615 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.615 INFO:teuthology.orchestra.run.smithi105.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-16T02:05:54.616 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-16T02:05:54.616 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.616 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.616 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.616 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.616 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.616 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.616 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.616 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.616 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.616 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.616 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.616 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.617 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-16T02:05:54.617 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-16T02:05:54.617 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.617 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.617 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.617 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.617 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.617 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.617 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.617 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.617 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.617 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.617 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.617 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.618 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-16T02:05:54.618 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-16T02:05:54.618 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.618 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.618 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.618 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.618 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.618 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.618 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.618 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.618 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.618 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.618 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.618 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.619 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-16T02:05:54.619 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-16T02:05:54.619 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.619 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.619 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.619 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.619 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.619 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.619 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.619 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.619 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.619 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.619 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.619 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.620 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-16T02:05:54.620 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-16T02:05:54.620 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.620 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.620 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.620 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.620 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.620 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.620 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.620 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.620 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.620 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.620 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.621 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.621 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-16T02:05:54.621 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-16T02:05:54.621 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.621 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.621 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.621 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.621 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.621 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.621 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.621 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.621 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.621 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.621 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.622 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.622 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-16T02:05:54.622 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-16T02:05:54.622 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.622 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.622 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.622 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.622 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.622 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.622 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.622 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.622 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.622 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.622 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.623 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.623 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-16T02:05:54.623 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-16T02:05:54.623 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.623 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.623 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.623 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.623 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.623 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.623 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.623 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.623 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.623 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.623 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.624 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.624 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_ENABLED": { 2024-09-16T02:05:54.624 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-16T02:05:54.624 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.624 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.624 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.624 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.624 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.624 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.624 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.624 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.624 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.624 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.625 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.625 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.625 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-16T02:05:54.625 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-16T02:05:54.625 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.625 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.625 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.625 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-16T02:05:54.625 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.625 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.625 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.625 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.625 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.625 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.626 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.626 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.626 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-16T02:05:54.626 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-16T02:05:54.626 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.626 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.626 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.626 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-09-16T02:05:54.626 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.626 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.626 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.626 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.626 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.626 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.627 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.627 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.627 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-16T02:05:54.627 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-16T02:05:54.627 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.627 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.627 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.627 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "8", 2024-09-16T02:05:54.627 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.627 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.627 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.627 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.627 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.627 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.628 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.628 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.628 INFO:teuthology.orchestra.run.smithi105.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-16T02:05:54.628 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-16T02:05:54.628 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.628 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.628 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.628 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "45", 2024-09-16T02:05:54.628 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.628 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.628 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.628 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.628 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.628 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.629 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.629 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.629 INFO:teuthology.orchestra.run.smithi105.stdout: "RGW_API_ACCESS_KEY": { 2024-09-16T02:05:54.629 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-16T02:05:54.629 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.629 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.629 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.629 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.629 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.629 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.629 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.629 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.629 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.630 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.630 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.630 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.630 INFO:teuthology.orchestra.run.smithi105.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-16T02:05:54.630 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-16T02:05:54.630 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.630 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.630 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.630 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "admin", 2024-09-16T02:05:54.630 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.630 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.630 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.630 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.630 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.630 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.631 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.631 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.631 INFO:teuthology.orchestra.run.smithi105.stdout: "RGW_API_SECRET_KEY": { 2024-09-16T02:05:54.631 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-16T02:05:54.631 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.631 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.631 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.631 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.631 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.631 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.631 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.631 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.631 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.631 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.632 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.632 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.632 INFO:teuthology.orchestra.run.smithi105.stdout: "RGW_API_SSL_VERIFY": { 2024-09-16T02:05:54.632 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-16T02:05:54.632 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.632 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.632 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.632 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.632 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.632 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.632 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.632 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.632 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.632 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.633 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.633 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.633 INFO:teuthology.orchestra.run.smithi105.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-16T02:05:54.633 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-16T02:05:54.633 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.633 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.633 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.633 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0", 2024-09-16T02:05:54.633 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.633 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.633 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.633 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.633 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.633 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.634 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.634 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.634 INFO:teuthology.orchestra.run.smithi105.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-16T02:05:54.634 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-16T02:05:54.634 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.634 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.634 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.634 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-09-16T02:05:54.634 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.634 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.634 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.634 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.634 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.634 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.635 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.635 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.635 INFO:teuthology.orchestra.run.smithi105.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-16T02:05:54.635 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-16T02:05:54.635 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.635 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.635 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.635 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "5", 2024-09-16T02:05:54.635 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.635 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.635 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.635 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.635 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.635 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.635 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.636 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.636 INFO:teuthology.orchestra.run.smithi105.stdout: "cross_origin_url": { 2024-09-16T02:05:54.636 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "cross_origin_url", 2024-09-16T02:05:54.636 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.636 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.636 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.636 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.636 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.636 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.636 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.636 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.636 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.636 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.637 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.637 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.637 INFO:teuthology.orchestra.run.smithi105.stdout: "crt_file": { 2024-09-16T02:05:54.637 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "crt_file", 2024-09-16T02:05:54.637 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.637 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.637 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.637 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.637 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.637 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.637 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.637 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.637 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.637 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.637 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.638 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.638 INFO:teuthology.orchestra.run.smithi105.stdout: "debug": { 2024-09-16T02:05:54.638 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "debug", 2024-09-16T02:05:54.638 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.638 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.638 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.638 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.638 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.638 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.638 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.638 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Enable/disable debug options", 2024-09-16T02:05:54.638 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.638 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.639 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.639 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.639 INFO:teuthology.orchestra.run.smithi105.stdout: "jwt_token_ttl": { 2024-09-16T02:05:54.639 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "jwt_token_ttl", 2024-09-16T02:05:54.639 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.639 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.639 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.639 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "28800", 2024-09-16T02:05:54.639 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.639 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.639 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.639 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.639 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.639 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.640 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.640 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.640 INFO:teuthology.orchestra.run.smithi105.stdout: "key_file": { 2024-09-16T02:05:54.640 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "key_file", 2024-09-16T02:05:54.640 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.640 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.640 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.640 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.640 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.640 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.640 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.640 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.640 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.641 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.641 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.641 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.641 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.641 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.641 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.641 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.641 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.641 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.641 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.641 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.641 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.641 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.641 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.642 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.642 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.642 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.642 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.642 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.642 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.642 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.642 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.642 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.642 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.642 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.642 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.642 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.642 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.643 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.643 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.643 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.643 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.643 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.643 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.643 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.643 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.643 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.643 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.643 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.643 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.643 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.643 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.644 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.644 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.644 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.644 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.644 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.644 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.644 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.644 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.644 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.644 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.644 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.644 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.644 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.645 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.645 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.645 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.645 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.645 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.645 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.645 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.645 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.645 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.645 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.645 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.645 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.645 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.645 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.646 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.646 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.646 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.646 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.646 INFO:teuthology.orchestra.run.smithi105.stdout: "motd": { 2024-09-16T02:05:54.646 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "motd", 2024-09-16T02:05:54.646 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.646 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.646 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.646 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.646 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.646 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.646 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.646 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "The message of the day", 2024-09-16T02:05:54.647 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.647 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.647 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.647 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.647 INFO:teuthology.orchestra.run.smithi105.stdout: "redirect_resolve_ip_addr": { 2024-09-16T02:05:54.647 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "redirect_resolve_ip_addr", 2024-09-16T02:05:54.647 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.647 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.647 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.647 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.647 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.647 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.647 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.647 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.648 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.648 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.648 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.648 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.648 INFO:teuthology.orchestra.run.smithi105.stdout: "server_addr": { 2024-09-16T02:05:54.648 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "server_addr", 2024-09-16T02:05:54.648 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.648 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.648 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.648 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "::", 2024-09-16T02:05:54.648 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.648 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.648 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.648 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.649 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.649 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.649 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.649 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.649 INFO:teuthology.orchestra.run.smithi105.stdout: "server_port": { 2024-09-16T02:05:54.649 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "server_port", 2024-09-16T02:05:54.649 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.649 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.649 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.649 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "8080", 2024-09-16T02:05:54.649 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.649 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.649 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.649 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.650 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.650 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.650 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.650 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.650 INFO:teuthology.orchestra.run.smithi105.stdout: "ssl": { 2024-09-16T02:05:54.650 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ssl", 2024-09-16T02:05:54.650 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.650 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.650 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.650 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.650 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.650 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.650 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.650 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.651 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.651 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.651 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.651 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.651 INFO:teuthology.orchestra.run.smithi105.stdout: "ssl_server_port": { 2024-09-16T02:05:54.651 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ssl_server_port", 2024-09-16T02:05:54.651 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.651 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.651 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.651 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "8443", 2024-09-16T02:05:54.651 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.651 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.651 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.651 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.651 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.652 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.652 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.652 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.652 INFO:teuthology.orchestra.run.smithi105.stdout: "standby_behaviour": { 2024-09-16T02:05:54.652 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "standby_behaviour", 2024-09-16T02:05:54.652 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.652 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.652 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.652 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "redirect", 2024-09-16T02:05:54.652 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.652 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.652 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.652 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.652 INFO:teuthology.orchestra.run.smithi105.stdout: "redirect" 2024-09-16T02:05:54.653 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.653 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.653 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.653 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.653 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.653 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.653 INFO:teuthology.orchestra.run.smithi105.stdout: "standby_error_status_code": { 2024-09-16T02:05:54.653 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "standby_error_status_code", 2024-09-16T02:05:54.653 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.653 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.653 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.653 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "500", 2024-09-16T02:05:54.653 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "400", 2024-09-16T02:05:54.653 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "599", 2024-09-16T02:05:54.654 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.654 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.654 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.654 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.654 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.654 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.654 INFO:teuthology.orchestra.run.smithi105.stdout: "url_prefix": { 2024-09-16T02:05:54.654 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "url_prefix", 2024-09-16T02:05:54.654 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.654 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.654 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.654 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.654 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.654 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.655 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.655 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.655 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.655 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.655 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.655 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.655 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.655 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.655 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.655 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "devicehealth", 2024-09-16T02:05:54.655 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.655 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.655 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.655 INFO:teuthology.orchestra.run.smithi105.stdout: "enable_monitoring": { 2024-09-16T02:05:54.656 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "enable_monitoring", 2024-09-16T02:05:54.656 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.656 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.656 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.656 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.656 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.656 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.656 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.656 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "monitor device health metrics", 2024-09-16T02:05:54.656 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.656 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.656 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.656 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.656 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.657 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.657 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.657 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.657 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.657 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.657 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.657 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.657 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.657 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.657 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.657 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.657 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.657 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.657 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.658 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.658 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.658 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.658 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.658 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.658 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.658 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.658 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.658 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.658 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.658 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.658 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.658 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.658 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.659 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.659 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.659 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.659 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.659 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.659 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.659 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.659 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.659 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.659 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.659 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.659 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.659 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.659 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.660 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.660 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.660 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.660 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.660 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.660 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.660 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.660 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.660 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.660 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.660 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.660 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.660 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.660 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.661 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.661 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.661 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.661 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.661 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.661 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.661 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.661 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.661 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.661 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.661 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.661 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.661 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.661 INFO:teuthology.orchestra.run.smithi105.stdout: "mark_out_threshold": { 2024-09-16T02:05:54.662 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "mark_out_threshold", 2024-09-16T02:05:54.662 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.662 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.662 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.662 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "2419200", 2024-09-16T02:05:54.662 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.662 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.662 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.662 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-16T02:05:54.662 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.662 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.662 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.662 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.663 INFO:teuthology.orchestra.run.smithi105.stdout: "pool_name": { 2024-09-16T02:05:54.663 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "pool_name", 2024-09-16T02:05:54.663 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.663 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.663 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.663 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "device_health_metrics", 2024-09-16T02:05:54.663 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.663 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.663 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.663 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-16T02:05:54.663 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.663 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.663 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.663 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.663 INFO:teuthology.orchestra.run.smithi105.stdout: "retention_period": { 2024-09-16T02:05:54.664 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "retention_period", 2024-09-16T02:05:54.664 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.664 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.664 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.664 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "15552000", 2024-09-16T02:05:54.664 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.664 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.664 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.664 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how long to retain device health metrics", 2024-09-16T02:05:54.664 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.664 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.664 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.664 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.664 INFO:teuthology.orchestra.run.smithi105.stdout: "scrape_frequency": { 2024-09-16T02:05:54.665 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "scrape_frequency", 2024-09-16T02:05:54.665 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.665 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.665 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.665 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "86400", 2024-09-16T02:05:54.665 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.665 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.665 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.665 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-16T02:05:54.665 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.665 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.665 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.665 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.665 INFO:teuthology.orchestra.run.smithi105.stdout: "self_heal": { 2024-09-16T02:05:54.666 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "self_heal", 2024-09-16T02:05:54.666 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.666 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.666 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.666 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.666 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.666 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.666 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.666 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-16T02:05:54.666 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.666 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.666 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.666 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.666 INFO:teuthology.orchestra.run.smithi105.stdout: "sleep_interval": { 2024-09-16T02:05:54.667 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "sleep_interval", 2024-09-16T02:05:54.667 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.667 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.667 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.667 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "600", 2024-09-16T02:05:54.667 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.667 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.667 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.667 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how frequently to wake up and check device health", 2024-09-16T02:05:54.667 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.667 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.667 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.667 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.667 INFO:teuthology.orchestra.run.smithi105.stdout: "warn_threshold": { 2024-09-16T02:05:54.668 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "warn_threshold", 2024-09-16T02:05:54.668 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.668 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.668 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.668 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "7257600", 2024-09-16T02:05:54.668 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.668 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.668 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.668 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-16T02:05:54.668 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.668 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.668 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.668 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.669 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.669 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.669 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.669 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "diskprediction_local", 2024-09-16T02:05:54.669 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.669 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.669 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.669 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.669 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.669 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.669 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.669 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.669 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.669 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.670 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.670 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.670 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.670 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.670 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.670 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.670 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.670 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.670 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.670 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.670 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.670 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.670 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.670 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.671 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.671 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.671 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.671 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.671 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.671 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.671 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.671 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.671 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.671 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.671 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.671 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.671 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.671 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.672 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.672 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.672 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.672 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.672 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.672 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.672 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.672 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.672 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.672 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.672 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.672 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.672 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.673 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.673 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.673 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.673 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.673 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.673 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.673 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.673 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.673 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.673 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.673 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.673 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.673 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.673 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.674 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.674 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.674 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.674 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.674 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.674 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.674 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.674 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.674 INFO:teuthology.orchestra.run.smithi105.stdout: "predict_interval": { 2024-09-16T02:05:54.674 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "predict_interval", 2024-09-16T02:05:54.674 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.674 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.674 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.675 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "86400", 2024-09-16T02:05:54.675 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.675 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.675 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.675 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.675 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.675 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.675 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.675 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.675 INFO:teuthology.orchestra.run.smithi105.stdout: "predictor_model": { 2024-09-16T02:05:54.675 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "predictor_model", 2024-09-16T02:05:54.675 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.675 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.675 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.676 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "prophetstor", 2024-09-16T02:05:54.676 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.676 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.676 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.676 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.676 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.676 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.676 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.676 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.676 INFO:teuthology.orchestra.run.smithi105.stdout: "sleep_interval": { 2024-09-16T02:05:54.676 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "sleep_interval", 2024-09-16T02:05:54.676 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.676 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.676 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.677 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "600", 2024-09-16T02:05:54.677 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.677 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.677 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.677 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.677 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.677 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.677 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.677 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.677 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.677 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.677 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.677 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "influx", 2024-09-16T02:05:54.677 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": false, 2024-09-16T02:05:54.678 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "influxdb python module not found", 2024-09-16T02:05:54.678 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.678 INFO:teuthology.orchestra.run.smithi105.stdout: "batch_size": { 2024-09-16T02:05:54.678 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "batch_size", 2024-09-16T02:05:54.678 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.678 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.678 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.678 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "5000", 2024-09-16T02:05:54.678 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.678 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.678 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.678 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-16T02:05:54.678 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.679 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.679 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.679 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.679 INFO:teuthology.orchestra.run.smithi105.stdout: "database": { 2024-09-16T02:05:54.679 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "database", 2024-09-16T02:05:54.679 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.679 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.679 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.679 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "ceph", 2024-09-16T02:05:54.679 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.679 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.679 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.679 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-16T02:05:54.679 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.679 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.680 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.680 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.680 INFO:teuthology.orchestra.run.smithi105.stdout: "hostname": { 2024-09-16T02:05:54.680 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "hostname", 2024-09-16T02:05:54.680 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.680 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.680 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.680 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.680 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.680 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.680 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.680 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "InfluxDB server hostname", 2024-09-16T02:05:54.680 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.681 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.681 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.681 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.681 INFO:teuthology.orchestra.run.smithi105.stdout: "interval": { 2024-09-16T02:05:54.681 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "interval", 2024-09-16T02:05:54.681 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.681 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.681 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.681 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "30", 2024-09-16T02:05:54.681 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "5", 2024-09-16T02:05:54.681 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.681 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.681 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-16T02:05:54.681 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.682 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.682 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.682 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.682 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.682 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.682 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.682 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.682 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.682 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.682 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.682 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.682 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.682 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.682 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.683 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.683 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.683 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.683 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.683 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.683 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.683 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.683 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.683 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.683 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.683 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.683 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.683 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.683 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.684 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.684 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.684 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.684 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.684 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.684 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.684 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.684 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.684 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.684 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.684 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.684 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.684 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.684 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.685 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.685 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.685 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.685 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.685 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.685 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.685 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.685 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.685 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.685 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.685 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.685 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.685 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.685 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.686 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.686 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.686 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.686 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.686 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.686 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.686 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.686 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.686 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.686 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.686 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.686 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.686 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.686 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.687 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.687 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.687 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.687 INFO:teuthology.orchestra.run.smithi105.stdout: "password": { 2024-09-16T02:05:54.687 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "password", 2024-09-16T02:05:54.687 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.687 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.687 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.687 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.687 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.687 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.687 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.687 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "password of InfluxDB server user", 2024-09-16T02:05:54.687 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.687 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.688 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.688 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.688 INFO:teuthology.orchestra.run.smithi105.stdout: "port": { 2024-09-16T02:05:54.688 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "port", 2024-09-16T02:05:54.688 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.688 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.688 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.688 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "8086", 2024-09-16T02:05:54.688 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.688 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.688 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.688 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "InfluxDB server port", 2024-09-16T02:05:54.688 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.689 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.689 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.689 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.689 INFO:teuthology.orchestra.run.smithi105.stdout: "ssl": { 2024-09-16T02:05:54.689 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ssl", 2024-09-16T02:05:54.689 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.689 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.689 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.689 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "false", 2024-09-16T02:05:54.689 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.689 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.689 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.689 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T02:05:54.689 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.690 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.690 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.690 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.690 INFO:teuthology.orchestra.run.smithi105.stdout: "threads": { 2024-09-16T02:05:54.690 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "threads", 2024-09-16T02:05:54.690 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.690 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.690 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.690 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "5", 2024-09-16T02:05:54.690 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "1", 2024-09-16T02:05:54.690 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "32", 2024-09-16T02:05:54.690 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.690 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-16T02:05:54.690 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.691 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.691 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.691 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.691 INFO:teuthology.orchestra.run.smithi105.stdout: "username": { 2024-09-16T02:05:54.691 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "username", 2024-09-16T02:05:54.691 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.691 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.691 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.691 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.691 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.691 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.691 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.691 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "username of InfluxDB server user", 2024-09-16T02:05:54.692 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.692 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.692 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.692 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.692 INFO:teuthology.orchestra.run.smithi105.stdout: "verify_ssl": { 2024-09-16T02:05:54.692 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "verify_ssl", 2024-09-16T02:05:54.692 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.692 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.692 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.692 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "true", 2024-09-16T02:05:54.692 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.692 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.692 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.692 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T02:05:54.693 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.693 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.693 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.693 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.693 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.693 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.693 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.693 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "insights", 2024-09-16T02:05:54.693 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.693 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.693 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.693 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.693 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.693 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.694 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.694 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.694 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.694 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.694 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.694 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.694 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.694 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.694 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.694 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.694 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.694 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.694 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.695 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.695 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.695 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.695 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.695 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.695 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.695 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.695 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.695 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.695 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.695 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.695 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.695 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.695 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.696 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.696 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.696 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.696 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.696 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.696 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.696 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.696 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.696 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.696 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.696 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.696 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.696 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.696 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.697 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.697 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.697 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.697 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.697 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.697 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.697 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.697 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.697 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.697 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.697 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.697 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.697 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.697 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.698 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.698 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.698 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.698 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.698 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.698 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.698 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.698 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.698 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.698 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.698 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.698 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.698 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.698 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.698 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.699 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "iostat", 2024-09-16T02:05:54.699 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.699 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.699 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.699 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.699 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.699 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.699 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.699 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.699 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.699 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.699 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.699 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.699 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.700 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.700 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.700 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.700 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.700 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.700 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.700 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.700 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.700 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.700 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.700 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.700 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.700 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.700 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.701 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.701 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.701 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.701 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.701 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.701 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.701 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.701 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.701 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.701 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.701 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.701 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.701 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.701 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.702 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.702 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.702 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.702 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.702 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.702 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.702 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.702 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.702 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.702 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.702 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.702 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.702 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.703 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.703 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.703 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.703 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.703 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.703 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.703 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.703 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.703 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.703 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.703 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.703 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.703 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.703 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.704 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.704 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.704 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.704 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.704 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.704 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.704 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.704 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.704 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "k8sevents", 2024-09-16T02:05:54.704 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.704 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.704 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.704 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_event_retention_days": { 2024-09-16T02:05:54.704 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ceph_event_retention_days", 2024-09-16T02:05:54.705 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.705 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.705 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.705 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "7", 2024-09-16T02:05:54.705 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.705 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.705 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.705 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-16T02:05:54.705 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.705 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.705 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.705 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.705 INFO:teuthology.orchestra.run.smithi105.stdout: "config_check_secs": { 2024-09-16T02:05:54.705 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "config_check_secs", 2024-09-16T02:05:54.706 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.706 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.706 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.706 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-09-16T02:05:54.706 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "10", 2024-09-16T02:05:54.706 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.706 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.706 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-16T02:05:54.706 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.706 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.706 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.706 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.706 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.706 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.707 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.707 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.707 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.707 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.707 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.707 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.707 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.707 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.707 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.707 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.707 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.707 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.707 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.708 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.708 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.708 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.708 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.708 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.708 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.708 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.708 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.708 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.708 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.708 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.708 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.708 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.708 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.709 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.709 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.709 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.709 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.709 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.709 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.709 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.709 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.709 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.709 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.709 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.709 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.709 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.709 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.709 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.710 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.710 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.710 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.710 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.710 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.710 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.710 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.710 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.710 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.710 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.710 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.710 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.710 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.710 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.711 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.711 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.711 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.711 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.711 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.711 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.711 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.711 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.711 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.711 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.711 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.711 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.711 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.711 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.712 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.712 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "localpool", 2024-09-16T02:05:54.712 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.712 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.712 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.712 INFO:teuthology.orchestra.run.smithi105.stdout: "failure_domain": { 2024-09-16T02:05:54.712 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "failure_domain", 2024-09-16T02:05:54.712 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.712 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.712 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.712 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "host", 2024-09-16T02:05:54.712 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.712 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.713 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.713 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "failure domain for any created local pool", 2024-09-16T02:05:54.713 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-16T02:05:54.713 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.713 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.713 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.713 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.713 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.713 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.713 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.713 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.713 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.713 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.713 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.714 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.714 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.714 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.714 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.714 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.714 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.714 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.714 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.714 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.714 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.714 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.714 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.714 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.714 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.715 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.715 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.715 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.715 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.715 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.715 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.715 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.715 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.715 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.715 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.715 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.715 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.715 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.715 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.716 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.716 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.716 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.716 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.716 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.716 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.716 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.716 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.716 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.716 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.716 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.716 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.716 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.716 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.717 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.717 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.717 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.717 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.717 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.717 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.717 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.717 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.717 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.717 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.717 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.717 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.717 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.717 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.717 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.718 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.718 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.718 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.718 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.718 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.718 INFO:teuthology.orchestra.run.smithi105.stdout: "min_size": { 2024-09-16T02:05:54.718 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "min_size", 2024-09-16T02:05:54.718 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.718 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.718 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.718 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.718 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.718 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.718 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.719 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "default min_size for any created local pool", 2024-09-16T02:05:54.719 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-16T02:05:54.719 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.719 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.719 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.719 INFO:teuthology.orchestra.run.smithi105.stdout: "num_rep": { 2024-09-16T02:05:54.719 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "num_rep", 2024-09-16T02:05:54.719 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.719 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.719 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.719 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "3", 2024-09-16T02:05:54.719 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.719 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.719 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.720 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "default replica count for any created local pool", 2024-09-16T02:05:54.720 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.720 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.720 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.720 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.720 INFO:teuthology.orchestra.run.smithi105.stdout: "pg_num": { 2024-09-16T02:05:54.720 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "pg_num", 2024-09-16T02:05:54.720 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.720 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.720 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.720 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "128", 2024-09-16T02:05:54.720 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.720 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.720 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.721 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "default pg_num for any created local pool", 2024-09-16T02:05:54.721 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.721 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.721 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.721 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.721 INFO:teuthology.orchestra.run.smithi105.stdout: "prefix": { 2024-09-16T02:05:54.721 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "prefix", 2024-09-16T02:05:54.721 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.721 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.721 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.721 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.721 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.721 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.721 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.722 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "name prefix for any created local pool", 2024-09-16T02:05:54.722 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.722 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.722 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.722 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.722 INFO:teuthology.orchestra.run.smithi105.stdout: "subtree": { 2024-09-16T02:05:54.722 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "subtree", 2024-09-16T02:05:54.722 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.722 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.722 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.722 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "rack", 2024-09-16T02:05:54.722 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.722 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.723 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.723 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-16T02:05:54.723 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-16T02:05:54.723 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.723 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.723 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.723 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.723 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.723 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.723 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "mds_autoscaler", 2024-09-16T02:05:54.723 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.723 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.723 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.723 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.724 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.724 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.724 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.724 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.724 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.724 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.724 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.724 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.724 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.724 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.724 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.724 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.724 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.724 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.725 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.725 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.725 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.725 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.725 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.725 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.725 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.725 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.725 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.725 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.725 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.725 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.725 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.725 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.726 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.726 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.726 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.726 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.726 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.726 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.726 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.726 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.726 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.726 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.726 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.726 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.726 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.726 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.727 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.727 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.727 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.727 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.727 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.727 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.727 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.727 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.727 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.727 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.727 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.727 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.727 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.727 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.728 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.728 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.728 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.728 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.728 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.728 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.728 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.728 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.728 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.728 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.728 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.728 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.728 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.728 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.729 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.729 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.729 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "mirroring", 2024-09-16T02:05:54.729 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.729 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.729 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.729 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.729 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.729 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.729 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.729 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.729 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.729 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.730 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.730 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.730 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.730 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.730 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.730 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.730 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.730 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.730 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.730 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.730 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.730 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.730 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.730 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.731 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.731 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.731 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.731 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.731 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.731 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.731 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.731 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.731 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.731 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.731 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.731 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.731 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.731 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.732 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.732 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.732 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.732 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.732 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.732 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.732 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.732 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.732 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.732 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.732 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.732 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.732 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.732 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.732 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.733 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.733 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.733 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.733 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.733 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.733 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.733 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.733 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.733 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.733 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.733 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.733 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.733 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.734 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.734 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.734 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.734 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.734 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.734 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.734 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.734 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.734 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.734 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.734 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "nfs", 2024-09-16T02:05:54.734 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.734 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.734 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.735 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.735 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.735 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.735 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.735 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.735 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.735 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.735 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.735 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.735 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.735 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.735 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.735 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.735 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.736 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.736 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.736 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.736 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.736 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.736 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.736 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.736 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.736 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.736 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.736 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.736 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.736 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.737 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.737 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.737 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.737 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.737 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.737 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.737 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.737 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.737 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.737 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.737 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.737 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.737 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.737 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.738 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.738 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.738 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.738 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.738 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.738 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.738 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.738 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.738 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.738 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.738 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.738 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.738 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.738 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.739 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.739 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.739 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.739 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.739 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.739 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.739 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.739 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.739 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.739 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.739 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.739 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.739 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.740 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.740 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.740 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.740 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.740 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.740 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "orchestrator", 2024-09-16T02:05:54.740 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.740 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.740 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.740 INFO:teuthology.orchestra.run.smithi105.stdout: "fail_fs": { 2024-09-16T02:05:54.740 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "fail_fs", 2024-09-16T02:05:54.740 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.740 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.740 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.741 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.741 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.741 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.741 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.741 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-16T02:05:54.741 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.741 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.741 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.741 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.741 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.741 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.741 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.741 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.742 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.742 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.742 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.742 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.742 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.742 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.742 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.742 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.742 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.742 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.742 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.742 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.742 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.742 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.743 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.743 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.743 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.743 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.743 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.743 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.743 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.743 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.743 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.743 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.743 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.743 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.743 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.743 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.744 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.744 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.744 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.744 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.744 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.744 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.744 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.744 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.744 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.744 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.744 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.744 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.744 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.745 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.745 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.745 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.745 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.745 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.745 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.745 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.745 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.745 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.745 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.745 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.745 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.745 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.745 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.746 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.746 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.746 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.746 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.746 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.746 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.746 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.746 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.746 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.746 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.746 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.746 INFO:teuthology.orchestra.run.smithi105.stdout: "orchestrator": { 2024-09-16T02:05:54.746 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "orchestrator", 2024-09-16T02:05:54.746 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.747 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.747 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.747 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.747 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.747 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.747 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.747 INFO:teuthology.orchestra.run.smithi105.stdout: "cephadm", 2024-09-16T02:05:54.747 INFO:teuthology.orchestra.run.smithi105.stdout: "rook", 2024-09-16T02:05:54.747 INFO:teuthology.orchestra.run.smithi105.stdout: "test_orchestrator" 2024-09-16T02:05:54.747 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.747 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Orchestrator backend", 2024-09-16T02:05:54.747 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.747 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.747 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.748 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.748 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.748 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.748 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.748 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "osd_perf_query", 2024-09-16T02:05:54.748 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.748 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.748 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.748 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.748 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.748 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.748 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.748 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.748 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.749 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.749 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.749 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.749 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.749 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.749 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.749 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.749 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.749 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.749 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.749 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.749 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.749 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.749 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.750 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.750 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.750 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.750 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.750 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.750 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.750 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.750 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.750 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.750 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.750 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.750 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.750 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.750 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.751 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.751 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.751 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.751 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.751 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.751 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.751 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.751 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.751 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.751 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.751 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.751 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.751 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.751 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.752 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.752 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.752 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.752 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.752 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.752 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.752 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.752 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.752 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.752 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.752 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.752 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.752 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.753 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.753 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.753 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.753 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.753 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.753 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.753 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.753 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.753 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.753 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.753 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.753 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.753 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "osd_support", 2024-09-16T02:05:54.753 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.754 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.754 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.754 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.754 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.754 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.754 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.754 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.754 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.754 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.754 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.754 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.754 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.754 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.754 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.755 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.755 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.755 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.755 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.755 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.755 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.755 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.755 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.755 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.755 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.755 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.755 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.755 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.755 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.756 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.756 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.756 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.756 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.756 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.756 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.756 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.756 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.756 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.756 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.756 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.756 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.756 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.757 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.757 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.757 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.757 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.757 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.757 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.757 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.757 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.757 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.757 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.757 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.757 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.757 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.757 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.758 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.758 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.758 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.758 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.758 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.758 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.758 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.758 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.758 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.758 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.758 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.758 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.758 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.758 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.759 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.759 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.759 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.759 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.759 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.759 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.759 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "pg_autoscaler", 2024-09-16T02:05:54.759 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.759 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.759 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.759 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.759 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.759 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.759 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.760 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.760 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.760 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.760 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.760 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.760 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.760 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.760 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.760 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.760 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.760 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.760 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.760 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.760 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.761 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.761 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.761 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.761 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.761 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.761 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.761 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.761 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.761 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.761 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.761 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.761 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.761 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.761 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.762 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.762 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.762 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.762 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.762 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.762 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.762 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.762 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.762 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.762 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.762 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.762 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.762 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.762 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.763 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.763 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.763 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.763 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.763 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.763 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.763 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.763 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.763 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.763 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.763 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.763 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.763 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.763 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.764 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.764 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.764 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.764 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.764 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.764 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.764 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.764 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.764 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.764 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.764 INFO:teuthology.orchestra.run.smithi105.stdout: "sleep_interval": { 2024-09-16T02:05:54.764 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "sleep_interval", 2024-09-16T02:05:54.764 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.765 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.765 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.765 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "60", 2024-09-16T02:05:54.765 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.765 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.765 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.765 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.765 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.765 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.765 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.765 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.765 INFO:teuthology.orchestra.run.smithi105.stdout: "threshold": { 2024-09-16T02:05:54.765 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "threshold", 2024-09-16T02:05:54.765 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-09-16T02:05:54.767 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.768 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.768 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "3.0", 2024-09-16T02:05:54.768 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "1.0", 2024-09-16T02:05:54.768 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.768 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.768 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "scaling threshold", 2024-09-16T02:05:54.768 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-16T02:05:54.768 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.768 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.768 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.768 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.768 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.768 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.768 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "progress", 2024-09-16T02:05:54.769 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.769 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.769 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.769 INFO:teuthology.orchestra.run.smithi105.stdout: "allow_pg_recovery_event": { 2024-09-16T02:05:54.769 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "allow_pg_recovery_event", 2024-09-16T02:05:54.769 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.769 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.769 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.769 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.769 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.769 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.769 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.769 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-16T02:05:54.770 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.770 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.770 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.770 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.770 INFO:teuthology.orchestra.run.smithi105.stdout: "enabled": { 2024-09-16T02:05:54.770 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "enabled", 2024-09-16T02:05:54.770 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.770 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.770 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.770 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.770 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.770 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.770 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.770 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.770 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.771 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.771 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.771 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.771 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.771 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.771 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.771 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.771 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.771 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.771 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.771 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.771 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.771 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.772 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.772 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.772 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.772 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.772 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.772 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.772 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.772 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.772 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.772 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.772 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.772 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.772 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.772 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.773 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.773 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.773 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.773 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.773 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.773 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.773 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.773 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.773 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.773 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.773 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.773 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.773 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.773 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.774 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.774 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.774 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.774 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.774 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.774 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.774 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.774 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.774 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.774 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.774 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.774 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.774 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.774 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.775 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.775 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.775 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.775 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.775 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.775 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.775 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.775 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.775 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.775 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.775 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.775 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.775 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.775 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.776 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.776 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.776 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.776 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.776 INFO:teuthology.orchestra.run.smithi105.stdout: "max_completed_events": { 2024-09-16T02:05:54.776 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "max_completed_events", 2024-09-16T02:05:54.776 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.776 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.776 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.776 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "50", 2024-09-16T02:05:54.776 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.776 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.776 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.777 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "number of past completed events to remember", 2024-09-16T02:05:54.777 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.777 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.777 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.777 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.777 INFO:teuthology.orchestra.run.smithi105.stdout: "sleep_interval": { 2024-09-16T02:05:54.777 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "sleep_interval", 2024-09-16T02:05:54.777 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.777 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.777 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.777 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "5", 2024-09-16T02:05:54.777 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.777 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.777 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.778 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how long the module is going to sleep", 2024-09-16T02:05:54.778 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.778 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.778 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.778 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.778 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.778 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.778 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.778 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "prometheus", 2024-09-16T02:05:54.778 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.778 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.778 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.778 INFO:teuthology.orchestra.run.smithi105.stdout: "cache": { 2024-09-16T02:05:54.778 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "cache", 2024-09-16T02:05:54.779 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.779 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.779 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.779 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.779 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.779 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.779 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.779 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.779 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.779 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.779 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.779 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.779 INFO:teuthology.orchestra.run.smithi105.stdout: "exclude_perf_counters": { 2024-09-16T02:05:54.779 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "exclude_perf_counters", 2024-09-16T02:05:54.780 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.780 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.780 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.780 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.780 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.780 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.780 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.780 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-16T02:05:54.780 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-16T02:05:54.780 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.780 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.780 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.780 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.781 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.781 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.781 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.781 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.781 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.781 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.781 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.781 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.781 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.781 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.781 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.781 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.781 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.781 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.782 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.782 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.782 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.782 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.782 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.782 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.782 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.782 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.782 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.782 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.782 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.782 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.782 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.782 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.783 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.783 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.783 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.783 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.783 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.783 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.783 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.783 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.783 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.783 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.783 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.783 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.783 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.783 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.784 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.784 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.784 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.784 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.784 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.784 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.784 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.784 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.784 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.784 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.784 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.784 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.784 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.785 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.785 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.785 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.785 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.785 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.785 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.785 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.785 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.785 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.785 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.785 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.785 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.785 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.785 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.786 INFO:teuthology.orchestra.run.smithi105.stdout: "rbd_stats_pools": { 2024-09-16T02:05:54.786 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rbd_stats_pools", 2024-09-16T02:05:54.786 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.786 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.786 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.786 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.786 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.786 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.786 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.786 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.786 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.786 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.786 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.786 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.787 INFO:teuthology.orchestra.run.smithi105.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-16T02:05:54.787 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-16T02:05:54.787 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.787 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.787 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.787 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "300", 2024-09-16T02:05:54.787 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.787 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.787 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.787 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.787 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.787 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.787 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.787 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.788 INFO:teuthology.orchestra.run.smithi105.stdout: "scrape_interval": { 2024-09-16T02:05:54.788 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "scrape_interval", 2024-09-16T02:05:54.788 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-09-16T02:05:54.788 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.788 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.788 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "15.0", 2024-09-16T02:05:54.788 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.788 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.788 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.788 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.788 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.788 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.788 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.789 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.789 INFO:teuthology.orchestra.run.smithi105.stdout: "server_addr": { 2024-09-16T02:05:54.789 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "server_addr", 2024-09-16T02:05:54.789 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.789 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.789 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.789 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "::", 2024-09-16T02:05:54.789 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.789 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.789 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.789 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-16T02:05:54.789 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.789 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.789 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.790 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.790 INFO:teuthology.orchestra.run.smithi105.stdout: "server_port": { 2024-09-16T02:05:54.790 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "server_port", 2024-09-16T02:05:54.790 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.790 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.790 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.790 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "9283", 2024-09-16T02:05:54.790 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.790 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.790 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.790 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-16T02:05:54.790 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.790 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.790 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.791 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.791 INFO:teuthology.orchestra.run.smithi105.stdout: "stale_cache_strategy": { 2024-09-16T02:05:54.791 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "stale_cache_strategy", 2024-09-16T02:05:54.791 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.791 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.791 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.791 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "log", 2024-09-16T02:05:54.791 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.791 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.791 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.791 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.791 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.791 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.791 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.792 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.792 INFO:teuthology.orchestra.run.smithi105.stdout: "standby_behaviour": { 2024-09-16T02:05:54.792 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "standby_behaviour", 2024-09-16T02:05:54.792 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.792 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.792 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.792 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "default", 2024-09-16T02:05:54.792 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.792 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.792 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.792 INFO:teuthology.orchestra.run.smithi105.stdout: "default", 2024-09-16T02:05:54.792 INFO:teuthology.orchestra.run.smithi105.stdout: "error" 2024-09-16T02:05:54.792 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.793 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.793 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.793 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.793 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.793 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.793 INFO:teuthology.orchestra.run.smithi105.stdout: "standby_error_status_code": { 2024-09-16T02:05:54.793 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "standby_error_status_code", 2024-09-16T02:05:54.793 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.793 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.793 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.793 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "500", 2024-09-16T02:05:54.793 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "400", 2024-09-16T02:05:54.793 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "599", 2024-09-16T02:05:54.793 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.793 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.794 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.794 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.794 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.794 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.794 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.794 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.794 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.794 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rbd_support", 2024-09-16T02:05:54.794 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.794 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.794 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.794 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.794 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.795 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.795 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.795 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.795 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.795 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.795 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.795 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.795 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.795 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.795 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.795 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.795 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.795 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.795 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.796 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.796 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.796 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.796 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.796 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.796 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.796 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.796 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.796 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.796 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.796 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.796 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.796 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.796 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.797 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.797 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.797 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.797 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.797 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.797 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.797 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.797 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.797 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.797 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.797 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.797 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.797 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.797 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.798 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.798 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.798 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.798 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.798 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.798 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.798 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.798 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.798 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.798 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.798 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.798 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.798 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.798 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.799 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.799 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.799 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.799 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.799 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.799 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.799 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.799 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.799 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.799 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.799 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.799 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.799 INFO:teuthology.orchestra.run.smithi105.stdout: "max_concurrent_snap_create": { 2024-09-16T02:05:54.800 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "max_concurrent_snap_create", 2024-09-16T02:05:54.800 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.800 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.800 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.800 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-09-16T02:05:54.800 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.800 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.800 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.800 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.800 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.800 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.800 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.800 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.801 INFO:teuthology.orchestra.run.smithi105.stdout: "mirror_snapshot_schedule": { 2024-09-16T02:05:54.801 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "mirror_snapshot_schedule", 2024-09-16T02:05:54.801 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.801 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.801 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.801 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.801 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.801 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.801 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.801 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.801 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.801 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.801 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.801 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.802 INFO:teuthology.orchestra.run.smithi105.stdout: "trash_purge_schedule": { 2024-09-16T02:05:54.802 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "trash_purge_schedule", 2024-09-16T02:05:54.802 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.802 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.802 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.802 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.802 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.802 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.802 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.802 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.802 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.802 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.802 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.802 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.803 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.803 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.803 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.803 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "restful", 2024-09-16T02:05:54.803 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.803 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.803 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.803 INFO:teuthology.orchestra.run.smithi105.stdout: "enable_auth": { 2024-09-16T02:05:54.803 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "enable_auth", 2024-09-16T02:05:54.803 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.803 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.803 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.803 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.803 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.804 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.804 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.804 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.804 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.804 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.804 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.804 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.804 INFO:teuthology.orchestra.run.smithi105.stdout: "key_file": { 2024-09-16T02:05:54.804 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "key_file", 2024-09-16T02:05:54.804 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.804 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.804 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.804 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.805 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.805 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.805 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.805 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.805 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.805 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.805 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.805 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.805 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.805 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.805 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.805 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.805 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.805 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.806 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.806 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.806 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.806 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.806 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.806 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.806 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.806 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.806 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.806 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.806 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.806 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.806 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.806 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.807 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.807 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.807 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.807 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.807 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.807 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.807 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.807 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.807 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.807 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.807 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.807 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.807 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.807 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.807 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.808 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.808 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.808 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.808 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.808 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.808 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.808 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.808 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.808 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.808 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.808 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.808 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.808 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.809 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.809 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.809 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.809 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.809 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.809 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.809 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.809 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.809 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.809 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.809 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.809 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.809 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.809 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.810 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.810 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.810 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.810 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.810 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.810 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.810 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.810 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.810 INFO:teuthology.orchestra.run.smithi105.stdout: "server_addr": { 2024-09-16T02:05:54.810 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "server_addr", 2024-09-16T02:05:54.810 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.810 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.810 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.810 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.811 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.811 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.811 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.811 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.811 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.811 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.811 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.811 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.811 INFO:teuthology.orchestra.run.smithi105.stdout: "server_port": { 2024-09-16T02:05:54.811 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "server_port", 2024-09-16T02:05:54.811 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.811 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.811 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.811 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.812 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.812 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.812 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.812 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.812 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.812 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.812 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.812 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.812 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.812 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.812 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.812 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rgw", 2024-09-16T02:05:54.812 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.813 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.813 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.813 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.813 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.813 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.813 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.813 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.813 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.813 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.813 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.813 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.813 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.813 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.813 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.814 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.814 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.814 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.814 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.814 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.814 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.814 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.814 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.814 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.814 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.814 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.814 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.814 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.814 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.815 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.815 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.815 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.815 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.815 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.815 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.815 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.815 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.815 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.815 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.815 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.815 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.815 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.815 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.816 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.816 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.816 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.816 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.816 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.816 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.816 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.816 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.816 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.816 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.816 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.816 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.816 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.816 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.817 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.817 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.817 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.817 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.817 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.817 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.817 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.817 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.817 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.817 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.817 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.817 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.817 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.818 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.818 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.818 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.818 INFO:teuthology.orchestra.run.smithi105.stdout: "secondary_zone_period_retry_limit": { 2024-09-16T02:05:54.818 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-16T02:05:54.818 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.818 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.818 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.818 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "5", 2024-09-16T02:05:54.818 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.818 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.818 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.818 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-16T02:05:54.818 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.818 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.819 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.819 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.819 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.819 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.819 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.819 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rook", 2024-09-16T02:05:54.819 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.819 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.819 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.819 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.819 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.819 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.819 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.819 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.820 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.820 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.820 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.820 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.820 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.820 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.820 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.820 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.820 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.820 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.820 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.820 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.820 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.820 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.821 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.821 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.821 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.821 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.821 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.821 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.821 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.821 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.821 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.821 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.821 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.821 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.821 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.821 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.822 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.822 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.822 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.822 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.822 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.822 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.822 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.822 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.822 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.822 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.822 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.822 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.822 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.822 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.823 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.823 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.823 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.823 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.823 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.823 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.823 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.823 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.823 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.823 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.823 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.823 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.823 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.823 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.823 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.824 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.824 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.824 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.824 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.824 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.824 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.824 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.824 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.824 INFO:teuthology.orchestra.run.smithi105.stdout: "storage_class": { 2024-09-16T02:05:54.824 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "storage_class", 2024-09-16T02:05:54.824 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.824 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.824 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.824 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "local", 2024-09-16T02:05:54.825 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.825 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.825 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.825 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-16T02:05:54.825 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.825 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.825 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.825 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.825 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.825 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.825 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.825 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "selftest", 2024-09-16T02:05:54.825 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.825 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.826 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.826 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.826 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.826 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.826 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.826 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.826 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.826 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.826 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.826 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.826 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.826 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.826 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.826 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.827 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.827 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.827 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.827 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.827 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.827 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.827 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.827 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.827 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.827 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.827 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.827 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.827 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.827 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.828 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.828 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.828 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.828 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.828 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.828 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.828 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.828 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.828 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.828 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.828 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.828 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.828 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.829 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.829 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.829 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.829 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.829 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.829 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.829 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.829 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.829 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.829 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.829 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.829 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.829 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.829 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.830 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.830 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.830 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.830 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.830 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.830 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.830 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.830 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.830 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.830 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.830 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.830 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.830 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.830 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.831 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.831 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.831 INFO:teuthology.orchestra.run.smithi105.stdout: "roption1": { 2024-09-16T02:05:54.831 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "roption1", 2024-09-16T02:05:54.831 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.831 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.831 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.831 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.831 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.831 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.831 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.831 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.831 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.831 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.832 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.832 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.832 INFO:teuthology.orchestra.run.smithi105.stdout: "roption2": { 2024-09-16T02:05:54.832 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "roption2", 2024-09-16T02:05:54.832 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.832 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.832 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.832 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "xyz", 2024-09-16T02:05:54.832 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.832 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.832 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.832 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.832 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.833 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.833 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.833 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.833 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption1": { 2024-09-16T02:05:54.833 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption1", 2024-09-16T02:05:54.833 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.833 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.833 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.833 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.833 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.833 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.833 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.833 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.833 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.834 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.834 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.834 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.834 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption2": { 2024-09-16T02:05:54.834 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption2", 2024-09-16T02:05:54.834 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.834 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.834 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.834 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.834 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.834 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.834 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.834 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.835 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.835 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.835 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.835 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.835 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption3": { 2024-09-16T02:05:54.835 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption3", 2024-09-16T02:05:54.835 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-09-16T02:05:54.835 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.835 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.835 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.835 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.835 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.835 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.835 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.836 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.836 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.836 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.836 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.836 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption4": { 2024-09-16T02:05:54.836 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption4", 2024-09-16T02:05:54.836 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.836 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.836 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.836 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.836 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.836 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.836 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.836 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.837 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.837 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.837 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.837 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.837 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption5": { 2024-09-16T02:05:54.837 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption5", 2024-09-16T02:05:54.837 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.837 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.837 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.837 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.837 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.837 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.837 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.838 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.838 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.838 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.838 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.838 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.838 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption6": { 2024-09-16T02:05:54.838 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption6", 2024-09-16T02:05:54.838 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.838 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.838 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.838 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.838 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.838 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.838 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.838 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.839 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.839 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.839 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.839 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.839 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption7": { 2024-09-16T02:05:54.839 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption7", 2024-09-16T02:05:54.839 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.839 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.839 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.839 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.839 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "1", 2024-09-16T02:05:54.839 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "42", 2024-09-16T02:05:54.839 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.840 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.840 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.840 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.840 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.840 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.840 INFO:teuthology.orchestra.run.smithi105.stdout: "testkey": { 2024-09-16T02:05:54.840 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "testkey", 2024-09-16T02:05:54.840 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.840 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.840 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.840 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.840 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.840 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.840 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.841 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.841 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.841 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.841 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.841 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.841 INFO:teuthology.orchestra.run.smithi105.stdout: "testlkey": { 2024-09-16T02:05:54.841 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "testlkey", 2024-09-16T02:05:54.841 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.841 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.841 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.841 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.841 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.841 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.841 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.842 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.842 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.842 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.842 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.842 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.842 INFO:teuthology.orchestra.run.smithi105.stdout: "testnewline": { 2024-09-16T02:05:54.842 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "testnewline", 2024-09-16T02:05:54.842 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.842 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.842 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.842 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.842 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.842 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.842 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.843 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.843 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.843 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.843 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.843 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.843 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.843 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.843 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.843 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "snap_schedule", 2024-09-16T02:05:54.843 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.843 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.843 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.843 INFO:teuthology.orchestra.run.smithi105.stdout: "allow_m_granularity": { 2024-09-16T02:05:54.843 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "allow_m_granularity", 2024-09-16T02:05:54.844 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.844 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.844 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.844 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.844 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.844 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.844 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.844 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "allow minute scheduled snapshots", 2024-09-16T02:05:54.844 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.844 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.844 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.844 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.844 INFO:teuthology.orchestra.run.smithi105.stdout: "dump_on_update": { 2024-09-16T02:05:54.845 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "dump_on_update", 2024-09-16T02:05:54.845 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.845 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.845 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.845 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.845 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.845 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.845 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.845 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "dump database to debug log on update", 2024-09-16T02:05:54.845 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.845 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.845 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.845 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.845 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.846 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.846 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.846 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.846 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.846 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.846 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.846 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.846 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.846 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.846 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.846 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.846 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.846 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.846 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.847 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.847 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.847 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.847 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.847 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.847 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.847 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.847 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.847 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.847 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.847 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.847 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.847 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.848 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.848 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.848 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.848 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.848 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.848 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.848 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.848 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.848 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.848 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.848 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.848 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.848 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.848 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.849 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.849 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.849 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.849 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.849 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.849 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.849 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.849 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.849 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.849 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.849 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.849 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.849 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.849 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.850 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.850 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.850 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.850 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.850 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.850 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.850 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.850 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.850 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.850 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.850 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.850 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.850 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.851 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.851 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.851 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.851 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.851 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "stats", 2024-09-16T02:05:54.851 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.851 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.851 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.851 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.851 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.851 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.851 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.851 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.851 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.852 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.852 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.852 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.852 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.852 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.852 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.852 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.852 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.852 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.852 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.852 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.852 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.852 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.853 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.853 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.853 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.853 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.853 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.853 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.853 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.853 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.853 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.853 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.853 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.853 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.853 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.853 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.854 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.854 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.854 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.854 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.854 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.854 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.854 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.854 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.854 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.854 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.854 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.854 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.854 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.854 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.855 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.855 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.855 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.855 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.855 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.855 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.855 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.855 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.855 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.855 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.855 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.855 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.855 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.855 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.856 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.856 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.856 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.856 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.856 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.856 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.856 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.856 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.856 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.856 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.856 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.856 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.856 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "status", 2024-09-16T02:05:54.856 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.857 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.857 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.857 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.857 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.857 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.857 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.857 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.857 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.857 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.857 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.857 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.857 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.857 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.857 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.858 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.858 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.858 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.858 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.858 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.858 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.858 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.858 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.858 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.858 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.858 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.858 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.858 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.858 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.859 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.859 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.859 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.859 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.859 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.859 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.859 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.859 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.859 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.859 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.859 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.859 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.859 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.859 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.860 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.860 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.860 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.860 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.860 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.860 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.860 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.860 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.860 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.860 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.860 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.860 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.860 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.860 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.861 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.861 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.861 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.861 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.861 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.861 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.861 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.861 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.861 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.861 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.861 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.861 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.861 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.861 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.862 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.862 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.862 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.862 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.862 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.862 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "telegraf", 2024-09-16T02:05:54.862 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.862 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.862 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.862 INFO:teuthology.orchestra.run.smithi105.stdout: "address": { 2024-09-16T02:05:54.862 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "address", 2024-09-16T02:05:54.862 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.862 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.863 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.863 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-16T02:05:54.863 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.863 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.863 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.863 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.863 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.863 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.863 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.863 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.863 INFO:teuthology.orchestra.run.smithi105.stdout: "interval": { 2024-09-16T02:05:54.863 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "interval", 2024-09-16T02:05:54.863 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.863 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.864 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.864 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "15", 2024-09-16T02:05:54.864 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.864 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.864 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.864 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.864 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.864 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.864 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.864 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.864 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.864 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.864 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.865 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.865 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.865 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.865 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.865 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.865 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.865 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.865 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.865 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.865 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.865 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.865 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.865 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.865 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.866 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.866 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.866 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.866 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.866 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.866 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.866 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.866 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.866 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.866 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.866 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.866 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.866 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.866 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.867 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.867 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.867 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.867 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.867 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.867 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.867 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.867 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.867 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.867 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.867 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.867 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.867 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.867 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.868 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.868 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.868 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.868 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.868 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.868 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.868 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.868 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.868 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.868 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.868 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.868 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.868 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.868 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.869 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.869 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.869 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.869 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.869 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.869 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.869 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.869 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.869 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.869 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.869 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.869 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.869 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.869 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.870 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "telemetry", 2024-09-16T02:05:54.870 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.870 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.870 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.870 INFO:teuthology.orchestra.run.smithi105.stdout: "channel_basic": { 2024-09-16T02:05:54.870 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "channel_basic", 2024-09-16T02:05:54.870 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.870 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.870 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.870 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.870 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.870 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.870 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.871 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-16T02:05:54.871 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.871 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.871 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.871 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.871 INFO:teuthology.orchestra.run.smithi105.stdout: "channel_crash": { 2024-09-16T02:05:54.871 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "channel_crash", 2024-09-16T02:05:54.871 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.871 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.871 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.871 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.871 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.871 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.871 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.872 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-16T02:05:54.872 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.872 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.872 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.872 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.872 INFO:teuthology.orchestra.run.smithi105.stdout: "channel_device": { 2024-09-16T02:05:54.872 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "channel_device", 2024-09-16T02:05:54.872 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.872 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.872 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.872 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.872 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.872 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.872 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.873 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-16T02:05:54.873 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.873 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.873 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.873 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.873 INFO:teuthology.orchestra.run.smithi105.stdout: "channel_ident": { 2024-09-16T02:05:54.873 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "channel_ident", 2024-09-16T02:05:54.873 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.873 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.873 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.873 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.873 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.873 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.874 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.874 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-16T02:05:54.874 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.874 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.874 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.874 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.874 INFO:teuthology.orchestra.run.smithi105.stdout: "channel_perf": { 2024-09-16T02:05:54.874 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "channel_perf", 2024-09-16T02:05:54.874 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.874 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.874 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.874 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.874 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.874 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.875 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.875 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-16T02:05:54.875 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.875 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.875 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.875 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.875 INFO:teuthology.orchestra.run.smithi105.stdout: "contact": { 2024-09-16T02:05:54.875 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "contact", 2024-09-16T02:05:54.875 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.875 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.875 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.875 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.875 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.875 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.876 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.876 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.876 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.876 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.876 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.876 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.876 INFO:teuthology.orchestra.run.smithi105.stdout: "description": { 2024-09-16T02:05:54.876 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "description", 2024-09-16T02:05:54.876 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.876 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.876 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.876 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.876 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.877 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.877 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.877 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.877 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.877 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.877 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.877 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.877 INFO:teuthology.orchestra.run.smithi105.stdout: "device_url": { 2024-09-16T02:05:54.877 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "device_url", 2024-09-16T02:05:54.877 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.877 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.877 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.877 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-16T02:05:54.877 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.878 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.878 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.878 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.878 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.878 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.878 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.878 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.878 INFO:teuthology.orchestra.run.smithi105.stdout: "enabled": { 2024-09-16T02:05:54.878 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "enabled", 2024-09-16T02:05:54.878 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.878 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.878 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.878 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.878 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.879 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.879 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.879 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.879 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.879 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.879 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.879 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.879 INFO:teuthology.orchestra.run.smithi105.stdout: "interval": { 2024-09-16T02:05:54.879 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "interval", 2024-09-16T02:05:54.879 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.879 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.879 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.879 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "24", 2024-09-16T02:05:54.879 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "8", 2024-09-16T02:05:54.880 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.880 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.880 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.880 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.880 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.880 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.880 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.880 INFO:teuthology.orchestra.run.smithi105.stdout: "last_opt_revision": { 2024-09-16T02:05:54.880 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "last_opt_revision", 2024-09-16T02:05:54.880 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.880 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.880 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.880 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "1", 2024-09-16T02:05:54.881 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.881 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.881 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.881 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.881 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.881 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.881 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.881 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.881 INFO:teuthology.orchestra.run.smithi105.stdout: "leaderboard": { 2024-09-16T02:05:54.881 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "leaderboard", 2024-09-16T02:05:54.881 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.881 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.881 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.881 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.882 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.882 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.882 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.882 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.882 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.882 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.882 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.882 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.882 INFO:teuthology.orchestra.run.smithi105.stdout: "leaderboard_description": { 2024-09-16T02:05:54.882 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "leaderboard_description", 2024-09-16T02:05:54.882 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.882 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.882 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.882 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.883 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.883 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.883 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.883 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.883 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.883 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.883 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.883 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.883 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.883 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.883 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.883 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.883 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.883 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.884 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.884 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.884 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.884 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.884 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.884 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.884 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.884 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.884 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.884 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.884 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.884 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.884 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.885 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.885 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.885 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.885 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.885 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.885 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.885 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.885 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.885 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.885 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.885 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.885 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.885 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.885 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.886 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.886 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.886 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.886 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.886 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.886 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.886 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.886 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.886 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.886 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.886 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.886 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.886 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.886 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.887 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.887 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.887 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.887 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.887 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.887 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.887 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.887 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.887 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.887 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.887 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.887 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.887 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.887 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.888 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.888 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.888 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.888 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.888 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.888 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.888 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.888 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.888 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.888 INFO:teuthology.orchestra.run.smithi105.stdout: "organization": { 2024-09-16T02:05:54.888 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "organization", 2024-09-16T02:05:54.888 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.888 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.889 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.889 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.889 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.889 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.889 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.889 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.889 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.889 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.889 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.889 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.889 INFO:teuthology.orchestra.run.smithi105.stdout: "proxy": { 2024-09-16T02:05:54.889 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "proxy", 2024-09-16T02:05:54.889 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.889 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.889 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.890 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.890 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.890 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.890 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.890 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.890 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.890 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.890 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.890 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.890 INFO:teuthology.orchestra.run.smithi105.stdout: "url": { 2024-09-16T02:05:54.890 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "url", 2024-09-16T02:05:54.890 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.890 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.891 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.891 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-16T02:05:54.891 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.891 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.891 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.891 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.891 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.891 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.891 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.891 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.891 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.891 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.891 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.891 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "test_orchestrator", 2024-09-16T02:05:54.891 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.892 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.892 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.892 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.892 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.892 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.892 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.892 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.892 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.892 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.892 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.892 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.892 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.892 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.893 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.893 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.893 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.893 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.893 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.893 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.893 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.893 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.893 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.893 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.893 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.893 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.893 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.893 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.894 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.894 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.894 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.894 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.894 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.894 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.894 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.894 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.894 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.894 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.894 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.894 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.894 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.894 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.894 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.895 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.895 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.895 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.895 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.895 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.895 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.895 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.895 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.895 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.895 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.895 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.895 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.895 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.896 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.896 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.896 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.896 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.896 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.896 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.896 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.896 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.896 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.896 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.896 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.896 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.896 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.897 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.897 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.897 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.897 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.897 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.897 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.897 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.897 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "volumes", 2024-09-16T02:05:54.897 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.897 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.897 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.897 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.897 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.897 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.898 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.898 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.898 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.898 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.898 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.898 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.898 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.898 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.898 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.898 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.898 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.898 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.898 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.898 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.899 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.899 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.899 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.899 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.899 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.899 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.899 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.899 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.899 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.899 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.899 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.899 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.899 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.899 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.900 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.900 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.900 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.900 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.900 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.900 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.900 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.900 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.900 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.900 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.900 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.900 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.900 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.901 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.901 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.901 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.901 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.901 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.901 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.901 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.901 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.901 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.901 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.901 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.901 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.901 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.901 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.902 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.902 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.902 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.902 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.902 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.902 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.902 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.902 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.902 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.902 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.902 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.902 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.902 INFO:teuthology.orchestra.run.smithi105.stdout: "max_concurrent_clones": { 2024-09-16T02:05:54.902 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "max_concurrent_clones", 2024-09-16T02:05:54.903 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.903 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.903 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.903 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "4", 2024-09-16T02:05:54.903 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.903 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.903 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.903 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-16T02:05:54.903 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.903 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.903 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.903 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.903 INFO:teuthology.orchestra.run.smithi105.stdout: "periodic_async_work": { 2024-09-16T02:05:54.903 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "periodic_async_work", 2024-09-16T02:05:54.904 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.904 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.904 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.904 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.904 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.904 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.904 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.904 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Periodically check for async work", 2024-09-16T02:05:54.904 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.904 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.904 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.904 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.904 INFO:teuthology.orchestra.run.smithi105.stdout: "snapshot_clone_delay": { 2024-09-16T02:05:54.905 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "snapshot_clone_delay", 2024-09-16T02:05:54.905 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.905 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.905 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.905 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0", 2024-09-16T02:05:54.905 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.905 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.905 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.905 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-16T02:05:54.905 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.905 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.905 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.905 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.905 INFO:teuthology.orchestra.run.smithi105.stdout: "snapshot_clone_no_wait": { 2024-09-16T02:05:54.905 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "snapshot_clone_no_wait", 2024-09-16T02:05:54.906 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.906 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.906 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.906 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-09-16T02:05:54.906 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.906 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.906 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.906 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-16T02:05:54.906 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.906 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.906 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.906 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.906 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.906 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.907 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.907 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "zabbix", 2024-09-16T02:05:54.907 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-09-16T02:05:54.907 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-09-16T02:05:54.907 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-09-16T02:05:54.907 INFO:teuthology.orchestra.run.smithi105.stdout: "discovery_interval": { 2024-09-16T02:05:54.907 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "discovery_interval", 2024-09-16T02:05:54.907 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "uint", 2024-09-16T02:05:54.907 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.907 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.907 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "100", 2024-09-16T02:05:54.907 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.907 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.907 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.908 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.908 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.908 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.908 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.908 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.908 INFO:teuthology.orchestra.run.smithi105.stdout: "identifier": { 2024-09-16T02:05:54.908 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "identifier", 2024-09-16T02:05:54.908 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.908 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.908 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.908 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.908 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.908 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.908 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.909 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.909 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.909 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.909 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.909 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.909 INFO:teuthology.orchestra.run.smithi105.stdout: "interval": { 2024-09-16T02:05:54.909 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "interval", 2024-09-16T02:05:54.909 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-09-16T02:05:54.909 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.909 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.909 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "60", 2024-09-16T02:05:54.909 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.909 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.909 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.910 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.910 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.910 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.910 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.910 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.910 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-09-16T02:05:54.910 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-09-16T02:05:54.910 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.910 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.910 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.910 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.910 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.910 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.910 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.911 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.911 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.911 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.911 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.911 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.911 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.911 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.911 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.911 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.911 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.911 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.911 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.911 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-09-16T02:05:54.911 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-09-16T02:05:54.912 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.912 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.912 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.912 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.912 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.912 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.912 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.912 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.912 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.912 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.912 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.912 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.912 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-09-16T02:05:54.912 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-09-16T02:05:54.913 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.913 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.913 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.913 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-09-16T02:05:54.913 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.913 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.913 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-09-16T02:05:54.913 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-09-16T02:05:54.913 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-09-16T02:05:54.913 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-09-16T02:05:54.913 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-09-16T02:05:54.913 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-09-16T02:05:54.913 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-09-16T02:05:54.914 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.914 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.914 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.914 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.914 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.914 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.914 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-09-16T02:05:54.914 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-09-16T02:05:54.914 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-09-16T02:05:54.914 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.914 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-09-16T02:05:54.914 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-09-16T02:05:54.914 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.914 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.915 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.915 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.915 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.915 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.915 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.915 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.915 INFO:teuthology.orchestra.run.smithi105.stdout: "zabbix_host": { 2024-09-16T02:05:54.915 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "zabbix_host", 2024-09-16T02:05:54.915 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.915 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.915 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.915 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-09-16T02:05:54.915 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.915 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.915 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.916 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.916 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.916 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.916 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.916 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.916 INFO:teuthology.orchestra.run.smithi105.stdout: "zabbix_port": { 2024-09-16T02:05:54.916 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "zabbix_port", 2024-09-16T02:05:54.916 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-09-16T02:05:54.916 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.916 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.916 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10051", 2024-09-16T02:05:54.916 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.916 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.917 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.917 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.917 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.917 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.917 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.917 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.917 INFO:teuthology.orchestra.run.smithi105.stdout: "zabbix_sender": { 2024-09-16T02:05:54.917 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "zabbix_sender", 2024-09-16T02:05:54.917 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-09-16T02:05:54.917 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-09-16T02:05:54.917 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-09-16T02:05:54.917 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-16T02:05:54.917 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-09-16T02:05:54.917 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-09-16T02:05:54.917 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-09-16T02:05:54.918 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-09-16T02:05:54.918 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-09-16T02:05:54.918 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-09-16T02:05:54.918 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-09-16T02:05:54.918 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.918 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.918 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.918 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.918 INFO:teuthology.orchestra.run.smithi105.stdout: "services": { 2024-09-16T02:05:54.918 INFO:teuthology.orchestra.run.smithi105.stdout: "dashboard": "https://172.21.15.105:8443/" 2024-09-16T02:05:54.918 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.918 INFO:teuthology.orchestra.run.smithi105.stdout: "always_on_modules": { 2024-09-16T02:05:54.918 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus": [ 2024-09-16T02:05:54.918 INFO:teuthology.orchestra.run.smithi105.stdout: "balancer", 2024-09-16T02:05:54.919 INFO:teuthology.orchestra.run.smithi105.stdout: "crash", 2024-09-16T02:05:54.919 INFO:teuthology.orchestra.run.smithi105.stdout: "devicehealth", 2024-09-16T02:05:54.919 INFO:teuthology.orchestra.run.smithi105.stdout: "orchestrator", 2024-09-16T02:05:54.919 INFO:teuthology.orchestra.run.smithi105.stdout: "pg_autoscaler", 2024-09-16T02:05:54.919 INFO:teuthology.orchestra.run.smithi105.stdout: "progress", 2024-09-16T02:05:54.919 INFO:teuthology.orchestra.run.smithi105.stdout: "rbd_support", 2024-09-16T02:05:54.919 INFO:teuthology.orchestra.run.smithi105.stdout: "status", 2024-09-16T02:05:54.919 INFO:teuthology.orchestra.run.smithi105.stdout: "telemetry", 2024-09-16T02:05:54.919 INFO:teuthology.orchestra.run.smithi105.stdout: "volumes" 2024-09-16T02:05:54.919 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.919 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific": [ 2024-09-16T02:05:54.919 INFO:teuthology.orchestra.run.smithi105.stdout: "balancer", 2024-09-16T02:05:54.919 INFO:teuthology.orchestra.run.smithi105.stdout: "crash", 2024-09-16T02:05:54.919 INFO:teuthology.orchestra.run.smithi105.stdout: "devicehealth", 2024-09-16T02:05:54.920 INFO:teuthology.orchestra.run.smithi105.stdout: "orchestrator", 2024-09-16T02:05:54.920 INFO:teuthology.orchestra.run.smithi105.stdout: "pg_autoscaler", 2024-09-16T02:05:54.920 INFO:teuthology.orchestra.run.smithi105.stdout: "progress", 2024-09-16T02:05:54.920 INFO:teuthology.orchestra.run.smithi105.stdout: "rbd_support", 2024-09-16T02:05:54.920 INFO:teuthology.orchestra.run.smithi105.stdout: "status", 2024-09-16T02:05:54.920 INFO:teuthology.orchestra.run.smithi105.stdout: "telemetry", 2024-09-16T02:05:54.920 INFO:teuthology.orchestra.run.smithi105.stdout: "volumes" 2024-09-16T02:05:54.920 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.920 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy": [ 2024-09-16T02:05:54.920 INFO:teuthology.orchestra.run.smithi105.stdout: "balancer", 2024-09-16T02:05:54.920 INFO:teuthology.orchestra.run.smithi105.stdout: "crash", 2024-09-16T02:05:54.920 INFO:teuthology.orchestra.run.smithi105.stdout: "devicehealth", 2024-09-16T02:05:54.920 INFO:teuthology.orchestra.run.smithi105.stdout: "orchestrator", 2024-09-16T02:05:54.920 INFO:teuthology.orchestra.run.smithi105.stdout: "pg_autoscaler", 2024-09-16T02:05:54.921 INFO:teuthology.orchestra.run.smithi105.stdout: "progress", 2024-09-16T02:05:54.921 INFO:teuthology.orchestra.run.smithi105.stdout: "rbd_support", 2024-09-16T02:05:54.921 INFO:teuthology.orchestra.run.smithi105.stdout: "status", 2024-09-16T02:05:54.921 INFO:teuthology.orchestra.run.smithi105.stdout: "telemetry", 2024-09-16T02:05:54.921 INFO:teuthology.orchestra.run.smithi105.stdout: "volumes" 2024-09-16T02:05:54.921 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.921 INFO:teuthology.orchestra.run.smithi105.stdout: "reef": [ 2024-09-16T02:05:54.921 INFO:teuthology.orchestra.run.smithi105.stdout: "balancer", 2024-09-16T02:05:54.921 INFO:teuthology.orchestra.run.smithi105.stdout: "crash", 2024-09-16T02:05:54.921 INFO:teuthology.orchestra.run.smithi105.stdout: "devicehealth", 2024-09-16T02:05:54.921 INFO:teuthology.orchestra.run.smithi105.stdout: "orchestrator", 2024-09-16T02:05:54.921 INFO:teuthology.orchestra.run.smithi105.stdout: "pg_autoscaler", 2024-09-16T02:05:54.921 INFO:teuthology.orchestra.run.smithi105.stdout: "progress", 2024-09-16T02:05:54.921 INFO:teuthology.orchestra.run.smithi105.stdout: "rbd_support", 2024-09-16T02:05:54.922 INFO:teuthology.orchestra.run.smithi105.stdout: "status", 2024-09-16T02:05:54.922 INFO:teuthology.orchestra.run.smithi105.stdout: "telemetry", 2024-09-16T02:05:54.922 INFO:teuthology.orchestra.run.smithi105.stdout: "volumes" 2024-09-16T02:05:54.922 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-16T02:05:54.922 INFO:teuthology.orchestra.run.smithi105.stdout: "squid": [ 2024-09-16T02:05:54.922 INFO:teuthology.orchestra.run.smithi105.stdout: "balancer", 2024-09-16T02:05:54.922 INFO:teuthology.orchestra.run.smithi105.stdout: "crash", 2024-09-16T02:05:54.922 INFO:teuthology.orchestra.run.smithi105.stdout: "devicehealth", 2024-09-16T02:05:54.922 INFO:teuthology.orchestra.run.smithi105.stdout: "orchestrator", 2024-09-16T02:05:54.922 INFO:teuthology.orchestra.run.smithi105.stdout: "pg_autoscaler", 2024-09-16T02:05:54.922 INFO:teuthology.orchestra.run.smithi105.stdout: "progress", 2024-09-16T02:05:54.922 INFO:teuthology.orchestra.run.smithi105.stdout: "rbd_support", 2024-09-16T02:05:54.922 INFO:teuthology.orchestra.run.smithi105.stdout: "status", 2024-09-16T02:05:54.922 INFO:teuthology.orchestra.run.smithi105.stdout: "telemetry", 2024-09-16T02:05:54.922 INFO:teuthology.orchestra.run.smithi105.stdout: "volumes" 2024-09-16T02:05:54.923 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-16T02:05:54.923 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.923 INFO:teuthology.orchestra.run.smithi105.stdout: "last_failure_osd_epoch": 3, 2024-09-16T02:05:54.923 INFO:teuthology.orchestra.run.smithi105.stdout: "active_clients": [ 2024-09-16T02:05:54.923 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.923 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "devicehealth", 2024-09-16T02:05:54.923 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-16T02:05:54.923 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.923 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-16T02:05:54.923 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:0", 2024-09-16T02:05:54.923 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 3952217751 2024-09-16T02:05:54.923 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.923 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-16T02:05:54.923 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.924 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.924 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "libcephsqlite", 2024-09-16T02:05:54.924 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-16T02:05:54.924 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.924 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-16T02:05:54.924 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:0", 2024-09-16T02:05:54.924 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 1478019961 2024-09-16T02:05:54.924 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.924 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-16T02:05:54.924 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.924 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.924 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rbd_support", 2024-09-16T02:05:54.924 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-16T02:05:54.924 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.925 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-16T02:05:54.925 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:0", 2024-09-16T02:05:54.925 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 2130860808 2024-09-16T02:05:54.925 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.925 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-16T02:05:54.925 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:54.925 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.925 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "volumes", 2024-09-16T02:05:54.925 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-16T02:05:54.925 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-16T02:05:54.925 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-16T02:05:54.925 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:0", 2024-09-16T02:05:54.925 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 2972851893 2024-09-16T02:05:54.925 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.926 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-16T02:05:54.926 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:54.926 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-16T02:05:54.926 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-16T02:05:54.981 DEBUG:tasks.check_counter:Getting stats from a 2024-09-16T02:05:54.982 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-16T02:05:54.982 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:53 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/1057314558' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T02:05:54.982 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:53 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2897089704' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T02:05:54.982 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:53 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/4017115349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T02:05:54.982 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:53 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/4022527764' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-16T02:05:54.983 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:53 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/1057314558' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T02:05:54.983 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:53 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/2897089704' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T02:05:54.983 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:53 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/4017115349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T02:05:54.984 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:53 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/4022527764' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-16T02:05:54.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:53 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/1057314558' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T02:05:54.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:53 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/2897089704' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T02:05:54.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:53 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/4017115349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T02:05:54.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:53 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/4022527764' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-16T02:05:54.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:54 smithi137 ceph-mon[34099]: pgmap v440: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 2.9 KiB/s wr, 221 op/s 2024-09-16T02:05:55.009 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:54 smithi105 ceph-mon[31197]: pgmap v440: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 2.9 KiB/s wr, 221 op/s 2024-09-16T02:05:55.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:54 smithi190 ceph-mon[34762]: pgmap v440: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 2.9 KiB/s wr, 221 op/s 2024-09-16T02:05:55.240 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:55.850 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-16T02:05:55.850 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T02:05:55.850 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 618, 2024-09-16T02:05:55.850 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 619, 2024-09-16T02:05:55.850 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 185844, 2024-09-16T02:05:55.850 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 196207, 2024-09-16T02:05:55.850 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 6, 2024-09-16T02:05:55.851 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 2, 2024-09-16T02:05:55.851 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.051883229, 2024-09-16T02:05:55.851 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.016547423, 2024-09-16T02:05:55.851 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.013780934, 2024-09-16T02:05:55.851 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.016413895, 2024-09-16T02:05:55.851 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:05:55.851 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 617, 2024-09-16T02:05:55.851 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.020094562, 2024-09-16T02:05:55.851 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000032568 2024-09-16T02:05:55.851 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.851 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:05:55.851 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.851 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.851 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.852 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.852 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T02:05:55.852 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T02:05:55.852 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.852 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T02:05:55.852 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 304, 2024-09-16T02:05:55.852 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 300, 2024-09-16T02:05:55.852 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 517856, 2024-09-16T02:05:55.852 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 120600, 2024-09-16T02:05:55.852 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 4, 2024-09-16T02:05:55.852 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 3, 2024-09-16T02:05:55.852 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.033110226, 2024-09-16T02:05:55.852 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.010800539, 2024-09-16T02:05:55.853 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.016652531, 2024-09-16T02:05:55.853 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.008012061, 2024-09-16T02:05:55.853 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:05:55.853 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 299, 2024-09-16T02:05:55.853 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.013190102, 2024-09-16T02:05:55.853 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000044114 2024-09-16T02:05:55.853 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.853 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:05:55.853 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.853 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.853 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.853 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.853 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 468496, 2024-09-16T02:05:55.853 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 57568 2024-09-16T02:05:55.854 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.854 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T02:05:55.854 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 6, 2024-09-16T02:05:55.854 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 73, 2024-09-16T02:05:55.854 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 3420, 2024-09-16T02:05:55.854 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 201054, 2024-09-16T02:05:55.854 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 4, 2024-09-16T02:05:55.854 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 3, 2024-09-16T02:05:55.854 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.008308917, 2024-09-16T02:05:55.854 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.003989646, 2024-09-16T02:05:55.855 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.003731255, 2024-09-16T02:05:55.855 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000019137, 2024-09-16T02:05:55.855 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:05:55.855 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 72, 2024-09-16T02:05:55.855 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.007785871, 2024-09-16T02:05:55.855 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000108137 2024-09-16T02:05:55.855 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.855 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:05:55.855 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.856 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.856 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.856 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.856 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-16T02:05:55.856 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 200736 2024-09-16T02:05:55.856 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.856 INFO:teuthology.orchestra.run.smithi105.stdout: "cct": { 2024-09-16T02:05:55.856 INFO:teuthology.orchestra.run.smithi105.stdout: "total_workers": 1, 2024-09-16T02:05:55.856 INFO:teuthology.orchestra.run.smithi105.stdout: "unhealthy_workers": 0 2024-09-16T02:05:55.856 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.856 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-MDSRank": { 2024-09-16T02:05:55.856 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-09-16T02:05:55.856 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-09-16T02:05:55.857 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1393, 2024-09-16T02:05:55.857 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.035084050, 2024-09-16T02:05:55.857 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000025185 2024-09-16T02:05:55.857 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:55.857 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.857 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-PurgeQueue": { 2024-09-16T02:05:55.857 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-09-16T02:05:55.857 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-09-16T02:05:55.858 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.858 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.858 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.858 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:55.858 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.858 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": { 2024-09-16T02:05:55.858 INFO:teuthology.orchestra.run.smithi105.stdout: "request": 0, 2024-09-16T02:05:55.858 INFO:teuthology.orchestra.run.smithi105.stdout: "reply": 0, 2024-09-16T02:05:55.858 INFO:teuthology.orchestra.run.smithi105.stdout: "reply_latency": { 2024-09-16T02:05:55.858 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.858 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.858 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.859 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.859 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_reply": 0, 2024-09-16T02:05:55.859 INFO:teuthology.orchestra.run.smithi105.stdout: "forward": 0, 2024-09-16T02:05:55.859 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_complete": 0, 2024-09-16T02:05:55.859 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_keys": 0, 2024-09-16T02:05:55.859 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_commit": 0, 2024-09-16T02:05:55.859 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_split": 0, 2024-09-16T02:05:55.859 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_merge": 0, 2024-09-16T02:05:55.860 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes": 0, 2024-09-16T02:05:55.860 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_top": 0, 2024-09-16T02:05:55.860 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_bottom": 0, 2024-09-16T02:05:55.860 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pin_tail": 0, 2024-09-16T02:05:55.860 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pinned": 0, 2024-09-16T02:05:55.860 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_expired": 0, 2024-09-16T02:05:55.860 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_with_caps": 0, 2024-09-16T02:05:55.861 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": 0, 2024-09-16T02:05:55.861 INFO:teuthology.orchestra.run.smithi105.stdout: "subtrees": 1, 2024-09-16T02:05:55.861 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse": 0, 2024-09-16T02:05:55.861 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_hit": 0, 2024-09-16T02:05:55.861 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_forward": 0, 2024-09-16T02:05:55.861 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_discover": 0, 2024-09-16T02:05:55.861 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_dir_fetch": 0, 2024-09-16T02:05:55.861 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_remote_ino": 0, 2024-09-16T02:05:55.861 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_lock": 0, 2024-09-16T02:05:55.861 INFO:teuthology.orchestra.run.smithi105.stdout: "load_cent": 0, 2024-09-16T02:05:55.861 INFO:teuthology.orchestra.run.smithi105.stdout: "q": 0, 2024-09-16T02:05:55.861 INFO:teuthology.orchestra.run.smithi105.stdout: "exported": 0, 2024-09-16T02:05:55.861 INFO:teuthology.orchestra.run.smithi105.stdout: "exported_inodes": 0, 2024-09-16T02:05:55.862 INFO:teuthology.orchestra.run.smithi105.stdout: "imported": 0, 2024-09-16T02:05:55.862 INFO:teuthology.orchestra.run.smithi105.stdout: "imported_inodes": 0, 2024-09-16T02:05:55.862 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_dir_fetch": 0, 2024-09-16T02:05:55.862 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_backtrace_fetch": 0, 2024-09-16T02:05:55.862 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_peer_discover": 0, 2024-09-16T02:05:55.862 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rfiles": 0, 2024-09-16T02:05:55.862 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rbytes": 0, 2024-09-16T02:05:55.862 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rsnaps": 0, 2024-09-16T02:05:55.862 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T02:05:55.862 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_set_tag": 0, 2024-09-16T02:05:55.862 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T02:05:55.862 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_inotable_repaired": 0, 2024-09-16T02:05:55.862 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_inodes": 0, 2024-09-16T02:05:55.862 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T02:05:55.862 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T02:05:55.863 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_file_inodes": 0, 2024-09-16T02:05:55.863 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_inode_file_caps": 0, 2024-09-16T02:05:55.863 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T02:05:55.863 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_grant": 0, 2024-09-16T02:05:55.863 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T02:05:55.863 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T02:05:55.863 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T02:05:55.863 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps": 0, 2024-09-16T02:05:55.863 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps_dirty": 0, 2024-09-16T02:05:55.863 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_cap_release": 0, 2024-09-16T02:05:55.863 INFO:teuthology.orchestra.run.smithi105.stdout: "process_request_cap_release": 0 2024-09-16T02:05:55.863 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.863 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_cache": { 2024-09-16T02:05:55.863 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update": 0, 2024-09-16T02:05:55.864 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update_receipt": 0, 2024-09-16T02:05:55.864 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_try_discover": 0, 2024-09-16T02:05:55.864 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_send_discover": 0, 2024-09-16T02:05:55.864 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_handle_discover": 0, 2024-09-16T02:05:55.864 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays": 0, 2024-09-16T02:05:55.864 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_delayed": 0, 2024-09-16T02:05:55.864 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_enqueuing": 0, 2024-09-16T02:05:55.864 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_created": 0, 2024-09-16T02:05:55.864 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_enqueued": 0, 2024-09-16T02:05:55.864 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_reintegrated": 0, 2024-09-16T02:05:55.864 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_migrated": 0, 2024-09-16T02:05:55.864 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_processing": 0, 2024-09-16T02:05:55.864 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_enqueued": 0, 2024-09-16T02:05:55.864 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_prioritized": 0, 2024-09-16T02:05:55.865 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_started": 0, 2024-09-16T02:05:55.865 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_completed": 0, 2024-09-16T02:05:55.865 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_quiesce_path": 0, 2024-09-16T02:05:55.865 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_quiesce_inode": 0, 2024-09-16T02:05:55.865 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T02:05:55.865 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_exportdir": 0, 2024-09-16T02:05:55.865 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_flush": 0, 2024-09-16T02:05:55.865 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragmentdir": 0, 2024-09-16T02:05:55.865 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragstats": 0, 2024-09-16T02:05:55.865 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_inodestats": 0 2024-09-16T02:05:55.865 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.865 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_log": { 2024-09-16T02:05:55.865 INFO:teuthology.orchestra.run.smithi105.stdout: "evlrg": 0, 2024-09-16T02:05:55.865 INFO:teuthology.orchestra.run.smithi105.stdout: "evadd": 0, 2024-09-16T02:05:55.866 INFO:teuthology.orchestra.run.smithi105.stdout: "evex": 0, 2024-09-16T02:05:55.866 INFO:teuthology.orchestra.run.smithi105.stdout: "evtrm": 0, 2024-09-16T02:05:55.866 INFO:teuthology.orchestra.run.smithi105.stdout: "ev": 3, 2024-09-16T02:05:55.866 INFO:teuthology.orchestra.run.smithi105.stdout: "evexg": 0, 2024-09-16T02:05:55.866 INFO:teuthology.orchestra.run.smithi105.stdout: "evexd": 0, 2024-09-16T02:05:55.866 INFO:teuthology.orchestra.run.smithi105.stdout: "segadd": 0, 2024-09-16T02:05:55.866 INFO:teuthology.orchestra.run.smithi105.stdout: "segex": 0, 2024-09-16T02:05:55.866 INFO:teuthology.orchestra.run.smithi105.stdout: "segtrm": 0, 2024-09-16T02:05:55.866 INFO:teuthology.orchestra.run.smithi105.stdout: "seg": 2, 2024-09-16T02:05:55.866 INFO:teuthology.orchestra.run.smithi105.stdout: "segmjr": 2, 2024-09-16T02:05:55.866 INFO:teuthology.orchestra.run.smithi105.stdout: "segexg": 0, 2024-09-16T02:05:55.866 INFO:teuthology.orchestra.run.smithi105.stdout: "segexd": 0, 2024-09-16T02:05:55.866 INFO:teuthology.orchestra.run.smithi105.stdout: "expos": 4194304, 2024-09-16T02:05:55.866 INFO:teuthology.orchestra.run.smithi105.stdout: "wrpos": 4195387, 2024-09-16T02:05:55.866 INFO:teuthology.orchestra.run.smithi105.stdout: "rdpos": 4194892, 2024-09-16T02:05:55.867 INFO:teuthology.orchestra.run.smithi105.stdout: "jlat": { 2024-09-16T02:05:55.867 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.867 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.867 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.867 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.867 INFO:teuthology.orchestra.run.smithi105.stdout: "replayed": 3 2024-09-16T02:05:55.867 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.867 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_mem": { 2024-09-16T02:05:55.867 INFO:teuthology.orchestra.run.smithi105.stdout: "ino": 3, 2024-09-16T02:05:55.867 INFO:teuthology.orchestra.run.smithi105.stdout: "ino+": 3, 2024-09-16T02:05:55.867 INFO:teuthology.orchestra.run.smithi105.stdout: "ino-": 0, 2024-09-16T02:05:55.867 INFO:teuthology.orchestra.run.smithi105.stdout: "dir": 1, 2024-09-16T02:05:55.867 INFO:teuthology.orchestra.run.smithi105.stdout: "dir+": 1, 2024-09-16T02:05:55.867 INFO:teuthology.orchestra.run.smithi105.stdout: "dir-": 0, 2024-09-16T02:05:55.867 INFO:teuthology.orchestra.run.smithi105.stdout: "dn": 0, 2024-09-16T02:05:55.868 INFO:teuthology.orchestra.run.smithi105.stdout: "dn+": 0, 2024-09-16T02:05:55.868 INFO:teuthology.orchestra.run.smithi105.stdout: "dn-": 0, 2024-09-16T02:05:55.868 INFO:teuthology.orchestra.run.smithi105.stdout: "cap": 0, 2024-09-16T02:05:55.868 INFO:teuthology.orchestra.run.smithi105.stdout: "cap+": 0, 2024-09-16T02:05:55.868 INFO:teuthology.orchestra.run.smithi105.stdout: "cap-": 0, 2024-09-16T02:05:55.868 INFO:teuthology.orchestra.run.smithi105.stdout: "rss": 48372, 2024-09-16T02:05:55.868 INFO:teuthology.orchestra.run.smithi105.stdout: "heap": 215328 2024-09-16T02:05:55.868 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.868 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_server": { 2024-09-16T02:05:55.868 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_client_request": 0, 2024-09-16T02:05:55.868 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_server_request": 0, 2024-09-16T02:05:55.868 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_request": 0, 2024-09-16T02:05:55.868 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_session": 0, 2024-09-16T02:05:55.868 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_peer_request": 0, 2024-09-16T02:05:55.869 INFO:teuthology.orchestra.run.smithi105.stdout: "req_create_latency": { 2024-09-16T02:05:55.869 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.869 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.869 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.869 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.869 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getattr_latency": { 2024-09-16T02:05:55.869 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.869 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.869 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.869 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.869 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getfilelock_latency": { 2024-09-16T02:05:55.869 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.869 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.869 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.870 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.870 INFO:teuthology.orchestra.run.smithi105.stdout: "req_link_latency": { 2024-09-16T02:05:55.870 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.870 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.870 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.870 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.870 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookup_latency": { 2024-09-16T02:05:55.870 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.870 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.870 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.870 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.870 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookuphash_latency": { 2024-09-16T02:05:55.870 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.870 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.870 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.871 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.871 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupino_latency": { 2024-09-16T02:05:55.871 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.871 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.871 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.871 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.871 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupname_latency": { 2024-09-16T02:05:55.871 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.871 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.871 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.871 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.871 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupparent_latency": { 2024-09-16T02:05:55.871 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.871 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.872 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.872 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.872 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupsnap_latency": { 2024-09-16T02:05:55.872 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.872 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.872 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.872 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.872 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lssnap_latency": { 2024-09-16T02:05:55.872 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.872 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.872 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.872 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.872 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mkdir_latency": { 2024-09-16T02:05:55.872 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.873 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.873 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.873 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.873 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mknod_latency": { 2024-09-16T02:05:55.873 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.873 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.874 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.874 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.874 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mksnap_latency": { 2024-09-16T02:05:55.874 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.874 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.874 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.874 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.874 INFO:teuthology.orchestra.run.smithi105.stdout: "req_open_latency": { 2024-09-16T02:05:55.874 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.874 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.874 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.874 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.874 INFO:teuthology.orchestra.run.smithi105.stdout: "req_readdir_latency": { 2024-09-16T02:05:55.875 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.875 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.875 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.875 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.875 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rename_latency": { 2024-09-16T02:05:55.875 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.875 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.875 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.875 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.876 INFO:teuthology.orchestra.run.smithi105.stdout: "req_renamesnap_latency": { 2024-09-16T02:05:55.876 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.876 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.876 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.876 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.876 INFO:teuthology.orchestra.run.smithi105.stdout: "req_snapdiff_latency": { 2024-09-16T02:05:55.876 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.876 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.876 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.876 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.876 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmdir_latency": { 2024-09-16T02:05:55.876 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.876 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.876 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.877 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.877 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmsnap_latency": { 2024-09-16T02:05:55.877 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.877 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.877 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.877 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.877 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmxattr_latency": { 2024-09-16T02:05:55.877 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.877 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.877 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.877 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.877 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setattr_latency": { 2024-09-16T02:05:55.877 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.877 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.878 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.878 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.878 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setdirlayout_latency": { 2024-09-16T02:05:55.878 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.878 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.878 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.878 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.878 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setfilelock_latency": { 2024-09-16T02:05:55.878 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.878 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.878 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.878 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.878 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setlayout_latency": { 2024-09-16T02:05:55.878 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.878 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.879 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.879 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.879 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setxattr_latency": { 2024-09-16T02:05:55.879 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.879 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.879 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.879 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.879 INFO:teuthology.orchestra.run.smithi105.stdout: "req_symlink_latency": { 2024-09-16T02:05:55.879 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.879 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.879 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.879 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.879 INFO:teuthology.orchestra.run.smithi105.stdout: "req_unlink_latency": { 2024-09-16T02:05:55.879 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.880 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.880 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.880 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.880 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_revoke_eviction": 0, 2024-09-16T02:05:55.880 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_acquisition_throttle": 0, 2024-09-16T02:05:55.880 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getvxattr_latency": { 2024-09-16T02:05:55.880 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.880 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.880 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.880 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:55.880 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.880 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_sessions": { 2024-09-16T02:05:55.880 INFO:teuthology.orchestra.run.smithi105.stdout: "session_count": 1, 2024-09-16T02:05:55.880 INFO:teuthology.orchestra.run.smithi105.stdout: "session_add": 1, 2024-09-16T02:05:55.881 INFO:teuthology.orchestra.run.smithi105.stdout: "session_remove": 0, 2024-09-16T02:05:55.881 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_open": 1, 2024-09-16T02:05:55.881 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_stale": 0, 2024-09-16T02:05:55.881 INFO:teuthology.orchestra.run.smithi105.stdout: "total_load": 0, 2024-09-16T02:05:55.881 INFO:teuthology.orchestra.run.smithi105.stdout: "average_load": 0, 2024-09-16T02:05:55.881 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_session_uptime": 1216, 2024-09-16T02:05:55.881 INFO:teuthology.orchestra.run.smithi105.stdout: "mdthresh_evicted": 0 2024-09-16T02:05:55.881 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.881 INFO:teuthology.orchestra.run.smithi105.stdout: "mempool": { 2024-09-16T02:05:55.881 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_bytes": 0, 2024-09-16T02:05:55.881 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_items": 0, 2024-09-16T02:05:55.881 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T02:05:55.881 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_items": 0, 2024-09-16T02:05:55.881 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T02:05:55.882 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_items": 0, 2024-09-16T02:05:55.882 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T02:05:55.882 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T02:05:55.882 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T02:05:55.882 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T02:05:55.882 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T02:05:55.882 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_items": 0, 2024-09-16T02:05:55.882 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T02:05:55.882 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T02:05:55.882 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_bytes": 0, 2024-09-16T02:05:55.882 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_items": 0, 2024-09-16T02:05:55.882 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_bytes": 0, 2024-09-16T02:05:55.882 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_items": 0, 2024-09-16T02:05:55.882 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T02:05:55.883 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T02:05:55.883 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T02:05:55.883 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T02:05:55.883 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T02:05:55.883 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_items": 0, 2024-09-16T02:05:55.883 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_bytes": 0, 2024-09-16T02:05:55.883 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_items": 0, 2024-09-16T02:05:55.883 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T02:05:55.883 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T02:05:55.883 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_bytes": 0, 2024-09-16T02:05:55.883 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_items": 0, 2024-09-16T02:05:55.883 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_bytes": 0, 2024-09-16T02:05:55.883 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_items": 0, 2024-09-16T02:05:55.883 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T02:05:55.884 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_items": 0, 2024-09-16T02:05:55.884 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T02:05:55.884 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_items": 0, 2024-09-16T02:05:55.884 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_bytes": 189239, 2024-09-16T02:05:55.884 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_items": 48, 2024-09-16T02:05:55.884 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_bytes": 0, 2024-09-16T02:05:55.884 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_items": 0, 2024-09-16T02:05:55.884 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_bytes": 0, 2024-09-16T02:05:55.884 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_items": 0, 2024-09-16T02:05:55.884 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_bytes": 0, 2024-09-16T02:05:55.884 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_items": 0, 2024-09-16T02:05:55.884 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_bytes": 0, 2024-09-16T02:05:55.884 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_items": 0, 2024-09-16T02:05:55.884 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_bytes": 7888, 2024-09-16T02:05:55.884 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_items": 163, 2024-09-16T02:05:55.885 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T02:05:55.885 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_items": 0, 2024-09-16T02:05:55.885 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_bytes": 0, 2024-09-16T02:05:55.885 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_items": 0, 2024-09-16T02:05:55.885 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_bytes": 8512, 2024-09-16T02:05:55.885 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_items": 13, 2024-09-16T02:05:55.885 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_bytes": 0, 2024-09-16T02:05:55.885 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_items": 0, 2024-09-16T02:05:55.885 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_bytes": 0, 2024-09-16T02:05:55.885 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_items": 0 2024-09-16T02:05:55.885 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.885 INFO:teuthology.orchestra.run.smithi105.stdout: "objecter": { 2024-09-16T02:05:55.885 INFO:teuthology.orchestra.run.smithi105.stdout: "op_active": 0, 2024-09-16T02:05:55.885 INFO:teuthology.orchestra.run.smithi105.stdout: "op_laggy": 0, 2024-09-16T02:05:55.886 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send": 816, 2024-09-16T02:05:55.886 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send_bytes": 21, 2024-09-16T02:05:55.886 INFO:teuthology.orchestra.run.smithi105.stdout: "op_resend": 3, 2024-09-16T02:05:55.886 INFO:teuthology.orchestra.run.smithi105.stdout: "op_reply": 816, 2024-09-16T02:05:55.886 INFO:teuthology.orchestra.run.smithi105.stdout: "op_latency": { 2024-09-16T02:05:55.886 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 816, 2024-09-16T02:05:55.886 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 1.597968158, 2024-09-16T02:05:55.886 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.001958294 2024-09-16T02:05:55.886 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.886 INFO:teuthology.orchestra.run.smithi105.stdout: "op_inflight": 0, 2024-09-16T02:05:55.886 INFO:teuthology.orchestra.run.smithi105.stdout: "oplen_avg": { 2024-09-16T02:05:55.886 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 816, 2024-09-16T02:05:55.886 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 817 2024-09-16T02:05:55.886 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.887 INFO:teuthology.orchestra.run.smithi105.stdout: "op": 816, 2024-09-16T02:05:55.887 INFO:teuthology.orchestra.run.smithi105.stdout: "op_r": 816, 2024-09-16T02:05:55.887 INFO:teuthology.orchestra.run.smithi105.stdout: "op_w": 0, 2024-09-16T02:05:55.887 INFO:teuthology.orchestra.run.smithi105.stdout: "op_rmw": 0, 2024-09-16T02:05:55.887 INFO:teuthology.orchestra.run.smithi105.stdout: "op_pg": 0, 2024-09-16T02:05:55.887 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_stat": 502, 2024-09-16T02:05:55.887 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_create": 0, 2024-09-16T02:05:55.887 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_read": 312, 2024-09-16T02:05:55.887 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_write": 0, 2024-09-16T02:05:55.887 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writefull": 0, 2024-09-16T02:05:55.887 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writesame": 0, 2024-09-16T02:05:55.887 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_append": 0, 2024-09-16T02:05:55.887 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_zero": 0, 2024-09-16T02:05:55.887 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_truncate": 0, 2024-09-16T02:05:55.887 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_delete": 0, 2024-09-16T02:05:55.888 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_mapext": 0, 2024-09-16T02:05:55.888 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_sparse_read": 0, 2024-09-16T02:05:55.888 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_clonerange": 0, 2024-09-16T02:05:55.888 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_getxattr": 1, 2024-09-16T02:05:55.888 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_setxattr": 0, 2024-09-16T02:05:55.888 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_cmpxattr": 0, 2024-09-16T02:05:55.888 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_rmxattr": 0, 2024-09-16T02:05:55.888 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_resetxattrs": 0, 2024-09-16T02:05:55.888 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_call": 0, 2024-09-16T02:05:55.888 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_watch": 0, 2024-09-16T02:05:55.888 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_notify": 0, 2024-09-16T02:05:55.888 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T02:05:55.888 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls": 0, 2024-09-16T02:05:55.888 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls_filter": 0, 2024-09-16T02:05:55.889 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_other": 0, 2024-09-16T02:05:55.889 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_active": 0, 2024-09-16T02:05:55.889 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_send": 0, 2024-09-16T02:05:55.889 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_resend": 0, 2024-09-16T02:05:55.889 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_ping": 0, 2024-09-16T02:05:55.889 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_active": 0, 2024-09-16T02:05:55.889 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_send": 0, 2024-09-16T02:05:55.889 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_resend": 0, 2024-09-16T02:05:55.889 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_active": 0, 2024-09-16T02:05:55.889 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_send": 0, 2024-09-16T02:05:55.889 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_resend": 0, 2024-09-16T02:05:55.889 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_active": 0, 2024-09-16T02:05:55.889 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_send": 0, 2024-09-16T02:05:55.889 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_resend": 0, 2024-09-16T02:05:55.889 INFO:teuthology.orchestra.run.smithi105.stdout: "command_active": 0, 2024-09-16T02:05:55.890 INFO:teuthology.orchestra.run.smithi105.stdout: "command_send": 0, 2024-09-16T02:05:55.890 INFO:teuthology.orchestra.run.smithi105.stdout: "command_resend": 0, 2024-09-16T02:05:55.890 INFO:teuthology.orchestra.run.smithi105.stdout: "map_epoch": 83, 2024-09-16T02:05:55.890 INFO:teuthology.orchestra.run.smithi105.stdout: "map_full": 0, 2024-09-16T02:05:55.890 INFO:teuthology.orchestra.run.smithi105.stdout: "map_inc": 4, 2024-09-16T02:05:55.890 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_sessions": 5, 2024-09-16T02:05:55.890 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_open": 5, 2024-09-16T02:05:55.890 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_close": 0, 2024-09-16T02:05:55.890 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_laggy": 0, 2024-09-16T02:05:55.890 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_wr": 0, 2024-09-16T02:05:55.890 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_rd": 2, 2024-09-16T02:05:55.890 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_del": 0 2024-09-16T02:05:55.890 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.890 INFO:teuthology.orchestra.run.smithi105.stdout: "oft": { 2024-09-16T02:05:55.890 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_objs": 0, 2024-09-16T02:05:55.891 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_kv_pairs": 0, 2024-09-16T02:05:55.891 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_updates": 0, 2024-09-16T02:05:55.891 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_removes": 0 2024-09-16T02:05:55.891 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.891 INFO:teuthology.orchestra.run.smithi105.stdout: "osd-slow-ops": { 2024-09-16T02:05:55.891 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_ops_count": 0 2024-09-16T02:05:55.891 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.891 INFO:teuthology.orchestra.run.smithi105.stdout: "purge_queue": { 2024-09-16T02:05:55.891 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops": 0, 2024-09-16T02:05:55.891 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T02:05:55.891 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing": 0, 2024-09-16T02:05:55.891 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_high_water": 0, 2024-09-16T02:05:55.891 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed_ops": 0, 2024-09-16T02:05:55.891 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed": 0, 2024-09-16T02:05:55.892 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_item_in_journal": 0 2024-09-16T02:05:55.892 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.892 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T02:05:55.892 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-09-16T02:05:55.892 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-09-16T02:05:55.892 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-09-16T02:05:55.892 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 924, 2024-09-16T02:05:55.892 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 615534, 2024-09-16T02:05:55.892 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-09-16T02:05:55.892 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 924, 2024-09-16T02:05:55.892 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-09-16T02:05:55.892 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-09-16T02:05:55.892 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 924, 2024-09-16T02:05:55.892 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 615534, 2024-09-16T02:05:55.893 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-09-16T02:05:55.893 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.893 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.893 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.893 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:55.893 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.893 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_bytes": { 2024-09-16T02:05:55.893 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-09-16T02:05:55.893 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-09-16T02:05:55.893 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-09-16T02:05:55.893 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-09-16T02:05:55.893 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-09-16T02:05:55.893 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-09-16T02:05:55.893 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-09-16T02:05:55.893 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 816, 2024-09-16T02:05:55.894 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 1088, 2024-09-16T02:05:55.894 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 816, 2024-09-16T02:05:55.894 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 1088, 2024-09-16T02:05:55.894 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-09-16T02:05:55.894 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.894 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.894 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.894 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:55.894 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.894 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_ops": { 2024-09-16T02:05:55.894 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-09-16T02:05:55.894 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 1024, 2024-09-16T02:05:55.894 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-09-16T02:05:55.894 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-09-16T02:05:55.895 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-09-16T02:05:55.895 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-09-16T02:05:55.895 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-09-16T02:05:55.895 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 816, 2024-09-16T02:05:55.895 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 816, 2024-09-16T02:05:55.895 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 816, 2024-09-16T02:05:55.895 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 816, 2024-09-16T02:05:55.895 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-09-16T02:05:55.895 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.895 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.895 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.895 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:55.895 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.895 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle": { 2024-09-16T02:05:55.896 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-09-16T02:05:55.896 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-09-16T02:05:55.896 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-09-16T02:05:55.896 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-09-16T02:05:55.896 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-09-16T02:05:55.896 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-09-16T02:05:55.896 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-09-16T02:05:55.896 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-09-16T02:05:55.896 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-09-16T02:05:55.896 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 0, 2024-09-16T02:05:55.896 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 0, 2024-09-16T02:05:55.896 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-09-16T02:05:55.896 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.896 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.896 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.897 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:55.897 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:55.897 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle-0x55e48c2f2b40": { 2024-09-16T02:05:55.897 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-09-16T02:05:55.897 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-09-16T02:05:55.897 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-09-16T02:05:55.897 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-09-16T02:05:55.897 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-09-16T02:05:55.897 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-09-16T02:05:55.897 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-09-16T02:05:55.897 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-09-16T02:05:55.897 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-09-16T02:05:55.897 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 0, 2024-09-16T02:05:55.897 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 0, 2024-09-16T02:05:55.898 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-09-16T02:05:55.898 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:55.898 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:55.898 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:55.898 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:55.898 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:55.898 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-16T02:05:56.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:56 smithi105 ceph-mon[31197]: pgmap v441: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.1 KiB/s wr, 187 op/s 2024-09-16T02:05:56.856 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-16T02:05:56.857 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-16T02:05:56.857 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-09-16T02:05:56.857 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-09-16T02:05:56.857 DEBUG:tasks.check_counter:Getting stats from d 2024-09-16T02:05:56.857 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-16T02:05:56.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:56 smithi137 ceph-mon[34099]: pgmap v441: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.1 KiB/s wr, 187 op/s 2024-09-16T02:05:57.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:56 smithi190 ceph-mon[34762]: pgmap v441: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.1 KiB/s wr, 187 op/s 2024-09-16T02:05:57.143 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:57.649 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-16T02:05:57.649 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T02:05:57.649 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 0, 2024-09-16T02:05:57.649 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 2, 2024-09-16T02:05:57.649 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 0, 2024-09-16T02:05:57.649 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 0, 2024-09-16T02:05:57.649 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 4, 2024-09-16T02:05:57.649 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 0, 2024-09-16T02:05:57.649 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.001093857, 2024-09-16T02:05:57.649 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.000000000, 2024-09-16T02:05:57.650 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.000218482, 2024-09-16T02:05:57.650 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T02:05:57.650 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:05:57.650 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:57.650 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:57.650 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:57.650 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:57.650 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:05:57.651 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:57.651 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:57.651 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:57.651 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:57.651 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T02:05:57.651 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T02:05:57.651 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:57.651 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T02:05:57.651 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 5, 2024-09-16T02:05:57.651 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 76, 2024-09-16T02:05:57.651 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 3152, 2024-09-16T02:05:57.652 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 91968, 2024-09-16T02:05:57.652 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 3, 2024-09-16T02:05:57.652 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 2, 2024-09-16T02:05:57.652 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.008285289, 2024-09-16T02:05:57.652 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.004423318, 2024-09-16T02:05:57.652 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.003178223, 2024-09-16T02:05:57.652 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T02:05:57.652 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:05:57.652 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 75, 2024-09-16T02:05:57.653 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.007329483, 2024-09-16T02:05:57.653 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000097726 2024-09-16T02:05:57.653 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:57.653 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:05:57.653 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:57.653 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:57.653 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:57.653 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:57.653 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-16T02:05:57.653 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 91968 2024-09-16T02:05:57.654 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:57.654 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T02:05:57.654 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 101, 2024-09-16T02:05:57.654 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 98, 2024-09-16T02:05:57.654 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 445632, 2024-09-16T02:05:57.654 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 59232, 2024-09-16T02:05:57.654 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 2, 2024-09-16T02:05:57.654 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 1, 2024-09-16T02:05:57.654 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.014706334, 2024-09-16T02:05:57.655 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.006193852, 2024-09-16T02:05:57.655 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.012020073, 2024-09-16T02:05:57.655 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T02:05:57.655 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:05:57.655 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 97, 2024-09-16T02:05:57.655 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.005580451, 2024-09-16T02:05:57.655 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000057530 2024-09-16T02:05:57.655 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:57.655 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:05:57.655 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:57.655 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:57.655 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:57.655 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:57.656 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 445632, 2024-09-16T02:05:57.656 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 59232 2024-09-16T02:05:57.656 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:57.656 INFO:teuthology.orchestra.run.smithi105.stdout: "cct": { 2024-09-16T02:05:57.656 INFO:teuthology.orchestra.run.smithi105.stdout: "total_workers": 0, 2024-09-16T02:05:57.656 INFO:teuthology.orchestra.run.smithi105.stdout: "unhealthy_workers": 0 2024-09-16T02:05:57.656 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:57.656 INFO:teuthology.orchestra.run.smithi105.stdout: "mempool": { 2024-09-16T02:05:57.656 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_bytes": 0, 2024-09-16T02:05:57.657 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_items": 0, 2024-09-16T02:05:57.657 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T02:05:57.657 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_items": 0, 2024-09-16T02:05:57.657 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T02:05:57.657 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_items": 0, 2024-09-16T02:05:57.657 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T02:05:57.657 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T02:05:57.657 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T02:05:57.657 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T02:05:57.657 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T02:05:57.658 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_items": 0, 2024-09-16T02:05:57.658 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T02:05:57.658 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T02:05:57.658 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_bytes": 0, 2024-09-16T02:05:57.658 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_items": 0, 2024-09-16T02:05:57.658 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_bytes": 0, 2024-09-16T02:05:57.658 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_items": 0, 2024-09-16T02:05:57.658 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T02:05:57.658 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T02:05:57.658 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T02:05:57.659 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T02:05:57.659 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T02:05:57.659 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_items": 0, 2024-09-16T02:05:57.659 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_bytes": 0, 2024-09-16T02:05:57.659 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_items": 0, 2024-09-16T02:05:57.659 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T02:05:57.659 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T02:05:57.659 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_bytes": 0, 2024-09-16T02:05:57.659 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_items": 0, 2024-09-16T02:05:57.659 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_bytes": 0, 2024-09-16T02:05:57.660 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_items": 0, 2024-09-16T02:05:57.660 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T02:05:57.660 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_items": 0, 2024-09-16T02:05:57.660 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T02:05:57.660 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_items": 0, 2024-09-16T02:05:57.660 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_bytes": 148840, 2024-09-16T02:05:57.660 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_items": 28, 2024-09-16T02:05:57.660 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_bytes": 0, 2024-09-16T02:05:57.660 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_items": 0, 2024-09-16T02:05:57.661 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_bytes": 0, 2024-09-16T02:05:57.661 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_items": 0, 2024-09-16T02:05:57.661 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_bytes": 0, 2024-09-16T02:05:57.661 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_items": 0, 2024-09-16T02:05:57.661 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_bytes": 0, 2024-09-16T02:05:57.661 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_items": 0, 2024-09-16T02:05:57.661 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_bytes": 0, 2024-09-16T02:05:57.661 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_items": 0, 2024-09-16T02:05:57.661 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T02:05:57.661 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_items": 0, 2024-09-16T02:05:57.661 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_bytes": 0, 2024-09-16T02:05:57.661 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_items": 0, 2024-09-16T02:05:57.661 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_bytes": 928, 2024-09-16T02:05:57.661 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_items": 2, 2024-09-16T02:05:57.661 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_bytes": 0, 2024-09-16T02:05:57.662 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_items": 0, 2024-09-16T02:05:57.662 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_bytes": 0, 2024-09-16T02:05:57.662 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_items": 0 2024-09-16T02:05:57.662 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:57.662 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T02:05:57.662 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-09-16T02:05:57.662 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-09-16T02:05:57.662 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-09-16T02:05:57.662 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 102, 2024-09-16T02:05:57.662 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 431380, 2024-09-16T02:05:57.662 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-09-16T02:05:57.662 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 102, 2024-09-16T02:05:57.663 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-09-16T02:05:57.663 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-09-16T02:05:57.663 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 102, 2024-09-16T02:05:57.663 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 431380, 2024-09-16T02:05:57.663 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-09-16T02:05:57.663 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:57.663 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:57.663 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:57.663 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:57.663 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:57.663 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-16T02:05:58.494 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-16T02:05:58.494 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-16T02:05:58.494 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-16T02:05:58.494 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-16T02:05:58.495 DEBUG:tasks.check_counter:Getting stats from g 2024-09-16T02:05:58.495 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-16T02:05:58.743 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:58.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:05:58 smithi105 ceph-mon[31197]: pgmap v442: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-16T02:05:58.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:05:58 smithi137 ceph-mon[34099]: pgmap v442: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-16T02:05:59.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:05:58 smithi190 ceph-mon[34762]: pgmap v442: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-16T02:05:59.735 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-16T02:05:59.735 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T02:05:59.735 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 317, 2024-09-16T02:05:59.735 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 317, 2024-09-16T02:05:59.735 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 108105, 2024-09-16T02:05:59.735 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 99983, 2024-09-16T02:05:59.735 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 6, 2024-09-16T02:05:59.735 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 2, 2024-09-16T02:05:59.735 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.035430669, 2024-09-16T02:05:59.735 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.013896739, 2024-09-16T02:05:59.736 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.009362680, 2024-09-16T02:05:59.736 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.008289516, 2024-09-16T02:05:59.736 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:05:59.736 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 316, 2024-09-16T02:05:59.736 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.018620512, 2024-09-16T02:05:59.736 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000058925 2024-09-16T02:05:59.736 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.736 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:05:59.736 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.736 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.736 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.736 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.736 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-16T02:05:59.737 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 448 2024-09-16T02:05:59.737 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.737 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T02:05:59.737 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 623, 2024-09-16T02:05:59.737 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 619, 2024-09-16T02:05:59.737 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 601869, 2024-09-16T02:05:59.737 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 222675, 2024-09-16T02:05:59.737 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 4, 2024-09-16T02:05:59.737 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 3, 2024-09-16T02:05:59.737 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.047527426, 2024-09-16T02:05:59.737 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.015649825, 2024-09-16T02:05:59.737 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.020953393, 2024-09-16T02:05:59.737 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.011949483, 2024-09-16T02:05:59.738 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:05:59.738 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 618, 2024-09-16T02:05:59.738 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.020515450, 2024-09-16T02:05:59.738 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000033196 2024-09-16T02:05:59.738 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.738 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:05:59.738 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.738 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.738 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.738 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.739 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 469040, 2024-09-16T02:05:59.739 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 58848 2024-09-16T02:05:59.739 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.739 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T02:05:59.739 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 4, 2024-09-16T02:05:59.739 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 75, 2024-09-16T02:05:59.739 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 1508, 2024-09-16T02:05:59.739 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 203221, 2024-09-16T02:05:59.739 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 5, 2024-09-16T02:05:59.739 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 4, 2024-09-16T02:05:59.740 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.006368127, 2024-09-16T02:05:59.740 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.003643274, 2024-09-16T02:05:59.740 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.001987828, 2024-09-16T02:05:59.740 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000062925, 2024-09-16T02:05:59.740 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:05:59.740 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 73, 2024-09-16T02:05:59.740 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.004639075, 2024-09-16T02:05:59.740 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000063548 2024-09-16T02:05:59.740 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.740 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:05:59.740 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.740 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.741 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.741 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.741 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-16T02:05:59.741 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 202256 2024-09-16T02:05:59.741 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.741 INFO:teuthology.orchestra.run.smithi105.stdout: "cct": { 2024-09-16T02:05:59.741 INFO:teuthology.orchestra.run.smithi105.stdout: "total_workers": 1, 2024-09-16T02:05:59.741 INFO:teuthology.orchestra.run.smithi105.stdout: "unhealthy_workers": 0 2024-09-16T02:05:59.741 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.741 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-MDSRank": { 2024-09-16T02:05:59.741 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-09-16T02:05:59.741 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-09-16T02:05:59.741 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1445, 2024-09-16T02:05:59.741 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.033388117, 2024-09-16T02:05:59.742 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000023105 2024-09-16T02:05:59.742 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:59.742 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.742 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-PurgeQueue": { 2024-09-16T02:05:59.742 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-09-16T02:05:59.742 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-09-16T02:05:59.742 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.742 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.742 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.742 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:59.742 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.742 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": { 2024-09-16T02:05:59.742 INFO:teuthology.orchestra.run.smithi105.stdout: "request": 0, 2024-09-16T02:05:59.743 INFO:teuthology.orchestra.run.smithi105.stdout: "reply": 0, 2024-09-16T02:05:59.743 INFO:teuthology.orchestra.run.smithi105.stdout: "reply_latency": { 2024-09-16T02:05:59.743 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.743 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.743 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.743 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.743 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_reply": 0, 2024-09-16T02:05:59.743 INFO:teuthology.orchestra.run.smithi105.stdout: "forward": 0, 2024-09-16T02:05:59.743 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_complete": 0, 2024-09-16T02:05:59.743 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_keys": 0, 2024-09-16T02:05:59.743 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_commit": 0, 2024-09-16T02:05:59.743 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_split": 0, 2024-09-16T02:05:59.744 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_merge": 0, 2024-09-16T02:05:59.744 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes": 0, 2024-09-16T02:05:59.744 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_top": 0, 2024-09-16T02:05:59.744 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_bottom": 0, 2024-09-16T02:05:59.744 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pin_tail": 0, 2024-09-16T02:05:59.744 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pinned": 0, 2024-09-16T02:05:59.744 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_expired": 0, 2024-09-16T02:05:59.744 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_with_caps": 0, 2024-09-16T02:05:59.744 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": 0, 2024-09-16T02:05:59.744 INFO:teuthology.orchestra.run.smithi105.stdout: "subtrees": 1, 2024-09-16T02:05:59.744 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse": 0, 2024-09-16T02:05:59.744 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_hit": 0, 2024-09-16T02:05:59.745 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_forward": 0, 2024-09-16T02:05:59.745 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_discover": 0, 2024-09-16T02:05:59.745 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_dir_fetch": 0, 2024-09-16T02:05:59.745 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_remote_ino": 0, 2024-09-16T02:05:59.745 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_lock": 0, 2024-09-16T02:05:59.745 INFO:teuthology.orchestra.run.smithi105.stdout: "load_cent": 0, 2024-09-16T02:05:59.745 INFO:teuthology.orchestra.run.smithi105.stdout: "q": 0, 2024-09-16T02:05:59.745 INFO:teuthology.orchestra.run.smithi105.stdout: "exported": 0, 2024-09-16T02:05:59.746 INFO:teuthology.orchestra.run.smithi105.stdout: "exported_inodes": 0, 2024-09-16T02:05:59.746 INFO:teuthology.orchestra.run.smithi105.stdout: "imported": 0, 2024-09-16T02:05:59.746 INFO:teuthology.orchestra.run.smithi105.stdout: "imported_inodes": 0, 2024-09-16T02:05:59.746 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_dir_fetch": 0, 2024-09-16T02:05:59.746 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_backtrace_fetch": 0, 2024-09-16T02:05:59.746 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_peer_discover": 0, 2024-09-16T02:05:59.746 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rfiles": 0, 2024-09-16T02:05:59.746 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rbytes": 0, 2024-09-16T02:05:59.746 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rsnaps": 0, 2024-09-16T02:05:59.746 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T02:05:59.747 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_set_tag": 0, 2024-09-16T02:05:59.747 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T02:05:59.747 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_inotable_repaired": 0, 2024-09-16T02:05:59.747 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_inodes": 0, 2024-09-16T02:05:59.747 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T02:05:59.747 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T02:05:59.747 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_file_inodes": 0, 2024-09-16T02:05:59.747 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_inode_file_caps": 0, 2024-09-16T02:05:59.747 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T02:05:59.747 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_grant": 0, 2024-09-16T02:05:59.747 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T02:05:59.747 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T02:05:59.747 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T02:05:59.748 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps": 0, 2024-09-16T02:05:59.748 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps_dirty": 0, 2024-09-16T02:05:59.748 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_cap_release": 0, 2024-09-16T02:05:59.748 INFO:teuthology.orchestra.run.smithi105.stdout: "process_request_cap_release": 0 2024-09-16T02:05:59.748 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.748 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_cache": { 2024-09-16T02:05:59.748 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update": 0, 2024-09-16T02:05:59.748 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update_receipt": 0, 2024-09-16T02:05:59.748 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_try_discover": 0, 2024-09-16T02:05:59.748 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_send_discover": 0, 2024-09-16T02:05:59.748 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_handle_discover": 0, 2024-09-16T02:05:59.748 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays": 0, 2024-09-16T02:05:59.748 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_delayed": 0, 2024-09-16T02:05:59.749 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_enqueuing": 0, 2024-09-16T02:05:59.749 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_created": 0, 2024-09-16T02:05:59.749 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_enqueued": 0, 2024-09-16T02:05:59.749 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_reintegrated": 0, 2024-09-16T02:05:59.749 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_migrated": 0, 2024-09-16T02:05:59.749 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_processing": 0, 2024-09-16T02:05:59.749 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_enqueued": 0, 2024-09-16T02:05:59.749 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_prioritized": 0, 2024-09-16T02:05:59.749 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_started": 0, 2024-09-16T02:05:59.749 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_completed": 0, 2024-09-16T02:05:59.749 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_quiesce_path": 0, 2024-09-16T02:05:59.750 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_quiesce_inode": 0, 2024-09-16T02:05:59.750 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T02:05:59.750 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_exportdir": 0, 2024-09-16T02:05:59.750 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_flush": 0, 2024-09-16T02:05:59.750 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragmentdir": 0, 2024-09-16T02:05:59.750 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragstats": 0, 2024-09-16T02:05:59.750 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_inodestats": 0 2024-09-16T02:05:59.750 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.750 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_log": { 2024-09-16T02:05:59.750 INFO:teuthology.orchestra.run.smithi105.stdout: "evlrg": 0, 2024-09-16T02:05:59.750 INFO:teuthology.orchestra.run.smithi105.stdout: "evadd": 0, 2024-09-16T02:05:59.750 INFO:teuthology.orchestra.run.smithi105.stdout: "evex": 0, 2024-09-16T02:05:59.750 INFO:teuthology.orchestra.run.smithi105.stdout: "evtrm": 0, 2024-09-16T02:05:59.750 INFO:teuthology.orchestra.run.smithi105.stdout: "ev": 3, 2024-09-16T02:05:59.751 INFO:teuthology.orchestra.run.smithi105.stdout: "evexg": 0, 2024-09-16T02:05:59.751 INFO:teuthology.orchestra.run.smithi105.stdout: "evexd": 0, 2024-09-16T02:05:59.751 INFO:teuthology.orchestra.run.smithi105.stdout: "segadd": 0, 2024-09-16T02:05:59.751 INFO:teuthology.orchestra.run.smithi105.stdout: "segex": 0, 2024-09-16T02:05:59.751 INFO:teuthology.orchestra.run.smithi105.stdout: "segtrm": 0, 2024-09-16T02:05:59.751 INFO:teuthology.orchestra.run.smithi105.stdout: "seg": 2, 2024-09-16T02:05:59.751 INFO:teuthology.orchestra.run.smithi105.stdout: "segmjr": 2, 2024-09-16T02:05:59.751 INFO:teuthology.orchestra.run.smithi105.stdout: "segexg": 0, 2024-09-16T02:05:59.751 INFO:teuthology.orchestra.run.smithi105.stdout: "segexd": 0, 2024-09-16T02:05:59.751 INFO:teuthology.orchestra.run.smithi105.stdout: "expos": 4194304, 2024-09-16T02:05:59.751 INFO:teuthology.orchestra.run.smithi105.stdout: "wrpos": 4195387, 2024-09-16T02:05:59.751 INFO:teuthology.orchestra.run.smithi105.stdout: "rdpos": 4194892, 2024-09-16T02:05:59.751 INFO:teuthology.orchestra.run.smithi105.stdout: "jlat": { 2024-09-16T02:05:59.751 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.752 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.752 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.752 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.752 INFO:teuthology.orchestra.run.smithi105.stdout: "replayed": 3 2024-09-16T02:05:59.752 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.752 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_mem": { 2024-09-16T02:05:59.752 INFO:teuthology.orchestra.run.smithi105.stdout: "ino": 3, 2024-09-16T02:05:59.752 INFO:teuthology.orchestra.run.smithi105.stdout: "ino+": 3, 2024-09-16T02:05:59.752 INFO:teuthology.orchestra.run.smithi105.stdout: "ino-": 0, 2024-09-16T02:05:59.752 INFO:teuthology.orchestra.run.smithi105.stdout: "dir": 1, 2024-09-16T02:05:59.752 INFO:teuthology.orchestra.run.smithi105.stdout: "dir+": 1, 2024-09-16T02:05:59.752 INFO:teuthology.orchestra.run.smithi105.stdout: "dir-": 0, 2024-09-16T02:05:59.753 INFO:teuthology.orchestra.run.smithi105.stdout: "dn": 0, 2024-09-16T02:05:59.753 INFO:teuthology.orchestra.run.smithi105.stdout: "dn+": 0, 2024-09-16T02:05:59.753 INFO:teuthology.orchestra.run.smithi105.stdout: "dn-": 0, 2024-09-16T02:05:59.753 INFO:teuthology.orchestra.run.smithi105.stdout: "cap": 0, 2024-09-16T02:05:59.753 INFO:teuthology.orchestra.run.smithi105.stdout: "cap+": 0, 2024-09-16T02:05:59.753 INFO:teuthology.orchestra.run.smithi105.stdout: "cap-": 0, 2024-09-16T02:05:59.753 INFO:teuthology.orchestra.run.smithi105.stdout: "rss": 47544, 2024-09-16T02:05:59.753 INFO:teuthology.orchestra.run.smithi105.stdout: "heap": 215328 2024-09-16T02:05:59.753 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.753 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_server": { 2024-09-16T02:05:59.753 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_client_request": 0, 2024-09-16T02:05:59.753 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_server_request": 0, 2024-09-16T02:05:59.753 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_request": 0, 2024-09-16T02:05:59.753 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_session": 0, 2024-09-16T02:05:59.754 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_peer_request": 0, 2024-09-16T02:05:59.754 INFO:teuthology.orchestra.run.smithi105.stdout: "req_create_latency": { 2024-09-16T02:05:59.754 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.754 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.754 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.754 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.754 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getattr_latency": { 2024-09-16T02:05:59.754 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.754 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.754 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.754 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.754 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getfilelock_latency": { 2024-09-16T02:05:59.754 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.755 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.755 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.755 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.755 INFO:teuthology.orchestra.run.smithi105.stdout: "req_link_latency": { 2024-09-16T02:05:59.755 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.755 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.755 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.755 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.755 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookup_latency": { 2024-09-16T02:05:59.755 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.755 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.756 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.756 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.756 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookuphash_latency": { 2024-09-16T02:05:59.756 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.756 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.756 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.756 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.756 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupino_latency": { 2024-09-16T02:05:59.756 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.756 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.756 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.756 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.756 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupname_latency": { 2024-09-16T02:05:59.757 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.757 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.757 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.757 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.757 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupparent_latency": { 2024-09-16T02:05:59.757 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.757 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.757 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.757 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.757 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupsnap_latency": { 2024-09-16T02:05:59.757 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.757 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.757 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lssnap_latency": { 2024-09-16T02:05:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mkdir_latency": { 2024-09-16T02:05:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mknod_latency": { 2024-09-16T02:05:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mksnap_latency": { 2024-09-16T02:05:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: "req_open_latency": { 2024-09-16T02:05:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: "req_readdir_latency": { 2024-09-16T02:05:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rename_latency": { 2024-09-16T02:05:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: "req_renamesnap_latency": { 2024-09-16T02:05:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.761 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.761 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.761 INFO:teuthology.orchestra.run.smithi105.stdout: "req_snapdiff_latency": { 2024-09-16T02:05:59.761 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.761 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.761 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.761 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.761 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmdir_latency": { 2024-09-16T02:05:59.761 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.761 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.761 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmsnap_latency": { 2024-09-16T02:05:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmxattr_latency": { 2024-09-16T02:05:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setattr_latency": { 2024-09-16T02:05:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setdirlayout_latency": { 2024-09-16T02:05:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setfilelock_latency": { 2024-09-16T02:05:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setlayout_latency": { 2024-09-16T02:05:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setxattr_latency": { 2024-09-16T02:05:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.765 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.765 INFO:teuthology.orchestra.run.smithi105.stdout: "req_symlink_latency": { 2024-09-16T02:05:59.765 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.765 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.765 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.765 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.765 INFO:teuthology.orchestra.run.smithi105.stdout: "req_unlink_latency": { 2024-09-16T02:05:59.765 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.765 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.766 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.766 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.766 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_revoke_eviction": 0, 2024-09-16T02:05:59.766 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_acquisition_throttle": 0, 2024-09-16T02:05:59.766 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getvxattr_latency": { 2024-09-16T02:05:59.766 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.766 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.766 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.766 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:59.766 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.766 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_sessions": { 2024-09-16T02:05:59.766 INFO:teuthology.orchestra.run.smithi105.stdout: "session_count": 1, 2024-09-16T02:05:59.767 INFO:teuthology.orchestra.run.smithi105.stdout: "session_add": 1, 2024-09-16T02:05:59.767 INFO:teuthology.orchestra.run.smithi105.stdout: "session_remove": 0, 2024-09-16T02:05:59.767 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_open": 1, 2024-09-16T02:05:59.767 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_stale": 0, 2024-09-16T02:05:59.767 INFO:teuthology.orchestra.run.smithi105.stdout: "total_load": 0, 2024-09-16T02:05:59.767 INFO:teuthology.orchestra.run.smithi105.stdout: "average_load": 0, 2024-09-16T02:05:59.767 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_session_uptime": 1223, 2024-09-16T02:05:59.767 INFO:teuthology.orchestra.run.smithi105.stdout: "mdthresh_evicted": 0 2024-09-16T02:05:59.767 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.767 INFO:teuthology.orchestra.run.smithi105.stdout: "mempool": { 2024-09-16T02:05:59.767 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_bytes": 0, 2024-09-16T02:05:59.767 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_items": 0, 2024-09-16T02:05:59.767 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T02:05:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_items": 0, 2024-09-16T02:05:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T02:05:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_items": 0, 2024-09-16T02:05:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T02:05:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T02:05:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T02:05:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T02:05:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T02:05:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_items": 0, 2024-09-16T02:05:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T02:05:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T02:05:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_bytes": 0, 2024-09-16T02:05:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_items": 0, 2024-09-16T02:05:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_bytes": 0, 2024-09-16T02:05:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_items": 0, 2024-09-16T02:05:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T02:05:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T02:05:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T02:05:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T02:05:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T02:05:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_items": 0, 2024-09-16T02:05:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_bytes": 0, 2024-09-16T02:05:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_items": 0, 2024-09-16T02:05:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T02:05:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T02:05:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_bytes": 0, 2024-09-16T02:05:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_items": 0, 2024-09-16T02:05:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_bytes": 0, 2024-09-16T02:05:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_items": 0, 2024-09-16T02:05:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T02:05:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_items": 0, 2024-09-16T02:05:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T02:05:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_items": 0, 2024-09-16T02:05:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_bytes": 197336, 2024-09-16T02:05:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_items": 53, 2024-09-16T02:05:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_bytes": 0, 2024-09-16T02:05:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_items": 0, 2024-09-16T02:05:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_bytes": 0, 2024-09-16T02:05:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_items": 0, 2024-09-16T02:05:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_bytes": 0, 2024-09-16T02:05:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_items": 0, 2024-09-16T02:05:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_bytes": 0, 2024-09-16T02:05:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_items": 0, 2024-09-16T02:05:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_bytes": 7888, 2024-09-16T02:05:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_items": 163, 2024-09-16T02:05:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T02:05:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_items": 0, 2024-09-16T02:05:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_bytes": 0, 2024-09-16T02:05:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_items": 0, 2024-09-16T02:05:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_bytes": 8512, 2024-09-16T02:05:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_items": 13, 2024-09-16T02:05:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_bytes": 0, 2024-09-16T02:05:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_items": 0, 2024-09-16T02:05:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_bytes": 0, 2024-09-16T02:05:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_items": 0 2024-09-16T02:05:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: "objecter": { 2024-09-16T02:05:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: "op_active": 0, 2024-09-16T02:05:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: "op_laggy": 0, 2024-09-16T02:05:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send": 831, 2024-09-16T02:05:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send_bytes": 21, 2024-09-16T02:05:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: "op_resend": 3, 2024-09-16T02:05:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: "op_reply": 831, 2024-09-16T02:05:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: "op_latency": { 2024-09-16T02:05:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 831, 2024-09-16T02:05:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.435991308, 2024-09-16T02:05:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000524658 2024-09-16T02:05:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: "op_inflight": 0, 2024-09-16T02:05:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: "oplen_avg": { 2024-09-16T02:05:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 831, 2024-09-16T02:05:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 832 2024-09-16T02:05:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: "op": 831, 2024-09-16T02:05:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: "op_r": 831, 2024-09-16T02:05:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: "op_w": 0, 2024-09-16T02:05:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: "op_rmw": 0, 2024-09-16T02:05:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: "op_pg": 0, 2024-09-16T02:05:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_stat": 512, 2024-09-16T02:05:59.774 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_create": 0, 2024-09-16T02:05:59.774 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_read": 317, 2024-09-16T02:05:59.774 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_write": 0, 2024-09-16T02:05:59.774 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writefull": 0, 2024-09-16T02:05:59.774 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writesame": 0, 2024-09-16T02:05:59.774 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_append": 0, 2024-09-16T02:05:59.774 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_zero": 0, 2024-09-16T02:05:59.775 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_truncate": 0, 2024-09-16T02:05:59.775 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_delete": 0, 2024-09-16T02:05:59.775 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_mapext": 0, 2024-09-16T02:05:59.775 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_sparse_read": 0, 2024-09-16T02:05:59.775 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_clonerange": 0, 2024-09-16T02:05:59.775 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_getxattr": 1, 2024-09-16T02:05:59.775 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_setxattr": 0, 2024-09-16T02:05:59.775 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_cmpxattr": 0, 2024-09-16T02:05:59.775 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_rmxattr": 0, 2024-09-16T02:05:59.775 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_resetxattrs": 0, 2024-09-16T02:05:59.775 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_call": 0, 2024-09-16T02:05:59.775 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_watch": 0, 2024-09-16T02:05:59.776 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_notify": 0, 2024-09-16T02:05:59.776 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T02:05:59.776 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls": 0, 2024-09-16T02:05:59.776 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls_filter": 0, 2024-09-16T02:05:59.776 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_other": 0, 2024-09-16T02:05:59.776 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_active": 0, 2024-09-16T02:05:59.776 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_send": 0, 2024-09-16T02:05:59.776 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_resend": 0, 2024-09-16T02:05:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_ping": 0, 2024-09-16T02:05:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_active": 0, 2024-09-16T02:05:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_send": 0, 2024-09-16T02:05:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_resend": 0, 2024-09-16T02:05:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_active": 0, 2024-09-16T02:05:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_send": 0, 2024-09-16T02:05:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_resend": 0, 2024-09-16T02:05:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_active": 0, 2024-09-16T02:05:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_send": 0, 2024-09-16T02:05:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_resend": 0, 2024-09-16T02:05:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: "command_active": 0, 2024-09-16T02:05:59.778 INFO:teuthology.orchestra.run.smithi105.stdout: "command_send": 0, 2024-09-16T02:05:59.778 INFO:teuthology.orchestra.run.smithi105.stdout: "command_resend": 0, 2024-09-16T02:05:59.778 INFO:teuthology.orchestra.run.smithi105.stdout: "map_epoch": 83, 2024-09-16T02:05:59.778 INFO:teuthology.orchestra.run.smithi105.stdout: "map_full": 0, 2024-09-16T02:05:59.778 INFO:teuthology.orchestra.run.smithi105.stdout: "map_inc": 4, 2024-09-16T02:05:59.778 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_sessions": 6, 2024-09-16T02:05:59.778 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_open": 6, 2024-09-16T02:05:59.778 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_close": 0, 2024-09-16T02:05:59.779 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_laggy": 0, 2024-09-16T02:05:59.779 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_wr": 0, 2024-09-16T02:05:59.779 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_rd": 2, 2024-09-16T02:05:59.779 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_del": 0 2024-09-16T02:05:59.779 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.779 INFO:teuthology.orchestra.run.smithi105.stdout: "oft": { 2024-09-16T02:05:59.780 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_objs": 0, 2024-09-16T02:05:59.780 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_kv_pairs": 0, 2024-09-16T02:05:59.780 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_updates": 0, 2024-09-16T02:05:59.780 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_removes": 0 2024-09-16T02:05:59.780 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.780 INFO:teuthology.orchestra.run.smithi105.stdout: "osd-slow-ops": { 2024-09-16T02:05:59.780 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_ops_count": 0 2024-09-16T02:05:59.780 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.781 INFO:teuthology.orchestra.run.smithi105.stdout: "purge_queue": { 2024-09-16T02:05:59.781 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops": 0, 2024-09-16T02:05:59.781 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T02:05:59.781 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing": 0, 2024-09-16T02:05:59.781 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_high_water": 0, 2024-09-16T02:05:59.781 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed_ops": 0, 2024-09-16T02:05:59.781 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed": 0, 2024-09-16T02:05:59.781 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_item_in_journal": 0 2024-09-16T02:05:59.781 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.781 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T02:05:59.781 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-09-16T02:05:59.782 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-09-16T02:05:59.782 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-09-16T02:05:59.782 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 940, 2024-09-16T02:05:59.782 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 618352, 2024-09-16T02:05:59.782 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-09-16T02:05:59.782 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 940, 2024-09-16T02:05:59.782 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-09-16T02:05:59.782 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-09-16T02:05:59.783 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 940, 2024-09-16T02:05:59.783 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 618352, 2024-09-16T02:05:59.783 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-09-16T02:05:59.783 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.783 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.783 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.783 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:59.783 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.784 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_bytes": { 2024-09-16T02:05:59.784 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-09-16T02:05:59.784 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-09-16T02:05:59.784 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-09-16T02:05:59.784 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-09-16T02:05:59.784 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-09-16T02:05:59.784 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-09-16T02:05:59.784 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-09-16T02:05:59.784 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 831, 2024-09-16T02:05:59.784 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 1088, 2024-09-16T02:05:59.784 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 831, 2024-09-16T02:05:59.785 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 1088, 2024-09-16T02:05:59.785 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-09-16T02:05:59.785 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.785 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.785 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.785 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:59.785 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.785 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_ops": { 2024-09-16T02:05:59.786 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-09-16T02:05:59.786 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 1024, 2024-09-16T02:05:59.786 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-09-16T02:05:59.786 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-09-16T02:05:59.786 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-09-16T02:05:59.787 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-09-16T02:05:59.787 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-09-16T02:05:59.787 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 831, 2024-09-16T02:05:59.787 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 831, 2024-09-16T02:05:59.787 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 831, 2024-09-16T02:05:59.787 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 831, 2024-09-16T02:05:59.788 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-09-16T02:05:59.788 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.788 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.789 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.789 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:59.789 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.789 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle": { 2024-09-16T02:05:59.789 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-09-16T02:05:59.789 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-09-16T02:05:59.790 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-09-16T02:05:59.790 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-09-16T02:05:59.790 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-09-16T02:05:59.791 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-09-16T02:05:59.791 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-09-16T02:05:59.791 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-09-16T02:05:59.791 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-09-16T02:05:59.792 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 0, 2024-09-16T02:05:59.792 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 0, 2024-09-16T02:05:59.792 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-09-16T02:05:59.792 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.792 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.792 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.793 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:59.793 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:05:59.793 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle-0x55bba784ab40": { 2024-09-16T02:05:59.793 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-09-16T02:05:59.794 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-09-16T02:05:59.794 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-09-16T02:05:59.794 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-09-16T02:05:59.794 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-09-16T02:05:59.794 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-09-16T02:05:59.794 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-09-16T02:05:59.795 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-09-16T02:05:59.795 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-09-16T02:05:59.795 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 0, 2024-09-16T02:05:59.795 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 0, 2024-09-16T02:05:59.795 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-09-16T02:05:59.796 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:05:59.796 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:05:59.796 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:05:59.796 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:59.796 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:05:59.796 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-16T02:06:00.685 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-16T02:06:00.686 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-16T02:06:00.686 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-09-16T02:06:00.686 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-09-16T02:06:00.686 DEBUG:tasks.check_counter:Getting stats from j 2024-09-16T02:06:00.686 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-16T02:06:00.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:00 smithi137 ceph-mon[34099]: pgmap v443: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-16T02:06:00.936 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:06:01.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:00 smithi105 ceph-mon[31197]: pgmap v443: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-16T02:06:01.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:00 smithi190 ceph-mon[34762]: pgmap v443: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-16T02:06:01.439 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-16T02:06:01.440 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T02:06:01.440 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 4, 2024-09-16T02:06:01.440 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 4, 2024-09-16T02:06:01.440 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 2992, 2024-09-16T02:06:01.440 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 448, 2024-09-16T02:06:01.440 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 4, 2024-09-16T02:06:01.440 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 0, 2024-09-16T02:06:01.440 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.003112682, 2024-09-16T02:06:01.440 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.000029706, 2024-09-16T02:06:01.440 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.002177162, 2024-09-16T02:06:01.440 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T02:06:01.440 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:06:01.440 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 3, 2024-09-16T02:06:01.441 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.003969852, 2024-09-16T02:06:01.441 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.001323284 2024-09-16T02:06:01.441 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:06:01.441 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:06:01.441 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:06:01.441 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:06:01.441 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:06:01.441 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:06:01.441 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-16T02:06:01.441 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 448 2024-09-16T02:06:01.441 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:06:01.441 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T02:06:01.441 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 1, 2024-09-16T02:06:01.442 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 76, 2024-09-16T02:06:01.442 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 160, 2024-09-16T02:06:01.442 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 91872, 2024-09-16T02:06:01.442 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 3, 2024-09-16T02:06:01.442 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 2, 2024-09-16T02:06:01.442 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.006195221, 2024-09-16T02:06:01.442 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.004372652, 2024-09-16T02:06:01.442 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.001052436, 2024-09-16T02:06:01.442 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T02:06:01.442 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:06:01.442 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 74, 2024-09-16T02:06:01.442 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.003751703, 2024-09-16T02:06:01.442 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000050698 2024-09-16T02:06:01.442 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:06:01.443 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:06:01.443 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:06:01.443 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:06:01.443 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:06:01.443 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:06:01.443 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-16T02:06:01.443 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 91872 2024-09-16T02:06:01.443 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:06:01.443 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T02:06:01.443 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 104, 2024-09-16T02:06:01.443 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 101, 2024-09-16T02:06:01.443 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 447072, 2024-09-16T02:06:01.443 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 60848, 2024-09-16T02:06:01.444 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 2, 2024-09-16T02:06:01.444 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 1, 2024-09-16T02:06:01.444 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.017640885, 2024-09-16T02:06:01.444 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.007016875, 2024-09-16T02:06:01.444 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.015128298, 2024-09-16T02:06:01.444 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T02:06:01.444 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:06:01.444 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 100, 2024-09-16T02:06:01.444 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.005445628, 2024-09-16T02:06:01.444 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000054456 2024-09-16T02:06:01.444 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:06:01.444 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:06:01.444 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:06:01.444 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:06:01.445 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:06:01.445 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:06:01.445 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 447072, 2024-09-16T02:06:01.445 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 60848 2024-09-16T02:06:01.445 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:06:01.445 INFO:teuthology.orchestra.run.smithi105.stdout: "cct": { 2024-09-16T02:06:01.445 INFO:teuthology.orchestra.run.smithi105.stdout: "total_workers": 0, 2024-09-16T02:06:01.445 INFO:teuthology.orchestra.run.smithi105.stdout: "unhealthy_workers": 0 2024-09-16T02:06:01.445 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:06:01.445 INFO:teuthology.orchestra.run.smithi105.stdout: "mempool": { 2024-09-16T02:06:01.445 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_bytes": 0, 2024-09-16T02:06:01.445 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_items": 0, 2024-09-16T02:06:01.445 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T02:06:01.445 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_items": 0, 2024-09-16T02:06:01.446 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T02:06:01.446 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_items": 0, 2024-09-16T02:06:01.446 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T02:06:01.446 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T02:06:01.446 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T02:06:01.446 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T02:06:01.446 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T02:06:01.446 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_items": 0, 2024-09-16T02:06:01.446 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T02:06:01.447 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T02:06:01.447 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_bytes": 0, 2024-09-16T02:06:01.447 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_items": 0, 2024-09-16T02:06:01.447 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_bytes": 0, 2024-09-16T02:06:01.447 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_items": 0, 2024-09-16T02:06:01.447 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T02:06:01.447 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T02:06:01.447 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T02:06:01.447 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T02:06:01.447 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T02:06:01.447 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_items": 0, 2024-09-16T02:06:01.447 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_bytes": 0, 2024-09-16T02:06:01.447 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_items": 0, 2024-09-16T02:06:01.448 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T02:06:01.448 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T02:06:01.448 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_bytes": 0, 2024-09-16T02:06:01.448 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_items": 0, 2024-09-16T02:06:01.448 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_bytes": 0, 2024-09-16T02:06:01.448 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_items": 0, 2024-09-16T02:06:01.448 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T02:06:01.448 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_items": 0, 2024-09-16T02:06:01.448 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T02:06:01.448 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_items": 0, 2024-09-16T02:06:01.448 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_bytes": 148823, 2024-09-16T02:06:01.448 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_items": 27, 2024-09-16T02:06:01.448 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_bytes": 0, 2024-09-16T02:06:01.448 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_items": 0, 2024-09-16T02:06:01.449 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_bytes": 0, 2024-09-16T02:06:01.449 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_items": 0, 2024-09-16T02:06:01.449 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_bytes": 0, 2024-09-16T02:06:01.449 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_items": 0, 2024-09-16T02:06:01.449 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_bytes": 0, 2024-09-16T02:06:01.449 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_items": 0, 2024-09-16T02:06:01.449 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_bytes": 0, 2024-09-16T02:06:01.449 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_items": 0, 2024-09-16T02:06:01.449 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T02:06:01.449 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_items": 0, 2024-09-16T02:06:01.449 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_bytes": 0, 2024-09-16T02:06:01.449 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_items": 0, 2024-09-16T02:06:01.449 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_bytes": 928, 2024-09-16T02:06:01.449 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_items": 2, 2024-09-16T02:06:01.449 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_bytes": 0, 2024-09-16T02:06:01.450 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_items": 0, 2024-09-16T02:06:01.450 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_bytes": 0, 2024-09-16T02:06:01.450 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_items": 0 2024-09-16T02:06:01.450 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-16T02:06:01.450 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T02:06:01.450 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-09-16T02:06:01.450 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-09-16T02:06:01.450 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-09-16T02:06:01.450 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 105, 2024-09-16T02:06:01.450 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 432407, 2024-09-16T02:06:01.450 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:01.450 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 105, 2024-09-16T02:06:01.450 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-09-16T02:06:01.450 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-09-16T02:06:01.451 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 105, 2024-09-16T02:06:01.451 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 432407, 2024-09-16T02:06:01.451 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-09-16T02:06:01.451 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-09-16T02:06:01.451 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-09-16T02:06:01.451 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-09-16T02:06:01.451 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:06:01.451 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-16T02:06:01.451 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-16T02:06:02.380 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-16T02:06:02.380 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-16T02:06:02.380 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-16T02:06:02.380 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-16T02:06:02.380 DEBUG:tasks.check_counter:Getting stats from b 2024-09-16T02:06:02.381 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-16T02:06:02.642 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T02:06:02.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:02 smithi137 ceph-mon[34099]: pgmap v444: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T02:06:03.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:02 smithi105 ceph-mon[31197]: pgmap v444: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T02:06:03.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:02 smithi190 ceph-mon[34762]: pgmap v444: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T02:06:03.105 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-16T02:06:03.105 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T02:06:03.105 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 1837, 2024-09-16T02:06:03.105 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 1578, 2024-09-16T02:06:03.105 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 1013180, 2024-09-16T02:06:03.105 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 609993, 2024-09-16T02:06:03.105 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 14, 2024-09-16T02:06:03.105 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 4, 2024-09-16T02:06:03.105 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.157253915, 2024-09-16T02:06:03.105 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.043357280, 2024-09-16T02:06:03.105 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.078104634, 2024-09-16T02:06:03.106 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.039271450, 2024-09-16T02:06:03.106 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:06:03.106 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1580, 2024-09-16T02:06:03.106 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.505266323, 2024-09-16T02:06:03.106 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000319788 2024-09-16T02:06:03.106 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.106 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:06:03.106 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1083, 2024-09-16T02:06:03.106 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000221191, 2024-09-16T02:06:03.106 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000204 2024-09-16T02:06:03.106 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.106 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 557888, 2024-09-16T02:06:03.106 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 99088 2024-09-16T02:06:03.107 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.107 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T02:06:03.107 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 2039, 2024-09-16T02:06:03.107 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 2076, 2024-09-16T02:06:03.107 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 556474, 2024-09-16T02:06:03.107 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 866726, 2024-09-16T02:06:03.107 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 9, 2024-09-16T02:06:03.107 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 6, 2024-09-16T02:06:03.107 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.132177566, 2024-09-16T02:06:03.107 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.043558209, 2024-09-16T02:06:03.107 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.044045374, 2024-09-16T02:06:03.107 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.041403002, 2024-09-16T02:06:03.107 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:06:03.108 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2077, 2024-09-16T02:06:03.108 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.601661120, 2024-09-16T02:06:03.108 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000289677 2024-09-16T02:06:03.108 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.108 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:06:03.108 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 444, 2024-09-16T02:06:03.108 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000110414, 2024-09-16T02:06:03.108 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000248 2024-09-16T02:06:03.108 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.108 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-16T02:06:03.108 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 208256 2024-09-16T02:06:03.108 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.108 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T02:06:03.109 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 1569, 2024-09-16T02:06:03.109 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 1506, 2024-09-16T02:06:03.109 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 449068, 2024-09-16T02:06:03.109 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 1315006, 2024-09-16T02:06:03.109 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 47, 2024-09-16T02:06:03.109 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 9, 2024-09-16T02:06:03.109 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.125423547, 2024-09-16T02:06:03.109 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.037174710, 2024-09-16T02:06:03.109 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.061855845, 2024-09-16T02:06:03.109 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.028814474, 2024-09-16T02:06:03.109 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:06:03.109 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1499, 2024-09-16T02:06:03.109 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.239548928, 2024-09-16T02:06:03.110 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000159805 2024-09-16T02:06:03.110 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.110 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:06:03.110 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 894, 2024-09-16T02:06:03.110 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000155189, 2024-09-16T02:06:03.110 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000173 2024-09-16T02:06:03.110 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.110 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-16T02:06:03.110 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 448 2024-09-16T02:06:03.110 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.110 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-09-16T02:06:03.110 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 1, 2024-09-16T02:06:03.110 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-09-16T02:06:03.110 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.111 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-MDSRank": { 2024-09-16T02:06:03.111 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-09-16T02:06:03.111 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-09-16T02:06:03.111 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 332, 2024-09-16T02:06:03.111 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.076971967, 2024-09-16T02:06:03.111 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000231843 2024-09-16T02:06:03.111 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-16T02:06:03.111 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.111 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-PurgeQueue": { 2024-09-16T02:06:03.111 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-09-16T02:06:03.111 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-09-16T02:06:03.111 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2871, 2024-09-16T02:06:03.111 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.077677996, 2024-09-16T02:06:03.111 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000027056 2024-09-16T02:06:03.112 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-16T02:06:03.112 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.112 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-09-16T02:06:03.112 INFO:teuthology.orchestra.run.smithi137.stdout: "request": 49, 2024-09-16T02:06:03.112 INFO:teuthology.orchestra.run.smithi137.stdout: "reply": 49, 2024-09-16T02:06:03.112 INFO:teuthology.orchestra.run.smithi137.stdout: "reply_latency": { 2024-09-16T02:06:03.112 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 49, 2024-09-16T02:06:03.112 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.066659636, 2024-09-16T02:06:03.112 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001360400 2024-09-16T02:06:03.112 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.112 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_reply": 0, 2024-09-16T02:06:03.112 INFO:teuthology.orchestra.run.smithi137.stdout: "forward": 0, 2024-09-16T02:06:03.112 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_complete": 0, 2024-09-16T02:06:03.112 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_keys": 0, 2024-09-16T02:06:03.113 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_commit": 12, 2024-09-16T02:06:03.113 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_split": 0, 2024-09-16T02:06:03.113 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_merge": 0, 2024-09-16T02:06:03.113 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes": 23, 2024-09-16T02:06:03.113 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_top": 0, 2024-09-16T02:06:03.113 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_bottom": 23, 2024-09-16T02:06:03.113 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pin_tail": 0, 2024-09-16T02:06:03.113 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pinned": 23, 2024-09-16T02:06:03.113 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_expired": 0, 2024-09-16T02:06:03.113 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_with_caps": 2, 2024-09-16T02:06:03.113 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": 2, 2024-09-16T02:06:03.113 INFO:teuthology.orchestra.run.smithi137.stdout: "subtrees": 2, 2024-09-16T02:06:03.113 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse": 72, 2024-09-16T02:06:03.114 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_hit": 62, 2024-09-16T02:06:03.114 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_forward": 0, 2024-09-16T02:06:03.114 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_discover": 0, 2024-09-16T02:06:03.114 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_dir_fetch": 0, 2024-09-16T02:06:03.114 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_remote_ino": 0, 2024-09-16T02:06:03.114 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_lock": 0, 2024-09-16T02:06:03.114 INFO:teuthology.orchestra.run.smithi137.stdout: "load_cent": 27, 2024-09-16T02:06:03.114 INFO:teuthology.orchestra.run.smithi137.stdout: "q": 0, 2024-09-16T02:06:03.114 INFO:teuthology.orchestra.run.smithi137.stdout: "exported": 0, 2024-09-16T02:06:03.114 INFO:teuthology.orchestra.run.smithi137.stdout: "exported_inodes": 0, 2024-09-16T02:06:03.114 INFO:teuthology.orchestra.run.smithi137.stdout: "imported": 0, 2024-09-16T02:06:03.114 INFO:teuthology.orchestra.run.smithi137.stdout: "imported_inodes": 0, 2024-09-16T02:06:03.114 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_dir_fetch": 0, 2024-09-16T02:06:03.114 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_backtrace_fetch": 64, 2024-09-16T02:06:03.115 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_peer_discover": 0, 2024-09-16T02:06:03.115 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rfiles": 4, 2024-09-16T02:06:03.115 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rbytes": 234, 2024-09-16T02:06:03.115 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rsnaps": 0, 2024-09-16T02:06:03.115 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_fetch": 64, 2024-09-16T02:06:03.115 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_set_tag": 0, 2024-09-16T02:06:03.115 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T02:06:03.115 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_inotable_repaired": 0, 2024-09-16T02:06:03.115 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_inodes": 42, 2024-09-16T02:06:03.115 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_base_inodes": 5, 2024-09-16T02:06:03.115 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dirfrag_rstats": 37, 2024-09-16T02:06:03.115 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_file_inodes": 22, 2024-09-16T02:06:03.115 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_inode_file_caps": 0, 2024-09-16T02:06:03.116 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_revoke": 15, 2024-09-16T02:06:03.116 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_grant": 71, 2024-09-16T02:06:03.116 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T02:06:03.116 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T02:06:03.116 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T02:06:03.116 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps": 80, 2024-09-16T02:06:03.116 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps_dirty": 59, 2024-09-16T02:06:03.116 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_cap_release": 4, 2024-09-16T02:06:03.116 INFO:teuthology.orchestra.run.smithi137.stdout: "process_request_cap_release": 31 2024-09-16T02:06:03.117 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.117 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_cache": { 2024-09-16T02:06:03.117 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update": 15, 2024-09-16T02:06:03.117 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update_receipt": 0, 2024-09-16T02:06:03.117 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_try_discover": 0, 2024-09-16T02:06:03.117 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_send_discover": 0, 2024-09-16T02:06:03.117 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_handle_discover": 32, 2024-09-16T02:06:03.117 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays": 0, 2024-09-16T02:06:03.117 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_delayed": 0, 2024-09-16T02:06:03.117 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_enqueuing": 0, 2024-09-16T02:06:03.117 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_created": 5, 2024-09-16T02:06:03.117 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_enqueued": 5, 2024-09-16T02:06:03.117 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_reintegrated": 0, 2024-09-16T02:06:03.118 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_migrated": 0, 2024-09-16T02:06:03.118 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_processing": 0, 2024-09-16T02:06:03.118 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_enqueued": 0, 2024-09-16T02:06:03.118 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_prioritized": 0, 2024-09-16T02:06:03.118 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_started": 0, 2024-09-16T02:06:03.118 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_completed": 0, 2024-09-16T02:06:03.118 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_path": 0, 2024-09-16T02:06:03.118 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_inode": 0, 2024-09-16T02:06:03.118 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_enqueue_scrub": 5, 2024-09-16T02:06:03.118 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_exportdir": 0, 2024-09-16T02:06:03.118 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_flush": 0, 2024-09-16T02:06:03.118 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragmentdir": 0, 2024-09-16T02:06:03.118 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragstats": 0, 2024-09-16T02:06:03.118 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_inodestats": 0 2024-09-16T02:06:03.119 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.119 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_log": { 2024-09-16T02:06:03.119 INFO:teuthology.orchestra.run.smithi137.stdout: "evlrg": 0, 2024-09-16T02:06:03.119 INFO:teuthology.orchestra.run.smithi137.stdout: "evadd": 122, 2024-09-16T02:06:03.119 INFO:teuthology.orchestra.run.smithi137.stdout: "evex": 0, 2024-09-16T02:06:03.119 INFO:teuthology.orchestra.run.smithi137.stdout: "evtrm": 0, 2024-09-16T02:06:03.119 INFO:teuthology.orchestra.run.smithi137.stdout: "ev": 122, 2024-09-16T02:06:03.119 INFO:teuthology.orchestra.run.smithi137.stdout: "evexg": 0, 2024-09-16T02:06:03.119 INFO:teuthology.orchestra.run.smithi137.stdout: "evexd": 0, 2024-09-16T02:06:03.119 INFO:teuthology.orchestra.run.smithi137.stdout: "segadd": 2, 2024-09-16T02:06:03.119 INFO:teuthology.orchestra.run.smithi137.stdout: "segex": 0, 2024-09-16T02:06:03.119 INFO:teuthology.orchestra.run.smithi137.stdout: "segtrm": 0, 2024-09-16T02:06:03.120 INFO:teuthology.orchestra.run.smithi137.stdout: "seg": 2, 2024-09-16T02:06:03.120 INFO:teuthology.orchestra.run.smithi137.stdout: "segmjr": 2, 2024-09-16T02:06:03.120 INFO:teuthology.orchestra.run.smithi137.stdout: "segexg": 0, 2024-09-16T02:06:03.120 INFO:teuthology.orchestra.run.smithi137.stdout: "segexd": 0, 2024-09-16T02:06:03.120 INFO:teuthology.orchestra.run.smithi137.stdout: "expos": 4194304, 2024-09-16T02:06:03.120 INFO:teuthology.orchestra.run.smithi137.stdout: "wrpos": 4583151, 2024-09-16T02:06:03.120 INFO:teuthology.orchestra.run.smithi137.stdout: "rdpos": 0, 2024-09-16T02:06:03.120 INFO:teuthology.orchestra.run.smithi137.stdout: "jlat": { 2024-09-16T02:06:03.120 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 102, 2024-09-16T02:06:03.120 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.194353586, 2024-09-16T02:06:03.120 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001905427 2024-09-16T02:06:03.120 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.120 INFO:teuthology.orchestra.run.smithi137.stdout: "replayed": 0 2024-09-16T02:06:03.120 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.121 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_mem": { 2024-09-16T02:06:03.121 INFO:teuthology.orchestra.run.smithi137.stdout: "ino": 23, 2024-09-16T02:06:03.121 INFO:teuthology.orchestra.run.smithi137.stdout: "ino+": 33, 2024-09-16T02:06:03.121 INFO:teuthology.orchestra.run.smithi137.stdout: "ino-": 10, 2024-09-16T02:06:03.121 INFO:teuthology.orchestra.run.smithi137.stdout: "dir": 18, 2024-09-16T02:06:03.121 INFO:teuthology.orchestra.run.smithi137.stdout: "dir+": 20, 2024-09-16T02:06:03.121 INFO:teuthology.orchestra.run.smithi137.stdout: "dir-": 2, 2024-09-16T02:06:03.121 INFO:teuthology.orchestra.run.smithi137.stdout: "dn": 23, 2024-09-16T02:06:03.121 INFO:teuthology.orchestra.run.smithi137.stdout: "dn+": 32, 2024-09-16T02:06:03.121 INFO:teuthology.orchestra.run.smithi137.stdout: "dn-": 9, 2024-09-16T02:06:03.121 INFO:teuthology.orchestra.run.smithi137.stdout: "cap": 2, 2024-09-16T02:06:03.121 INFO:teuthology.orchestra.run.smithi137.stdout: "cap+": 18, 2024-09-16T02:06:03.121 INFO:teuthology.orchestra.run.smithi137.stdout: "cap-": 16, 2024-09-16T02:06:03.122 INFO:teuthology.orchestra.run.smithi137.stdout: "rss": 53452, 2024-09-16T02:06:03.122 INFO:teuthology.orchestra.run.smithi137.stdout: "heap": 231712 2024-09-16T02:06:03.122 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.122 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_server": { 2024-09-16T02:06:03.122 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_client_request": 74, 2024-09-16T02:06:03.122 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_server_request": 1, 2024-09-16T02:06:03.122 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_request": 49, 2024-09-16T02:06:03.122 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_session": 11, 2024-09-16T02:06:03.122 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_peer_request": 2, 2024-09-16T02:06:03.122 INFO:teuthology.orchestra.run.smithi137.stdout: "req_create_latency": { 2024-09-16T02:06:03.122 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 7, 2024-09-16T02:06:03.122 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.006807858, 2024-09-16T02:06:03.122 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000972551 2024-09-16T02:06:03.122 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.122 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getattr_latency": { 2024-09-16T02:06:03.123 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 15, 2024-09-16T02:06:03.123 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.003624958, 2024-09-16T02:06:03.123 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000241663 2024-09-16T02:06:03.123 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.123 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getfilelock_latency": { 2024-09-16T02:06:03.123 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:03.123 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:03.123 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:03.123 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.123 INFO:teuthology.orchestra.run.smithi137.stdout: "req_link_latency": { 2024-09-16T02:06:03.123 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:03.123 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:03.123 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:03.123 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.124 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookup_latency": { 2024-09-16T02:06:03.124 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:03.124 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:03.124 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:03.124 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.124 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookuphash_latency": { 2024-09-16T02:06:03.124 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:03.124 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:03.124 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:03.124 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.124 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupino_latency": { 2024-09-16T02:06:03.124 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:03.124 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:03.125 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:03.125 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.125 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupname_latency": { 2024-09-16T02:06:03.125 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:03.125 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:03.125 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:03.125 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.125 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupparent_latency": { 2024-09-16T02:06:03.125 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:03.125 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:03.125 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:03.125 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.125 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupsnap_latency": { 2024-09-16T02:06:03.125 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:03.125 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:03.126 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:03.126 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.126 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lssnap_latency": { 2024-09-16T02:06:03.126 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:03.126 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:03.126 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:03.126 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.126 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mkdir_latency": { 2024-09-16T02:06:03.127 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 8, 2024-09-16T02:06:03.127 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.011451769, 2024-09-16T02:06:03.127 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001431471 2024-09-16T02:06:03.127 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.127 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mknod_latency": { 2024-09-16T02:06:03.127 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:03.127 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:03.127 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:03.127 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.127 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mksnap_latency": { 2024-09-16T02:06:03.127 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:03.127 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:03.127 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:03.127 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.128 INFO:teuthology.orchestra.run.smithi137.stdout: "req_open_latency": { 2024-09-16T02:06:03.128 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2, 2024-09-16T02:06:03.128 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.001921482, 2024-09-16T02:06:03.128 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000960741 2024-09-16T02:06:03.128 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.128 INFO:teuthology.orchestra.run.smithi137.stdout: "req_readdir_latency": { 2024-09-16T02:06:03.128 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2, 2024-09-16T02:06:03.128 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.007189230, 2024-09-16T02:06:03.128 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.003594615 2024-09-16T02:06:03.128 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.128 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rename_latency": { 2024-09-16T02:06:03.128 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2, 2024-09-16T02:06:03.128 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.001996106, 2024-09-16T02:06:03.128 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000998053 2024-09-16T02:06:03.129 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.129 INFO:teuthology.orchestra.run.smithi137.stdout: "req_renamesnap_latency": { 2024-09-16T02:06:03.129 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:03.129 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:03.129 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:03.129 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.129 INFO:teuthology.orchestra.run.smithi137.stdout: "req_snapdiff_latency": { 2024-09-16T02:06:03.129 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:03.129 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:03.129 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:03.129 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.129 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmdir_latency": { 2024-09-16T02:06:03.129 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2, 2024-09-16T02:06:03.129 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.009495466, 2024-09-16T02:06:03.130 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.004747733 2024-09-16T02:06:03.130 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.130 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmsnap_latency": { 2024-09-16T02:06:03.130 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:03.130 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:03.130 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:03.130 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.130 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmxattr_latency": { 2024-09-16T02:06:03.130 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:03.130 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:03.130 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:03.130 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.130 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setattr_latency": { 2024-09-16T02:06:03.130 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1, 2024-09-16T02:06:03.131 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000934058, 2024-09-16T02:06:03.131 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000934058 2024-09-16T02:06:03.131 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.131 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setdirlayout_latency": { 2024-09-16T02:06:03.131 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:03.131 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:03.131 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:03.131 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.131 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setfilelock_latency": { 2024-09-16T02:06:03.131 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:03.131 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:03.131 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:03.131 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.131 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setlayout_latency": { 2024-09-16T02:06:03.132 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:03.132 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:03.132 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:03.132 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.132 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setxattr_latency": { 2024-09-16T02:06:03.132 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 7, 2024-09-16T02:06:03.132 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.017409204, 2024-09-16T02:06:03.132 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.002487029 2024-09-16T02:06:03.132 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.132 INFO:teuthology.orchestra.run.smithi137.stdout: "req_symlink_latency": { 2024-09-16T02:06:03.132 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:03.132 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:03.132 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:03.132 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.133 INFO:teuthology.orchestra.run.smithi137.stdout: "req_unlink_latency": { 2024-09-16T02:06:03.133 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 3, 2024-09-16T02:06:03.133 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.005829505, 2024-09-16T02:06:03.133 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001943168 2024-09-16T02:06:03.133 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.133 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_revoke_eviction": 0, 2024-09-16T02:06:03.133 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_acquisition_throttle": 0, 2024-09-16T02:06:03.133 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getvxattr_latency": { 2024-09-16T02:06:03.133 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:03.133 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:03.133 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:03.133 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-16T02:06:03.133 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.133 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_sessions": { 2024-09-16T02:06:03.134 INFO:teuthology.orchestra.run.smithi137.stdout: "session_count": 1, 2024-09-16T02:06:03.134 INFO:teuthology.orchestra.run.smithi137.stdout: "session_add": 2, 2024-09-16T02:06:03.134 INFO:teuthology.orchestra.run.smithi137.stdout: "session_remove": 1, 2024-09-16T02:06:03.134 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_open": 1, 2024-09-16T02:06:03.134 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_stale": 0, 2024-09-16T02:06:03.134 INFO:teuthology.orchestra.run.smithi137.stdout: "total_load": 14, 2024-09-16T02:06:03.134 INFO:teuthology.orchestra.run.smithi137.stdout: "average_load": 14, 2024-09-16T02:06:03.134 INFO:teuthology.orchestra.run.smithi137.stdout: "avg_session_uptime": 200, 2024-09-16T02:06:03.134 INFO:teuthology.orchestra.run.smithi137.stdout: "mdthresh_evicted": 0 2024-09-16T02:06:03.134 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.134 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-09-16T02:06:03.134 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-09-16T02:06:03.134 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-09-16T02:06:03.135 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T02:06:03.135 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-09-16T02:06:03.135 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T02:06:03.135 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-09-16T02:06:03.135 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T02:06:03.135 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T02:06:03.135 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T02:06:03.135 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T02:06:03.135 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T02:06:03.135 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-09-16T02:06:03.135 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T02:06:03.135 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T02:06:03.135 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-09-16T02:06:03.135 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-09-16T02:06:03.135 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-09-16T02:06:03.136 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-09-16T02:06:03.136 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T02:06:03.136 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T02:06:03.136 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T02:06:03.136 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T02:06:03.136 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T02:06:03.136 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-09-16T02:06:03.136 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-09-16T02:06:03.136 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-09-16T02:06:03.136 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T02:06:03.136 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T02:06:03.136 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-09-16T02:06:03.136 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-09-16T02:06:03.137 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-09-16T02:06:03.137 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-09-16T02:06:03.137 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T02:06:03.137 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-09-16T02:06:03.137 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T02:06:03.137 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-09-16T02:06:03.137 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 291645, 2024-09-16T02:06:03.137 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 119, 2024-09-16T02:06:03.137 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 88, 2024-09-16T02:06:03.137 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 1, 2024-09-16T02:06:03.137 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-09-16T02:06:03.137 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-09-16T02:06:03.137 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-09-16T02:06:03.137 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-09-16T02:06:03.138 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-09-16T02:06:03.138 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-09-16T02:06:03.138 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 7888, 2024-09-16T02:06:03.138 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 163, 2024-09-16T02:06:03.138 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T02:06:03.138 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-09-16T02:06:03.138 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-09-16T02:06:03.138 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-09-16T02:06:03.138 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 96786, 2024-09-16T02:06:03.138 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 698, 2024-09-16T02:06:03.138 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-09-16T02:06:03.138 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-09-16T02:06:03.138 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-09-16T02:06:03.139 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-09-16T02:06:03.139 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.139 INFO:teuthology.orchestra.run.smithi137.stdout: "objecter": { 2024-09-16T02:06:03.139 INFO:teuthology.orchestra.run.smithi137.stdout: "op_active": 0, 2024-09-16T02:06:03.139 INFO:teuthology.orchestra.run.smithi137.stdout: "op_laggy": 0, 2024-09-16T02:06:03.139 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send": 3846, 2024-09-16T02:06:03.139 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send_bytes": 406023, 2024-09-16T02:06:03.139 INFO:teuthology.orchestra.run.smithi137.stdout: "op_resend": 37, 2024-09-16T02:06:03.139 INFO:teuthology.orchestra.run.smithi137.stdout: "op_reply": 3846, 2024-09-16T02:06:03.139 INFO:teuthology.orchestra.run.smithi137.stdout: "op_latency": { 2024-09-16T02:06:03.139 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 3846, 2024-09-16T02:06:03.139 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 30.743448275, 2024-09-16T02:06:03.139 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.007993616 2024-09-16T02:06:03.140 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.140 INFO:teuthology.orchestra.run.smithi137.stdout: "op_inflight": 0, 2024-09-16T02:06:03.140 INFO:teuthology.orchestra.run.smithi137.stdout: "oplen_avg": { 2024-09-16T02:06:03.140 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 3846, 2024-09-16T02:06:03.140 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 3897 2024-09-16T02:06:03.140 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.140 INFO:teuthology.orchestra.run.smithi137.stdout: "op": 3846, 2024-09-16T02:06:03.140 INFO:teuthology.orchestra.run.smithi137.stdout: "op_r": 79, 2024-09-16T02:06:03.140 INFO:teuthology.orchestra.run.smithi137.stdout: "op_w": 3767, 2024-09-16T02:06:03.140 INFO:teuthology.orchestra.run.smithi137.stdout: "op_rmw": 0, 2024-09-16T02:06:03.140 INFO:teuthology.orchestra.run.smithi137.stdout: "op_pg": 0, 2024-09-16T02:06:03.140 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_stat": 0, 2024-09-16T02:06:03.140 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_create": 11, 2024-09-16T02:06:03.141 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_read": 10, 2024-09-16T02:06:03.141 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_write": 107, 2024-09-16T02:06:03.141 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writefull": 23, 2024-09-16T02:06:03.141 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writesame": 0, 2024-09-16T02:06:03.141 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_append": 0, 2024-09-16T02:06:03.141 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_zero": 0, 2024-09-16T02:06:03.141 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_truncate": 0, 2024-09-16T02:06:03.141 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_delete": 3586, 2024-09-16T02:06:03.141 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_mapext": 0, 2024-09-16T02:06:03.141 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_sparse_read": 0, 2024-09-16T02:06:03.141 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_clonerange": 0, 2024-09-16T02:06:03.141 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_getxattr": 69, 2024-09-16T02:06:03.141 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_setxattr": 22, 2024-09-16T02:06:03.142 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_cmpxattr": 0, 2024-09-16T02:06:03.142 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_rmxattr": 0, 2024-09-16T02:06:03.142 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_resetxattrs": 0, 2024-09-16T02:06:03.142 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_call": 0, 2024-09-16T02:06:03.142 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_watch": 0, 2024-09-16T02:06:03.142 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_notify": 0, 2024-09-16T02:06:03.142 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T02:06:03.142 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls": 0, 2024-09-16T02:06:03.142 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls_filter": 0, 2024-09-16T02:06:03.142 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_other": 0, 2024-09-16T02:06:03.142 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_active": 0, 2024-09-16T02:06:03.142 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_send": 0, 2024-09-16T02:06:03.142 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_resend": 0, 2024-09-16T02:06:03.142 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_ping": 0, 2024-09-16T02:06:03.143 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_active": 0, 2024-09-16T02:06:03.143 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_send": 0, 2024-09-16T02:06:03.143 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_resend": 0, 2024-09-16T02:06:03.143 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_active": 0, 2024-09-16T02:06:03.143 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_send": 0, 2024-09-16T02:06:03.143 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_resend": 0, 2024-09-16T02:06:03.143 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_active": 0, 2024-09-16T02:06:03.143 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_send": 0, 2024-09-16T02:06:03.143 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_resend": 0, 2024-09-16T02:06:03.143 INFO:teuthology.orchestra.run.smithi137.stdout: "command_active": 0, 2024-09-16T02:06:03.143 INFO:teuthology.orchestra.run.smithi137.stdout: "command_send": 0, 2024-09-16T02:06:03.143 INFO:teuthology.orchestra.run.smithi137.stdout: "command_resend": 0, 2024-09-16T02:06:03.143 INFO:teuthology.orchestra.run.smithi137.stdout: "map_epoch": 83, 2024-09-16T02:06:03.143 INFO:teuthology.orchestra.run.smithi137.stdout: "map_full": 0, 2024-09-16T02:06:03.144 INFO:teuthology.orchestra.run.smithi137.stdout: "map_inc": 4, 2024-09-16T02:06:03.144 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_sessions": 12, 2024-09-16T02:06:03.144 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_open": 12, 2024-09-16T02:06:03.144 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_close": 0, 2024-09-16T02:06:03.144 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_laggy": 0, 2024-09-16T02:06:03.144 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_wr": 53, 2024-09-16T02:06:03.144 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_rd": 0, 2024-09-16T02:06:03.144 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_del": 16 2024-09-16T02:06:03.144 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.144 INFO:teuthology.orchestra.run.smithi137.stdout: "oft": { 2024-09-16T02:06:03.144 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_objs": 1, 2024-09-16T02:06:03.144 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_kv_pairs": 0, 2024-09-16T02:06:03.144 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_updates": 0, 2024-09-16T02:06:03.144 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_removes": 0 2024-09-16T02:06:03.145 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.145 INFO:teuthology.orchestra.run.smithi137.stdout: "osd-slow-ops": { 2024-09-16T02:06:03.145 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_ops_count": 0 2024-09-16T02:06:03.145 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.145 INFO:teuthology.orchestra.run.smithi137.stdout: "purge_queue": { 2024-09-16T02:06:03.145 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops": 0, 2024-09-16T02:06:03.145 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops_high_water": 2560, 2024-09-16T02:06:03.145 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing": 0, 2024-09-16T02:06:03.145 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_high_water": 2, 2024-09-16T02:06:03.145 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed_ops": 3074, 2024-09-16T02:06:03.145 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed": 5, 2024-09-16T02:06:03.145 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_item_in_journal": 0 2024-09-16T02:06:03.145 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.145 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T02:06:03.146 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-16T02:06:03.146 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-09-16T02:06:03.146 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-16T02:06:03.146 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 5441, 2024-09-16T02:06:03.146 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 1512100, 2024-09-16T02:06:03.146 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:03.146 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 5441, 2024-09-16T02:06:03.146 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-09-16T02:06:03.146 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-09-16T02:06:03.146 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 5441, 2024-09-16T02:06:03.146 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 1512100, 2024-09-16T02:06:03.146 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-16T02:06:03.146 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:03.146 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:03.146 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:03.147 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-16T02:06:03.147 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.147 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_bytes": { 2024-09-16T02:06:03.147 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-16T02:06:03.147 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-09-16T02:06:03.147 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-16T02:06:03.147 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-09-16T02:06:03.147 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-09-16T02:06:03.147 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:03.147 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-09-16T02:06:03.147 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 3846, 2024-09-16T02:06:03.147 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 406528, 2024-09-16T02:06:03.147 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 3846, 2024-09-16T02:06:03.147 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 406528, 2024-09-16T02:06:03.147 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-16T02:06:03.148 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:03.148 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:03.148 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:03.148 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-16T02:06:03.148 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.148 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_ops": { 2024-09-16T02:06:03.148 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-16T02:06:03.148 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 1024, 2024-09-16T02:06:03.148 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-16T02:06:03.148 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-09-16T02:06:03.148 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-09-16T02:06:03.148 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:03.149 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-09-16T02:06:03.149 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 3846, 2024-09-16T02:06:03.149 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 3846, 2024-09-16T02:06:03.149 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 3846, 2024-09-16T02:06:03.149 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 3846, 2024-09-16T02:06:03.149 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-16T02:06:03.149 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:03.149 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:03.149 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:03.149 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-16T02:06:03.149 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.149 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle": { 2024-09-16T02:06:03.149 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-16T02:06:03.149 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-09-16T02:06:03.150 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-16T02:06:03.150 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 5, 2024-09-16T02:06:03.150 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 505, 2024-09-16T02:06:03.150 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:03.150 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 5, 2024-09-16T02:06:03.150 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-09-16T02:06:03.150 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-09-16T02:06:03.150 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 5, 2024-09-16T02:06:03.150 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 505, 2024-09-16T02:06:03.150 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-16T02:06:03.150 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:03.150 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:03.150 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:03.150 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-16T02:06:03.152 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:03.152 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle-0x5632dfcae960": { 2024-09-16T02:06:03.152 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-16T02:06:03.152 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-09-16T02:06:03.152 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-16T02:06:03.152 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 122, 2024-09-16T02:06:03.153 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 388847, 2024-09-16T02:06:03.153 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:03.153 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 122, 2024-09-16T02:06:03.153 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-09-16T02:06:03.153 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-09-16T02:06:03.153 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 102, 2024-09-16T02:06:03.153 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 388847, 2024-09-16T02:06:03.153 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-16T02:06:03.153 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:03.154 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:03.154 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:03.154 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-16T02:06:03.154 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-16T02:06:03.154 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-16T02:06:04.877 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:04 smithi137 ceph-mon[34099]: pgmap v445: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-16T02:06:04.878 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-09-16T02:06:04.878 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-09-16T02:06:04.878 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=15 2024-09-16T02:06:04.878 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-09-16T02:06:04.878 DEBUG:tasks.check_counter:Getting stats from e 2024-09-16T02:06:04.879 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-16T02:06:05.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:04 smithi105 ceph-mon[31197]: pgmap v445: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-16T02:06:05.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:04 smithi190 ceph-mon[34762]: pgmap v445: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-16T02:06:05.132 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T02:06:05.708 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-16T02:06:05.709 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T02:06:05.709 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 804, 2024-09-16T02:06:05.709 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 798, 2024-09-16T02:06:05.709 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 1066099, 2024-09-16T02:06:05.709 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 280826, 2024-09-16T02:06:05.709 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 5, 2024-09-16T02:06:05.709 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 1, 2024-09-16T02:06:05.709 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.076646269, 2024-09-16T02:06:05.709 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.029228795, 2024-09-16T02:06:05.709 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.030662729, 2024-09-16T02:06:05.709 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.015846562, 2024-09-16T02:06:05.709 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:06:05.709 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 798, 2024-09-16T02:06:05.710 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.032031931, 2024-09-16T02:06:05.710 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000040140 2024-09-16T02:06:05.710 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.710 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:06:05.710 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.710 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.710 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.710 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.710 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 474448, 2024-09-16T02:06:05.710 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 62992 2024-09-16T02:06:05.710 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.710 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T02:06:05.710 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 5, 2024-09-16T02:06:05.711 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 80, 2024-09-16T02:06:05.711 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 1816, 2024-09-16T02:06:05.711 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 205914, 2024-09-16T02:06:05.711 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 5, 2024-09-16T02:06:05.711 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 4, 2024-09-16T02:06:05.711 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.005710832, 2024-09-16T02:06:05.711 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.003223329, 2024-09-16T02:06:05.711 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.001556280, 2024-09-16T02:06:05.711 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000097240, 2024-09-16T02:06:05.711 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:06:05.711 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 78, 2024-09-16T02:06:05.711 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.008549648, 2024-09-16T02:06:05.711 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000109610 2024-09-16T02:06:05.712 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.712 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:06:05.712 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.712 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.712 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.712 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.712 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-16T02:06:05.712 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 204576 2024-09-16T02:06:05.712 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.712 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T02:06:05.712 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 317, 2024-09-16T02:06:05.712 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 319, 2024-09-16T02:06:05.713 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 78088, 2024-09-16T02:06:05.713 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 100813, 2024-09-16T02:06:05.713 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 4, 2024-09-16T02:06:05.713 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 3, 2024-09-16T02:06:05.713 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.026539913, 2024-09-16T02:06:05.713 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.007574648, 2024-09-16T02:06:05.713 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.006875219, 2024-09-16T02:06:05.713 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.009736396, 2024-09-16T02:06:05.713 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:06:05.713 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 317, 2024-09-16T02:06:05.713 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.010878063, 2024-09-16T02:06:05.713 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000034315 2024-09-16T02:06:05.713 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.714 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:06:05.714 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.714 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.714 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.714 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.714 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T02:06:05.714 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T02:06:05.714 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.714 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-09-16T02:06:05.714 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 1, 2024-09-16T02:06:05.714 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-09-16T02:06:05.714 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.714 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-MDSRank": { 2024-09-16T02:06:05.714 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-09-16T02:06:05.715 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-09-16T02:06:05.715 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1683, 2024-09-16T02:06:05.715 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.038460661, 2024-09-16T02:06:05.715 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000022852 2024-09-16T02:06:05.715 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-16T02:06:05.715 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.715 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-PurgeQueue": { 2024-09-16T02:06:05.715 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-09-16T02:06:05.715 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-09-16T02:06:05.715 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.715 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.715 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.715 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-16T02:06:05.716 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.716 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-09-16T02:06:05.716 INFO:teuthology.orchestra.run.smithi137.stdout: "request": 0, 2024-09-16T02:06:05.716 INFO:teuthology.orchestra.run.smithi137.stdout: "reply": 0, 2024-09-16T02:06:05.716 INFO:teuthology.orchestra.run.smithi137.stdout: "reply_latency": { 2024-09-16T02:06:05.716 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.716 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.716 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.716 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.716 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_reply": 0, 2024-09-16T02:06:05.716 INFO:teuthology.orchestra.run.smithi137.stdout: "forward": 0, 2024-09-16T02:06:05.716 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_complete": 0, 2024-09-16T02:06:05.717 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_keys": 0, 2024-09-16T02:06:05.717 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_commit": 0, 2024-09-16T02:06:05.717 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_split": 0, 2024-09-16T02:06:05.717 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_merge": 0, 2024-09-16T02:06:05.717 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes": 19, 2024-09-16T02:06:05.717 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_top": 0, 2024-09-16T02:06:05.717 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_bottom": 19, 2024-09-16T02:06:05.717 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pin_tail": 0, 2024-09-16T02:06:05.717 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pinned": 19, 2024-09-16T02:06:05.717 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_expired": 0, 2024-09-16T02:06:05.717 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_with_caps": 0, 2024-09-16T02:06:05.717 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": 0, 2024-09-16T02:06:05.717 INFO:teuthology.orchestra.run.smithi137.stdout: "subtrees": 2, 2024-09-16T02:06:05.718 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse": 0, 2024-09-16T02:06:05.718 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_hit": 0, 2024-09-16T02:06:05.718 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_forward": 0, 2024-09-16T02:06:05.718 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_discover": 0, 2024-09-16T02:06:05.718 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_dir_fetch": 0, 2024-09-16T02:06:05.718 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_remote_ino": 0, 2024-09-16T02:06:05.718 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_lock": 0, 2024-09-16T02:06:05.718 INFO:teuthology.orchestra.run.smithi137.stdout: "load_cent": 0, 2024-09-16T02:06:05.718 INFO:teuthology.orchestra.run.smithi137.stdout: "q": 0, 2024-09-16T02:06:05.718 INFO:teuthology.orchestra.run.smithi137.stdout: "exported": 0, 2024-09-16T02:06:05.718 INFO:teuthology.orchestra.run.smithi137.stdout: "exported_inodes": 0, 2024-09-16T02:06:05.718 INFO:teuthology.orchestra.run.smithi137.stdout: "imported": 0, 2024-09-16T02:06:05.718 INFO:teuthology.orchestra.run.smithi137.stdout: "imported_inodes": 0, 2024-09-16T02:06:05.719 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_dir_fetch": 0, 2024-09-16T02:06:05.719 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_backtrace_fetch": 0, 2024-09-16T02:06:05.719 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_peer_discover": 0, 2024-09-16T02:06:05.719 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rfiles": 4, 2024-09-16T02:06:05.719 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rbytes": 234, 2024-09-16T02:06:05.719 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rsnaps": 0, 2024-09-16T02:06:05.719 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T02:06:05.719 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_set_tag": 0, 2024-09-16T02:06:05.719 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T02:06:05.719 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_inotable_repaired": 0, 2024-09-16T02:06:05.719 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_inodes": 0, 2024-09-16T02:06:05.719 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T02:06:05.719 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T02:06:05.720 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_file_inodes": 0, 2024-09-16T02:06:05.720 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_inode_file_caps": 0, 2024-09-16T02:06:05.720 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T02:06:05.720 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_grant": 0, 2024-09-16T02:06:05.720 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T02:06:05.720 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T02:06:05.720 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T02:06:05.720 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps": 0, 2024-09-16T02:06:05.720 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps_dirty": 0, 2024-09-16T02:06:05.720 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_cap_release": 0, 2024-09-16T02:06:05.720 INFO:teuthology.orchestra.run.smithi137.stdout: "process_request_cap_release": 0 2024-09-16T02:06:05.720 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.720 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_cache": { 2024-09-16T02:06:05.720 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update": 0, 2024-09-16T02:06:05.721 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update_receipt": 0, 2024-09-16T02:06:05.721 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_try_discover": 0, 2024-09-16T02:06:05.721 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_send_discover": 0, 2024-09-16T02:06:05.721 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_handle_discover": 0, 2024-09-16T02:06:05.721 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays": 0, 2024-09-16T02:06:05.721 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_delayed": 0, 2024-09-16T02:06:05.721 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_enqueuing": 0, 2024-09-16T02:06:05.721 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_created": 0, 2024-09-16T02:06:05.721 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_enqueued": 0, 2024-09-16T02:06:05.721 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_reintegrated": 0, 2024-09-16T02:06:05.721 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_migrated": 0, 2024-09-16T02:06:05.721 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_processing": 0, 2024-09-16T02:06:05.721 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_enqueued": 0, 2024-09-16T02:06:05.721 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_prioritized": 0, 2024-09-16T02:06:05.722 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_started": 0, 2024-09-16T02:06:05.722 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_completed": 0, 2024-09-16T02:06:05.722 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_path": 0, 2024-09-16T02:06:05.722 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_inode": 0, 2024-09-16T02:06:05.722 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T02:06:05.722 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_exportdir": 0, 2024-09-16T02:06:05.722 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_flush": 0, 2024-09-16T02:06:05.722 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragmentdir": 0, 2024-09-16T02:06:05.722 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragstats": 0, 2024-09-16T02:06:05.722 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_inodestats": 0 2024-09-16T02:06:05.722 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.722 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_log": { 2024-09-16T02:06:05.722 INFO:teuthology.orchestra.run.smithi137.stdout: "evlrg": 0, 2024-09-16T02:06:05.722 INFO:teuthology.orchestra.run.smithi137.stdout: "evadd": 0, 2024-09-16T02:06:05.722 INFO:teuthology.orchestra.run.smithi137.stdout: "evex": 0, 2024-09-16T02:06:05.723 INFO:teuthology.orchestra.run.smithi137.stdout: "evtrm": 0, 2024-09-16T02:06:05.723 INFO:teuthology.orchestra.run.smithi137.stdout: "ev": 122, 2024-09-16T02:06:05.723 INFO:teuthology.orchestra.run.smithi137.stdout: "evexg": 0, 2024-09-16T02:06:05.723 INFO:teuthology.orchestra.run.smithi137.stdout: "evexd": 0, 2024-09-16T02:06:05.723 INFO:teuthology.orchestra.run.smithi137.stdout: "segadd": 0, 2024-09-16T02:06:05.723 INFO:teuthology.orchestra.run.smithi137.stdout: "segex": 0, 2024-09-16T02:06:05.723 INFO:teuthology.orchestra.run.smithi137.stdout: "segtrm": 0, 2024-09-16T02:06:05.723 INFO:teuthology.orchestra.run.smithi137.stdout: "seg": 2, 2024-09-16T02:06:05.723 INFO:teuthology.orchestra.run.smithi137.stdout: "segmjr": 2, 2024-09-16T02:06:05.723 INFO:teuthology.orchestra.run.smithi137.stdout: "segexg": 0, 2024-09-16T02:06:05.723 INFO:teuthology.orchestra.run.smithi137.stdout: "segexd": 0, 2024-09-16T02:06:05.723 INFO:teuthology.orchestra.run.smithi137.stdout: "expos": 4194304, 2024-09-16T02:06:05.723 INFO:teuthology.orchestra.run.smithi137.stdout: "wrpos": 4583151, 2024-09-16T02:06:05.723 INFO:teuthology.orchestra.run.smithi137.stdout: "rdpos": 4582013, 2024-09-16T02:06:05.724 INFO:teuthology.orchestra.run.smithi137.stdout: "jlat": { 2024-09-16T02:06:05.724 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.724 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.724 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.724 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.724 INFO:teuthology.orchestra.run.smithi137.stdout: "replayed": 122 2024-09-16T02:06:05.724 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.724 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_mem": { 2024-09-16T02:06:05.724 INFO:teuthology.orchestra.run.smithi137.stdout: "ino": 14, 2024-09-16T02:06:05.724 INFO:teuthology.orchestra.run.smithi137.stdout: "ino+": 19, 2024-09-16T02:06:05.724 INFO:teuthology.orchestra.run.smithi137.stdout: "ino-": 5, 2024-09-16T02:06:05.724 INFO:teuthology.orchestra.run.smithi137.stdout: "dir": 9, 2024-09-16T02:06:05.724 INFO:teuthology.orchestra.run.smithi137.stdout: "dir+": 11, 2024-09-16T02:06:05.724 INFO:teuthology.orchestra.run.smithi137.stdout: "dir-": 2, 2024-09-16T02:06:05.725 INFO:teuthology.orchestra.run.smithi137.stdout: "dn": 19, 2024-09-16T02:06:05.725 INFO:teuthology.orchestra.run.smithi137.stdout: "dn+": 23, 2024-09-16T02:06:05.725 INFO:teuthology.orchestra.run.smithi137.stdout: "dn-": 4, 2024-09-16T02:06:05.725 INFO:teuthology.orchestra.run.smithi137.stdout: "cap": 0, 2024-09-16T02:06:05.725 INFO:teuthology.orchestra.run.smithi137.stdout: "cap+": 0, 2024-09-16T02:06:05.725 INFO:teuthology.orchestra.run.smithi137.stdout: "cap-": 0, 2024-09-16T02:06:05.725 INFO:teuthology.orchestra.run.smithi137.stdout: "rss": 48740, 2024-09-16T02:06:05.725 INFO:teuthology.orchestra.run.smithi137.stdout: "heap": 215328 2024-09-16T02:06:05.725 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.725 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_server": { 2024-09-16T02:06:05.725 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_client_request": 0, 2024-09-16T02:06:05.725 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_server_request": 0, 2024-09-16T02:06:05.725 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_request": 0, 2024-09-16T02:06:05.725 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_session": 0, 2024-09-16T02:06:05.726 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_peer_request": 0, 2024-09-16T02:06:05.726 INFO:teuthology.orchestra.run.smithi137.stdout: "req_create_latency": { 2024-09-16T02:06:05.726 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.726 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.726 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.726 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.726 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getattr_latency": { 2024-09-16T02:06:05.726 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.726 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.727 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.727 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.727 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getfilelock_latency": { 2024-09-16T02:06:05.727 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.727 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.727 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.727 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.727 INFO:teuthology.orchestra.run.smithi137.stdout: "req_link_latency": { 2024-09-16T02:06:05.727 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.727 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.727 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.727 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.727 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookup_latency": { 2024-09-16T02:06:05.727 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.728 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.728 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.728 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.728 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookuphash_latency": { 2024-09-16T02:06:05.728 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.728 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.728 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.728 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.728 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupino_latency": { 2024-09-16T02:06:05.728 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.728 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.728 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.728 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.728 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupname_latency": { 2024-09-16T02:06:05.728 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.729 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.729 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.729 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.729 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupparent_latency": { 2024-09-16T02:06:05.729 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.729 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.729 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.729 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.729 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupsnap_latency": { 2024-09-16T02:06:05.729 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.729 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.729 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.729 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.730 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lssnap_latency": { 2024-09-16T02:06:05.730 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.730 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.730 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.730 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.730 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mkdir_latency": { 2024-09-16T02:06:05.730 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.730 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.731 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.731 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.731 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mknod_latency": { 2024-09-16T02:06:05.731 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.731 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.731 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.731 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.731 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mksnap_latency": { 2024-09-16T02:06:05.731 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.731 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.731 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.731 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.731 INFO:teuthology.orchestra.run.smithi137.stdout: "req_open_latency": { 2024-09-16T02:06:05.731 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.732 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.732 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.732 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.732 INFO:teuthology.orchestra.run.smithi137.stdout: "req_readdir_latency": { 2024-09-16T02:06:05.732 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.732 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.732 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.732 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.732 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rename_latency": { 2024-09-16T02:06:05.732 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.732 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.732 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.732 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.733 INFO:teuthology.orchestra.run.smithi137.stdout: "req_renamesnap_latency": { 2024-09-16T02:06:05.733 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.733 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.733 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.733 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.733 INFO:teuthology.orchestra.run.smithi137.stdout: "req_snapdiff_latency": { 2024-09-16T02:06:05.733 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.733 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.733 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.733 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.733 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmdir_latency": { 2024-09-16T02:06:05.733 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.733 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.734 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.734 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.734 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmsnap_latency": { 2024-09-16T02:06:05.734 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.734 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.734 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.734 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.734 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmxattr_latency": { 2024-09-16T02:06:05.734 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.734 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.734 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.734 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.734 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setattr_latency": { 2024-09-16T02:06:05.735 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.735 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.735 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.735 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.735 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setdirlayout_latency": { 2024-09-16T02:06:05.735 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.735 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.735 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.735 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.735 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setfilelock_latency": { 2024-09-16T02:06:05.735 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.735 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.735 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.735 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.735 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setlayout_latency": { 2024-09-16T02:06:05.736 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.736 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.736 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.736 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.736 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setxattr_latency": { 2024-09-16T02:06:05.736 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.736 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.736 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.736 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.736 INFO:teuthology.orchestra.run.smithi137.stdout: "req_symlink_latency": { 2024-09-16T02:06:05.736 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.736 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.736 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.736 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.737 INFO:teuthology.orchestra.run.smithi137.stdout: "req_unlink_latency": { 2024-09-16T02:06:05.737 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.737 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.737 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.737 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.737 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_revoke_eviction": 0, 2024-09-16T02:06:05.737 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_acquisition_throttle": 0, 2024-09-16T02:06:05.737 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getvxattr_latency": { 2024-09-16T02:06:05.737 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.737 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.737 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.737 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-16T02:06:05.737 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.737 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_sessions": { 2024-09-16T02:06:05.738 INFO:teuthology.orchestra.run.smithi137.stdout: "session_count": 1, 2024-09-16T02:06:05.738 INFO:teuthology.orchestra.run.smithi137.stdout: "session_add": 2, 2024-09-16T02:06:05.738 INFO:teuthology.orchestra.run.smithi137.stdout: "session_remove": 1, 2024-09-16T02:06:05.738 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_open": 1, 2024-09-16T02:06:05.738 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_stale": 0, 2024-09-16T02:06:05.738 INFO:teuthology.orchestra.run.smithi137.stdout: "total_load": 0, 2024-09-16T02:06:05.738 INFO:teuthology.orchestra.run.smithi137.stdout: "average_load": 0, 2024-09-16T02:06:05.738 INFO:teuthology.orchestra.run.smithi137.stdout: "avg_session_uptime": 1231, 2024-09-16T02:06:05.738 INFO:teuthology.orchestra.run.smithi137.stdout: "mdthresh_evicted": 0 2024-09-16T02:06:05.738 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.738 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-09-16T02:06:05.738 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-09-16T02:06:05.738 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-09-16T02:06:05.739 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T02:06:05.739 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-09-16T02:06:05.739 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T02:06:05.739 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-09-16T02:06:05.739 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T02:06:05.739 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T02:06:05.739 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T02:06:05.739 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T02:06:05.739 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T02:06:05.739 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-09-16T02:06:05.739 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T02:06:05.739 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T02:06:05.739 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-09-16T02:06:05.739 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-09-16T02:06:05.740 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-09-16T02:06:05.740 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-09-16T02:06:05.740 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T02:06:05.740 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T02:06:05.740 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T02:06:05.740 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T02:06:05.740 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T02:06:05.740 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-09-16T02:06:05.740 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-09-16T02:06:05.740 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-09-16T02:06:05.740 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T02:06:05.740 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T02:06:05.740 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-09-16T02:06:05.740 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-09-16T02:06:05.741 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-09-16T02:06:05.741 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-09-16T02:06:05.741 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T02:06:05.741 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-09-16T02:06:05.741 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T02:06:05.741 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-09-16T02:06:05.741 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 189279, 2024-09-16T02:06:05.741 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 49, 2024-09-16T02:06:05.741 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 0, 2024-09-16T02:06:05.741 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 0, 2024-09-16T02:06:05.741 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-09-16T02:06:05.741 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-09-16T02:06:05.741 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-09-16T02:06:05.742 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-09-16T02:06:05.742 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-09-16T02:06:05.742 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-09-16T02:06:05.742 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 7888, 2024-09-16T02:06:05.742 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 163, 2024-09-16T02:06:05.742 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T02:06:05.742 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-09-16T02:06:05.742 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-09-16T02:06:05.742 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-09-16T02:06:05.742 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 56234, 2024-09-16T02:06:05.742 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 266, 2024-09-16T02:06:05.742 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-09-16T02:06:05.742 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-09-16T02:06:05.742 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-09-16T02:06:05.742 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-09-16T02:06:05.743 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.743 INFO:teuthology.orchestra.run.smithi137.stdout: "objecter": { 2024-09-16T02:06:05.743 INFO:teuthology.orchestra.run.smithi137.stdout: "op_active": 0, 2024-09-16T02:06:05.743 INFO:teuthology.orchestra.run.smithi137.stdout: "op_laggy": 0, 2024-09-16T02:06:05.743 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send": 1006, 2024-09-16T02:06:05.743 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send_bytes": 26, 2024-09-16T02:06:05.743 INFO:teuthology.orchestra.run.smithi137.stdout: "op_resend": 4, 2024-09-16T02:06:05.743 INFO:teuthology.orchestra.run.smithi137.stdout: "op_reply": 1006, 2024-09-16T02:06:05.743 INFO:teuthology.orchestra.run.smithi137.stdout: "op_latency": { 2024-09-16T02:06:05.743 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1006, 2024-09-16T02:06:05.743 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.389993030, 2024-09-16T02:06:05.743 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000387667 2024-09-16T02:06:05.743 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.743 INFO:teuthology.orchestra.run.smithi137.stdout: "op_inflight": 0, 2024-09-16T02:06:05.744 INFO:teuthology.orchestra.run.smithi137.stdout: "oplen_avg": { 2024-09-16T02:06:05.744 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1006, 2024-09-16T02:06:05.744 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 1007 2024-09-16T02:06:05.744 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.744 INFO:teuthology.orchestra.run.smithi137.stdout: "op": 1006, 2024-09-16T02:06:05.744 INFO:teuthology.orchestra.run.smithi137.stdout: "op_r": 1006, 2024-09-16T02:06:05.744 INFO:teuthology.orchestra.run.smithi137.stdout: "op_w": 0, 2024-09-16T02:06:05.744 INFO:teuthology.orchestra.run.smithi137.stdout: "op_rmw": 0, 2024-09-16T02:06:05.744 INFO:teuthology.orchestra.run.smithi137.stdout: "op_pg": 0, 2024-09-16T02:06:05.744 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_stat": 632, 2024-09-16T02:06:05.744 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_create": 0, 2024-09-16T02:06:05.744 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_read": 371, 2024-09-16T02:06:05.744 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_write": 0, 2024-09-16T02:06:05.745 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writefull": 0, 2024-09-16T02:06:05.745 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writesame": 0, 2024-09-16T02:06:05.745 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_append": 0, 2024-09-16T02:06:05.745 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_zero": 0, 2024-09-16T02:06:05.745 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_truncate": 0, 2024-09-16T02:06:05.745 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_delete": 0, 2024-09-16T02:06:05.745 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_mapext": 0, 2024-09-16T02:06:05.745 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_sparse_read": 0, 2024-09-16T02:06:05.745 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_clonerange": 0, 2024-09-16T02:06:05.745 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_getxattr": 2, 2024-09-16T02:06:05.745 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_setxattr": 0, 2024-09-16T02:06:05.745 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_cmpxattr": 0, 2024-09-16T02:06:05.745 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_rmxattr": 0, 2024-09-16T02:06:05.745 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_resetxattrs": 0, 2024-09-16T02:06:05.746 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_call": 0, 2024-09-16T02:06:05.746 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_watch": 0, 2024-09-16T02:06:05.746 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_notify": 0, 2024-09-16T02:06:05.746 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T02:06:05.746 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls": 0, 2024-09-16T02:06:05.746 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls_filter": 0, 2024-09-16T02:06:05.746 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_other": 0, 2024-09-16T02:06:05.746 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_active": 0, 2024-09-16T02:06:05.746 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_send": 0, 2024-09-16T02:06:05.746 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_resend": 0, 2024-09-16T02:06:05.746 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_ping": 0, 2024-09-16T02:06:05.746 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_active": 0, 2024-09-16T02:06:05.746 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_send": 0, 2024-09-16T02:06:05.746 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_resend": 0, 2024-09-16T02:06:05.746 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_active": 0, 2024-09-16T02:06:05.747 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_send": 0, 2024-09-16T02:06:05.747 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_resend": 0, 2024-09-16T02:06:05.747 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_active": 0, 2024-09-16T02:06:05.747 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_send": 0, 2024-09-16T02:06:05.747 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_resend": 0, 2024-09-16T02:06:05.747 INFO:teuthology.orchestra.run.smithi137.stdout: "command_active": 0, 2024-09-16T02:06:05.747 INFO:teuthology.orchestra.run.smithi137.stdout: "command_send": 0, 2024-09-16T02:06:05.747 INFO:teuthology.orchestra.run.smithi137.stdout: "command_resend": 0, 2024-09-16T02:06:05.747 INFO:teuthology.orchestra.run.smithi137.stdout: "map_epoch": 83, 2024-09-16T02:06:05.747 INFO:teuthology.orchestra.run.smithi137.stdout: "map_full": 0, 2024-09-16T02:06:05.747 INFO:teuthology.orchestra.run.smithi137.stdout: "map_inc": 4, 2024-09-16T02:06:05.747 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_sessions": 5, 2024-09-16T02:06:05.747 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_open": 5, 2024-09-16T02:06:05.747 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_close": 0, 2024-09-16T02:06:05.748 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_laggy": 0, 2024-09-16T02:06:05.748 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_wr": 0, 2024-09-16T02:06:05.748 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_rd": 2, 2024-09-16T02:06:05.748 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_del": 0 2024-09-16T02:06:05.748 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.748 INFO:teuthology.orchestra.run.smithi137.stdout: "oft": { 2024-09-16T02:06:05.748 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_objs": 0, 2024-09-16T02:06:05.748 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_kv_pairs": 0, 2024-09-16T02:06:05.748 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_updates": 0, 2024-09-16T02:06:05.748 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_removes": 0 2024-09-16T02:06:05.748 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.748 INFO:teuthology.orchestra.run.smithi137.stdout: "osd-slow-ops": { 2024-09-16T02:06:05.748 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_ops_count": 0 2024-09-16T02:06:05.748 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.749 INFO:teuthology.orchestra.run.smithi137.stdout: "purge_queue": { 2024-09-16T02:06:05.749 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops": 0, 2024-09-16T02:06:05.749 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T02:06:05.749 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing": 0, 2024-09-16T02:06:05.749 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_high_water": 0, 2024-09-16T02:06:05.749 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed_ops": 0, 2024-09-16T02:06:05.749 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed": 0, 2024-09-16T02:06:05.749 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_item_in_journal": 0 2024-09-16T02:06:05.749 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.749 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T02:06:05.749 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-16T02:06:05.749 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-09-16T02:06:05.749 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-16T02:06:05.749 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 1122, 2024-09-16T02:06:05.750 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 1036148, 2024-09-16T02:06:05.750 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:05.750 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 1122, 2024-09-16T02:06:05.750 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-09-16T02:06:05.750 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-09-16T02:06:05.750 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 1122, 2024-09-16T02:06:05.750 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 1036148, 2024-09-16T02:06:05.750 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-16T02:06:05.750 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.750 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.750 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.750 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-16T02:06:05.750 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.750 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_bytes": { 2024-09-16T02:06:05.751 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-16T02:06:05.751 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-09-16T02:06:05.751 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-16T02:06:05.751 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-09-16T02:06:05.751 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-09-16T02:06:05.751 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:05.751 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-09-16T02:06:05.751 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 1006, 2024-09-16T02:06:05.751 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 388857, 2024-09-16T02:06:05.751 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 1006, 2024-09-16T02:06:05.751 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 388857, 2024-09-16T02:06:05.751 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-16T02:06:05.751 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.751 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.752 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.752 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-16T02:06:05.752 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.752 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_ops": { 2024-09-16T02:06:05.752 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-16T02:06:05.752 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 1024, 2024-09-16T02:06:05.752 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-16T02:06:05.752 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-09-16T02:06:05.752 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-09-16T02:06:05.752 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:05.752 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-09-16T02:06:05.752 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 1006, 2024-09-16T02:06:05.752 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 1006, 2024-09-16T02:06:05.753 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 1006, 2024-09-16T02:06:05.753 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 1006, 2024-09-16T02:06:05.753 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-16T02:06:05.753 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.753 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.753 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.753 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-16T02:06:05.753 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.753 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle": { 2024-09-16T02:06:05.753 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-16T02:06:05.753 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-09-16T02:06:05.753 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-16T02:06:05.753 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-09-16T02:06:05.753 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-09-16T02:06:05.754 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:05.754 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-09-16T02:06:05.754 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-09-16T02:06:05.754 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-09-16T02:06:05.754 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 0, 2024-09-16T02:06:05.754 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 0, 2024-09-16T02:06:05.754 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-16T02:06:05.754 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.754 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.754 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.754 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-16T02:06:05.754 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:05.754 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle-0x55af84b1eb40": { 2024-09-16T02:06:05.754 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-16T02:06:05.755 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-09-16T02:06:05.755 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-16T02:06:05.755 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-09-16T02:06:05.755 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-09-16T02:06:05.755 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:05.755 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-09-16T02:06:05.755 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-09-16T02:06:05.755 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-09-16T02:06:05.755 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 0, 2024-09-16T02:06:05.755 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 0, 2024-09-16T02:06:05.755 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-16T02:06:05.755 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:05.755 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:05.755 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:05.755 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-16T02:06:05.756 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-16T02:06:05.756 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-16T02:06:06.522 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-16T02:06:06.522 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-16T02:06:06.522 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-09-16T02:06:06.523 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-09-16T02:06:06.523 DEBUG:tasks.check_counter:Getting stats from h 2024-09-16T02:06:06.523 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-16T02:06:06.770 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T02:06:06.891 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:06 smithi137 ceph-mon[34099]: pgmap v446: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-16T02:06:07.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:06 smithi105 ceph-mon[31197]: pgmap v446: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-16T02:06:07.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:06 smithi190 ceph-mon[34762]: pgmap v446: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-16T02:06:07.417 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-16T02:06:07.417 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T02:06:07.417 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 190, 2024-09-16T02:06:07.417 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 178, 2024-09-16T02:06:07.417 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 513932, 2024-09-16T02:06:07.417 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 111041, 2024-09-16T02:06:07.417 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 13, 2024-09-16T02:06:07.417 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 2, 2024-09-16T02:06:07.418 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.027831050, 2024-09-16T02:06:07.418 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.008028816, 2024-09-16T02:06:07.418 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.023355005, 2024-09-16T02:06:07.418 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000376197, 2024-09-16T02:06:07.418 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:06:07.418 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 178, 2024-09-16T02:06:07.418 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.026155713, 2024-09-16T02:06:07.418 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000146942 2024-09-16T02:06:07.418 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.418 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:06:07.418 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 49, 2024-09-16T02:06:07.418 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000008719, 2024-09-16T02:06:07.418 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000177 2024-09-16T02:06:07.419 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.419 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 497696, 2024-09-16T02:06:07.419 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 66656 2024-09-16T02:06:07.419 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.419 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T02:06:07.419 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 44, 2024-09-16T02:06:07.419 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 117, 2024-09-16T02:06:07.419 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 16980, 2024-09-16T02:06:07.419 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 374333, 2024-09-16T02:06:07.419 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 20, 2024-09-16T02:06:07.419 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 5, 2024-09-16T02:06:07.419 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.013639054, 2024-09-16T02:06:07.420 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.004230312, 2024-09-16T02:06:07.420 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.007953746, 2024-09-16T02:06:07.420 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000379931, 2024-09-16T02:06:07.420 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:06:07.420 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 115, 2024-09-16T02:06:07.420 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.020506364, 2024-09-16T02:06:07.420 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000178316 2024-09-16T02:06:07.420 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.420 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:06:07.420 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 38, 2024-09-16T02:06:07.420 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000006572, 2024-09-16T02:06:07.420 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000172 2024-09-16T02:06:07.421 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.421 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-16T02:06:07.421 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 204224 2024-09-16T02:06:07.421 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.421 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T02:06:07.421 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 241, 2024-09-16T02:06:07.421 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 274, 2024-09-16T02:06:07.421 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 83821, 2024-09-16T02:06:07.421 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 117077, 2024-09-16T02:06:07.421 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 8, 2024-09-16T02:06:07.421 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 5, 2024-09-16T02:06:07.421 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.032125283, 2024-09-16T02:06:07.421 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.013319569, 2024-09-16T02:06:07.422 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.017500634, 2024-09-16T02:06:07.422 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.002181894, 2024-09-16T02:06:07.422 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:06:07.422 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 272, 2024-09-16T02:06:07.422 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.029072715, 2024-09-16T02:06:07.422 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000106884 2024-09-16T02:06:07.422 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.422 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:06:07.422 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 477, 2024-09-16T02:06:07.422 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000099279, 2024-09-16T02:06:07.422 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000208 2024-09-16T02:06:07.422 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.422 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T02:06:07.422 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T02:06:07.423 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.423 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-09-16T02:06:07.423 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 1, 2024-09-16T02:06:07.423 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-09-16T02:06:07.423 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.423 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-MDSRank": { 2024-09-16T02:06:07.423 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-09-16T02:06:07.423 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-09-16T02:06:07.423 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 37, 2024-09-16T02:06:07.423 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.001853268, 2024-09-16T02:06:07.423 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000050088 2024-09-16T02:06:07.423 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-16T02:06:07.423 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.423 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-PurgeQueue": { 2024-09-16T02:06:07.424 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-09-16T02:06:07.424 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-09-16T02:06:07.424 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2, 2024-09-16T02:06:07.424 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000014971, 2024-09-16T02:06:07.424 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000007485 2024-09-16T02:06:07.424 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-16T02:06:07.424 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.424 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-09-16T02:06:07.424 INFO:teuthology.orchestra.run.smithi137.stdout: "request": 3, 2024-09-16T02:06:07.424 INFO:teuthology.orchestra.run.smithi137.stdout: "reply": 2, 2024-09-16T02:06:07.424 INFO:teuthology.orchestra.run.smithi137.stdout: "reply_latency": { 2024-09-16T02:06:07.424 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2, 2024-09-16T02:06:07.424 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000360202, 2024-09-16T02:06:07.425 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000180101 2024-09-16T02:06:07.425 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.425 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_reply": 0, 2024-09-16T02:06:07.425 INFO:teuthology.orchestra.run.smithi137.stdout: "forward": 1, 2024-09-16T02:06:07.425 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_complete": 0, 2024-09-16T02:06:07.425 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_keys": 0, 2024-09-16T02:06:07.425 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_commit": 11, 2024-09-16T02:06:07.425 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_split": 0, 2024-09-16T02:06:07.425 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_merge": 0, 2024-09-16T02:06:07.426 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes": 16, 2024-09-16T02:06:07.426 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_top": 0, 2024-09-16T02:06:07.426 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_bottom": 16, 2024-09-16T02:06:07.426 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pin_tail": 0, 2024-09-16T02:06:07.426 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pinned": 13, 2024-09-16T02:06:07.426 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_expired": 4, 2024-09-16T02:06:07.426 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_with_caps": 1, 2024-09-16T02:06:07.426 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": 1, 2024-09-16T02:06:07.426 INFO:teuthology.orchestra.run.smithi137.stdout: "subtrees": 3, 2024-09-16T02:06:07.426 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse": 11, 2024-09-16T02:06:07.426 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_hit": 6, 2024-09-16T02:06:07.426 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_forward": 0, 2024-09-16T02:06:07.426 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_discover": 4, 2024-09-16T02:06:07.426 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_dir_fetch": 0, 2024-09-16T02:06:07.427 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_remote_ino": 0, 2024-09-16T02:06:07.427 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_lock": 0, 2024-09-16T02:06:07.427 INFO:teuthology.orchestra.run.smithi137.stdout: "load_cent": 0, 2024-09-16T02:06:07.427 INFO:teuthology.orchestra.run.smithi137.stdout: "q": 0, 2024-09-16T02:06:07.427 INFO:teuthology.orchestra.run.smithi137.stdout: "exported": 0, 2024-09-16T02:06:07.427 INFO:teuthology.orchestra.run.smithi137.stdout: "exported_inodes": 0, 2024-09-16T02:06:07.427 INFO:teuthology.orchestra.run.smithi137.stdout: "imported": 0, 2024-09-16T02:06:07.427 INFO:teuthology.orchestra.run.smithi137.stdout: "imported_inodes": 0, 2024-09-16T02:06:07.427 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_dir_fetch": 0, 2024-09-16T02:06:07.427 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_backtrace_fetch": 0, 2024-09-16T02:06:07.428 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_peer_discover": 0, 2024-09-16T02:06:07.428 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rfiles": 4, 2024-09-16T02:06:07.428 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rbytes": 234, 2024-09-16T02:06:07.428 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rsnaps": 0, 2024-09-16T02:06:07.428 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T02:06:07.428 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_set_tag": 0, 2024-09-16T02:06:07.428 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T02:06:07.428 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_inotable_repaired": 0, 2024-09-16T02:06:07.428 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_inodes": 0, 2024-09-16T02:06:07.428 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T02:06:07.428 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T02:06:07.428 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_file_inodes": 0, 2024-09-16T02:06:07.428 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_inode_file_caps": 0, 2024-09-16T02:06:07.429 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T02:06:07.429 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_grant": 0, 2024-09-16T02:06:07.429 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T02:06:07.429 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T02:06:07.429 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T02:06:07.429 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps": 0, 2024-09-16T02:06:07.429 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps_dirty": 0, 2024-09-16T02:06:07.429 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_cap_release": 0, 2024-09-16T02:06:07.430 INFO:teuthology.orchestra.run.smithi137.stdout: "process_request_cap_release": 1 2024-09-16T02:06:07.430 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.430 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_cache": { 2024-09-16T02:06:07.430 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update": 0, 2024-09-16T02:06:07.430 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update_receipt": 9, 2024-09-16T02:06:07.430 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_try_discover": 6, 2024-09-16T02:06:07.430 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_send_discover": 8, 2024-09-16T02:06:07.430 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_handle_discover": 0, 2024-09-16T02:06:07.430 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays": 0, 2024-09-16T02:06:07.430 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_delayed": 0, 2024-09-16T02:06:07.430 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_enqueuing": 0, 2024-09-16T02:06:07.430 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_created": 0, 2024-09-16T02:06:07.430 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_enqueued": 0, 2024-09-16T02:06:07.430 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_reintegrated": 0, 2024-09-16T02:06:07.431 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_migrated": 0, 2024-09-16T02:06:07.431 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_processing": 0, 2024-09-16T02:06:07.431 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_enqueued": 0, 2024-09-16T02:06:07.431 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_prioritized": 0, 2024-09-16T02:06:07.431 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_started": 0, 2024-09-16T02:06:07.431 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_completed": 0, 2024-09-16T02:06:07.431 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_path": 0, 2024-09-16T02:06:07.431 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_inode": 0, 2024-09-16T02:06:07.431 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T02:06:07.431 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_exportdir": 0, 2024-09-16T02:06:07.431 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_flush": 0, 2024-09-16T02:06:07.431 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragmentdir": 0, 2024-09-16T02:06:07.431 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragstats": 0, 2024-09-16T02:06:07.432 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_inodestats": 0 2024-09-16T02:06:07.432 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.432 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_log": { 2024-09-16T02:06:07.432 INFO:teuthology.orchestra.run.smithi137.stdout: "evlrg": 0, 2024-09-16T02:06:07.432 INFO:teuthology.orchestra.run.smithi137.stdout: "evadd": 5, 2024-09-16T02:06:07.432 INFO:teuthology.orchestra.run.smithi137.stdout: "evex": 0, 2024-09-16T02:06:07.432 INFO:teuthology.orchestra.run.smithi137.stdout: "evtrm": 0, 2024-09-16T02:06:07.432 INFO:teuthology.orchestra.run.smithi137.stdout: "ev": 5, 2024-09-16T02:06:07.432 INFO:teuthology.orchestra.run.smithi137.stdout: "evexg": 0, 2024-09-16T02:06:07.432 INFO:teuthology.orchestra.run.smithi137.stdout: "evexd": 0, 2024-09-16T02:06:07.432 INFO:teuthology.orchestra.run.smithi137.stdout: "segadd": 2, 2024-09-16T02:06:07.432 INFO:teuthology.orchestra.run.smithi137.stdout: "segex": 0, 2024-09-16T02:06:07.432 INFO:teuthology.orchestra.run.smithi137.stdout: "segtrm": 0, 2024-09-16T02:06:07.433 INFO:teuthology.orchestra.run.smithi137.stdout: "seg": 2, 2024-09-16T02:06:07.433 INFO:teuthology.orchestra.run.smithi137.stdout: "segmjr": 2, 2024-09-16T02:06:07.433 INFO:teuthology.orchestra.run.smithi137.stdout: "segexg": 0, 2024-09-16T02:06:07.433 INFO:teuthology.orchestra.run.smithi137.stdout: "segexd": 0, 2024-09-16T02:06:07.433 INFO:teuthology.orchestra.run.smithi137.stdout: "expos": 4194304, 2024-09-16T02:06:07.433 INFO:teuthology.orchestra.run.smithi137.stdout: "wrpos": 4195921, 2024-09-16T02:06:07.433 INFO:teuthology.orchestra.run.smithi137.stdout: "rdpos": 0, 2024-09-16T02:06:07.433 INFO:teuthology.orchestra.run.smithi137.stdout: "jlat": { 2024-09-16T02:06:07.433 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 4, 2024-09-16T02:06:07.433 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.006807127, 2024-09-16T02:06:07.433 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001701781 2024-09-16T02:06:07.433 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.433 INFO:teuthology.orchestra.run.smithi137.stdout: "replayed": 0 2024-09-16T02:06:07.433 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.434 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_mem": { 2024-09-16T02:06:07.434 INFO:teuthology.orchestra.run.smithi137.stdout: "ino": 20, 2024-09-16T02:06:07.434 INFO:teuthology.orchestra.run.smithi137.stdout: "ino+": 22, 2024-09-16T02:06:07.434 INFO:teuthology.orchestra.run.smithi137.stdout: "ino-": 2, 2024-09-16T02:06:07.434 INFO:teuthology.orchestra.run.smithi137.stdout: "dir": 19, 2024-09-16T02:06:07.434 INFO:teuthology.orchestra.run.smithi137.stdout: "dir+": 22, 2024-09-16T02:06:07.434 INFO:teuthology.orchestra.run.smithi137.stdout: "dir-": 3, 2024-09-16T02:06:07.434 INFO:teuthology.orchestra.run.smithi137.stdout: "dn": 16, 2024-09-16T02:06:07.434 INFO:teuthology.orchestra.run.smithi137.stdout: "dn+": 20, 2024-09-16T02:06:07.434 INFO:teuthology.orchestra.run.smithi137.stdout: "dn-": 4, 2024-09-16T02:06:07.434 INFO:teuthology.orchestra.run.smithi137.stdout: "cap": 1, 2024-09-16T02:06:07.434 INFO:teuthology.orchestra.run.smithi137.stdout: "cap+": 2, 2024-09-16T02:06:07.434 INFO:teuthology.orchestra.run.smithi137.stdout: "cap-": 1, 2024-09-16T02:06:07.434 INFO:teuthology.orchestra.run.smithi137.stdout: "rss": 50292, 2024-09-16T02:06:07.435 INFO:teuthology.orchestra.run.smithi137.stdout: "heap": 223520 2024-09-16T02:06:07.435 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.435 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_server": { 2024-09-16T02:06:07.435 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_client_request": 4, 2024-09-16T02:06:07.435 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_server_request": 0, 2024-09-16T02:06:07.435 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_request": 3, 2024-09-16T02:06:07.435 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_session": 9, 2024-09-16T02:06:07.435 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_peer_request": 0, 2024-09-16T02:06:07.435 INFO:teuthology.orchestra.run.smithi137.stdout: "req_create_latency": { 2024-09-16T02:06:07.435 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:07.435 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:07.435 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:07.435 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.435 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getattr_latency": { 2024-09-16T02:06:07.436 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2, 2024-09-16T02:06:07.436 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000360202, 2024-09-16T02:06:07.436 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000180101 2024-09-16T02:06:07.436 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.436 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getfilelock_latency": { 2024-09-16T02:06:07.436 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:07.436 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:07.436 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:07.436 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.436 INFO:teuthology.orchestra.run.smithi137.stdout: "req_link_latency": { 2024-09-16T02:06:07.436 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:07.436 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:07.436 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:07.436 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.436 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookup_latency": { 2024-09-16T02:06:07.437 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:07.437 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:07.437 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:07.437 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.437 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookuphash_latency": { 2024-09-16T02:06:07.437 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:07.437 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:07.437 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:07.437 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.437 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupino_latency": { 2024-09-16T02:06:07.437 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:07.437 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:07.437 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:07.437 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.438 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupname_latency": { 2024-09-16T02:06:07.438 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:07.438 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:07.438 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:07.438 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.438 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupparent_latency": { 2024-09-16T02:06:07.438 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:07.438 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:07.439 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:07.439 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.439 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupsnap_latency": { 2024-09-16T02:06:07.439 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:07.439 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:07.439 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:07.439 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.439 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lssnap_latency": { 2024-09-16T02:06:07.439 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:07.439 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:07.439 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:07.439 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.439 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mkdir_latency": { 2024-09-16T02:06:07.439 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:07.440 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:07.440 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:07.440 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.440 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mknod_latency": { 2024-09-16T02:06:07.441 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:07.441 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:07.441 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:07.441 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.441 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mksnap_latency": { 2024-09-16T02:06:07.441 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:07.441 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:07.441 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:07.441 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.441 INFO:teuthology.orchestra.run.smithi137.stdout: "req_open_latency": { 2024-09-16T02:06:07.441 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:07.441 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:07.441 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:07.442 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.442 INFO:teuthology.orchestra.run.smithi137.stdout: "req_readdir_latency": { 2024-09-16T02:06:07.442 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:07.442 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:07.442 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:07.442 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.442 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rename_latency": { 2024-09-16T02:06:07.442 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:07.442 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:07.442 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:07.442 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.443 INFO:teuthology.orchestra.run.smithi137.stdout: "req_renamesnap_latency": { 2024-09-16T02:06:07.443 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:07.443 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:07.443 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:07.443 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.443 INFO:teuthology.orchestra.run.smithi137.stdout: "req_snapdiff_latency": { 2024-09-16T02:06:07.443 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:07.443 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:07.443 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:07.443 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.443 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmdir_latency": { 2024-09-16T02:06:07.443 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:07.443 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:07.444 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:07.444 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.444 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmsnap_latency": { 2024-09-16T02:06:07.444 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:07.444 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:07.444 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:07.444 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.444 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmxattr_latency": { 2024-09-16T02:06:07.444 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:07.444 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:07.444 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:07.444 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.444 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setattr_latency": { 2024-09-16T02:06:07.444 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:07.445 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:07.445 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:07.445 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.445 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setdirlayout_latency": { 2024-09-16T02:06:07.445 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:07.445 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:07.445 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:07.445 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.445 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setfilelock_latency": { 2024-09-16T02:06:07.445 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:07.445 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:07.446 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:07.446 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.446 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setlayout_latency": { 2024-09-16T02:06:07.446 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:07.446 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:07.446 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:07.446 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.446 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setxattr_latency": { 2024-09-16T02:06:07.446 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:07.446 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:07.446 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:07.446 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.447 INFO:teuthology.orchestra.run.smithi137.stdout: "req_symlink_latency": { 2024-09-16T02:06:07.447 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:07.447 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:07.447 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:07.447 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.447 INFO:teuthology.orchestra.run.smithi137.stdout: "req_unlink_latency": { 2024-09-16T02:06:07.447 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:07.447 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:07.447 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:07.447 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.447 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_revoke_eviction": 0, 2024-09-16T02:06:07.447 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_acquisition_throttle": 0, 2024-09-16T02:06:07.447 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getvxattr_latency": { 2024-09-16T02:06:07.447 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:07.448 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:07.448 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:07.448 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-16T02:06:07.448 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.448 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_sessions": { 2024-09-16T02:06:07.448 INFO:teuthology.orchestra.run.smithi137.stdout: "session_count": 1, 2024-09-16T02:06:07.448 INFO:teuthology.orchestra.run.smithi137.stdout: "session_add": 2, 2024-09-16T02:06:07.448 INFO:teuthology.orchestra.run.smithi137.stdout: "session_remove": 1, 2024-09-16T02:06:07.448 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_open": 1, 2024-09-16T02:06:07.448 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_stale": 0, 2024-09-16T02:06:07.448 INFO:teuthology.orchestra.run.smithi137.stdout: "total_load": 1, 2024-09-16T02:06:07.449 INFO:teuthology.orchestra.run.smithi137.stdout: "average_load": 1, 2024-09-16T02:06:07.449 INFO:teuthology.orchestra.run.smithi137.stdout: "avg_session_uptime": 203, 2024-09-16T02:06:07.449 INFO:teuthology.orchestra.run.smithi137.stdout: "mdthresh_evicted": 0 2024-09-16T02:06:07.449 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.449 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-09-16T02:06:07.449 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-09-16T02:06:07.449 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-09-16T02:06:07.449 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T02:06:07.449 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-09-16T02:06:07.449 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T02:06:07.449 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-09-16T02:06:07.449 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T02:06:07.449 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T02:06:07.449 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T02:06:07.450 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T02:06:07.450 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T02:06:07.450 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-09-16T02:06:07.450 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T02:06:07.450 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T02:06:07.450 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-09-16T02:06:07.450 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-09-16T02:06:07.450 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-09-16T02:06:07.450 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-09-16T02:06:07.450 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T02:06:07.450 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T02:06:07.450 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T02:06:07.450 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T02:06:07.452 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T02:06:07.453 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-09-16T02:06:07.453 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-09-16T02:06:07.453 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-09-16T02:06:07.453 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T02:06:07.453 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T02:06:07.453 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-09-16T02:06:07.453 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-09-16T02:06:07.453 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-09-16T02:06:07.453 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-09-16T02:06:07.453 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T02:06:07.453 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-09-16T02:06:07.453 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T02:06:07.453 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-09-16T02:06:07.454 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 254838, 2024-09-16T02:06:07.454 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 87, 2024-09-16T02:06:07.454 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 0, 2024-09-16T02:06:07.454 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 0, 2024-09-16T02:06:07.454 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-09-16T02:06:07.454 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-09-16T02:06:07.454 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-09-16T02:06:07.454 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-09-16T02:06:07.454 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-09-16T02:06:07.454 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-09-16T02:06:07.454 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 7888, 2024-09-16T02:06:07.454 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 163, 2024-09-16T02:06:07.454 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T02:06:07.454 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-09-16T02:06:07.455 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-09-16T02:06:07.455 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-09-16T02:06:07.455 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 81325, 2024-09-16T02:06:07.455 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 328, 2024-09-16T02:06:07.455 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-09-16T02:06:07.455 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-09-16T02:06:07.455 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-09-16T02:06:07.455 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-09-16T02:06:07.455 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.455 INFO:teuthology.orchestra.run.smithi137.stdout: "objecter": { 2024-09-16T02:06:07.455 INFO:teuthology.orchestra.run.smithi137.stdout: "op_active": 0, 2024-09-16T02:06:07.455 INFO:teuthology.orchestra.run.smithi137.stdout: "op_laggy": 0, 2024-09-16T02:06:07.455 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send": 41, 2024-09-16T02:06:07.455 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send_bytes": 12013, 2024-09-16T02:06:07.456 INFO:teuthology.orchestra.run.smithi137.stdout: "op_resend": 28, 2024-09-16T02:06:07.456 INFO:teuthology.orchestra.run.smithi137.stdout: "op_reply": 41, 2024-09-16T02:06:07.456 INFO:teuthology.orchestra.run.smithi137.stdout: "op_latency": { 2024-09-16T02:06:07.456 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 41, 2024-09-16T02:06:07.456 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.200996451, 2024-09-16T02:06:07.456 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.004902352 2024-09-16T02:06:07.456 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.456 INFO:teuthology.orchestra.run.smithi137.stdout: "op_inflight": 0, 2024-09-16T02:06:07.457 INFO:teuthology.orchestra.run.smithi137.stdout: "oplen_avg": { 2024-09-16T02:06:07.457 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 41, 2024-09-16T02:06:07.457 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 64 2024-09-16T02:06:07.457 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.457 INFO:teuthology.orchestra.run.smithi137.stdout: "op": 41, 2024-09-16T02:06:07.457 INFO:teuthology.orchestra.run.smithi137.stdout: "op_r": 0, 2024-09-16T02:06:07.457 INFO:teuthology.orchestra.run.smithi137.stdout: "op_w": 41, 2024-09-16T02:06:07.457 INFO:teuthology.orchestra.run.smithi137.stdout: "op_rmw": 0, 2024-09-16T02:06:07.457 INFO:teuthology.orchestra.run.smithi137.stdout: "op_pg": 0, 2024-09-16T02:06:07.457 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_stat": 0, 2024-09-16T02:06:07.458 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_create": 10, 2024-09-16T02:06:07.458 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_read": 0, 2024-09-16T02:06:07.458 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_write": 4, 2024-09-16T02:06:07.458 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writefull": 7, 2024-09-16T02:06:07.458 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writesame": 0, 2024-09-16T02:06:07.458 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_append": 0, 2024-09-16T02:06:07.458 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_zero": 0, 2024-09-16T02:06:07.458 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_truncate": 0, 2024-09-16T02:06:07.458 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_delete": 6, 2024-09-16T02:06:07.458 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_mapext": 0, 2024-09-16T02:06:07.458 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_sparse_read": 0, 2024-09-16T02:06:07.458 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_clonerange": 0, 2024-09-16T02:06:07.458 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_getxattr": 0, 2024-09-16T02:06:07.459 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_setxattr": 20, 2024-09-16T02:06:07.459 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_cmpxattr": 0, 2024-09-16T02:06:07.459 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_rmxattr": 0, 2024-09-16T02:06:07.459 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_resetxattrs": 0, 2024-09-16T02:06:07.459 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_call": 0, 2024-09-16T02:06:07.459 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_watch": 0, 2024-09-16T02:06:07.459 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_notify": 0, 2024-09-16T02:06:07.459 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T02:06:07.459 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls": 0, 2024-09-16T02:06:07.459 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls_filter": 0, 2024-09-16T02:06:07.459 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_other": 0, 2024-09-16T02:06:07.459 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_active": 0, 2024-09-16T02:06:07.460 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_send": 0, 2024-09-16T02:06:07.460 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_resend": 0, 2024-09-16T02:06:07.460 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_ping": 0, 2024-09-16T02:06:07.460 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_active": 0, 2024-09-16T02:06:07.460 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_send": 0, 2024-09-16T02:06:07.460 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_resend": 0, 2024-09-16T02:06:07.460 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_active": 0, 2024-09-16T02:06:07.460 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_send": 0, 2024-09-16T02:06:07.460 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_resend": 0, 2024-09-16T02:06:07.460 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_active": 0, 2024-09-16T02:06:07.460 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_send": 0, 2024-09-16T02:06:07.460 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_resend": 0, 2024-09-16T02:06:07.460 INFO:teuthology.orchestra.run.smithi137.stdout: "command_active": 0, 2024-09-16T02:06:07.460 INFO:teuthology.orchestra.run.smithi137.stdout: "command_send": 0, 2024-09-16T02:06:07.461 INFO:teuthology.orchestra.run.smithi137.stdout: "command_resend": 0, 2024-09-16T02:06:07.461 INFO:teuthology.orchestra.run.smithi137.stdout: "map_epoch": 83, 2024-09-16T02:06:07.461 INFO:teuthology.orchestra.run.smithi137.stdout: "map_full": 0, 2024-09-16T02:06:07.461 INFO:teuthology.orchestra.run.smithi137.stdout: "map_inc": 4, 2024-09-16T02:06:07.461 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_sessions": 8, 2024-09-16T02:06:07.461 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_open": 8, 2024-09-16T02:06:07.461 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_close": 0, 2024-09-16T02:06:07.461 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_laggy": 0, 2024-09-16T02:06:07.461 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_wr": 16, 2024-09-16T02:06:07.461 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_rd": 0, 2024-09-16T02:06:07.461 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_del": 1 2024-09-16T02:06:07.461 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.461 INFO:teuthology.orchestra.run.smithi137.stdout: "oft": { 2024-09-16T02:06:07.461 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_objs": 1, 2024-09-16T02:06:07.462 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_kv_pairs": 0, 2024-09-16T02:06:07.462 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_updates": 0, 2024-09-16T02:06:07.462 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_removes": 0 2024-09-16T02:06:07.462 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.462 INFO:teuthology.orchestra.run.smithi137.stdout: "osd-slow-ops": { 2024-09-16T02:06:07.462 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_ops_count": 0 2024-09-16T02:06:07.462 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.462 INFO:teuthology.orchestra.run.smithi137.stdout: "purge_queue": { 2024-09-16T02:06:07.462 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops": 0, 2024-09-16T02:06:07.462 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T02:06:07.462 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing": 0, 2024-09-16T02:06:07.462 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_high_water": 0, 2024-09-16T02:06:07.462 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed_ops": 0, 2024-09-16T02:06:07.462 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed": 0, 2024-09-16T02:06:07.463 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_item_in_journal": 0 2024-09-16T02:06:07.463 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.463 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T02:06:07.463 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-16T02:06:07.463 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-09-16T02:06:07.463 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-16T02:06:07.463 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 471, 2024-09-16T02:06:07.463 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 561974, 2024-09-16T02:06:07.463 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:07.463 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 471, 2024-09-16T02:06:07.463 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-09-16T02:06:07.463 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-09-16T02:06:07.463 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 471, 2024-09-16T02:06:07.463 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 561974, 2024-09-16T02:06:07.463 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-16T02:06:07.464 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:07.464 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:07.464 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:07.464 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-16T02:06:07.464 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.464 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_bytes": { 2024-09-16T02:06:07.464 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-16T02:06:07.464 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-09-16T02:06:07.464 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-16T02:06:07.464 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-09-16T02:06:07.464 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-09-16T02:06:07.464 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:07.464 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-09-16T02:06:07.464 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 41, 2024-09-16T02:06:07.465 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 12013, 2024-09-16T02:06:07.465 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 41, 2024-09-16T02:06:07.465 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 12013, 2024-09-16T02:06:07.465 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-16T02:06:07.465 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:07.465 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:07.465 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:07.465 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-16T02:06:07.465 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.465 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_ops": { 2024-09-16T02:06:07.465 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-16T02:06:07.465 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 1024, 2024-09-16T02:06:07.465 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-16T02:06:07.466 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-09-16T02:06:07.466 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-09-16T02:06:07.466 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:07.466 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-09-16T02:06:07.466 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 41, 2024-09-16T02:06:07.466 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 41, 2024-09-16T02:06:07.466 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 41, 2024-09-16T02:06:07.466 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 41, 2024-09-16T02:06:07.466 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-16T02:06:07.466 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:07.466 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:07.466 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:07.466 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-16T02:06:07.466 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.466 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle": { 2024-09-16T02:06:07.467 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-16T02:06:07.467 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-09-16T02:06:07.467 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-16T02:06:07.467 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-09-16T02:06:07.467 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-09-16T02:06:07.467 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:07.467 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-09-16T02:06:07.467 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-09-16T02:06:07.467 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-09-16T02:06:07.467 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 0, 2024-09-16T02:06:07.468 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 0, 2024-09-16T02:06:07.468 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-16T02:06:07.468 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:07.468 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:07.468 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:07.468 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-16T02:06:07.468 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:07.468 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle-0x55cb18d0e960": { 2024-09-16T02:06:07.468 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-16T02:06:07.468 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-09-16T02:06:07.468 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-16T02:06:07.468 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 5, 2024-09-16T02:06:07.468 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 1617, 2024-09-16T02:06:07.468 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:07.469 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 5, 2024-09-16T02:06:07.469 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-09-16T02:06:07.469 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-09-16T02:06:07.469 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 4, 2024-09-16T02:06:07.469 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 1617, 2024-09-16T02:06:07.469 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-16T02:06:07.469 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:07.469 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:07.469 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:07.469 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-16T02:06:07.469 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-16T02:06:07.469 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-16T02:06:08.309 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-16T02:06:08.533 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-16T02:06:08.533 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-09-16T02:06:08.533 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=9 2024-09-16T02:06:08.533 DEBUG:tasks.check_counter:Getting stats from k 2024-09-16T02:06:08.533 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-16T02:06:08.781 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T02:06:08.902 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:08 smithi137 ceph-mon[34099]: pgmap v447: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-16T02:06:09.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:08 smithi105 ceph-mon[31197]: pgmap v447: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-16T02:06:09.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:08 smithi190 ceph-mon[34762]: pgmap v447: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-16T02:06:09.304 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-16T02:06:09.304 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T02:06:09.304 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 860, 2024-09-16T02:06:09.304 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 859, 2024-09-16T02:06:09.304 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 249456, 2024-09-16T02:06:09.304 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 272345, 2024-09-16T02:06:09.304 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 6, 2024-09-16T02:06:09.305 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 2, 2024-09-16T02:06:09.305 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.067508098, 2024-09-16T02:06:09.305 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.022008702, 2024-09-16T02:06:09.305 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.018540684, 2024-09-16T02:06:09.305 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.021357056, 2024-09-16T02:06:09.305 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:06:09.305 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 858, 2024-09-16T02:06:09.305 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.031239994, 2024-09-16T02:06:09.305 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000036410 2024-09-16T02:06:09.305 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.305 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:06:09.305 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.305 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.306 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.306 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.306 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-16T02:06:09.306 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 448 2024-09-16T02:06:09.306 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.306 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T02:06:09.306 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 3, 2024-09-16T02:06:09.306 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 80, 2024-09-16T02:06:09.306 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 716, 2024-09-16T02:06:09.306 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 205626, 2024-09-16T02:06:09.306 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 4, 2024-09-16T02:06:09.306 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 3, 2024-09-16T02:06:09.306 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.005749206, 2024-09-16T02:06:09.306 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.003392370, 2024-09-16T02:06:09.307 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.001470378, 2024-09-16T02:06:09.307 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000079289, 2024-09-16T02:06:09.307 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:06:09.307 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 78, 2024-09-16T02:06:09.307 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.005309290, 2024-09-16T02:06:09.307 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000068067 2024-09-16T02:06:09.307 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.307 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:06:09.307 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.307 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.307 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.307 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.307 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-16T02:06:09.308 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 204928 2024-09-16T02:06:09.308 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.308 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T02:06:09.308 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 118, 2024-09-16T02:06:09.308 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 114, 2024-09-16T02:06:09.308 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 472380, 2024-09-16T02:06:09.308 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 64622, 2024-09-16T02:06:09.308 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 3, 2024-09-16T02:06:09.308 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 2, 2024-09-16T02:06:09.308 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.017951644, 2024-09-16T02:06:09.308 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.006975182, 2024-09-16T02:06:09.308 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.016279014, 2024-09-16T02:06:09.308 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000024735, 2024-09-16T02:06:09.309 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:06:09.309 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 113, 2024-09-16T02:06:09.309 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.006998323, 2024-09-16T02:06:09.309 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000061932 2024-09-16T02:06:09.309 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.309 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:06:09.309 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.309 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.309 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.310 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.310 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 472112, 2024-09-16T02:06:09.310 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 64304 2024-09-16T02:06:09.310 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.310 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-09-16T02:06:09.310 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 1, 2024-09-16T02:06:09.310 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-09-16T02:06:09.310 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.310 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-MDSRank": { 2024-09-16T02:06:09.310 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-09-16T02:06:09.310 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-09-16T02:06:09.310 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1496, 2024-09-16T02:06:09.310 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.034434821, 2024-09-16T02:06:09.310 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000023017 2024-09-16T02:06:09.311 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-16T02:06:09.311 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.311 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-PurgeQueue": { 2024-09-16T02:06:09.311 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-09-16T02:06:09.311 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-09-16T02:06:09.311 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.311 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.311 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.311 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-16T02:06:09.311 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.311 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-09-16T02:06:09.311 INFO:teuthology.orchestra.run.smithi137.stdout: "request": 0, 2024-09-16T02:06:09.311 INFO:teuthology.orchestra.run.smithi137.stdout: "reply": 0, 2024-09-16T02:06:09.311 INFO:teuthology.orchestra.run.smithi137.stdout: "reply_latency": { 2024-09-16T02:06:09.312 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.312 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.312 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.312 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.312 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_reply": 0, 2024-09-16T02:06:09.312 INFO:teuthology.orchestra.run.smithi137.stdout: "forward": 0, 2024-09-16T02:06:09.312 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_complete": 0, 2024-09-16T02:06:09.312 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_keys": 0, 2024-09-16T02:06:09.312 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_commit": 0, 2024-09-16T02:06:09.312 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_split": 0, 2024-09-16T02:06:09.312 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_merge": 0, 2024-09-16T02:06:09.312 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes": 0, 2024-09-16T02:06:09.312 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_top": 0, 2024-09-16T02:06:09.313 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_bottom": 0, 2024-09-16T02:06:09.313 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pin_tail": 0, 2024-09-16T02:06:09.313 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pinned": 0, 2024-09-16T02:06:09.313 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_expired": 0, 2024-09-16T02:06:09.313 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_with_caps": 0, 2024-09-16T02:06:09.313 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": 0, 2024-09-16T02:06:09.313 INFO:teuthology.orchestra.run.smithi137.stdout: "subtrees": 1, 2024-09-16T02:06:09.313 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse": 0, 2024-09-16T02:06:09.313 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_hit": 0, 2024-09-16T02:06:09.313 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_forward": 0, 2024-09-16T02:06:09.313 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_discover": 0, 2024-09-16T02:06:09.313 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_dir_fetch": 0, 2024-09-16T02:06:09.313 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_remote_ino": 0, 2024-09-16T02:06:09.314 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_lock": 0, 2024-09-16T02:06:09.314 INFO:teuthology.orchestra.run.smithi137.stdout: "load_cent": 0, 2024-09-16T02:06:09.314 INFO:teuthology.orchestra.run.smithi137.stdout: "q": 0, 2024-09-16T02:06:09.314 INFO:teuthology.orchestra.run.smithi137.stdout: "exported": 0, 2024-09-16T02:06:09.314 INFO:teuthology.orchestra.run.smithi137.stdout: "exported_inodes": 0, 2024-09-16T02:06:09.314 INFO:teuthology.orchestra.run.smithi137.stdout: "imported": 0, 2024-09-16T02:06:09.314 INFO:teuthology.orchestra.run.smithi137.stdout: "imported_inodes": 0, 2024-09-16T02:06:09.314 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_dir_fetch": 0, 2024-09-16T02:06:09.314 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_backtrace_fetch": 0, 2024-09-16T02:06:09.314 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_peer_discover": 0, 2024-09-16T02:06:09.314 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rfiles": 0, 2024-09-16T02:06:09.314 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rbytes": 0, 2024-09-16T02:06:09.315 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rsnaps": 0, 2024-09-16T02:06:09.315 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T02:06:09.315 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_set_tag": 0, 2024-09-16T02:06:09.315 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T02:06:09.315 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_inotable_repaired": 0, 2024-09-16T02:06:09.315 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_inodes": 0, 2024-09-16T02:06:09.315 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T02:06:09.315 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T02:06:09.315 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_file_inodes": 0, 2024-09-16T02:06:09.315 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_inode_file_caps": 0, 2024-09-16T02:06:09.315 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T02:06:09.315 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_grant": 0, 2024-09-16T02:06:09.315 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T02:06:09.315 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T02:06:09.316 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T02:06:09.316 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps": 0, 2024-09-16T02:06:09.316 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps_dirty": 0, 2024-09-16T02:06:09.316 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_cap_release": 0, 2024-09-16T02:06:09.316 INFO:teuthology.orchestra.run.smithi137.stdout: "process_request_cap_release": 0 2024-09-16T02:06:09.316 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.316 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_cache": { 2024-09-16T02:06:09.316 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update": 0, 2024-09-16T02:06:09.316 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update_receipt": 0, 2024-09-16T02:06:09.316 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_try_discover": 0, 2024-09-16T02:06:09.316 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_send_discover": 0, 2024-09-16T02:06:09.316 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_handle_discover": 0, 2024-09-16T02:06:09.316 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays": 0, 2024-09-16T02:06:09.316 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_delayed": 0, 2024-09-16T02:06:09.317 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_enqueuing": 0, 2024-09-16T02:06:09.317 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_created": 0, 2024-09-16T02:06:09.317 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_enqueued": 0, 2024-09-16T02:06:09.317 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_reintegrated": 0, 2024-09-16T02:06:09.317 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_migrated": 0, 2024-09-16T02:06:09.317 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_processing": 0, 2024-09-16T02:06:09.317 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_enqueued": 0, 2024-09-16T02:06:09.317 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_prioritized": 0, 2024-09-16T02:06:09.317 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_started": 0, 2024-09-16T02:06:09.317 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_completed": 0, 2024-09-16T02:06:09.317 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_path": 0, 2024-09-16T02:06:09.318 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_inode": 0, 2024-09-16T02:06:09.318 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T02:06:09.318 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_exportdir": 0, 2024-09-16T02:06:09.318 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_flush": 0, 2024-09-16T02:06:09.318 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragmentdir": 0, 2024-09-16T02:06:09.318 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragstats": 0, 2024-09-16T02:06:09.318 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_inodestats": 0 2024-09-16T02:06:09.318 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.318 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_log": { 2024-09-16T02:06:09.318 INFO:teuthology.orchestra.run.smithi137.stdout: "evlrg": 0, 2024-09-16T02:06:09.318 INFO:teuthology.orchestra.run.smithi137.stdout: "evadd": 0, 2024-09-16T02:06:09.318 INFO:teuthology.orchestra.run.smithi137.stdout: "evex": 0, 2024-09-16T02:06:09.319 INFO:teuthology.orchestra.run.smithi137.stdout: "evtrm": 0, 2024-09-16T02:06:09.319 INFO:teuthology.orchestra.run.smithi137.stdout: "ev": 3, 2024-09-16T02:06:09.319 INFO:teuthology.orchestra.run.smithi137.stdout: "evexg": 0, 2024-09-16T02:06:09.319 INFO:teuthology.orchestra.run.smithi137.stdout: "evexd": 0, 2024-09-16T02:06:09.319 INFO:teuthology.orchestra.run.smithi137.stdout: "segadd": 0, 2024-09-16T02:06:09.319 INFO:teuthology.orchestra.run.smithi137.stdout: "segex": 0, 2024-09-16T02:06:09.319 INFO:teuthology.orchestra.run.smithi137.stdout: "segtrm": 0, 2024-09-16T02:06:09.319 INFO:teuthology.orchestra.run.smithi137.stdout: "seg": 2, 2024-09-16T02:06:09.319 INFO:teuthology.orchestra.run.smithi137.stdout: "segmjr": 2, 2024-09-16T02:06:09.319 INFO:teuthology.orchestra.run.smithi137.stdout: "segexg": 0, 2024-09-16T02:06:09.319 INFO:teuthology.orchestra.run.smithi137.stdout: "segexd": 0, 2024-09-16T02:06:09.319 INFO:teuthology.orchestra.run.smithi137.stdout: "expos": 4194304, 2024-09-16T02:06:09.319 INFO:teuthology.orchestra.run.smithi137.stdout: "wrpos": 4195387, 2024-09-16T02:06:09.319 INFO:teuthology.orchestra.run.smithi137.stdout: "rdpos": 4194892, 2024-09-16T02:06:09.320 INFO:teuthology.orchestra.run.smithi137.stdout: "jlat": { 2024-09-16T02:06:09.320 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.320 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.320 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.320 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.320 INFO:teuthology.orchestra.run.smithi137.stdout: "replayed": 3 2024-09-16T02:06:09.320 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.320 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_mem": { 2024-09-16T02:06:09.320 INFO:teuthology.orchestra.run.smithi137.stdout: "ino": 3, 2024-09-16T02:06:09.320 INFO:teuthology.orchestra.run.smithi137.stdout: "ino+": 3, 2024-09-16T02:06:09.320 INFO:teuthology.orchestra.run.smithi137.stdout: "ino-": 0, 2024-09-16T02:06:09.320 INFO:teuthology.orchestra.run.smithi137.stdout: "dir": 1, 2024-09-16T02:06:09.320 INFO:teuthology.orchestra.run.smithi137.stdout: "dir+": 1, 2024-09-16T02:06:09.321 INFO:teuthology.orchestra.run.smithi137.stdout: "dir-": 0, 2024-09-16T02:06:09.321 INFO:teuthology.orchestra.run.smithi137.stdout: "dn": 0, 2024-09-16T02:06:09.321 INFO:teuthology.orchestra.run.smithi137.stdout: "dn+": 0, 2024-09-16T02:06:09.321 INFO:teuthology.orchestra.run.smithi137.stdout: "dn-": 0, 2024-09-16T02:06:09.321 INFO:teuthology.orchestra.run.smithi137.stdout: "cap": 0, 2024-09-16T02:06:09.321 INFO:teuthology.orchestra.run.smithi137.stdout: "cap+": 0, 2024-09-16T02:06:09.321 INFO:teuthology.orchestra.run.smithi137.stdout: "cap-": 0, 2024-09-16T02:06:09.321 INFO:teuthology.orchestra.run.smithi137.stdout: "rss": 50500, 2024-09-16T02:06:09.321 INFO:teuthology.orchestra.run.smithi137.stdout: "heap": 215328 2024-09-16T02:06:09.321 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.321 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_server": { 2024-09-16T02:06:09.321 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_client_request": 0, 2024-09-16T02:06:09.321 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_server_request": 0, 2024-09-16T02:06:09.322 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_request": 0, 2024-09-16T02:06:09.322 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_session": 0, 2024-09-16T02:06:09.322 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_peer_request": 0, 2024-09-16T02:06:09.322 INFO:teuthology.orchestra.run.smithi137.stdout: "req_create_latency": { 2024-09-16T02:06:09.322 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.322 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.322 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.322 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.322 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getattr_latency": { 2024-09-16T02:06:09.322 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.322 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.322 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.322 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.322 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getfilelock_latency": { 2024-09-16T02:06:09.323 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.323 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.323 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.323 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.323 INFO:teuthology.orchestra.run.smithi137.stdout: "req_link_latency": { 2024-09-16T02:06:09.323 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.323 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.323 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.323 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.323 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookup_latency": { 2024-09-16T02:06:09.323 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.323 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.323 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.323 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.324 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookuphash_latency": { 2024-09-16T02:06:09.324 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.324 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.324 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.324 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.324 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupino_latency": { 2024-09-16T02:06:09.324 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.324 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.324 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.324 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.324 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupname_latency": { 2024-09-16T02:06:09.324 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.324 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.324 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.325 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.325 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupparent_latency": { 2024-09-16T02:06:09.325 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.325 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.325 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.325 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.325 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupsnap_latency": { 2024-09-16T02:06:09.325 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.325 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.325 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.325 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.325 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lssnap_latency": { 2024-09-16T02:06:09.325 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.325 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.326 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.326 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.326 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mkdir_latency": { 2024-09-16T02:06:09.326 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.326 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.326 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.327 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.327 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mknod_latency": { 2024-09-16T02:06:09.327 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.327 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.327 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.327 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.327 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mksnap_latency": { 2024-09-16T02:06:09.327 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.327 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.327 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.327 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.327 INFO:teuthology.orchestra.run.smithi137.stdout: "req_open_latency": { 2024-09-16T02:06:09.327 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.327 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.328 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.328 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.328 INFO:teuthology.orchestra.run.smithi137.stdout: "req_readdir_latency": { 2024-09-16T02:06:09.328 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.328 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.328 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.328 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.328 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rename_latency": { 2024-09-16T02:06:09.328 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.328 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.328 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.328 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.328 INFO:teuthology.orchestra.run.smithi137.stdout: "req_renamesnap_latency": { 2024-09-16T02:06:09.328 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.329 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.329 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.329 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.329 INFO:teuthology.orchestra.run.smithi137.stdout: "req_snapdiff_latency": { 2024-09-16T02:06:09.329 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.329 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.329 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.329 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.329 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmdir_latency": { 2024-09-16T02:06:09.329 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.329 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.329 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.329 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.329 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmsnap_latency": { 2024-09-16T02:06:09.330 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.330 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.330 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.330 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.330 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmxattr_latency": { 2024-09-16T02:06:09.330 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.330 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.330 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.330 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.330 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setattr_latency": { 2024-09-16T02:06:09.330 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.330 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.330 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.331 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.331 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setdirlayout_latency": { 2024-09-16T02:06:09.331 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.331 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.331 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.331 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.331 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setfilelock_latency": { 2024-09-16T02:06:09.331 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.331 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.331 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.331 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.331 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setlayout_latency": { 2024-09-16T02:06:09.331 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.332 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.332 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.332 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.332 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setxattr_latency": { 2024-09-16T02:06:09.332 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.332 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.332 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.332 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.332 INFO:teuthology.orchestra.run.smithi137.stdout: "req_symlink_latency": { 2024-09-16T02:06:09.332 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.332 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.332 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.332 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.332 INFO:teuthology.orchestra.run.smithi137.stdout: "req_unlink_latency": { 2024-09-16T02:06:09.333 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.333 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.333 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.333 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.333 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_revoke_eviction": 0, 2024-09-16T02:06:09.333 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_acquisition_throttle": 0, 2024-09-16T02:06:09.333 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getvxattr_latency": { 2024-09-16T02:06:09.333 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.333 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.333 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.333 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-16T02:06:09.333 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.333 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_sessions": { 2024-09-16T02:06:09.333 INFO:teuthology.orchestra.run.smithi137.stdout: "session_count": 1, 2024-09-16T02:06:09.333 INFO:teuthology.orchestra.run.smithi137.stdout: "session_add": 1, 2024-09-16T02:06:09.334 INFO:teuthology.orchestra.run.smithi137.stdout: "session_remove": 0, 2024-09-16T02:06:09.334 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_open": 1, 2024-09-16T02:06:09.334 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_stale": 0, 2024-09-16T02:06:09.334 INFO:teuthology.orchestra.run.smithi137.stdout: "total_load": 0, 2024-09-16T02:06:09.334 INFO:teuthology.orchestra.run.smithi137.stdout: "average_load": 0, 2024-09-16T02:06:09.334 INFO:teuthology.orchestra.run.smithi137.stdout: "avg_session_uptime": 1232, 2024-09-16T02:06:09.334 INFO:teuthology.orchestra.run.smithi137.stdout: "mdthresh_evicted": 0 2024-09-16T02:06:09.334 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.334 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-09-16T02:06:09.334 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-09-16T02:06:09.334 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-09-16T02:06:09.334 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T02:06:09.334 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-09-16T02:06:09.334 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T02:06:09.335 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-09-16T02:06:09.335 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T02:06:09.335 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T02:06:09.335 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T02:06:09.335 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T02:06:09.335 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T02:06:09.335 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-09-16T02:06:09.335 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T02:06:09.335 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T02:06:09.335 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-09-16T02:06:09.335 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-09-16T02:06:09.335 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-09-16T02:06:09.335 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-09-16T02:06:09.335 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T02:06:09.336 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T02:06:09.336 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T02:06:09.336 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T02:06:09.336 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T02:06:09.336 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-09-16T02:06:09.336 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-09-16T02:06:09.336 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-09-16T02:06:09.336 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T02:06:09.336 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T02:06:09.336 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-09-16T02:06:09.336 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-09-16T02:06:09.336 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-09-16T02:06:09.336 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-09-16T02:06:09.336 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T02:06:09.337 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-09-16T02:06:09.337 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T02:06:09.337 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-09-16T02:06:09.337 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 182230, 2024-09-16T02:06:09.337 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 46, 2024-09-16T02:06:09.337 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 0, 2024-09-16T02:06:09.337 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 0, 2024-09-16T02:06:09.337 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-09-16T02:06:09.337 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-09-16T02:06:09.337 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-09-16T02:06:09.337 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-09-16T02:06:09.337 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-09-16T02:06:09.337 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-09-16T02:06:09.337 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 7888, 2024-09-16T02:06:09.337 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 163, 2024-09-16T02:06:09.338 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T02:06:09.338 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-09-16T02:06:09.338 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-09-16T02:06:09.338 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-09-16T02:06:09.338 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 8512, 2024-09-16T02:06:09.338 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 13, 2024-09-16T02:06:09.338 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-09-16T02:06:09.338 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-09-16T02:06:09.338 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-09-16T02:06:09.339 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-09-16T02:06:09.339 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.339 INFO:teuthology.orchestra.run.smithi137.stdout: "objecter": { 2024-09-16T02:06:09.339 INFO:teuthology.orchestra.run.smithi137.stdout: "op_active": 0, 2024-09-16T02:06:09.339 INFO:teuthology.orchestra.run.smithi137.stdout: "op_laggy": 0, 2024-09-16T02:06:09.339 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send": 858, 2024-09-16T02:06:09.339 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send_bytes": 21, 2024-09-16T02:06:09.339 INFO:teuthology.orchestra.run.smithi137.stdout: "op_resend": 3, 2024-09-16T02:06:09.339 INFO:teuthology.orchestra.run.smithi137.stdout: "op_reply": 858, 2024-09-16T02:06:09.339 INFO:teuthology.orchestra.run.smithi137.stdout: "op_latency": { 2024-09-16T02:06:09.339 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 858, 2024-09-16T02:06:09.340 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 1.352975793, 2024-09-16T02:06:09.340 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001576894 2024-09-16T02:06:09.340 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.340 INFO:teuthology.orchestra.run.smithi137.stdout: "op_inflight": 0, 2024-09-16T02:06:09.340 INFO:teuthology.orchestra.run.smithi137.stdout: "oplen_avg": { 2024-09-16T02:06:09.340 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 858, 2024-09-16T02:06:09.340 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 859 2024-09-16T02:06:09.340 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.340 INFO:teuthology.orchestra.run.smithi137.stdout: "op": 858, 2024-09-16T02:06:09.340 INFO:teuthology.orchestra.run.smithi137.stdout: "op_r": 858, 2024-09-16T02:06:09.341 INFO:teuthology.orchestra.run.smithi137.stdout: "op_w": 0, 2024-09-16T02:06:09.341 INFO:teuthology.orchestra.run.smithi137.stdout: "op_rmw": 0, 2024-09-16T02:06:09.341 INFO:teuthology.orchestra.run.smithi137.stdout: "op_pg": 0, 2024-09-16T02:06:09.341 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_stat": 530, 2024-09-16T02:06:09.341 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_create": 0, 2024-09-16T02:06:09.341 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_read": 326, 2024-09-16T02:06:09.341 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_write": 0, 2024-09-16T02:06:09.341 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writefull": 0, 2024-09-16T02:06:09.341 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writesame": 0, 2024-09-16T02:06:09.341 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_append": 0, 2024-09-16T02:06:09.341 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_zero": 0, 2024-09-16T02:06:09.341 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_truncate": 0, 2024-09-16T02:06:09.341 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_delete": 0, 2024-09-16T02:06:09.341 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_mapext": 0, 2024-09-16T02:06:09.341 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_sparse_read": 0, 2024-09-16T02:06:09.342 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_clonerange": 0, 2024-09-16T02:06:09.342 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_getxattr": 1, 2024-09-16T02:06:09.342 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_setxattr": 0, 2024-09-16T02:06:09.342 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_cmpxattr": 0, 2024-09-16T02:06:09.342 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_rmxattr": 0, 2024-09-16T02:06:09.342 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_resetxattrs": 0, 2024-09-16T02:06:09.342 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_call": 0, 2024-09-16T02:06:09.342 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_watch": 0, 2024-09-16T02:06:09.342 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_notify": 0, 2024-09-16T02:06:09.342 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T02:06:09.342 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls": 0, 2024-09-16T02:06:09.343 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls_filter": 0, 2024-09-16T02:06:09.343 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_other": 0, 2024-09-16T02:06:09.343 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_active": 0, 2024-09-16T02:06:09.343 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_send": 0, 2024-09-16T02:06:09.343 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_resend": 0, 2024-09-16T02:06:09.343 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_ping": 0, 2024-09-16T02:06:09.343 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_active": 0, 2024-09-16T02:06:09.343 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_send": 0, 2024-09-16T02:06:09.343 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_resend": 0, 2024-09-16T02:06:09.343 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_active": 0, 2024-09-16T02:06:09.343 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_send": 0, 2024-09-16T02:06:09.343 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_resend": 0, 2024-09-16T02:06:09.343 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_active": 0, 2024-09-16T02:06:09.343 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_send": 0, 2024-09-16T02:06:09.344 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_resend": 0, 2024-09-16T02:06:09.344 INFO:teuthology.orchestra.run.smithi137.stdout: "command_active": 0, 2024-09-16T02:06:09.344 INFO:teuthology.orchestra.run.smithi137.stdout: "command_send": 0, 2024-09-16T02:06:09.344 INFO:teuthology.orchestra.run.smithi137.stdout: "command_resend": 0, 2024-09-16T02:06:09.344 INFO:teuthology.orchestra.run.smithi137.stdout: "map_epoch": 83, 2024-09-16T02:06:09.344 INFO:teuthology.orchestra.run.smithi137.stdout: "map_full": 0, 2024-09-16T02:06:09.344 INFO:teuthology.orchestra.run.smithi137.stdout: "map_inc": 4, 2024-09-16T02:06:09.344 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_sessions": 4, 2024-09-16T02:06:09.344 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_open": 4, 2024-09-16T02:06:09.344 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_close": 0, 2024-09-16T02:06:09.344 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_laggy": 0, 2024-09-16T02:06:09.344 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_wr": 0, 2024-09-16T02:06:09.344 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_rd": 2, 2024-09-16T02:06:09.345 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_del": 0 2024-09-16T02:06:09.345 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.345 INFO:teuthology.orchestra.run.smithi137.stdout: "oft": { 2024-09-16T02:06:09.345 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_objs": 0, 2024-09-16T02:06:09.345 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_kv_pairs": 0, 2024-09-16T02:06:09.345 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_updates": 0, 2024-09-16T02:06:09.345 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_removes": 0 2024-09-16T02:06:09.345 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.345 INFO:teuthology.orchestra.run.smithi137.stdout: "osd-slow-ops": { 2024-09-16T02:06:09.345 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_ops_count": 0 2024-09-16T02:06:09.345 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.345 INFO:teuthology.orchestra.run.smithi137.stdout: "purge_queue": { 2024-09-16T02:06:09.345 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops": 0, 2024-09-16T02:06:09.345 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T02:06:09.345 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing": 0, 2024-09-16T02:06:09.346 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_high_water": 0, 2024-09-16T02:06:09.346 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed_ops": 0, 2024-09-16T02:06:09.346 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed": 0, 2024-09-16T02:06:09.346 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_item_in_journal": 0 2024-09-16T02:06:09.346 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.346 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T02:06:09.346 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-16T02:06:09.346 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-09-16T02:06:09.346 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-16T02:06:09.346 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 977, 2024-09-16T02:06:09.346 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 625637, 2024-09-16T02:06:09.346 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:09.346 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 977, 2024-09-16T02:06:09.346 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-09-16T02:06:09.346 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-09-16T02:06:09.347 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 977, 2024-09-16T02:06:09.347 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 625637, 2024-09-16T02:06:09.347 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-16T02:06:09.347 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.347 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.347 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.347 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-16T02:06:09.347 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.347 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_bytes": { 2024-09-16T02:06:09.347 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-16T02:06:09.347 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-09-16T02:06:09.347 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-16T02:06:09.347 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-09-16T02:06:09.347 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-09-16T02:06:09.347 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:09.348 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-09-16T02:06:09.348 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 858, 2024-09-16T02:06:09.348 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 1088, 2024-09-16T02:06:09.348 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 858, 2024-09-16T02:06:09.348 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 1088, 2024-09-16T02:06:09.348 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-16T02:06:09.348 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.348 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.348 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.348 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-16T02:06:09.348 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.348 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_ops": { 2024-09-16T02:06:09.348 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-16T02:06:09.348 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 1024, 2024-09-16T02:06:09.349 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-16T02:06:09.349 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-09-16T02:06:09.349 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-09-16T02:06:09.349 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:09.349 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-09-16T02:06:09.349 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 858, 2024-09-16T02:06:09.349 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 858, 2024-09-16T02:06:09.349 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 858, 2024-09-16T02:06:09.349 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 858, 2024-09-16T02:06:09.349 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-16T02:06:09.349 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.349 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.349 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.349 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-16T02:06:09.350 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.350 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle": { 2024-09-16T02:06:09.350 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-16T02:06:09.350 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-09-16T02:06:09.350 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-16T02:06:09.350 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-09-16T02:06:09.350 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-09-16T02:06:09.350 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:09.350 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-09-16T02:06:09.350 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-09-16T02:06:09.350 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-09-16T02:06:09.350 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 0, 2024-09-16T02:06:09.350 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 0, 2024-09-16T02:06:09.350 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-16T02:06:09.350 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.351 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.351 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.351 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-16T02:06:09.351 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-16T02:06:09.351 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle-0x5627d438eb40": { 2024-09-16T02:06:09.351 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-16T02:06:09.351 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-09-16T02:06:09.351 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-16T02:06:09.351 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-09-16T02:06:09.351 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-09-16T02:06:09.351 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:09.351 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-09-16T02:06:09.351 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-09-16T02:06:09.351 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-09-16T02:06:09.351 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 0, 2024-09-16T02:06:09.352 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 0, 2024-09-16T02:06:09.352 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-16T02:06:09.352 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-16T02:06:09.352 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-16T02:06:09.352 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-16T02:06:09.352 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-16T02:06:09.352 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-16T02:06:09.352 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-16T02:06:10.055 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-16T02:06:10.055 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-16T02:06:10.055 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-09-16T02:06:10.055 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-09-16T02:06:10.056 DEBUG:tasks.check_counter:Getting stats from c 2024-09-16T02:06:10.056 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-16T02:06:10.313 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T02:06:10.743 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:10 smithi190 ceph-mon[34762]: pgmap v448: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-16T02:06:10.852 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-16T02:06:10.852 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T02:06:10.852 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 183, 2024-09-16T02:06:10.852 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 175, 2024-09-16T02:06:10.852 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 505884, 2024-09-16T02:06:10.852 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 134416, 2024-09-16T02:06:10.852 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 14, 2024-09-16T02:06:10.853 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 2, 2024-09-16T02:06:10.853 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.026424365, 2024-09-16T02:06:10.853 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.008244611, 2024-09-16T02:06:10.853 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.020149419, 2024-09-16T02:06:10.853 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.001335477, 2024-09-16T02:06:10.853 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:06:10.853 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 174, 2024-09-16T02:06:10.853 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.018151223, 2024-09-16T02:06:10.853 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000104317 2024-09-16T02:06:10.853 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.853 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:06:10.853 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 45, 2024-09-16T02:06:10.853 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000008383, 2024-09-16T02:06:10.853 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000186 2024-09-16T02:06:10.854 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.854 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 489520, 2024-09-16T02:06:10.854 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 66544 2024-09-16T02:06:10.854 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.854 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T02:06:10.854 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 39, 2024-09-16T02:06:10.854 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 114, 2024-09-16T02:06:10.854 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 15903, 2024-09-16T02:06:10.854 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 344720, 2024-09-16T02:06:10.854 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 18, 2024-09-16T02:06:10.854 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 4, 2024-09-16T02:06:10.854 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.013048541, 2024-09-16T02:06:10.854 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.004665075, 2024-09-16T02:06:10.854 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.007114791, 2024-09-16T02:06:10.855 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.000273555, 2024-09-16T02:06:10.855 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:06:10.855 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 112, 2024-09-16T02:06:10.855 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.017528363, 2024-09-16T02:06:10.855 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000156503 2024-09-16T02:06:10.855 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.855 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:06:10.855 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 39, 2024-09-16T02:06:10.855 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000018386, 2024-09-16T02:06:10.855 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000471 2024-09-16T02:06:10.855 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.855 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-16T02:06:10.855 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 206192 2024-09-16T02:06:10.856 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.856 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T02:06:10.856 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 237, 2024-09-16T02:06:10.856 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 275, 2024-09-16T02:06:10.856 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 86147, 2024-09-16T02:06:10.856 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 111237, 2024-09-16T02:06:10.856 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 8, 2024-09-16T02:06:10.856 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 4, 2024-09-16T02:06:10.856 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.028843589, 2024-09-16T02:06:10.856 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.009834452, 2024-09-16T02:06:10.856 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.017000707, 2024-09-16T02:06:10.856 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.002174113, 2024-09-16T02:06:10.856 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:06:10.856 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 274, 2024-09-16T02:06:10.857 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.029384319, 2024-09-16T02:06:10.857 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000107242 2024-09-16T02:06:10.857 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.857 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:06:10.857 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 474, 2024-09-16T02:06:10.857 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000118169, 2024-09-16T02:06:10.857 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000249 2024-09-16T02:06:10.857 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.857 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-16T02:06:10.857 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 448 2024-09-16T02:06:10.857 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.857 INFO:teuthology.orchestra.run.smithi190.stdout: "cct": { 2024-09-16T02:06:10.857 INFO:teuthology.orchestra.run.smithi190.stdout: "total_workers": 1, 2024-09-16T02:06:10.858 INFO:teuthology.orchestra.run.smithi190.stdout: "unhealthy_workers": 0 2024-09-16T02:06:10.858 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.858 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-MDSRank": { 2024-09-16T02:06:10.858 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-09-16T02:06:10.858 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-09-16T02:06:10.858 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 38, 2024-09-16T02:06:10.858 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.001740380, 2024-09-16T02:06:10.858 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000045799 2024-09-16T02:06:10.858 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-16T02:06:10.858 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.858 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-PurgeQueue": { 2024-09-16T02:06:10.858 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-09-16T02:06:10.858 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-09-16T02:06:10.858 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2, 2024-09-16T02:06:10.858 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000012787, 2024-09-16T02:06:10.859 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000006393 2024-09-16T02:06:10.859 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-16T02:06:10.859 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.859 INFO:teuthology.orchestra.run.smithi190.stdout: "mds": { 2024-09-16T02:06:10.859 INFO:teuthology.orchestra.run.smithi190.stdout: "request": 1, 2024-09-16T02:06:10.859 INFO:teuthology.orchestra.run.smithi190.stdout: "reply": 0, 2024-09-16T02:06:10.859 INFO:teuthology.orchestra.run.smithi190.stdout: "reply_latency": { 2024-09-16T02:06:10.859 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:10.859 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:10.859 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:10.859 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.859 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_reply": 0, 2024-09-16T02:06:10.859 INFO:teuthology.orchestra.run.smithi190.stdout: "forward": 1, 2024-09-16T02:06:10.859 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_complete": 0, 2024-09-16T02:06:10.860 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_keys": 0, 2024-09-16T02:06:10.860 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_commit": 11, 2024-09-16T02:06:10.860 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_split": 0, 2024-09-16T02:06:10.860 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_merge": 0, 2024-09-16T02:06:10.860 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes": 16, 2024-09-16T02:06:10.860 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_top": 0, 2024-09-16T02:06:10.860 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_bottom": 16, 2024-09-16T02:06:10.860 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pin_tail": 0, 2024-09-16T02:06:10.860 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pinned": 13, 2024-09-16T02:06:10.860 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_expired": 4, 2024-09-16T02:06:10.860 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_with_caps": 0, 2024-09-16T02:06:10.860 INFO:teuthology.orchestra.run.smithi190.stdout: "caps": 0, 2024-09-16T02:06:10.860 INFO:teuthology.orchestra.run.smithi190.stdout: "subtrees": 3, 2024-09-16T02:06:10.860 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse": 8, 2024-09-16T02:06:10.861 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_hit": 4, 2024-09-16T02:06:10.861 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_forward": 1, 2024-09-16T02:06:10.861 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_discover": 3, 2024-09-16T02:06:10.861 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_dir_fetch": 0, 2024-09-16T02:06:10.861 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_remote_ino": 0, 2024-09-16T02:06:10.861 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_lock": 0, 2024-09-16T02:06:10.861 INFO:teuthology.orchestra.run.smithi190.stdout: "load_cent": 1000, 2024-09-16T02:06:10.861 INFO:teuthology.orchestra.run.smithi190.stdout: "q": 1, 2024-09-16T02:06:10.862 INFO:teuthology.orchestra.run.smithi190.stdout: "exported": 0, 2024-09-16T02:06:10.862 INFO:teuthology.orchestra.run.smithi190.stdout: "exported_inodes": 0, 2024-09-16T02:06:10.862 INFO:teuthology.orchestra.run.smithi190.stdout: "imported": 0, 2024-09-16T02:06:10.862 INFO:teuthology.orchestra.run.smithi190.stdout: "imported_inodes": 0, 2024-09-16T02:06:10.862 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_dir_fetch": 0, 2024-09-16T02:06:10.862 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_backtrace_fetch": 0, 2024-09-16T02:06:10.862 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_peer_discover": 0, 2024-09-16T02:06:10.862 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rfiles": 4, 2024-09-16T02:06:10.862 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rbytes": 234, 2024-09-16T02:06:10.862 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rsnaps": 0, 2024-09-16T02:06:10.862 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T02:06:10.862 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_set_tag": 0, 2024-09-16T02:06:10.862 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T02:06:10.863 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_inotable_repaired": 0, 2024-09-16T02:06:10.863 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_inodes": 0, 2024-09-16T02:06:10.863 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T02:06:10.863 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T02:06:10.863 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_file_inodes": 0, 2024-09-16T02:06:10.863 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_inode_file_caps": 0, 2024-09-16T02:06:10.863 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T02:06:10.863 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_grant": 0, 2024-09-16T02:06:10.863 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T02:06:10.863 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T02:06:10.863 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T02:06:10.863 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps": 0, 2024-09-16T02:06:10.863 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps_dirty": 0, 2024-09-16T02:06:10.863 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_cap_release": 0, 2024-09-16T02:06:10.863 INFO:teuthology.orchestra.run.smithi190.stdout: "process_request_cap_release": 0 2024-09-16T02:06:10.864 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.864 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_cache": { 2024-09-16T02:06:10.864 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update": 0, 2024-09-16T02:06:10.864 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update_receipt": 11, 2024-09-16T02:06:10.864 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_try_discover": 7, 2024-09-16T02:06:10.864 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_send_discover": 8, 2024-09-16T02:06:10.864 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_handle_discover": 0, 2024-09-16T02:06:10.864 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays": 0, 2024-09-16T02:06:10.864 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_delayed": 0, 2024-09-16T02:06:10.864 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_enqueuing": 0, 2024-09-16T02:06:10.864 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_created": 0, 2024-09-16T02:06:10.864 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_enqueued": 0, 2024-09-16T02:06:10.864 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_reintegrated": 0, 2024-09-16T02:06:10.864 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_migrated": 0, 2024-09-16T02:06:10.864 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_processing": 0, 2024-09-16T02:06:10.865 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_enqueued": 0, 2024-09-16T02:06:10.865 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_prioritized": 0, 2024-09-16T02:06:10.865 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_started": 0, 2024-09-16T02:06:10.865 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_completed": 0, 2024-09-16T02:06:10.865 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_quiesce_path": 0, 2024-09-16T02:06:10.865 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_quiesce_inode": 0, 2024-09-16T02:06:10.865 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T02:06:10.865 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_exportdir": 0, 2024-09-16T02:06:10.865 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_flush": 0, 2024-09-16T02:06:10.865 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragmentdir": 0, 2024-09-16T02:06:10.865 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragstats": 0, 2024-09-16T02:06:10.865 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_inodestats": 0 2024-09-16T02:06:10.865 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.865 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_log": { 2024-09-16T02:06:10.866 INFO:teuthology.orchestra.run.smithi190.stdout: "evlrg": 0, 2024-09-16T02:06:10.866 INFO:teuthology.orchestra.run.smithi190.stdout: "evadd": 3, 2024-09-16T02:06:10.866 INFO:teuthology.orchestra.run.smithi190.stdout: "evex": 0, 2024-09-16T02:06:10.866 INFO:teuthology.orchestra.run.smithi190.stdout: "evtrm": 0, 2024-09-16T02:06:10.866 INFO:teuthology.orchestra.run.smithi190.stdout: "ev": 3, 2024-09-16T02:06:10.866 INFO:teuthology.orchestra.run.smithi190.stdout: "evexg": 0, 2024-09-16T02:06:10.866 INFO:teuthology.orchestra.run.smithi190.stdout: "evexd": 0, 2024-09-16T02:06:10.866 INFO:teuthology.orchestra.run.smithi190.stdout: "segadd": 2, 2024-09-16T02:06:10.866 INFO:teuthology.orchestra.run.smithi190.stdout: "segex": 0, 2024-09-16T02:06:10.866 INFO:teuthology.orchestra.run.smithi190.stdout: "segtrm": 0, 2024-09-16T02:06:10.866 INFO:teuthology.orchestra.run.smithi190.stdout: "seg": 2, 2024-09-16T02:06:10.866 INFO:teuthology.orchestra.run.smithi190.stdout: "segmjr": 2, 2024-09-16T02:06:10.866 INFO:teuthology.orchestra.run.smithi190.stdout: "segexg": 0, 2024-09-16T02:06:10.866 INFO:teuthology.orchestra.run.smithi190.stdout: "segexd": 0, 2024-09-16T02:06:10.866 INFO:teuthology.orchestra.run.smithi190.stdout: "expos": 4194304, 2024-09-16T02:06:10.867 INFO:teuthology.orchestra.run.smithi190.stdout: "wrpos": 4195387, 2024-09-16T02:06:10.867 INFO:teuthology.orchestra.run.smithi190.stdout: "rdpos": 0, 2024-09-16T02:06:10.867 INFO:teuthology.orchestra.run.smithi190.stdout: "jlat": { 2024-09-16T02:06:10.867 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2, 2024-09-16T02:06:10.867 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.003123592, 2024-09-16T02:06:10.867 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.001561796 2024-09-16T02:06:10.867 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.867 INFO:teuthology.orchestra.run.smithi190.stdout: "replayed": 0 2024-09-16T02:06:10.867 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.867 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_mem": { 2024-09-16T02:06:10.867 INFO:teuthology.orchestra.run.smithi190.stdout: "ino": 20, 2024-09-16T02:06:10.868 INFO:teuthology.orchestra.run.smithi190.stdout: "ino+": 22, 2024-09-16T02:06:10.868 INFO:teuthology.orchestra.run.smithi190.stdout: "ino-": 2, 2024-09-16T02:06:10.868 INFO:teuthology.orchestra.run.smithi190.stdout: "dir": 19, 2024-09-16T02:06:10.868 INFO:teuthology.orchestra.run.smithi190.stdout: "dir+": 23, 2024-09-16T02:06:10.868 INFO:teuthology.orchestra.run.smithi190.stdout: "dir-": 4, 2024-09-16T02:06:10.868 INFO:teuthology.orchestra.run.smithi190.stdout: "dn": 16, 2024-09-16T02:06:10.868 INFO:teuthology.orchestra.run.smithi190.stdout: "dn+": 20, 2024-09-16T02:06:10.868 INFO:teuthology.orchestra.run.smithi190.stdout: "dn-": 4, 2024-09-16T02:06:10.868 INFO:teuthology.orchestra.run.smithi190.stdout: "cap": 0, 2024-09-16T02:06:10.868 INFO:teuthology.orchestra.run.smithi190.stdout: "cap+": 0, 2024-09-16T02:06:10.868 INFO:teuthology.orchestra.run.smithi190.stdout: "cap-": 0, 2024-09-16T02:06:10.868 INFO:teuthology.orchestra.run.smithi190.stdout: "rss": 49424, 2024-09-16T02:06:10.868 INFO:teuthology.orchestra.run.smithi190.stdout: "heap": 223520 2024-09-16T02:06:10.869 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.869 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_server": { 2024-09-16T02:06:10.869 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_client_request": 1, 2024-09-16T02:06:10.869 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_server_request": 0, 2024-09-16T02:06:10.869 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_request": 1, 2024-09-16T02:06:10.869 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_session": 5, 2024-09-16T02:06:10.869 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_peer_request": 0, 2024-09-16T02:06:10.869 INFO:teuthology.orchestra.run.smithi190.stdout: "req_create_latency": { 2024-09-16T02:06:10.869 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:10.869 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:10.869 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:10.869 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.869 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getattr_latency": { 2024-09-16T02:06:10.869 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:10.869 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:10.870 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:10.870 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.870 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getfilelock_latency": { 2024-09-16T02:06:10.870 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:10.870 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:10.870 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:10.870 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.870 INFO:teuthology.orchestra.run.smithi190.stdout: "req_link_latency": { 2024-09-16T02:06:10.870 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:10.870 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:10.870 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:10.870 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.870 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookup_latency": { 2024-09-16T02:06:10.870 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:10.870 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:10.871 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:10.871 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.871 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookuphash_latency": { 2024-09-16T02:06:10.871 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:10.871 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:10.871 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:10.871 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.871 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupino_latency": { 2024-09-16T02:06:10.871 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:10.871 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:10.871 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:10.871 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.871 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupname_latency": { 2024-09-16T02:06:10.872 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:10.872 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:10.872 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:10.872 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.872 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupparent_latency": { 2024-09-16T02:06:10.872 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:10.872 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:10.872 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:10.872 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.872 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupsnap_latency": { 2024-09-16T02:06:10.872 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:10.872 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:10.872 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:10.873 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.873 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lssnap_latency": { 2024-09-16T02:06:10.873 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:10.873 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:10.873 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:10.873 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.873 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mkdir_latency": { 2024-09-16T02:06:10.873 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:10.873 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:10.873 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:10.873 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.873 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mknod_latency": { 2024-09-16T02:06:10.874 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:10.874 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:10.874 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:10.874 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.874 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mksnap_latency": { 2024-09-16T02:06:10.874 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:10.874 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:10.874 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:10.874 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.874 INFO:teuthology.orchestra.run.smithi190.stdout: "req_open_latency": { 2024-09-16T02:06:10.874 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:10.874 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:10.874 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:10.874 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.875 INFO:teuthology.orchestra.run.smithi190.stdout: "req_readdir_latency": { 2024-09-16T02:06:10.875 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:10.875 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:10.875 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:10.875 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.875 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rename_latency": { 2024-09-16T02:06:10.875 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:10.875 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:10.875 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:10.875 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.875 INFO:teuthology.orchestra.run.smithi190.stdout: "req_renamesnap_latency": { 2024-09-16T02:06:10.875 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:10.875 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:10.876 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:10.876 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.876 INFO:teuthology.orchestra.run.smithi190.stdout: "req_snapdiff_latency": { 2024-09-16T02:06:10.876 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:10.876 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:10.876 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:10.876 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.876 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmdir_latency": { 2024-09-16T02:06:10.876 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:10.876 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:10.876 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:10.876 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.876 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmsnap_latency": { 2024-09-16T02:06:10.876 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:10.877 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:10.877 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:10.877 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.877 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmxattr_latency": { 2024-09-16T02:06:10.877 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:10.877 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:10.877 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:10.877 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.877 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setattr_latency": { 2024-09-16T02:06:10.878 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:10.878 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:10.878 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:10.878 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.878 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setdirlayout_latency": { 2024-09-16T02:06:10.878 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:10.878 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:10.878 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:10.878 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.878 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setfilelock_latency": { 2024-09-16T02:06:10.878 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:10.878 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:10.879 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:10.879 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.879 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setlayout_latency": { 2024-09-16T02:06:10.879 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:10.879 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:10.879 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:10.879 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.879 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setxattr_latency": { 2024-09-16T02:06:10.879 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:10.879 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:10.879 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:10.879 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.879 INFO:teuthology.orchestra.run.smithi190.stdout: "req_symlink_latency": { 2024-09-16T02:06:10.879 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:10.880 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:10.880 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:10.880 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.880 INFO:teuthology.orchestra.run.smithi190.stdout: "req_unlink_latency": { 2024-09-16T02:06:10.880 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:10.880 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:10.880 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:10.880 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.880 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_revoke_eviction": 0, 2024-09-16T02:06:10.880 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_acquisition_throttle": 0, 2024-09-16T02:06:10.880 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getvxattr_latency": { 2024-09-16T02:06:10.880 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:10.880 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:10.881 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:10.881 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-16T02:06:10.881 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.881 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_sessions": { 2024-09-16T02:06:10.881 INFO:teuthology.orchestra.run.smithi190.stdout: "session_count": 1, 2024-09-16T02:06:10.881 INFO:teuthology.orchestra.run.smithi190.stdout: "session_add": 1, 2024-09-16T02:06:10.881 INFO:teuthology.orchestra.run.smithi190.stdout: "session_remove": 0, 2024-09-16T02:06:10.881 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_open": 1, 2024-09-16T02:06:10.881 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_stale": 0, 2024-09-16T02:06:10.881 INFO:teuthology.orchestra.run.smithi190.stdout: "total_load": 0, 2024-09-16T02:06:10.881 INFO:teuthology.orchestra.run.smithi190.stdout: "average_load": 0, 2024-09-16T02:06:10.881 INFO:teuthology.orchestra.run.smithi190.stdout: "avg_session_uptime": 210, 2024-09-16T02:06:10.881 INFO:teuthology.orchestra.run.smithi190.stdout: "mdthresh_evicted": 0 2024-09-16T02:06:10.881 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.882 INFO:teuthology.orchestra.run.smithi190.stdout: "mempool": { 2024-09-16T02:06:10.882 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_bytes": 0, 2024-09-16T02:06:10.882 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_items": 0, 2024-09-16T02:06:10.882 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T02:06:10.882 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_items": 0, 2024-09-16T02:06:10.882 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T02:06:10.882 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_items": 0, 2024-09-16T02:06:10.882 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T02:06:10.882 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T02:06:10.882 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T02:06:10.882 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T02:06:10.882 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T02:06:10.882 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_items": 0, 2024-09-16T02:06:10.882 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T02:06:10.883 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T02:06:10.883 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_bytes": 0, 2024-09-16T02:06:10.883 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_items": 0, 2024-09-16T02:06:10.883 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_bytes": 0, 2024-09-16T02:06:10.883 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_items": 0, 2024-09-16T02:06:10.883 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T02:06:10.883 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T02:06:10.883 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T02:06:10.883 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T02:06:10.883 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T02:06:10.883 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_items": 0, 2024-09-16T02:06:10.883 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_bytes": 0, 2024-09-16T02:06:10.883 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_items": 0, 2024-09-16T02:06:10.883 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T02:06:10.883 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T02:06:10.884 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_bytes": 0, 2024-09-16T02:06:10.884 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_items": 0, 2024-09-16T02:06:10.884 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_bytes": 0, 2024-09-16T02:06:10.884 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_items": 0, 2024-09-16T02:06:10.884 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T02:06:10.884 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_items": 0, 2024-09-16T02:06:10.884 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T02:06:10.884 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_items": 0, 2024-09-16T02:06:10.884 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_bytes": 254147, 2024-09-16T02:06:10.884 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_items": 81, 2024-09-16T02:06:10.884 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_bytes": 0, 2024-09-16T02:06:10.884 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_items": 0, 2024-09-16T02:06:10.884 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_bytes": 0, 2024-09-16T02:06:10.884 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_items": 0, 2024-09-16T02:06:10.885 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_bytes": 0, 2024-09-16T02:06:10.885 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_items": 0, 2024-09-16T02:06:10.885 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_bytes": 0, 2024-09-16T02:06:10.885 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_items": 0, 2024-09-16T02:06:10.885 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_bytes": 7888, 2024-09-16T02:06:10.885 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_items": 163, 2024-09-16T02:06:10.885 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T02:06:10.885 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_items": 0, 2024-09-16T02:06:10.885 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_bytes": 0, 2024-09-16T02:06:10.885 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_items": 0, 2024-09-16T02:06:10.885 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_bytes": 80933, 2024-09-16T02:06:10.885 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_items": 323, 2024-09-16T02:06:10.885 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_bytes": 0, 2024-09-16T02:06:10.885 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_items": 0, 2024-09-16T02:06:10.885 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_bytes": 0, 2024-09-16T02:06:10.886 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_items": 0 2024-09-16T02:06:10.886 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.886 INFO:teuthology.orchestra.run.smithi190.stdout: "objecter": { 2024-09-16T02:06:10.886 INFO:teuthology.orchestra.run.smithi190.stdout: "op_active": 0, 2024-09-16T02:06:10.886 INFO:teuthology.orchestra.run.smithi190.stdout: "op_laggy": 0, 2024-09-16T02:06:10.886 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send": 37, 2024-09-16T02:06:10.886 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send_bytes": 11295, 2024-09-16T02:06:10.886 INFO:teuthology.orchestra.run.smithi190.stdout: "op_resend": 29, 2024-09-16T02:06:10.886 INFO:teuthology.orchestra.run.smithi190.stdout: "op_reply": 37, 2024-09-16T02:06:10.886 INFO:teuthology.orchestra.run.smithi190.stdout: "op_latency": { 2024-09-16T02:06:10.886 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 37, 2024-09-16T02:06:10.886 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.190994526, 2024-09-16T02:06:10.886 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.005162014 2024-09-16T02:06:10.886 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.886 INFO:teuthology.orchestra.run.smithi190.stdout: "op_inflight": 0, 2024-09-16T02:06:10.887 INFO:teuthology.orchestra.run.smithi190.stdout: "oplen_avg": { 2024-09-16T02:06:10.887 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 37, 2024-09-16T02:06:10.887 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 59 2024-09-16T02:06:10.887 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.887 INFO:teuthology.orchestra.run.smithi190.stdout: "op": 37, 2024-09-16T02:06:10.887 INFO:teuthology.orchestra.run.smithi190.stdout: "op_r": 0, 2024-09-16T02:06:10.887 INFO:teuthology.orchestra.run.smithi190.stdout: "op_w": 37, 2024-09-16T02:06:10.887 INFO:teuthology.orchestra.run.smithi190.stdout: "op_rmw": 0, 2024-09-16T02:06:10.887 INFO:teuthology.orchestra.run.smithi190.stdout: "op_pg": 0, 2024-09-16T02:06:10.887 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_stat": 0, 2024-09-16T02:06:10.887 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_create": 10, 2024-09-16T02:06:10.887 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_read": 0, 2024-09-16T02:06:10.887 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_write": 2, 2024-09-16T02:06:10.887 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writefull": 6, 2024-09-16T02:06:10.887 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writesame": 0, 2024-09-16T02:06:10.888 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_append": 0, 2024-09-16T02:06:10.888 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_zero": 0, 2024-09-16T02:06:10.888 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_truncate": 0, 2024-09-16T02:06:10.888 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_delete": 6, 2024-09-16T02:06:10.888 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_mapext": 0, 2024-09-16T02:06:10.888 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_sparse_read": 0, 2024-09-16T02:06:10.888 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_clonerange": 0, 2024-09-16T02:06:10.888 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_getxattr": 0, 2024-09-16T02:06:10.888 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_setxattr": 20, 2024-09-16T02:06:10.888 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_cmpxattr": 0, 2024-09-16T02:06:10.888 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_rmxattr": 0, 2024-09-16T02:06:10.888 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_resetxattrs": 0, 2024-09-16T02:06:10.888 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_call": 0, 2024-09-16T02:06:10.888 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_watch": 0, 2024-09-16T02:06:10.889 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_notify": 0, 2024-09-16T02:06:10.889 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T02:06:10.889 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls": 0, 2024-09-16T02:06:10.889 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls_filter": 0, 2024-09-16T02:06:10.889 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_other": 0, 2024-09-16T02:06:10.889 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_active": 0, 2024-09-16T02:06:10.889 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_send": 0, 2024-09-16T02:06:10.889 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_resend": 0, 2024-09-16T02:06:10.889 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_ping": 0, 2024-09-16T02:06:10.889 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_active": 0, 2024-09-16T02:06:10.889 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_send": 0, 2024-09-16T02:06:10.889 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_resend": 0, 2024-09-16T02:06:10.889 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_active": 0, 2024-09-16T02:06:10.889 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_send": 0, 2024-09-16T02:06:10.890 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_resend": 0, 2024-09-16T02:06:10.890 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_active": 0, 2024-09-16T02:06:10.890 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_send": 0, 2024-09-16T02:06:10.890 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_resend": 0, 2024-09-16T02:06:10.890 INFO:teuthology.orchestra.run.smithi190.stdout: "command_active": 0, 2024-09-16T02:06:10.890 INFO:teuthology.orchestra.run.smithi190.stdout: "command_send": 0, 2024-09-16T02:06:10.890 INFO:teuthology.orchestra.run.smithi190.stdout: "command_resend": 0, 2024-09-16T02:06:10.890 INFO:teuthology.orchestra.run.smithi190.stdout: "map_epoch": 83, 2024-09-16T02:06:10.890 INFO:teuthology.orchestra.run.smithi190.stdout: "map_full": 0, 2024-09-16T02:06:10.890 INFO:teuthology.orchestra.run.smithi190.stdout: "map_inc": 4, 2024-09-16T02:06:10.890 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_sessions": 8, 2024-09-16T02:06:10.890 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_open": 8, 2024-09-16T02:06:10.890 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_close": 0, 2024-09-16T02:06:10.890 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_laggy": 0, 2024-09-16T02:06:10.891 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_wr": 14, 2024-09-16T02:06:10.891 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_rd": 0, 2024-09-16T02:06:10.891 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_del": 1 2024-09-16T02:06:10.891 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.891 INFO:teuthology.orchestra.run.smithi190.stdout: "oft": { 2024-09-16T02:06:10.891 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_objs": 1, 2024-09-16T02:06:10.891 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_kv_pairs": 0, 2024-09-16T02:06:10.891 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_updates": 0, 2024-09-16T02:06:10.891 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_removes": 0 2024-09-16T02:06:10.891 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.891 INFO:teuthology.orchestra.run.smithi190.stdout: "osd-slow-ops": { 2024-09-16T02:06:10.891 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_ops_count": 0 2024-09-16T02:06:10.891 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.891 INFO:teuthology.orchestra.run.smithi190.stdout: "purge_queue": { 2024-09-16T02:06:10.892 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops": 0, 2024-09-16T02:06:10.892 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T02:06:10.892 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing": 0, 2024-09-16T02:06:10.892 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_high_water": 0, 2024-09-16T02:06:10.892 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed_ops": 0, 2024-09-16T02:06:10.892 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed": 0, 2024-09-16T02:06:10.892 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_item_in_journal": 0 2024-09-16T02:06:10.892 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.892 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T02:06:10.892 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-16T02:06:10.892 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-09-16T02:06:10.892 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-16T02:06:10.892 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 455, 2024-09-16T02:06:10.892 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 556781, 2024-09-16T02:06:10.893 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:10.893 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 455, 2024-09-16T02:06:10.893 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-09-16T02:06:10.893 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-09-16T02:06:10.893 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 455, 2024-09-16T02:06:10.893 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 556781, 2024-09-16T02:06:10.893 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-16T02:06:10.893 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:10.893 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:10.893 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:10.893 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-16T02:06:10.893 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.893 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_bytes": { 2024-09-16T02:06:10.894 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-16T02:06:10.894 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-09-16T02:06:10.894 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-16T02:06:10.894 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-09-16T02:06:10.894 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-09-16T02:06:10.894 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:10.894 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-09-16T02:06:10.894 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 37, 2024-09-16T02:06:10.894 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 11295, 2024-09-16T02:06:10.894 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 37, 2024-09-16T02:06:10.894 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 11295, 2024-09-16T02:06:10.894 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-16T02:06:10.894 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:10.895 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:10.895 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:10.895 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-16T02:06:10.896 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.896 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_ops": { 2024-09-16T02:06:10.896 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-16T02:06:10.896 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 1024, 2024-09-16T02:06:10.896 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-16T02:06:10.896 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-09-16T02:06:10.896 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-09-16T02:06:10.896 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:10.896 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-09-16T02:06:10.896 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 37, 2024-09-16T02:06:10.896 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 37, 2024-09-16T02:06:10.896 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 37, 2024-09-16T02:06:10.896 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 37, 2024-09-16T02:06:10.897 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-16T02:06:10.897 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:10.897 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:10.897 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:10.897 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-16T02:06:10.897 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.897 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle": { 2024-09-16T02:06:10.897 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-16T02:06:10.897 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2024-09-16T02:06:10.897 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-16T02:06:10.897 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-09-16T02:06:10.897 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-09-16T02:06:10.897 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:10.897 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-09-16T02:06:10.898 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-09-16T02:06:10.898 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-09-16T02:06:10.898 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 0, 2024-09-16T02:06:10.898 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 0, 2024-09-16T02:06:10.898 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-16T02:06:10.898 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:10.898 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:10.898 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:10.898 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-16T02:06:10.898 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:10.898 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle-0x563f86aa6960": { 2024-09-16T02:06:10.898 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-16T02:06:10.898 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2024-09-16T02:06:10.898 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-16T02:06:10.899 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 3, 2024-09-16T02:06:10.899 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 1083, 2024-09-16T02:06:10.899 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:10.899 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 3, 2024-09-16T02:06:10.899 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-09-16T02:06:10.899 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-09-16T02:06:10.899 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 2, 2024-09-16T02:06:10.899 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 1083, 2024-09-16T02:06:10.899 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-16T02:06:10.899 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:10.899 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:10.899 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:10.899 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-16T02:06:10.899 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-16T02:06:10.900 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-16T02:06:10.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:10 smithi137 ceph-mon[34099]: pgmap v448: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-16T02:06:11.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:10 smithi105 ceph-mon[31197]: pgmap v448: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-16T02:06:11.738 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-16T02:06:11.738 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-16T02:06:11.738 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-09-16T02:06:11.738 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=11 2024-09-16T02:06:11.738 DEBUG:tasks.check_counter:Getting stats from f 2024-09-16T02:06:11.739 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-16T02:06:11.989 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T02:06:12.570 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-16T02:06:12.570 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T02:06:12.570 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 376, 2024-09-16T02:06:12.571 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 407, 2024-09-16T02:06:12.571 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 563280, 2024-09-16T02:06:12.571 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 169586, 2024-09-16T02:06:12.571 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 9, 2024-09-16T02:06:12.571 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 4, 2024-09-16T02:06:12.571 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.045992828, 2024-09-16T02:06:12.571 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.017351130, 2024-09-16T02:06:12.571 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.031099882, 2024-09-16T02:06:12.571 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.002077094, 2024-09-16T02:06:12.571 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:06:12.571 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 406, 2024-09-16T02:06:12.571 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.031905564, 2024-09-16T02:06:12.571 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000078585 2024-09-16T02:06:12.571 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.572 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:06:12.572 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 467, 2024-09-16T02:06:12.572 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000107739, 2024-09-16T02:06:12.572 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000230 2024-09-16T02:06:12.572 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.572 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 484016, 2024-09-16T02:06:12.572 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 67824 2024-09-16T02:06:12.572 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.572 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T02:06:12.572 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 39, 2024-09-16T02:06:12.572 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 113, 2024-09-16T02:06:12.572 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 18218, 2024-09-16T02:06:12.572 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 279289, 2024-09-16T02:06:12.572 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 13, 2024-09-16T02:06:12.573 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 6, 2024-09-16T02:06:12.573 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.012162115, 2024-09-16T02:06:12.573 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.004280706, 2024-09-16T02:06:12.573 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.006813812, 2024-09-16T02:06:12.573 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.000251374, 2024-09-16T02:06:12.573 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:06:12.573 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 112, 2024-09-16T02:06:12.573 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.019938700, 2024-09-16T02:06:12.573 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000178024 2024-09-16T02:06:12.573 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.573 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:06:12.573 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 41, 2024-09-16T02:06:12.573 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000008542, 2024-09-16T02:06:12.573 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000208 2024-09-16T02:06:12.574 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.574 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-16T02:06:12.574 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 208432 2024-09-16T02:06:12.574 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.574 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T02:06:12.574 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 44, 2024-09-16T02:06:12.574 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 46, 2024-09-16T02:06:12.574 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 17271, 2024-09-16T02:06:12.574 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 141356, 2024-09-16T02:06:12.574 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 18, 2024-09-16T02:06:12.574 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 9, 2024-09-16T02:06:12.574 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.009775928, 2024-09-16T02:06:12.574 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.001029131, 2024-09-16T02:06:12.574 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.008038588, 2024-09-16T02:06:12.574 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.000530554, 2024-09-16T02:06:12.575 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:06:12.575 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 44, 2024-09-16T02:06:12.575 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.028805475, 2024-09-16T02:06:12.575 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000654669 2024-09-16T02:06:12.575 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.575 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:06:12.575 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 41, 2024-09-16T02:06:12.575 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000022067, 2024-09-16T02:06:12.575 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000538 2024-09-16T02:06:12.575 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.575 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T02:06:12.575 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T02:06:12.575 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.575 INFO:teuthology.orchestra.run.smithi190.stdout: "cct": { 2024-09-16T02:06:12.576 INFO:teuthology.orchestra.run.smithi190.stdout: "total_workers": 1, 2024-09-16T02:06:12.576 INFO:teuthology.orchestra.run.smithi190.stdout: "unhealthy_workers": 0 2024-09-16T02:06:12.576 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.576 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-MDSRank": { 2024-09-16T02:06:12.576 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-09-16T02:06:12.576 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-09-16T02:06:12.576 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 43, 2024-09-16T02:06:12.576 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.001336731, 2024-09-16T02:06:12.576 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000031086 2024-09-16T02:06:12.576 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-16T02:06:12.576 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.576 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-PurgeQueue": { 2024-09-16T02:06:12.576 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-09-16T02:06:12.576 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-09-16T02:06:12.577 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2, 2024-09-16T02:06:12.577 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000007712, 2024-09-16T02:06:12.577 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000003856 2024-09-16T02:06:12.577 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-16T02:06:12.577 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.577 INFO:teuthology.orchestra.run.smithi190.stdout: "mds": { 2024-09-16T02:06:12.577 INFO:teuthology.orchestra.run.smithi190.stdout: "request": 0, 2024-09-16T02:06:12.577 INFO:teuthology.orchestra.run.smithi190.stdout: "reply": 0, 2024-09-16T02:06:12.577 INFO:teuthology.orchestra.run.smithi190.stdout: "reply_latency": { 2024-09-16T02:06:12.577 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:12.577 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:12.577 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:12.577 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.577 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_reply": 0, 2024-09-16T02:06:12.578 INFO:teuthology.orchestra.run.smithi190.stdout: "forward": 0, 2024-09-16T02:06:12.578 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_complete": 0, 2024-09-16T02:06:12.578 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_keys": 0, 2024-09-16T02:06:12.578 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_commit": 11, 2024-09-16T02:06:12.578 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_split": 0, 2024-09-16T02:06:12.578 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_merge": 0, 2024-09-16T02:06:12.578 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes": 16, 2024-09-16T02:06:12.578 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_top": 0, 2024-09-16T02:06:12.578 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_bottom": 16, 2024-09-16T02:06:12.578 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pin_tail": 0, 2024-09-16T02:06:12.578 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pinned": 13, 2024-09-16T02:06:12.578 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_expired": 4, 2024-09-16T02:06:12.578 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_with_caps": 0, 2024-09-16T02:06:12.578 INFO:teuthology.orchestra.run.smithi190.stdout: "caps": 0, 2024-09-16T02:06:12.579 INFO:teuthology.orchestra.run.smithi190.stdout: "subtrees": 3, 2024-09-16T02:06:12.579 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse": 7, 2024-09-16T02:06:12.579 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_hit": 4, 2024-09-16T02:06:12.579 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_forward": 0, 2024-09-16T02:06:12.579 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_discover": 3, 2024-09-16T02:06:12.579 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_dir_fetch": 0, 2024-09-16T02:06:12.579 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_remote_ino": 0, 2024-09-16T02:06:12.579 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_lock": 0, 2024-09-16T02:06:12.579 INFO:teuthology.orchestra.run.smithi190.stdout: "load_cent": 0, 2024-09-16T02:06:12.579 INFO:teuthology.orchestra.run.smithi190.stdout: "q": 0, 2024-09-16T02:06:12.579 INFO:teuthology.orchestra.run.smithi190.stdout: "exported": 0, 2024-09-16T02:06:12.579 INFO:teuthology.orchestra.run.smithi190.stdout: "exported_inodes": 0, 2024-09-16T02:06:12.579 INFO:teuthology.orchestra.run.smithi190.stdout: "imported": 0, 2024-09-16T02:06:12.579 INFO:teuthology.orchestra.run.smithi190.stdout: "imported_inodes": 0, 2024-09-16T02:06:12.580 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_dir_fetch": 0, 2024-09-16T02:06:12.580 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_backtrace_fetch": 0, 2024-09-16T02:06:12.580 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_peer_discover": 0, 2024-09-16T02:06:12.580 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rfiles": 4, 2024-09-16T02:06:12.580 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rbytes": 234, 2024-09-16T02:06:12.580 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rsnaps": 0, 2024-09-16T02:06:12.580 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T02:06:12.580 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_set_tag": 0, 2024-09-16T02:06:12.580 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T02:06:12.580 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_inotable_repaired": 0, 2024-09-16T02:06:12.580 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_inodes": 0, 2024-09-16T02:06:12.580 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T02:06:12.580 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T02:06:12.580 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_file_inodes": 0, 2024-09-16T02:06:12.581 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_inode_file_caps": 0, 2024-09-16T02:06:12.581 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T02:06:12.581 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_grant": 0, 2024-09-16T02:06:12.581 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T02:06:12.581 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T02:06:12.581 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T02:06:12.581 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps": 0, 2024-09-16T02:06:12.581 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps_dirty": 0, 2024-09-16T02:06:12.581 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_cap_release": 0, 2024-09-16T02:06:12.581 INFO:teuthology.orchestra.run.smithi190.stdout: "process_request_cap_release": 0 2024-09-16T02:06:12.581 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.581 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_cache": { 2024-09-16T02:06:12.581 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update": 0, 2024-09-16T02:06:12.581 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update_receipt": 11, 2024-09-16T02:06:12.582 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_try_discover": 7, 2024-09-16T02:06:12.582 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_send_discover": 8, 2024-09-16T02:06:12.582 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_handle_discover": 0, 2024-09-16T02:06:12.582 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays": 0, 2024-09-16T02:06:12.582 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_delayed": 0, 2024-09-16T02:06:12.582 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_enqueuing": 0, 2024-09-16T02:06:12.582 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_created": 0, 2024-09-16T02:06:12.582 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_enqueued": 0, 2024-09-16T02:06:12.582 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_reintegrated": 0, 2024-09-16T02:06:12.582 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_migrated": 0, 2024-09-16T02:06:12.582 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_processing": 0, 2024-09-16T02:06:12.582 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_enqueued": 0, 2024-09-16T02:06:12.582 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_prioritized": 0, 2024-09-16T02:06:12.582 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_started": 0, 2024-09-16T02:06:12.583 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_completed": 0, 2024-09-16T02:06:12.583 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_quiesce_path": 0, 2024-09-16T02:06:12.583 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_quiesce_inode": 0, 2024-09-16T02:06:12.583 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T02:06:12.583 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_exportdir": 0, 2024-09-16T02:06:12.583 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_flush": 0, 2024-09-16T02:06:12.583 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragmentdir": 0, 2024-09-16T02:06:12.583 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragstats": 0, 2024-09-16T02:06:12.583 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_inodestats": 0 2024-09-16T02:06:12.583 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.583 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_log": { 2024-09-16T02:06:12.583 INFO:teuthology.orchestra.run.smithi190.stdout: "evlrg": 0, 2024-09-16T02:06:12.583 INFO:teuthology.orchestra.run.smithi190.stdout: "evadd": 3, 2024-09-16T02:06:12.583 INFO:teuthology.orchestra.run.smithi190.stdout: "evex": 0, 2024-09-16T02:06:12.583 INFO:teuthology.orchestra.run.smithi190.stdout: "evtrm": 0, 2024-09-16T02:06:12.584 INFO:teuthology.orchestra.run.smithi190.stdout: "ev": 3, 2024-09-16T02:06:12.584 INFO:teuthology.orchestra.run.smithi190.stdout: "evexg": 0, 2024-09-16T02:06:12.584 INFO:teuthology.orchestra.run.smithi190.stdout: "evexd": 0, 2024-09-16T02:06:12.584 INFO:teuthology.orchestra.run.smithi190.stdout: "segadd": 2, 2024-09-16T02:06:12.584 INFO:teuthology.orchestra.run.smithi190.stdout: "segex": 0, 2024-09-16T02:06:12.584 INFO:teuthology.orchestra.run.smithi190.stdout: "segtrm": 0, 2024-09-16T02:06:12.584 INFO:teuthology.orchestra.run.smithi190.stdout: "seg": 2, 2024-09-16T02:06:12.584 INFO:teuthology.orchestra.run.smithi190.stdout: "segmjr": 2, 2024-09-16T02:06:12.584 INFO:teuthology.orchestra.run.smithi190.stdout: "segexg": 0, 2024-09-16T02:06:12.584 INFO:teuthology.orchestra.run.smithi190.stdout: "segexd": 0, 2024-09-16T02:06:12.584 INFO:teuthology.orchestra.run.smithi190.stdout: "expos": 4194304, 2024-09-16T02:06:12.584 INFO:teuthology.orchestra.run.smithi190.stdout: "wrpos": 4195387, 2024-09-16T02:06:12.585 INFO:teuthology.orchestra.run.smithi190.stdout: "rdpos": 0, 2024-09-16T02:06:12.585 INFO:teuthology.orchestra.run.smithi190.stdout: "jlat": { 2024-09-16T02:06:12.585 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2, 2024-09-16T02:06:12.585 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.002951645, 2024-09-16T02:06:12.585 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.001475822 2024-09-16T02:06:12.585 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.585 INFO:teuthology.orchestra.run.smithi190.stdout: "replayed": 0 2024-09-16T02:06:12.585 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.585 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_mem": { 2024-09-16T02:06:12.585 INFO:teuthology.orchestra.run.smithi190.stdout: "ino": 20, 2024-09-16T02:06:12.585 INFO:teuthology.orchestra.run.smithi190.stdout: "ino+": 22, 2024-09-16T02:06:12.585 INFO:teuthology.orchestra.run.smithi190.stdout: "ino-": 2, 2024-09-16T02:06:12.585 INFO:teuthology.orchestra.run.smithi190.stdout: "dir": 19, 2024-09-16T02:06:12.585 INFO:teuthology.orchestra.run.smithi190.stdout: "dir+": 23, 2024-09-16T02:06:12.586 INFO:teuthology.orchestra.run.smithi190.stdout: "dir-": 4, 2024-09-16T02:06:12.586 INFO:teuthology.orchestra.run.smithi190.stdout: "dn": 16, 2024-09-16T02:06:12.586 INFO:teuthology.orchestra.run.smithi190.stdout: "dn+": 20, 2024-09-16T02:06:12.586 INFO:teuthology.orchestra.run.smithi190.stdout: "dn-": 4, 2024-09-16T02:06:12.586 INFO:teuthology.orchestra.run.smithi190.stdout: "cap": 0, 2024-09-16T02:06:12.586 INFO:teuthology.orchestra.run.smithi190.stdout: "cap+": 0, 2024-09-16T02:06:12.586 INFO:teuthology.orchestra.run.smithi190.stdout: "cap-": 0, 2024-09-16T02:06:12.586 INFO:teuthology.orchestra.run.smithi190.stdout: "rss": 52668, 2024-09-16T02:06:12.586 INFO:teuthology.orchestra.run.smithi190.stdout: "heap": 223520 2024-09-16T02:06:12.586 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.586 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_server": { 2024-09-16T02:06:12.586 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_client_request": 0, 2024-09-16T02:06:12.586 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_server_request": 0, 2024-09-16T02:06:12.586 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_request": 0, 2024-09-16T02:06:12.587 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_session": 5, 2024-09-16T02:06:12.587 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_peer_request": 0, 2024-09-16T02:06:12.587 INFO:teuthology.orchestra.run.smithi190.stdout: "req_create_latency": { 2024-09-16T02:06:12.587 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:12.587 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:12.587 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:12.587 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.587 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getattr_latency": { 2024-09-16T02:06:12.587 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:12.587 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:12.587 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:12.587 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.587 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getfilelock_latency": { 2024-09-16T02:06:12.587 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:12.587 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:12.588 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:12.588 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.588 INFO:teuthology.orchestra.run.smithi190.stdout: "req_link_latency": { 2024-09-16T02:06:12.588 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:12.588 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:12.588 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:12.588 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.588 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookup_latency": { 2024-09-16T02:06:12.588 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:12.588 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:12.588 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:12.588 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.588 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookuphash_latency": { 2024-09-16T02:06:12.588 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:12.588 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:12.589 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:12.589 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.589 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupino_latency": { 2024-09-16T02:06:12.589 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:12.589 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:12.589 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:12.589 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.589 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupname_latency": { 2024-09-16T02:06:12.589 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:12.589 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:12.589 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:12.589 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.589 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupparent_latency": { 2024-09-16T02:06:12.590 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:12.590 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:12.590 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:12.590 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.590 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupsnap_latency": { 2024-09-16T02:06:12.590 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:12.590 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:12.590 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:12.590 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.590 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lssnap_latency": { 2024-09-16T02:06:12.590 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:12.590 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:12.591 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:12.591 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.591 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mkdir_latency": { 2024-09-16T02:06:12.591 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:12.591 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:12.591 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:12.591 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.591 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mknod_latency": { 2024-09-16T02:06:12.591 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:12.591 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:12.591 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:12.591 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.591 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mksnap_latency": { 2024-09-16T02:06:12.591 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:12.591 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:12.592 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:12.592 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.592 INFO:teuthology.orchestra.run.smithi190.stdout: "req_open_latency": { 2024-09-16T02:06:12.592 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:12.592 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:12.592 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:12.592 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.592 INFO:teuthology.orchestra.run.smithi190.stdout: "req_readdir_latency": { 2024-09-16T02:06:12.592 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:12.592 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:12.592 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:12.592 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.592 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rename_latency": { 2024-09-16T02:06:12.592 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:12.593 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:12.593 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:12.593 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.593 INFO:teuthology.orchestra.run.smithi190.stdout: "req_renamesnap_latency": { 2024-09-16T02:06:12.593 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:12.593 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:12.593 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:12.593 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.593 INFO:teuthology.orchestra.run.smithi190.stdout: "req_snapdiff_latency": { 2024-09-16T02:06:12.593 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:12.593 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:12.593 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:12.593 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.593 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmdir_latency": { 2024-09-16T02:06:12.594 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:12.594 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:12.594 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:12.594 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.594 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmsnap_latency": { 2024-09-16T02:06:12.594 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:12.594 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:12.594 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:12.594 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.594 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmxattr_latency": { 2024-09-16T02:06:12.594 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:12.594 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:12.594 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:12.594 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.594 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setattr_latency": { 2024-09-16T02:06:12.595 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:12.595 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:12.595 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:12.595 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.595 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setdirlayout_latency": { 2024-09-16T02:06:12.595 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:12.595 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:12.595 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:12.595 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.595 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setfilelock_latency": { 2024-09-16T02:06:12.595 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:12.596 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:12.596 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:12.596 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.596 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setlayout_latency": { 2024-09-16T02:06:12.596 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:12.596 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:12.596 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:12.596 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.596 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setxattr_latency": { 2024-09-16T02:06:12.596 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:12.596 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:12.596 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:12.597 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.597 INFO:teuthology.orchestra.run.smithi190.stdout: "req_symlink_latency": { 2024-09-16T02:06:12.597 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:12.597 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:12.597 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:12.597 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.597 INFO:teuthology.orchestra.run.smithi190.stdout: "req_unlink_latency": { 2024-09-16T02:06:12.597 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:12.597 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:12.597 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:12.597 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.597 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_revoke_eviction": 0, 2024-09-16T02:06:12.597 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_acquisition_throttle": 0, 2024-09-16T02:06:12.597 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getvxattr_latency": { 2024-09-16T02:06:12.598 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:12.598 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:12.598 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:12.598 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-16T02:06:12.598 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.598 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_sessions": { 2024-09-16T02:06:12.598 INFO:teuthology.orchestra.run.smithi190.stdout: "session_count": 1, 2024-09-16T02:06:12.598 INFO:teuthology.orchestra.run.smithi190.stdout: "session_add": 1, 2024-09-16T02:06:12.598 INFO:teuthology.orchestra.run.smithi190.stdout: "session_remove": 0, 2024-09-16T02:06:12.598 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_open": 1, 2024-09-16T02:06:12.598 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_stale": 0, 2024-09-16T02:06:12.598 INFO:teuthology.orchestra.run.smithi190.stdout: "total_load": 0, 2024-09-16T02:06:12.598 INFO:teuthology.orchestra.run.smithi190.stdout: "average_load": 0, 2024-09-16T02:06:12.598 INFO:teuthology.orchestra.run.smithi190.stdout: "avg_session_uptime": 213, 2024-09-16T02:06:12.599 INFO:teuthology.orchestra.run.smithi190.stdout: "mdthresh_evicted": 0 2024-09-16T02:06:12.599 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.599 INFO:teuthology.orchestra.run.smithi190.stdout: "mempool": { 2024-09-16T02:06:12.599 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_bytes": 0, 2024-09-16T02:06:12.599 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_items": 0, 2024-09-16T02:06:12.599 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T02:06:12.599 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_items": 0, 2024-09-16T02:06:12.599 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T02:06:12.599 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_items": 0, 2024-09-16T02:06:12.599 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T02:06:12.599 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T02:06:12.599 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T02:06:12.599 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T02:06:12.599 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T02:06:12.599 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_items": 0, 2024-09-16T02:06:12.600 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T02:06:12.600 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T02:06:12.600 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_bytes": 0, 2024-09-16T02:06:12.600 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_items": 0, 2024-09-16T02:06:12.600 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_bytes": 0, 2024-09-16T02:06:12.600 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_items": 0, 2024-09-16T02:06:12.600 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T02:06:12.600 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T02:06:12.600 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T02:06:12.600 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T02:06:12.600 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T02:06:12.600 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_items": 0, 2024-09-16T02:06:12.600 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_bytes": 0, 2024-09-16T02:06:12.600 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_items": 0, 2024-09-16T02:06:12.601 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T02:06:12.601 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T02:06:12.601 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_bytes": 0, 2024-09-16T02:06:12.601 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_items": 0, 2024-09-16T02:06:12.601 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_bytes": 0, 2024-09-16T02:06:12.601 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_items": 0, 2024-09-16T02:06:12.601 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T02:06:12.601 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_items": 0, 2024-09-16T02:06:12.601 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T02:06:12.601 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_items": 0, 2024-09-16T02:06:12.601 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_bytes": 238455, 2024-09-16T02:06:12.601 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_items": 84, 2024-09-16T02:06:12.601 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_bytes": 0, 2024-09-16T02:06:12.601 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_items": 0, 2024-09-16T02:06:12.602 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_bytes": 0, 2024-09-16T02:06:12.602 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_items": 0, 2024-09-16T02:06:12.602 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_bytes": 0, 2024-09-16T02:06:12.602 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_items": 0, 2024-09-16T02:06:12.602 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_bytes": 0, 2024-09-16T02:06:12.602 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_items": 0, 2024-09-16T02:06:12.602 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_bytes": 7888, 2024-09-16T02:06:12.602 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_items": 163, 2024-09-16T02:06:12.602 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T02:06:12.602 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_items": 0, 2024-09-16T02:06:12.602 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_bytes": 0, 2024-09-16T02:06:12.602 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_items": 0, 2024-09-16T02:06:12.602 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_bytes": 80909, 2024-09-16T02:06:12.602 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_items": 320, 2024-09-16T02:06:12.603 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_bytes": 0, 2024-09-16T02:06:12.603 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_items": 0, 2024-09-16T02:06:12.603 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_bytes": 0, 2024-09-16T02:06:12.603 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_items": 0 2024-09-16T02:06:12.603 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.603 INFO:teuthology.orchestra.run.smithi190.stdout: "objecter": { 2024-09-16T02:06:12.603 INFO:teuthology.orchestra.run.smithi190.stdout: "op_active": 0, 2024-09-16T02:06:12.603 INFO:teuthology.orchestra.run.smithi190.stdout: "op_laggy": 0, 2024-09-16T02:06:12.603 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send": 37, 2024-09-16T02:06:12.603 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send_bytes": 11295, 2024-09-16T02:06:12.603 INFO:teuthology.orchestra.run.smithi190.stdout: "op_resend": 33, 2024-09-16T02:06:12.603 INFO:teuthology.orchestra.run.smithi190.stdout: "op_reply": 37, 2024-09-16T02:06:12.603 INFO:teuthology.orchestra.run.smithi190.stdout: "op_latency": { 2024-09-16T02:06:12.603 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 37, 2024-09-16T02:06:12.603 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.200994270, 2024-09-16T02:06:12.604 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.005432277 2024-09-16T02:06:12.604 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.604 INFO:teuthology.orchestra.run.smithi190.stdout: "op_inflight": 0, 2024-09-16T02:06:12.604 INFO:teuthology.orchestra.run.smithi190.stdout: "oplen_avg": { 2024-09-16T02:06:12.604 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 37, 2024-09-16T02:06:12.604 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 59 2024-09-16T02:06:12.604 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.604 INFO:teuthology.orchestra.run.smithi190.stdout: "op": 37, 2024-09-16T02:06:12.604 INFO:teuthology.orchestra.run.smithi190.stdout: "op_r": 0, 2024-09-16T02:06:12.604 INFO:teuthology.orchestra.run.smithi190.stdout: "op_w": 37, 2024-09-16T02:06:12.604 INFO:teuthology.orchestra.run.smithi190.stdout: "op_rmw": 0, 2024-09-16T02:06:12.604 INFO:teuthology.orchestra.run.smithi190.stdout: "op_pg": 0, 2024-09-16T02:06:12.604 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_stat": 0, 2024-09-16T02:06:12.604 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_create": 10, 2024-09-16T02:06:12.605 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_read": 0, 2024-09-16T02:06:12.605 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_write": 2, 2024-09-16T02:06:12.605 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writefull": 6, 2024-09-16T02:06:12.605 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writesame": 0, 2024-09-16T02:06:12.605 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_append": 0, 2024-09-16T02:06:12.605 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_zero": 0, 2024-09-16T02:06:12.605 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_truncate": 0, 2024-09-16T02:06:12.605 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_delete": 6, 2024-09-16T02:06:12.605 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_mapext": 0, 2024-09-16T02:06:12.605 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_sparse_read": 0, 2024-09-16T02:06:12.605 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_clonerange": 0, 2024-09-16T02:06:12.605 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_getxattr": 0, 2024-09-16T02:06:12.605 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_setxattr": 20, 2024-09-16T02:06:12.605 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_cmpxattr": 0, 2024-09-16T02:06:12.605 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_rmxattr": 0, 2024-09-16T02:06:12.606 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_resetxattrs": 0, 2024-09-16T02:06:12.606 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_call": 0, 2024-09-16T02:06:12.606 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_watch": 0, 2024-09-16T02:06:12.606 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_notify": 0, 2024-09-16T02:06:12.606 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T02:06:12.606 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls": 0, 2024-09-16T02:06:12.606 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls_filter": 0, 2024-09-16T02:06:12.606 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_other": 0, 2024-09-16T02:06:12.606 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_active": 0, 2024-09-16T02:06:12.606 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_send": 0, 2024-09-16T02:06:12.606 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_resend": 0, 2024-09-16T02:06:12.606 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_ping": 0, 2024-09-16T02:06:12.606 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_active": 0, 2024-09-16T02:06:12.606 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_send": 0, 2024-09-16T02:06:12.607 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_resend": 0, 2024-09-16T02:06:12.607 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_active": 0, 2024-09-16T02:06:12.607 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_send": 0, 2024-09-16T02:06:12.607 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_resend": 0, 2024-09-16T02:06:12.607 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_active": 0, 2024-09-16T02:06:12.607 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_send": 0, 2024-09-16T02:06:12.607 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_resend": 0, 2024-09-16T02:06:12.607 INFO:teuthology.orchestra.run.smithi190.stdout: "command_active": 0, 2024-09-16T02:06:12.607 INFO:teuthology.orchestra.run.smithi190.stdout: "command_send": 0, 2024-09-16T02:06:12.607 INFO:teuthology.orchestra.run.smithi190.stdout: "command_resend": 0, 2024-09-16T02:06:12.607 INFO:teuthology.orchestra.run.smithi190.stdout: "map_epoch": 83, 2024-09-16T02:06:12.607 INFO:teuthology.orchestra.run.smithi190.stdout: "map_full": 0, 2024-09-16T02:06:12.607 INFO:teuthology.orchestra.run.smithi190.stdout: "map_inc": 4, 2024-09-16T02:06:12.607 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_sessions": 9, 2024-09-16T02:06:12.608 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_open": 9, 2024-09-16T02:06:12.608 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_close": 0, 2024-09-16T02:06:12.608 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_laggy": 0, 2024-09-16T02:06:12.608 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_wr": 14, 2024-09-16T02:06:12.608 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_rd": 0, 2024-09-16T02:06:12.608 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_del": 1 2024-09-16T02:06:12.608 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.608 INFO:teuthology.orchestra.run.smithi190.stdout: "oft": { 2024-09-16T02:06:12.608 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_objs": 1, 2024-09-16T02:06:12.608 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_kv_pairs": 0, 2024-09-16T02:06:12.608 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_updates": 0, 2024-09-16T02:06:12.608 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_removes": 0 2024-09-16T02:06:12.608 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.608 INFO:teuthology.orchestra.run.smithi190.stdout: "osd-slow-ops": { 2024-09-16T02:06:12.608 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_ops_count": 0 2024-09-16T02:06:12.609 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.609 INFO:teuthology.orchestra.run.smithi190.stdout: "purge_queue": { 2024-09-16T02:06:12.609 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops": 0, 2024-09-16T02:06:12.609 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T02:06:12.609 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing": 0, 2024-09-16T02:06:12.609 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_high_water": 0, 2024-09-16T02:06:12.609 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed_ops": 0, 2024-09-16T02:06:12.609 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed": 0, 2024-09-16T02:06:12.609 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_item_in_journal": 0 2024-09-16T02:06:12.609 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.609 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T02:06:12.609 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-16T02:06:12.609 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-09-16T02:06:12.609 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-16T02:06:12.609 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 455, 2024-09-16T02:06:12.610 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 547466, 2024-09-16T02:06:12.610 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:12.610 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 455, 2024-09-16T02:06:12.610 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-09-16T02:06:12.610 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-09-16T02:06:12.610 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 455, 2024-09-16T02:06:12.610 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 547466, 2024-09-16T02:06:12.610 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-16T02:06:12.610 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:12.610 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:12.610 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:12.610 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-16T02:06:12.610 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.610 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_bytes": { 2024-09-16T02:06:12.611 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-16T02:06:12.611 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-09-16T02:06:12.611 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-16T02:06:12.611 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-09-16T02:06:12.611 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-09-16T02:06:12.611 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:12.611 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-09-16T02:06:12.611 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 37, 2024-09-16T02:06:12.611 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 11295, 2024-09-16T02:06:12.611 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 37, 2024-09-16T02:06:12.611 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 11295, 2024-09-16T02:06:12.611 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-16T02:06:12.611 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:12.611 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:12.612 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:12.612 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-16T02:06:12.612 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.612 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_ops": { 2024-09-16T02:06:12.612 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-16T02:06:12.612 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 1024, 2024-09-16T02:06:12.612 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-16T02:06:12.612 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-09-16T02:06:12.612 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-09-16T02:06:12.612 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:12.612 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-09-16T02:06:12.612 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 37, 2024-09-16T02:06:12.612 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 37, 2024-09-16T02:06:12.613 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 37, 2024-09-16T02:06:12.613 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 37, 2024-09-16T02:06:12.613 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-16T02:06:12.613 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:12.613 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:12.613 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:12.613 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-16T02:06:12.613 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.613 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle": { 2024-09-16T02:06:12.613 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-16T02:06:12.613 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2024-09-16T02:06:12.614 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-16T02:06:12.614 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-09-16T02:06:12.614 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-09-16T02:06:12.614 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:12.614 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-09-16T02:06:12.614 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-09-16T02:06:12.614 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-09-16T02:06:12.614 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 0, 2024-09-16T02:06:12.614 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 0, 2024-09-16T02:06:12.614 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-16T02:06:12.614 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:12.615 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:12.615 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:12.615 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-16T02:06:12.615 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:12.615 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle-0x55a48b1be960": { 2024-09-16T02:06:12.615 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-16T02:06:12.615 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2024-09-16T02:06:12.615 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-16T02:06:12.615 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 3, 2024-09-16T02:06:12.615 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 1083, 2024-09-16T02:06:12.616 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:12.616 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 3, 2024-09-16T02:06:12.616 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-09-16T02:06:12.616 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-09-16T02:06:12.616 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 2, 2024-09-16T02:06:12.616 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 1083, 2024-09-16T02:06:12.616 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-16T02:06:12.616 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:12.616 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:12.616 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:12.616 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-16T02:06:12.616 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-16T02:06:12.616 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-16T02:06:12.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:12 smithi190 ceph-mon[34762]: pgmap v449: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T02:06:12.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:12 smithi137 ceph-mon[34099]: pgmap v449: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T02:06:13.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:12 smithi105 ceph-mon[31197]: pgmap v449: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T02:06:13.361 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-16T02:06:13.361 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-16T02:06:13.361 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-09-16T02:06:13.362 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=11 2024-09-16T02:06:13.362 DEBUG:tasks.check_counter:Getting stats from i 2024-09-16T02:06:13.362 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-16T02:06:13.606 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T02:06:14.147 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-16T02:06:14.147 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T02:06:14.147 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 391, 2024-09-16T02:06:14.147 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 420, 2024-09-16T02:06:14.147 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 581345, 2024-09-16T02:06:14.147 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 172253, 2024-09-16T02:06:14.148 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 9, 2024-09-16T02:06:14.148 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 4, 2024-09-16T02:06:14.148 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.049649710, 2024-09-16T02:06:14.148 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.018176686, 2024-09-16T02:06:14.148 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.033748855, 2024-09-16T02:06:14.148 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.002238405, 2024-09-16T02:06:14.148 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:06:14.148 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 419, 2024-09-16T02:06:14.148 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.037656665, 2024-09-16T02:06:14.148 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000089872 2024-09-16T02:06:14.148 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.148 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:06:14.148 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 489, 2024-09-16T02:06:14.148 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000108857, 2024-09-16T02:06:14.149 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000222 2024-09-16T02:06:14.149 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.149 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 498272, 2024-09-16T02:06:14.149 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 66688 2024-09-16T02:06:14.149 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.149 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T02:06:14.149 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 42, 2024-09-16T02:06:14.149 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 114, 2024-09-16T02:06:14.149 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 19031, 2024-09-16T02:06:14.149 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 275215, 2024-09-16T02:06:14.149 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 14, 2024-09-16T02:06:14.149 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 4, 2024-09-16T02:06:14.149 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.012802304, 2024-09-16T02:06:14.150 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.004081973, 2024-09-16T02:06:14.150 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.007765685, 2024-09-16T02:06:14.150 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.000187186, 2024-09-16T02:06:14.150 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:06:14.150 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 113, 2024-09-16T02:06:14.150 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.021175809, 2024-09-16T02:06:14.150 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000187396 2024-09-16T02:06:14.150 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.150 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:06:14.150 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 45, 2024-09-16T02:06:14.150 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000007937, 2024-09-16T02:06:14.150 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000176 2024-09-16T02:06:14.150 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.150 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-16T02:06:14.151 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 208080 2024-09-16T02:06:14.151 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.151 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T02:06:14.151 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 41, 2024-09-16T02:06:14.151 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 43, 2024-09-16T02:06:14.151 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 16615, 2024-09-16T02:06:14.151 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 145427, 2024-09-16T02:06:14.151 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 18, 2024-09-16T02:06:14.151 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 5, 2024-09-16T02:06:14.151 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.008471966, 2024-09-16T02:06:14.151 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.001386621, 2024-09-16T02:06:14.151 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.006217549, 2024-09-16T02:06:14.151 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.000488999, 2024-09-16T02:06:14.151 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:06:14.151 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 41, 2024-09-16T02:06:14.152 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.013519554, 2024-09-16T02:06:14.152 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000329745 2024-09-16T02:06:14.152 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.152 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:06:14.152 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 36, 2024-09-16T02:06:14.152 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000006135, 2024-09-16T02:06:14.152 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000170 2024-09-16T02:06:14.152 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.152 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T02:06:14.152 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T02:06:14.152 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.152 INFO:teuthology.orchestra.run.smithi190.stdout: "cct": { 2024-09-16T02:06:14.153 INFO:teuthology.orchestra.run.smithi190.stdout: "total_workers": 1, 2024-09-16T02:06:14.153 INFO:teuthology.orchestra.run.smithi190.stdout: "unhealthy_workers": 0 2024-09-16T02:06:14.153 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.153 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-MDSRank": { 2024-09-16T02:06:14.153 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-09-16T02:06:14.153 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-09-16T02:06:14.153 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 36, 2024-09-16T02:06:14.153 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.001963886, 2024-09-16T02:06:14.153 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000054552 2024-09-16T02:06:14.153 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-16T02:06:14.153 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.153 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-PurgeQueue": { 2024-09-16T02:06:14.153 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-09-16T02:06:14.153 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-09-16T02:06:14.154 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2, 2024-09-16T02:06:14.154 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000012223, 2024-09-16T02:06:14.154 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000006111 2024-09-16T02:06:14.154 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-16T02:06:14.154 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.154 INFO:teuthology.orchestra.run.smithi190.stdout: "mds": { 2024-09-16T02:06:14.154 INFO:teuthology.orchestra.run.smithi190.stdout: "request": 2, 2024-09-16T02:06:14.154 INFO:teuthology.orchestra.run.smithi190.stdout: "reply": 2, 2024-09-16T02:06:14.154 INFO:teuthology.orchestra.run.smithi190.stdout: "reply_latency": { 2024-09-16T02:06:14.154 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2, 2024-09-16T02:06:14.154 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.001321907, 2024-09-16T02:06:14.154 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000660953 2024-09-16T02:06:14.154 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.154 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_reply": 0, 2024-09-16T02:06:14.155 INFO:teuthology.orchestra.run.smithi190.stdout: "forward": 0, 2024-09-16T02:06:14.155 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_complete": 0, 2024-09-16T02:06:14.155 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_keys": 0, 2024-09-16T02:06:14.155 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_commit": 11, 2024-09-16T02:06:14.155 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_split": 0, 2024-09-16T02:06:14.155 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_merge": 0, 2024-09-16T02:06:14.155 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes": 16, 2024-09-16T02:06:14.155 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_top": 0, 2024-09-16T02:06:14.155 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_bottom": 16, 2024-09-16T02:06:14.155 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pin_tail": 0, 2024-09-16T02:06:14.155 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pinned": 13, 2024-09-16T02:06:14.155 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_expired": 4, 2024-09-16T02:06:14.155 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_with_caps": 2, 2024-09-16T02:06:14.155 INFO:teuthology.orchestra.run.smithi190.stdout: "caps": 2, 2024-09-16T02:06:14.156 INFO:teuthology.orchestra.run.smithi190.stdout: "subtrees": 3, 2024-09-16T02:06:14.156 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse": 10, 2024-09-16T02:06:14.156 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_hit": 7, 2024-09-16T02:06:14.156 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_forward": 0, 2024-09-16T02:06:14.156 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_discover": 3, 2024-09-16T02:06:14.156 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_dir_fetch": 0, 2024-09-16T02:06:14.156 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_remote_ino": 0, 2024-09-16T02:06:14.156 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_lock": 0, 2024-09-16T02:06:14.156 INFO:teuthology.orchestra.run.smithi190.stdout: "load_cent": 1000, 2024-09-16T02:06:14.156 INFO:teuthology.orchestra.run.smithi190.stdout: "q": 1, 2024-09-16T02:06:14.156 INFO:teuthology.orchestra.run.smithi190.stdout: "exported": 0, 2024-09-16T02:06:14.156 INFO:teuthology.orchestra.run.smithi190.stdout: "exported_inodes": 0, 2024-09-16T02:06:14.156 INFO:teuthology.orchestra.run.smithi190.stdout: "imported": 0, 2024-09-16T02:06:14.157 INFO:teuthology.orchestra.run.smithi190.stdout: "imported_inodes": 0, 2024-09-16T02:06:14.157 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_dir_fetch": 0, 2024-09-16T02:06:14.157 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_backtrace_fetch": 0, 2024-09-16T02:06:14.157 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_peer_discover": 0, 2024-09-16T02:06:14.157 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rfiles": 4, 2024-09-16T02:06:14.157 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rbytes": 234, 2024-09-16T02:06:14.157 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rsnaps": 0, 2024-09-16T02:06:14.157 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T02:06:14.157 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_set_tag": 0, 2024-09-16T02:06:14.157 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T02:06:14.157 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_inotable_repaired": 0, 2024-09-16T02:06:14.157 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_inodes": 0, 2024-09-16T02:06:14.157 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T02:06:14.158 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T02:06:14.158 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_file_inodes": 0, 2024-09-16T02:06:14.158 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_inode_file_caps": 0, 2024-09-16T02:06:14.158 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T02:06:14.158 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_grant": 0, 2024-09-16T02:06:14.158 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T02:06:14.158 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T02:06:14.158 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T02:06:14.158 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps": 0, 2024-09-16T02:06:14.158 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps_dirty": 0, 2024-09-16T02:06:14.158 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_cap_release": 0, 2024-09-16T02:06:14.158 INFO:teuthology.orchestra.run.smithi190.stdout: "process_request_cap_release": 0 2024-09-16T02:06:14.158 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.158 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_cache": { 2024-09-16T02:06:14.159 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update": 0, 2024-09-16T02:06:14.159 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update_receipt": 11, 2024-09-16T02:06:14.159 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_try_discover": 7, 2024-09-16T02:06:14.159 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_send_discover": 8, 2024-09-16T02:06:14.159 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_handle_discover": 0, 2024-09-16T02:06:14.159 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays": 0, 2024-09-16T02:06:14.159 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_delayed": 0, 2024-09-16T02:06:14.159 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_enqueuing": 0, 2024-09-16T02:06:14.159 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_created": 0, 2024-09-16T02:06:14.159 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_enqueued": 0, 2024-09-16T02:06:14.159 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_reintegrated": 0, 2024-09-16T02:06:14.159 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_migrated": 0, 2024-09-16T02:06:14.159 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_processing": 0, 2024-09-16T02:06:14.160 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_enqueued": 0, 2024-09-16T02:06:14.160 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_prioritized": 0, 2024-09-16T02:06:14.160 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_started": 0, 2024-09-16T02:06:14.160 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_completed": 0, 2024-09-16T02:06:14.160 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_quiesce_path": 0, 2024-09-16T02:06:14.160 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_quiesce_inode": 0, 2024-09-16T02:06:14.160 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T02:06:14.160 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_exportdir": 0, 2024-09-16T02:06:14.160 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_flush": 0, 2024-09-16T02:06:14.160 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragmentdir": 0, 2024-09-16T02:06:14.160 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragstats": 0, 2024-09-16T02:06:14.160 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_inodestats": 0 2024-09-16T02:06:14.160 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.161 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_log": { 2024-09-16T02:06:14.161 INFO:teuthology.orchestra.run.smithi190.stdout: "evlrg": 0, 2024-09-16T02:06:14.161 INFO:teuthology.orchestra.run.smithi190.stdout: "evadd": 3, 2024-09-16T02:06:14.161 INFO:teuthology.orchestra.run.smithi190.stdout: "evex": 0, 2024-09-16T02:06:14.161 INFO:teuthology.orchestra.run.smithi190.stdout: "evtrm": 0, 2024-09-16T02:06:14.161 INFO:teuthology.orchestra.run.smithi190.stdout: "ev": 3, 2024-09-16T02:06:14.161 INFO:teuthology.orchestra.run.smithi190.stdout: "evexg": 0, 2024-09-16T02:06:14.161 INFO:teuthology.orchestra.run.smithi190.stdout: "evexd": 0, 2024-09-16T02:06:14.161 INFO:teuthology.orchestra.run.smithi190.stdout: "segadd": 2, 2024-09-16T02:06:14.161 INFO:teuthology.orchestra.run.smithi190.stdout: "segex": 0, 2024-09-16T02:06:14.161 INFO:teuthology.orchestra.run.smithi190.stdout: "segtrm": 0, 2024-09-16T02:06:14.161 INFO:teuthology.orchestra.run.smithi190.stdout: "seg": 2, 2024-09-16T02:06:14.161 INFO:teuthology.orchestra.run.smithi190.stdout: "segmjr": 2, 2024-09-16T02:06:14.161 INFO:teuthology.orchestra.run.smithi190.stdout: "segexg": 0, 2024-09-16T02:06:14.162 INFO:teuthology.orchestra.run.smithi190.stdout: "segexd": 0, 2024-09-16T02:06:14.162 INFO:teuthology.orchestra.run.smithi190.stdout: "expos": 4194304, 2024-09-16T02:06:14.162 INFO:teuthology.orchestra.run.smithi190.stdout: "wrpos": 4195387, 2024-09-16T02:06:14.162 INFO:teuthology.orchestra.run.smithi190.stdout: "rdpos": 0, 2024-09-16T02:06:14.162 INFO:teuthology.orchestra.run.smithi190.stdout: "jlat": { 2024-09-16T02:06:14.162 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2, 2024-09-16T02:06:14.162 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.003016545, 2024-09-16T02:06:14.162 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.001508272 2024-09-16T02:06:14.162 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.162 INFO:teuthology.orchestra.run.smithi190.stdout: "replayed": 0 2024-09-16T02:06:14.162 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.162 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_mem": { 2024-09-16T02:06:14.162 INFO:teuthology.orchestra.run.smithi190.stdout: "ino": 20, 2024-09-16T02:06:14.162 INFO:teuthology.orchestra.run.smithi190.stdout: "ino+": 22, 2024-09-16T02:06:14.163 INFO:teuthology.orchestra.run.smithi190.stdout: "ino-": 2, 2024-09-16T02:06:14.163 INFO:teuthology.orchestra.run.smithi190.stdout: "dir": 19, 2024-09-16T02:06:14.163 INFO:teuthology.orchestra.run.smithi190.stdout: "dir+": 23, 2024-09-16T02:06:14.163 INFO:teuthology.orchestra.run.smithi190.stdout: "dir-": 4, 2024-09-16T02:06:14.163 INFO:teuthology.orchestra.run.smithi190.stdout: "dn": 16, 2024-09-16T02:06:14.163 INFO:teuthology.orchestra.run.smithi190.stdout: "dn+": 20, 2024-09-16T02:06:14.163 INFO:teuthology.orchestra.run.smithi190.stdout: "dn-": 4, 2024-09-16T02:06:14.163 INFO:teuthology.orchestra.run.smithi190.stdout: "cap": 2, 2024-09-16T02:06:14.163 INFO:teuthology.orchestra.run.smithi190.stdout: "cap+": 2, 2024-09-16T02:06:14.163 INFO:teuthology.orchestra.run.smithi190.stdout: "cap-": 0, 2024-09-16T02:06:14.163 INFO:teuthology.orchestra.run.smithi190.stdout: "rss": 50552, 2024-09-16T02:06:14.163 INFO:teuthology.orchestra.run.smithi190.stdout: "heap": 223520 2024-09-16T02:06:14.163 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.163 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_server": { 2024-09-16T02:06:14.164 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_client_request": 5, 2024-09-16T02:06:14.164 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_server_request": 0, 2024-09-16T02:06:14.164 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_request": 2, 2024-09-16T02:06:14.164 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_session": 5, 2024-09-16T02:06:14.164 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_peer_request": 1, 2024-09-16T02:06:14.164 INFO:teuthology.orchestra.run.smithi190.stdout: "req_create_latency": { 2024-09-16T02:06:14.164 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:14.164 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:14.164 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:14.164 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.164 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getattr_latency": { 2024-09-16T02:06:14.164 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2, 2024-09-16T02:06:14.164 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.001321907, 2024-09-16T02:06:14.165 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000660953 2024-09-16T02:06:14.165 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.165 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getfilelock_latency": { 2024-09-16T02:06:14.165 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:14.165 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:14.165 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:14.165 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.165 INFO:teuthology.orchestra.run.smithi190.stdout: "req_link_latency": { 2024-09-16T02:06:14.165 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:14.165 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:14.165 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:14.166 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.166 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookup_latency": { 2024-09-16T02:06:14.166 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:14.166 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:14.166 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:14.166 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.166 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookuphash_latency": { 2024-09-16T02:06:14.166 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:14.166 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:14.166 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:14.166 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.166 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupino_latency": { 2024-09-16T02:06:14.166 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:14.167 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:14.167 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:14.167 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.167 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupname_latency": { 2024-09-16T02:06:14.167 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:14.167 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:14.167 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:14.167 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.167 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupparent_latency": { 2024-09-16T02:06:14.167 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:14.167 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:14.167 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:14.167 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.167 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupsnap_latency": { 2024-09-16T02:06:14.167 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:14.168 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:14.168 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:14.168 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.168 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lssnap_latency": { 2024-09-16T02:06:14.168 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:14.168 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:14.168 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:14.168 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.168 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mkdir_latency": { 2024-09-16T02:06:14.168 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:14.168 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:14.168 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:14.168 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.169 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mknod_latency": { 2024-09-16T02:06:14.169 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:14.169 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:14.169 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:14.169 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.169 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mksnap_latency": { 2024-09-16T02:06:14.169 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:14.169 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:14.169 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:14.169 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.169 INFO:teuthology.orchestra.run.smithi190.stdout: "req_open_latency": { 2024-09-16T02:06:14.169 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:14.169 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:14.169 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:14.170 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.170 INFO:teuthology.orchestra.run.smithi190.stdout: "req_readdir_latency": { 2024-09-16T02:06:14.170 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:14.170 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:14.170 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:14.170 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.170 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rename_latency": { 2024-09-16T02:06:14.170 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:14.170 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:14.170 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:14.170 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.170 INFO:teuthology.orchestra.run.smithi190.stdout: "req_renamesnap_latency": { 2024-09-16T02:06:14.170 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:14.170 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:14.171 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:14.171 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.171 INFO:teuthology.orchestra.run.smithi190.stdout: "req_snapdiff_latency": { 2024-09-16T02:06:14.171 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:14.171 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:14.171 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:14.171 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.171 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmdir_latency": { 2024-09-16T02:06:14.171 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:14.171 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:14.171 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:14.171 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.171 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmsnap_latency": { 2024-09-16T02:06:14.171 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:14.172 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:14.172 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:14.172 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.172 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmxattr_latency": { 2024-09-16T02:06:14.172 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:14.172 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:14.172 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:14.172 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.172 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setattr_latency": { 2024-09-16T02:06:14.172 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:14.172 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:14.172 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:14.172 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.172 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setdirlayout_latency": { 2024-09-16T02:06:14.173 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:14.173 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:14.173 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:14.173 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.173 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setfilelock_latency": { 2024-09-16T02:06:14.173 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:14.173 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:14.173 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:14.173 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.173 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setlayout_latency": { 2024-09-16T02:06:14.173 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:14.173 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:14.173 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:14.173 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.174 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setxattr_latency": { 2024-09-16T02:06:14.174 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:14.174 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:14.174 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:14.174 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.174 INFO:teuthology.orchestra.run.smithi190.stdout: "req_symlink_latency": { 2024-09-16T02:06:14.174 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:14.174 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:14.174 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:14.174 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.174 INFO:teuthology.orchestra.run.smithi190.stdout: "req_unlink_latency": { 2024-09-16T02:06:14.174 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:14.174 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:14.174 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:14.175 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.175 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_revoke_eviction": 0, 2024-09-16T02:06:14.175 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_acquisition_throttle": 0, 2024-09-16T02:06:14.175 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getvxattr_latency": { 2024-09-16T02:06:14.175 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:14.175 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:14.175 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:14.175 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-16T02:06:14.175 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.175 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_sessions": { 2024-09-16T02:06:14.175 INFO:teuthology.orchestra.run.smithi190.stdout: "session_count": 1, 2024-09-16T02:06:14.175 INFO:teuthology.orchestra.run.smithi190.stdout: "session_add": 1, 2024-09-16T02:06:14.175 INFO:teuthology.orchestra.run.smithi190.stdout: "session_remove": 0, 2024-09-16T02:06:14.175 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_open": 1, 2024-09-16T02:06:14.176 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_stale": 0, 2024-09-16T02:06:14.176 INFO:teuthology.orchestra.run.smithi190.stdout: "total_load": 1, 2024-09-16T02:06:14.176 INFO:teuthology.orchestra.run.smithi190.stdout: "average_load": 1, 2024-09-16T02:06:14.176 INFO:teuthology.orchestra.run.smithi190.stdout: "avg_session_uptime": 211, 2024-09-16T02:06:14.176 INFO:teuthology.orchestra.run.smithi190.stdout: "mdthresh_evicted": 0 2024-09-16T02:06:14.176 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.176 INFO:teuthology.orchestra.run.smithi190.stdout: "mempool": { 2024-09-16T02:06:14.176 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_bytes": 0, 2024-09-16T02:06:14.176 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_items": 0, 2024-09-16T02:06:14.176 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T02:06:14.176 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_items": 0, 2024-09-16T02:06:14.176 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T02:06:14.176 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_items": 0, 2024-09-16T02:06:14.176 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T02:06:14.177 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T02:06:14.177 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T02:06:14.177 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T02:06:14.177 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T02:06:14.177 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_items": 0, 2024-09-16T02:06:14.177 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T02:06:14.177 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T02:06:14.177 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_bytes": 0, 2024-09-16T02:06:14.177 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_items": 0, 2024-09-16T02:06:14.177 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_bytes": 0, 2024-09-16T02:06:14.177 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_items": 0, 2024-09-16T02:06:14.177 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T02:06:14.177 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T02:06:14.177 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T02:06:14.177 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T02:06:14.178 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T02:06:14.178 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_items": 0, 2024-09-16T02:06:14.178 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_bytes": 0, 2024-09-16T02:06:14.178 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_items": 0, 2024-09-16T02:06:14.178 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T02:06:14.178 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T02:06:14.178 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_bytes": 0, 2024-09-16T02:06:14.178 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_items": 0, 2024-09-16T02:06:14.178 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_bytes": 0, 2024-09-16T02:06:14.178 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_items": 0, 2024-09-16T02:06:14.178 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T02:06:14.178 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_items": 0, 2024-09-16T02:06:14.178 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T02:06:14.178 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_items": 0, 2024-09-16T02:06:14.178 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_bytes": 254273, 2024-09-16T02:06:14.179 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_items": 82, 2024-09-16T02:06:14.179 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_bytes": 0, 2024-09-16T02:06:14.179 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_items": 0, 2024-09-16T02:06:14.179 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_bytes": 0, 2024-09-16T02:06:14.179 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_items": 0, 2024-09-16T02:06:14.179 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_bytes": 0, 2024-09-16T02:06:14.179 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_items": 0, 2024-09-16T02:06:14.179 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_bytes": 0, 2024-09-16T02:06:14.179 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_items": 0, 2024-09-16T02:06:14.179 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_bytes": 7888, 2024-09-16T02:06:14.179 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_items": 163, 2024-09-16T02:06:14.179 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T02:06:14.179 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_items": 0, 2024-09-16T02:06:14.179 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_bytes": 0, 2024-09-16T02:06:14.180 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_items": 0, 2024-09-16T02:06:14.180 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_bytes": 81709, 2024-09-16T02:06:14.180 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_items": 332, 2024-09-16T02:06:14.180 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_bytes": 0, 2024-09-16T02:06:14.180 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_items": 0, 2024-09-16T02:06:14.180 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_bytes": 0, 2024-09-16T02:06:14.180 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_items": 0 2024-09-16T02:06:14.180 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.180 INFO:teuthology.orchestra.run.smithi190.stdout: "objecter": { 2024-09-16T02:06:14.180 INFO:teuthology.orchestra.run.smithi190.stdout: "op_active": 0, 2024-09-16T02:06:14.180 INFO:teuthology.orchestra.run.smithi190.stdout: "op_laggy": 0, 2024-09-16T02:06:14.180 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send": 38, 2024-09-16T02:06:14.180 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send_bytes": 11450, 2024-09-16T02:06:14.180 INFO:teuthology.orchestra.run.smithi190.stdout: "op_resend": 32, 2024-09-16T02:06:14.180 INFO:teuthology.orchestra.run.smithi190.stdout: "op_reply": 38, 2024-09-16T02:06:14.181 INFO:teuthology.orchestra.run.smithi190.stdout: "op_latency": { 2024-09-16T02:06:14.181 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 38, 2024-09-16T02:06:14.181 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.216993808, 2024-09-16T02:06:14.181 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.005710363 2024-09-16T02:06:14.181 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.181 INFO:teuthology.orchestra.run.smithi190.stdout: "op_inflight": 0, 2024-09-16T02:06:14.181 INFO:teuthology.orchestra.run.smithi190.stdout: "oplen_avg": { 2024-09-16T02:06:14.181 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 38, 2024-09-16T02:06:14.181 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 61 2024-09-16T02:06:14.181 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.181 INFO:teuthology.orchestra.run.smithi190.stdout: "op": 38, 2024-09-16T02:06:14.181 INFO:teuthology.orchestra.run.smithi190.stdout: "op_r": 0, 2024-09-16T02:06:14.181 INFO:teuthology.orchestra.run.smithi190.stdout: "op_w": 38, 2024-09-16T02:06:14.181 INFO:teuthology.orchestra.run.smithi190.stdout: "op_rmw": 0, 2024-09-16T02:06:14.182 INFO:teuthology.orchestra.run.smithi190.stdout: "op_pg": 0, 2024-09-16T02:06:14.182 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_stat": 0, 2024-09-16T02:06:14.182 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_create": 10, 2024-09-16T02:06:14.182 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_read": 0, 2024-09-16T02:06:14.182 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_write": 2, 2024-09-16T02:06:14.182 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writefull": 6, 2024-09-16T02:06:14.182 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writesame": 0, 2024-09-16T02:06:14.182 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_append": 0, 2024-09-16T02:06:14.182 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_zero": 0, 2024-09-16T02:06:14.182 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_truncate": 0, 2024-09-16T02:06:14.182 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_delete": 6, 2024-09-16T02:06:14.182 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_mapext": 0, 2024-09-16T02:06:14.183 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_sparse_read": 0, 2024-09-16T02:06:14.183 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_clonerange": 0, 2024-09-16T02:06:14.183 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_getxattr": 0, 2024-09-16T02:06:14.183 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_setxattr": 20, 2024-09-16T02:06:14.183 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_cmpxattr": 0, 2024-09-16T02:06:14.183 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_rmxattr": 0, 2024-09-16T02:06:14.183 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_resetxattrs": 0, 2024-09-16T02:06:14.183 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_call": 0, 2024-09-16T02:06:14.183 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_watch": 0, 2024-09-16T02:06:14.183 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_notify": 0, 2024-09-16T02:06:14.183 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T02:06:14.183 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls": 0, 2024-09-16T02:06:14.183 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls_filter": 0, 2024-09-16T02:06:14.183 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_other": 0, 2024-09-16T02:06:14.184 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_active": 0, 2024-09-16T02:06:14.184 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_send": 0, 2024-09-16T02:06:14.184 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_resend": 0, 2024-09-16T02:06:14.184 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_ping": 0, 2024-09-16T02:06:14.184 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_active": 0, 2024-09-16T02:06:14.184 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_send": 0, 2024-09-16T02:06:14.184 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_resend": 0, 2024-09-16T02:06:14.184 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_active": 0, 2024-09-16T02:06:14.184 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_send": 0, 2024-09-16T02:06:14.184 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_resend": 0, 2024-09-16T02:06:14.184 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_active": 0, 2024-09-16T02:06:14.184 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_send": 0, 2024-09-16T02:06:14.184 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_resend": 0, 2024-09-16T02:06:14.184 INFO:teuthology.orchestra.run.smithi190.stdout: "command_active": 0, 2024-09-16T02:06:14.184 INFO:teuthology.orchestra.run.smithi190.stdout: "command_send": 0, 2024-09-16T02:06:14.185 INFO:teuthology.orchestra.run.smithi190.stdout: "command_resend": 0, 2024-09-16T02:06:14.185 INFO:teuthology.orchestra.run.smithi190.stdout: "map_epoch": 83, 2024-09-16T02:06:14.185 INFO:teuthology.orchestra.run.smithi190.stdout: "map_full": 0, 2024-09-16T02:06:14.185 INFO:teuthology.orchestra.run.smithi190.stdout: "map_inc": 4, 2024-09-16T02:06:14.185 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_sessions": 9, 2024-09-16T02:06:14.185 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_open": 9, 2024-09-16T02:06:14.185 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_close": 0, 2024-09-16T02:06:14.185 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_laggy": 0, 2024-09-16T02:06:14.185 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_wr": 16, 2024-09-16T02:06:14.185 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_rd": 0, 2024-09-16T02:06:14.185 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_del": 1 2024-09-16T02:06:14.185 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.185 INFO:teuthology.orchestra.run.smithi190.stdout: "oft": { 2024-09-16T02:06:14.185 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_objs": 1, 2024-09-16T02:06:14.186 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_kv_pairs": 0, 2024-09-16T02:06:14.186 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_updates": 0, 2024-09-16T02:06:14.186 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_removes": 0 2024-09-16T02:06:14.186 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.186 INFO:teuthology.orchestra.run.smithi190.stdout: "osd-slow-ops": { 2024-09-16T02:06:14.186 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_ops_count": 0 2024-09-16T02:06:14.186 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.186 INFO:teuthology.orchestra.run.smithi190.stdout: "purge_queue": { 2024-09-16T02:06:14.186 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops": 0, 2024-09-16T02:06:14.186 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T02:06:14.186 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing": 0, 2024-09-16T02:06:14.186 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_high_water": 0, 2024-09-16T02:06:14.186 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed_ops": 0, 2024-09-16T02:06:14.186 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed": 0, 2024-09-16T02:06:14.187 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_item_in_journal": 0 2024-09-16T02:06:14.187 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.187 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T02:06:14.187 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-16T02:06:14.187 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-09-16T02:06:14.187 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-16T02:06:14.187 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 470, 2024-09-16T02:06:14.187 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 564203, 2024-09-16T02:06:14.187 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:14.187 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 470, 2024-09-16T02:06:14.187 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-09-16T02:06:14.187 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-09-16T02:06:14.187 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 470, 2024-09-16T02:06:14.187 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 564203, 2024-09-16T02:06:14.187 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-16T02:06:14.188 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:14.188 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:14.188 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:14.188 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-16T02:06:14.188 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.188 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_bytes": { 2024-09-16T02:06:14.188 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-16T02:06:14.188 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-09-16T02:06:14.188 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-16T02:06:14.188 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-09-16T02:06:14.188 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-09-16T02:06:14.188 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:14.188 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-09-16T02:06:14.188 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 38, 2024-09-16T02:06:14.189 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 11450, 2024-09-16T02:06:14.189 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 38, 2024-09-16T02:06:14.189 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 11450, 2024-09-16T02:06:14.189 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-16T02:06:14.189 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:14.189 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:14.189 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:14.189 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-16T02:06:14.189 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.189 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_ops": { 2024-09-16T02:06:14.189 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-16T02:06:14.189 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 1024, 2024-09-16T02:06:14.189 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-16T02:06:14.190 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-09-16T02:06:14.190 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-09-16T02:06:14.190 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:14.190 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-09-16T02:06:14.190 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 38, 2024-09-16T02:06:14.190 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 38, 2024-09-16T02:06:14.190 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 38, 2024-09-16T02:06:14.190 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 38, 2024-09-16T02:06:14.190 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-16T02:06:14.190 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:14.190 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:14.190 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:14.190 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-16T02:06:14.190 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.190 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle": { 2024-09-16T02:06:14.191 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-16T02:06:14.191 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2024-09-16T02:06:14.191 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-16T02:06:14.191 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-09-16T02:06:14.191 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-09-16T02:06:14.191 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:14.191 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-09-16T02:06:14.191 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-09-16T02:06:14.191 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-09-16T02:06:14.191 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 0, 2024-09-16T02:06:14.191 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 0, 2024-09-16T02:06:14.191 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-16T02:06:14.191 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:14.191 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:14.191 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:14.192 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-16T02:06:14.192 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:14.192 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle-0x55dda41de960": { 2024-09-16T02:06:14.192 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-16T02:06:14.192 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2024-09-16T02:06:14.192 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-16T02:06:14.192 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 3, 2024-09-16T02:06:14.192 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 1083, 2024-09-16T02:06:14.192 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:14.192 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 3, 2024-09-16T02:06:14.192 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-09-16T02:06:14.192 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-09-16T02:06:14.192 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 2, 2024-09-16T02:06:14.192 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 1083, 2024-09-16T02:06:14.192 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-16T02:06:14.193 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:14.193 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:14.193 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:14.193 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-16T02:06:14.193 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-16T02:06:14.193 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-16T02:06:14.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:14 smithi137 ceph-mon[34099]: pgmap v450: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-16T02:06:14.818 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:14 smithi190 ceph-mon[34762]: pgmap v450: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-16T02:06:14.845 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-16T02:06:14.845 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-09-16T02:06:14.845 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-09-16T02:06:14.845 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=11 2024-09-16T02:06:14.845 DEBUG:tasks.check_counter:Getting stats from l 2024-09-16T02:06:14.846 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-16T02:06:15.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:14 smithi105 ceph-mon[31197]: pgmap v450: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-16T02:06:15.089 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T02:06:15.731 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-16T02:06:15.731 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T02:06:15.731 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 120, 2024-09-16T02:06:15.731 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 116, 2024-09-16T02:06:15.731 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 473490, 2024-09-16T02:06:15.731 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 65908, 2024-09-16T02:06:15.731 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 5, 2024-09-16T02:06:15.731 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 1, 2024-09-16T02:06:15.731 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.019709882, 2024-09-16T02:06:15.731 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.007428666, 2024-09-16T02:06:15.732 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.017223399, 2024-09-16T02:06:15.732 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.000018776, 2024-09-16T02:06:15.732 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:06:15.732 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 115, 2024-09-16T02:06:15.732 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.007310399, 2024-09-16T02:06:15.732 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000063568 2024-09-16T02:06:15.732 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.732 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:06:15.732 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.732 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.732 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.732 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.732 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 472656, 2024-09-16T02:06:15.733 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 65584 2024-09-16T02:06:15.733 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.733 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T02:06:15.733 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 253, 2024-09-16T02:06:15.733 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 330, 2024-09-16T02:06:15.733 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 64182, 2024-09-16T02:06:15.733 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 287829, 2024-09-16T02:06:15.733 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 5, 2024-09-16T02:06:15.733 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 4, 2024-09-16T02:06:15.733 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.031092737, 2024-09-16T02:06:15.733 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.009610992, 2024-09-16T02:06:15.733 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.008937504, 2024-09-16T02:06:15.733 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.009656560, 2024-09-16T02:06:15.734 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:06:15.734 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 329, 2024-09-16T02:06:15.734 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.019186303, 2024-09-16T02:06:15.734 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000058317 2024-09-16T02:06:15.734 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.734 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:06:15.734 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.734 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.734 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.734 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.734 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-16T02:06:15.734 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 208960 2024-09-16T02:06:15.734 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.735 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T02:06:15.735 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 662, 2024-09-16T02:06:15.735 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 664, 2024-09-16T02:06:15.735 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 199336, 2024-09-16T02:06:15.735 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 210574, 2024-09-16T02:06:15.735 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 4, 2024-09-16T02:06:15.735 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 3, 2024-09-16T02:06:15.735 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.063337958, 2024-09-16T02:06:15.735 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.019177570, 2024-09-16T02:06:15.735 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.017770526, 2024-09-16T02:06:15.735 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.020580352, 2024-09-16T02:06:15.735 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:06:15.735 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 662, 2024-09-16T02:06:15.735 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.027807455, 2024-09-16T02:06:15.735 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000042005 2024-09-16T02:06:15.736 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.736 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:06:15.736 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.736 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.736 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.736 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.736 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T02:06:15.736 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T02:06:15.736 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.736 INFO:teuthology.orchestra.run.smithi190.stdout: "cct": { 2024-09-16T02:06:15.736 INFO:teuthology.orchestra.run.smithi190.stdout: "total_workers": 1, 2024-09-16T02:06:15.736 INFO:teuthology.orchestra.run.smithi190.stdout: "unhealthy_workers": 0 2024-09-16T02:06:15.737 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.737 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-MDSRank": { 2024-09-16T02:06:15.737 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-09-16T02:06:15.737 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-09-16T02:06:15.737 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1533, 2024-09-16T02:06:15.737 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.045459137, 2024-09-16T02:06:15.737 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000029653 2024-09-16T02:06:15.737 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-16T02:06:15.737 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.737 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-PurgeQueue": { 2024-09-16T02:06:15.737 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-09-16T02:06:15.737 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-09-16T02:06:15.737 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.737 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.738 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.738 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-16T02:06:15.738 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.738 INFO:teuthology.orchestra.run.smithi190.stdout: "mds": { 2024-09-16T02:06:15.738 INFO:teuthology.orchestra.run.smithi190.stdout: "request": 0, 2024-09-16T02:06:15.738 INFO:teuthology.orchestra.run.smithi190.stdout: "reply": 0, 2024-09-16T02:06:15.738 INFO:teuthology.orchestra.run.smithi190.stdout: "reply_latency": { 2024-09-16T02:06:15.738 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.738 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.738 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.738 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.738 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_reply": 0, 2024-09-16T02:06:15.738 INFO:teuthology.orchestra.run.smithi190.stdout: "forward": 0, 2024-09-16T02:06:15.739 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_complete": 0, 2024-09-16T02:06:15.739 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_keys": 0, 2024-09-16T02:06:15.739 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_commit": 0, 2024-09-16T02:06:15.739 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_split": 0, 2024-09-16T02:06:15.739 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_merge": 0, 2024-09-16T02:06:15.739 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes": 0, 2024-09-16T02:06:15.739 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_top": 0, 2024-09-16T02:06:15.739 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_bottom": 0, 2024-09-16T02:06:15.739 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pin_tail": 0, 2024-09-16T02:06:15.739 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pinned": 0, 2024-09-16T02:06:15.740 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_expired": 0, 2024-09-16T02:06:15.740 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_with_caps": 0, 2024-09-16T02:06:15.740 INFO:teuthology.orchestra.run.smithi190.stdout: "caps": 0, 2024-09-16T02:06:15.740 INFO:teuthology.orchestra.run.smithi190.stdout: "subtrees": 1, 2024-09-16T02:06:15.740 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse": 0, 2024-09-16T02:06:15.740 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_hit": 0, 2024-09-16T02:06:15.740 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_forward": 0, 2024-09-16T02:06:15.740 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_discover": 0, 2024-09-16T02:06:15.740 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_dir_fetch": 0, 2024-09-16T02:06:15.740 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_remote_ino": 0, 2024-09-16T02:06:15.740 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_lock": 0, 2024-09-16T02:06:15.740 INFO:teuthology.orchestra.run.smithi190.stdout: "load_cent": 0, 2024-09-16T02:06:15.740 INFO:teuthology.orchestra.run.smithi190.stdout: "q": 0, 2024-09-16T02:06:15.741 INFO:teuthology.orchestra.run.smithi190.stdout: "exported": 0, 2024-09-16T02:06:15.741 INFO:teuthology.orchestra.run.smithi190.stdout: "exported_inodes": 0, 2024-09-16T02:06:15.741 INFO:teuthology.orchestra.run.smithi190.stdout: "imported": 0, 2024-09-16T02:06:15.741 INFO:teuthology.orchestra.run.smithi190.stdout: "imported_inodes": 0, 2024-09-16T02:06:15.741 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_dir_fetch": 0, 2024-09-16T02:06:15.741 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_backtrace_fetch": 0, 2024-09-16T02:06:15.741 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_peer_discover": 0, 2024-09-16T02:06:15.741 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rfiles": 0, 2024-09-16T02:06:15.741 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rbytes": 0, 2024-09-16T02:06:15.741 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rsnaps": 0, 2024-09-16T02:06:15.741 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T02:06:15.741 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_set_tag": 0, 2024-09-16T02:06:15.741 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T02:06:15.741 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_inotable_repaired": 0, 2024-09-16T02:06:15.742 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_inodes": 0, 2024-09-16T02:06:15.742 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T02:06:15.742 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T02:06:15.742 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_file_inodes": 0, 2024-09-16T02:06:15.742 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_inode_file_caps": 0, 2024-09-16T02:06:15.742 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T02:06:15.742 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_grant": 0, 2024-09-16T02:06:15.742 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T02:06:15.742 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T02:06:15.742 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T02:06:15.742 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps": 0, 2024-09-16T02:06:15.742 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps_dirty": 0, 2024-09-16T02:06:15.742 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_cap_release": 0, 2024-09-16T02:06:15.742 INFO:teuthology.orchestra.run.smithi190.stdout: "process_request_cap_release": 0 2024-09-16T02:06:15.742 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.743 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_cache": { 2024-09-16T02:06:15.743 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update": 0, 2024-09-16T02:06:15.743 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update_receipt": 0, 2024-09-16T02:06:15.743 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_try_discover": 0, 2024-09-16T02:06:15.743 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_send_discover": 0, 2024-09-16T02:06:15.743 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_handle_discover": 0, 2024-09-16T02:06:15.743 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays": 0, 2024-09-16T02:06:15.743 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_delayed": 0, 2024-09-16T02:06:15.743 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_enqueuing": 0, 2024-09-16T02:06:15.743 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_created": 0, 2024-09-16T02:06:15.743 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_enqueued": 0, 2024-09-16T02:06:15.743 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_reintegrated": 0, 2024-09-16T02:06:15.743 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_migrated": 0, 2024-09-16T02:06:15.743 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_processing": 0, 2024-09-16T02:06:15.744 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_enqueued": 0, 2024-09-16T02:06:15.744 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_prioritized": 0, 2024-09-16T02:06:15.744 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_started": 0, 2024-09-16T02:06:15.744 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_completed": 0, 2024-09-16T02:06:15.744 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_quiesce_path": 0, 2024-09-16T02:06:15.744 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_quiesce_inode": 0, 2024-09-16T02:06:15.744 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T02:06:15.744 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_exportdir": 0, 2024-09-16T02:06:15.744 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_flush": 0, 2024-09-16T02:06:15.744 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragmentdir": 0, 2024-09-16T02:06:15.744 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragstats": 0, 2024-09-16T02:06:15.744 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_inodestats": 0 2024-09-16T02:06:15.744 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.745 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_log": { 2024-09-16T02:06:15.745 INFO:teuthology.orchestra.run.smithi190.stdout: "evlrg": 0, 2024-09-16T02:06:15.745 INFO:teuthology.orchestra.run.smithi190.stdout: "evadd": 0, 2024-09-16T02:06:15.745 INFO:teuthology.orchestra.run.smithi190.stdout: "evex": 0, 2024-09-16T02:06:15.745 INFO:teuthology.orchestra.run.smithi190.stdout: "evtrm": 0, 2024-09-16T02:06:15.745 INFO:teuthology.orchestra.run.smithi190.stdout: "ev": 5, 2024-09-16T02:06:15.745 INFO:teuthology.orchestra.run.smithi190.stdout: "evexg": 0, 2024-09-16T02:06:15.745 INFO:teuthology.orchestra.run.smithi190.stdout: "evexd": 0, 2024-09-16T02:06:15.745 INFO:teuthology.orchestra.run.smithi190.stdout: "segadd": 0, 2024-09-16T02:06:15.745 INFO:teuthology.orchestra.run.smithi190.stdout: "segex": 0, 2024-09-16T02:06:15.745 INFO:teuthology.orchestra.run.smithi190.stdout: "segtrm": 0, 2024-09-16T02:06:15.745 INFO:teuthology.orchestra.run.smithi190.stdout: "seg": 2, 2024-09-16T02:06:15.745 INFO:teuthology.orchestra.run.smithi190.stdout: "segmjr": 2, 2024-09-16T02:06:15.745 INFO:teuthology.orchestra.run.smithi190.stdout: "segexg": 0, 2024-09-16T02:06:15.746 INFO:teuthology.orchestra.run.smithi190.stdout: "segexd": 0, 2024-09-16T02:06:15.746 INFO:teuthology.orchestra.run.smithi190.stdout: "expos": 4194304, 2024-09-16T02:06:15.746 INFO:teuthology.orchestra.run.smithi190.stdout: "wrpos": 4195921, 2024-09-16T02:06:15.746 INFO:teuthology.orchestra.run.smithi190.stdout: "rdpos": 4195426, 2024-09-16T02:06:15.746 INFO:teuthology.orchestra.run.smithi190.stdout: "jlat": { 2024-09-16T02:06:15.746 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.746 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.746 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.746 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.746 INFO:teuthology.orchestra.run.smithi190.stdout: "replayed": 5 2024-09-16T02:06:15.746 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.746 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_mem": { 2024-09-16T02:06:15.746 INFO:teuthology.orchestra.run.smithi190.stdout: "ino": 3, 2024-09-16T02:06:15.746 INFO:teuthology.orchestra.run.smithi190.stdout: "ino+": 3, 2024-09-16T02:06:15.747 INFO:teuthology.orchestra.run.smithi190.stdout: "ino-": 0, 2024-09-16T02:06:15.747 INFO:teuthology.orchestra.run.smithi190.stdout: "dir": 1, 2024-09-16T02:06:15.747 INFO:teuthology.orchestra.run.smithi190.stdout: "dir+": 1, 2024-09-16T02:06:15.747 INFO:teuthology.orchestra.run.smithi190.stdout: "dir-": 0, 2024-09-16T02:06:15.747 INFO:teuthology.orchestra.run.smithi190.stdout: "dn": 0, 2024-09-16T02:06:15.747 INFO:teuthology.orchestra.run.smithi190.stdout: "dn+": 0, 2024-09-16T02:06:15.747 INFO:teuthology.orchestra.run.smithi190.stdout: "dn-": 0, 2024-09-16T02:06:15.747 INFO:teuthology.orchestra.run.smithi190.stdout: "cap": 0, 2024-09-16T02:06:15.747 INFO:teuthology.orchestra.run.smithi190.stdout: "cap+": 0, 2024-09-16T02:06:15.747 INFO:teuthology.orchestra.run.smithi190.stdout: "cap-": 0, 2024-09-16T02:06:15.747 INFO:teuthology.orchestra.run.smithi190.stdout: "rss": 48244, 2024-09-16T02:06:15.747 INFO:teuthology.orchestra.run.smithi190.stdout: "heap": 215328 2024-09-16T02:06:15.747 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.747 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_server": { 2024-09-16T02:06:15.748 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_client_request": 0, 2024-09-16T02:06:15.748 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_server_request": 0, 2024-09-16T02:06:15.748 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_request": 0, 2024-09-16T02:06:15.748 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_session": 0, 2024-09-16T02:06:15.748 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_peer_request": 0, 2024-09-16T02:06:15.748 INFO:teuthology.orchestra.run.smithi190.stdout: "req_create_latency": { 2024-09-16T02:06:15.748 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.748 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.748 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.748 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.748 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getattr_latency": { 2024-09-16T02:06:15.748 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.748 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.748 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.749 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.749 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getfilelock_latency": { 2024-09-16T02:06:15.749 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.749 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.749 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.749 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.749 INFO:teuthology.orchestra.run.smithi190.stdout: "req_link_latency": { 2024-09-16T02:06:15.749 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.749 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.749 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.749 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.749 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookup_latency": { 2024-09-16T02:06:15.749 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.749 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.750 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.750 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.750 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookuphash_latency": { 2024-09-16T02:06:15.750 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.750 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.750 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.750 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.750 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupino_latency": { 2024-09-16T02:06:15.750 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.750 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.750 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.750 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.750 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupname_latency": { 2024-09-16T02:06:15.750 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.751 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.751 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.751 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.751 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupparent_latency": { 2024-09-16T02:06:15.751 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.751 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.751 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.751 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.751 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupsnap_latency": { 2024-09-16T02:06:15.751 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.751 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.751 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.751 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.751 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lssnap_latency": { 2024-09-16T02:06:15.752 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.752 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.752 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.752 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.752 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mkdir_latency": { 2024-09-16T02:06:15.752 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.752 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.752 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.752 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.752 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mknod_latency": { 2024-09-16T02:06:15.752 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.752 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.752 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.752 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.753 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mksnap_latency": { 2024-09-16T02:06:15.753 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.753 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.753 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.753 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.753 INFO:teuthology.orchestra.run.smithi190.stdout: "req_open_latency": { 2024-09-16T02:06:15.753 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.753 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.753 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.753 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.753 INFO:teuthology.orchestra.run.smithi190.stdout: "req_readdir_latency": { 2024-09-16T02:06:15.753 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.753 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.753 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.754 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.754 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rename_latency": { 2024-09-16T02:06:15.754 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.754 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.754 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.754 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.754 INFO:teuthology.orchestra.run.smithi190.stdout: "req_renamesnap_latency": { 2024-09-16T02:06:15.754 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.754 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.754 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.754 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.754 INFO:teuthology.orchestra.run.smithi190.stdout: "req_snapdiff_latency": { 2024-09-16T02:06:15.754 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.755 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.755 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.755 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.755 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmdir_latency": { 2024-09-16T02:06:15.755 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.755 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.755 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.755 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.755 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmsnap_latency": { 2024-09-16T02:06:15.755 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.755 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.755 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.755 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.755 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmxattr_latency": { 2024-09-16T02:06:15.756 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.756 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.756 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.756 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.756 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setattr_latency": { 2024-09-16T02:06:15.756 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.756 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.756 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.756 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.756 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setdirlayout_latency": { 2024-09-16T02:06:15.756 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.756 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.756 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.757 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.757 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setfilelock_latency": { 2024-09-16T02:06:15.757 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.757 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.757 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.757 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.757 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setlayout_latency": { 2024-09-16T02:06:15.757 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.757 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.757 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.757 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.757 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setxattr_latency": { 2024-09-16T02:06:15.757 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.757 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.758 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.758 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.758 INFO:teuthology.orchestra.run.smithi190.stdout: "req_symlink_latency": { 2024-09-16T02:06:15.758 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.758 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.758 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.758 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.758 INFO:teuthology.orchestra.run.smithi190.stdout: "req_unlink_latency": { 2024-09-16T02:06:15.758 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.758 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.758 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.758 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.758 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_revoke_eviction": 0, 2024-09-16T02:06:15.759 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_acquisition_throttle": 0, 2024-09-16T02:06:15.759 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getvxattr_latency": { 2024-09-16T02:06:15.759 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.759 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.759 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.759 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-16T02:06:15.759 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.759 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_sessions": { 2024-09-16T02:06:15.759 INFO:teuthology.orchestra.run.smithi190.stdout: "session_count": 1, 2024-09-16T02:06:15.759 INFO:teuthology.orchestra.run.smithi190.stdout: "session_add": 2, 2024-09-16T02:06:15.759 INFO:teuthology.orchestra.run.smithi190.stdout: "session_remove": 1, 2024-09-16T02:06:15.759 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_open": 1, 2024-09-16T02:06:15.759 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_stale": 0, 2024-09-16T02:06:15.759 INFO:teuthology.orchestra.run.smithi190.stdout: "total_load": 0, 2024-09-16T02:06:15.760 INFO:teuthology.orchestra.run.smithi190.stdout: "average_load": 0, 2024-09-16T02:06:15.760 INFO:teuthology.orchestra.run.smithi190.stdout: "avg_session_uptime": 1232, 2024-09-16T02:06:15.760 INFO:teuthology.orchestra.run.smithi190.stdout: "mdthresh_evicted": 0 2024-09-16T02:06:15.760 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.760 INFO:teuthology.orchestra.run.smithi190.stdout: "mempool": { 2024-09-16T02:06:15.760 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_bytes": 0, 2024-09-16T02:06:15.760 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_items": 0, 2024-09-16T02:06:15.760 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T02:06:15.760 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_items": 0, 2024-09-16T02:06:15.760 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T02:06:15.760 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_items": 0, 2024-09-16T02:06:15.760 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T02:06:15.760 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T02:06:15.761 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T02:06:15.761 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T02:06:15.761 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T02:06:15.761 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_items": 0, 2024-09-16T02:06:15.761 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T02:06:15.761 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T02:06:15.761 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_bytes": 0, 2024-09-16T02:06:15.761 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_items": 0, 2024-09-16T02:06:15.761 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_bytes": 0, 2024-09-16T02:06:15.761 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_items": 0, 2024-09-16T02:06:15.761 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T02:06:15.761 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T02:06:15.761 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T02:06:15.762 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T02:06:15.762 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T02:06:15.762 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_items": 0, 2024-09-16T02:06:15.762 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_bytes": 0, 2024-09-16T02:06:15.762 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_items": 0, 2024-09-16T02:06:15.762 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T02:06:15.762 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T02:06:15.762 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_bytes": 0, 2024-09-16T02:06:15.762 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_items": 0, 2024-09-16T02:06:15.762 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_bytes": 0, 2024-09-16T02:06:15.762 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_items": 0, 2024-09-16T02:06:15.762 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T02:06:15.762 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_items": 0, 2024-09-16T02:06:15.762 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T02:06:15.763 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_items": 0, 2024-09-16T02:06:15.763 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_bytes": 189273, 2024-09-16T02:06:15.763 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_items": 50, 2024-09-16T02:06:15.763 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_bytes": 0, 2024-09-16T02:06:15.763 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_items": 0, 2024-09-16T02:06:15.763 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_bytes": 0, 2024-09-16T02:06:15.763 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_items": 0, 2024-09-16T02:06:15.763 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_bytes": 0, 2024-09-16T02:06:15.763 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_items": 0, 2024-09-16T02:06:15.763 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_bytes": 0, 2024-09-16T02:06:15.763 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_items": 0, 2024-09-16T02:06:15.763 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_bytes": 7888, 2024-09-16T02:06:15.763 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_items": 163, 2024-09-16T02:06:15.763 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T02:06:15.764 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_items": 0, 2024-09-16T02:06:15.764 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_bytes": 0, 2024-09-16T02:06:15.764 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_items": 0, 2024-09-16T02:06:15.764 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_bytes": 8512, 2024-09-16T02:06:15.764 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_items": 13, 2024-09-16T02:06:15.764 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_bytes": 0, 2024-09-16T02:06:15.764 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_items": 0, 2024-09-16T02:06:15.764 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_bytes": 0, 2024-09-16T02:06:15.764 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_items": 0 2024-09-16T02:06:15.764 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.764 INFO:teuthology.orchestra.run.smithi190.stdout: "objecter": { 2024-09-16T02:06:15.764 INFO:teuthology.orchestra.run.smithi190.stdout: "op_active": 0, 2024-09-16T02:06:15.764 INFO:teuthology.orchestra.run.smithi190.stdout: "op_laggy": 0, 2024-09-16T02:06:15.765 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send": 911, 2024-09-16T02:06:15.765 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send_bytes": 21, 2024-09-16T02:06:15.765 INFO:teuthology.orchestra.run.smithi190.stdout: "op_resend": 3, 2024-09-16T02:06:15.765 INFO:teuthology.orchestra.run.smithi190.stdout: "op_reply": 911, 2024-09-16T02:06:15.765 INFO:teuthology.orchestra.run.smithi190.stdout: "op_latency": { 2024-09-16T02:06:15.765 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 911, 2024-09-16T02:06:15.765 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.505985484, 2024-09-16T02:06:15.765 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000555417 2024-09-16T02:06:15.765 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.765 INFO:teuthology.orchestra.run.smithi190.stdout: "op_inflight": 0, 2024-09-16T02:06:15.765 INFO:teuthology.orchestra.run.smithi190.stdout: "oplen_avg": { 2024-09-16T02:06:15.765 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 911, 2024-09-16T02:06:15.765 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 912 2024-09-16T02:06:15.765 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.766 INFO:teuthology.orchestra.run.smithi190.stdout: "op": 911, 2024-09-16T02:06:15.766 INFO:teuthology.orchestra.run.smithi190.stdout: "op_r": 911, 2024-09-16T02:06:15.766 INFO:teuthology.orchestra.run.smithi190.stdout: "op_w": 0, 2024-09-16T02:06:15.766 INFO:teuthology.orchestra.run.smithi190.stdout: "op_rmw": 0, 2024-09-16T02:06:15.766 INFO:teuthology.orchestra.run.smithi190.stdout: "op_pg": 0, 2024-09-16T02:06:15.766 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_stat": 574, 2024-09-16T02:06:15.766 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_create": 0, 2024-09-16T02:06:15.766 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_read": 335, 2024-09-16T02:06:15.766 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_write": 0, 2024-09-16T02:06:15.766 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writefull": 0, 2024-09-16T02:06:15.766 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writesame": 0, 2024-09-16T02:06:15.766 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_append": 0, 2024-09-16T02:06:15.766 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_zero": 0, 2024-09-16T02:06:15.766 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_truncate": 0, 2024-09-16T02:06:15.767 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_delete": 0, 2024-09-16T02:06:15.767 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_mapext": 0, 2024-09-16T02:06:15.767 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_sparse_read": 0, 2024-09-16T02:06:15.767 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_clonerange": 0, 2024-09-16T02:06:15.767 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_getxattr": 1, 2024-09-16T02:06:15.767 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_setxattr": 0, 2024-09-16T02:06:15.767 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_cmpxattr": 0, 2024-09-16T02:06:15.767 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_rmxattr": 0, 2024-09-16T02:06:15.767 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_resetxattrs": 0, 2024-09-16T02:06:15.767 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_call": 0, 2024-09-16T02:06:15.767 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_watch": 0, 2024-09-16T02:06:15.767 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_notify": 0, 2024-09-16T02:06:15.767 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T02:06:15.767 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls": 0, 2024-09-16T02:06:15.767 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls_filter": 0, 2024-09-16T02:06:15.768 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_other": 0, 2024-09-16T02:06:15.768 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_active": 0, 2024-09-16T02:06:15.768 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_send": 0, 2024-09-16T02:06:15.768 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_resend": 0, 2024-09-16T02:06:15.768 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_ping": 0, 2024-09-16T02:06:15.768 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_active": 0, 2024-09-16T02:06:15.768 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_send": 0, 2024-09-16T02:06:15.768 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_resend": 0, 2024-09-16T02:06:15.768 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_active": 0, 2024-09-16T02:06:15.768 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_send": 0, 2024-09-16T02:06:15.768 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_resend": 0, 2024-09-16T02:06:15.768 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_active": 0, 2024-09-16T02:06:15.768 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_send": 0, 2024-09-16T02:06:15.768 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_resend": 0, 2024-09-16T02:06:15.769 INFO:teuthology.orchestra.run.smithi190.stdout: "command_active": 0, 2024-09-16T02:06:15.769 INFO:teuthology.orchestra.run.smithi190.stdout: "command_send": 0, 2024-09-16T02:06:15.769 INFO:teuthology.orchestra.run.smithi190.stdout: "command_resend": 0, 2024-09-16T02:06:15.769 INFO:teuthology.orchestra.run.smithi190.stdout: "map_epoch": 83, 2024-09-16T02:06:15.769 INFO:teuthology.orchestra.run.smithi190.stdout: "map_full": 0, 2024-09-16T02:06:15.769 INFO:teuthology.orchestra.run.smithi190.stdout: "map_inc": 4, 2024-09-16T02:06:15.769 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_sessions": 5, 2024-09-16T02:06:15.769 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_open": 5, 2024-09-16T02:06:15.769 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_close": 0, 2024-09-16T02:06:15.769 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_laggy": 0, 2024-09-16T02:06:15.769 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_wr": 0, 2024-09-16T02:06:15.769 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_rd": 2, 2024-09-16T02:06:15.769 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_del": 0 2024-09-16T02:06:15.770 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.770 INFO:teuthology.orchestra.run.smithi190.stdout: "oft": { 2024-09-16T02:06:15.770 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_objs": 0, 2024-09-16T02:06:15.770 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_kv_pairs": 0, 2024-09-16T02:06:15.770 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_updates": 0, 2024-09-16T02:06:15.770 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_removes": 0 2024-09-16T02:06:15.770 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.770 INFO:teuthology.orchestra.run.smithi190.stdout: "osd-slow-ops": { 2024-09-16T02:06:15.770 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_ops_count": 0 2024-09-16T02:06:15.770 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.770 INFO:teuthology.orchestra.run.smithi190.stdout: "purge_queue": { 2024-09-16T02:06:15.770 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops": 0, 2024-09-16T02:06:15.770 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T02:06:15.770 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing": 0, 2024-09-16T02:06:15.771 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_high_water": 0, 2024-09-16T02:06:15.771 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed_ops": 0, 2024-09-16T02:06:15.771 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed": 0, 2024-09-16T02:06:15.771 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_item_in_journal": 0 2024-09-16T02:06:15.771 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.771 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T02:06:15.771 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-16T02:06:15.771 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-09-16T02:06:15.771 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-16T02:06:15.771 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 1031, 2024-09-16T02:06:15.771 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 635129, 2024-09-16T02:06:15.771 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:15.771 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 1031, 2024-09-16T02:06:15.771 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-09-16T02:06:15.772 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-09-16T02:06:15.772 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 1031, 2024-09-16T02:06:15.772 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 635129, 2024-09-16T02:06:15.772 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-16T02:06:15.772 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.772 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.772 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.772 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-16T02:06:15.772 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.772 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_bytes": { 2024-09-16T02:06:15.772 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-16T02:06:15.772 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-09-16T02:06:15.772 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-16T02:06:15.772 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-09-16T02:06:15.773 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-09-16T02:06:15.773 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:15.773 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-09-16T02:06:15.773 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 911, 2024-09-16T02:06:15.773 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 1622, 2024-09-16T02:06:15.773 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 911, 2024-09-16T02:06:15.773 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 1622, 2024-09-16T02:06:15.773 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-16T02:06:15.773 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.773 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.773 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.773 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-16T02:06:15.773 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.773 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_ops": { 2024-09-16T02:06:15.774 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-16T02:06:15.774 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 1024, 2024-09-16T02:06:15.774 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-16T02:06:15.774 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-09-16T02:06:15.774 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-09-16T02:06:15.774 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:15.774 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-09-16T02:06:15.774 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 911, 2024-09-16T02:06:15.774 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 911, 2024-09-16T02:06:15.774 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 911, 2024-09-16T02:06:15.774 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 911, 2024-09-16T02:06:15.774 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-16T02:06:15.774 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.775 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.775 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.775 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-16T02:06:15.775 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.775 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle": { 2024-09-16T02:06:15.775 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-16T02:06:15.775 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2024-09-16T02:06:15.775 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-16T02:06:15.775 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-09-16T02:06:15.775 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-09-16T02:06:15.775 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:15.775 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-09-16T02:06:15.775 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-09-16T02:06:15.775 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-09-16T02:06:15.775 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 0, 2024-09-16T02:06:15.776 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 0, 2024-09-16T02:06:15.776 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-16T02:06:15.776 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.776 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.776 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.776 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-16T02:06:15.776 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T02:06:15.776 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle-0x55db64e26b40": { 2024-09-16T02:06:15.776 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-09-16T02:06:15.776 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2024-09-16T02:06:15.776 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-09-16T02:06:15.776 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-09-16T02:06:15.776 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-09-16T02:06:15.776 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-09-16T02:06:15.777 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-09-16T02:06:15.777 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-09-16T02:06:15.777 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-09-16T02:06:15.777 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 0, 2024-09-16T02:06:15.777 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 0, 2024-09-16T02:06:15.777 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-09-16T02:06:15.777 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-09-16T02:06:15.777 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-09-16T02:06:15.777 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-09-16T02:06:15.777 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-16T02:06:15.777 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-16T02:06:15.777 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-16T02:06:15.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:15 smithi190 ceph-mon[34762]: pgmap v451: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-16T02:06:16.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:15 smithi105 ceph-mon[31197]: pgmap v451: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-16T02:06:16.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:15 smithi137 ceph-mon[34099]: pgmap v451: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-16T02:06:16.473 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-16T02:06:16.473 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-16T02:06:16.473 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-09-16T02:06:16.473 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-09-16T02:06:16.473 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-16T02:06:16.484 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-16T02:06:16.484 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-16T02:06:16.484 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T02:06:16.507 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-16T02:06:16.507 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T02:06:16.566 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-16T02:06:16.863 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:06:17.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.364+0000 7f3b8ce83640 1 -- 172.21.15.105:0/3307497556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b881027a0 msgr2=0x7f3b881067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:17.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.364+0000 7f3b8ce83640 1 --2- 172.21.15.105:0/3307497556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b881027a0 0x7f3b881067f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c009900 tx=0x7f3b7c02a410 comp rx=0 tx=0).stop 2024-09-16T02:06:17.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.364+0000 7f3b8ce83640 1 -- 172.21.15.105:0/3307497556 shutdown_connections 2024-09-16T02:06:17.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.364+0000 7f3b8ce83640 1 --2- 172.21.15.105:0/3307497556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b88106f20 0x7f3b8810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:17.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.364+0000 7f3b8ce83640 1 --2- 172.21.15.105:0/3307497556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b881027a0 0x7f3b881067f0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:17.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.364+0000 7f3b8ce83640 1 --2- 172.21.15.105:0/3307497556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b88101df0 0x7f3b881021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:17.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.364+0000 7f3b8ce83640 1 -- 172.21.15.105:0/3307497556 >> 172.21.15.105:0/3307497556 conn(0x7f3b880fd8b0 msgr2=0x7f3b880ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:17.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.364+0000 7f3b8ce83640 1 -- 172.21.15.105:0/3307497556 shutdown_connections 2024-09-16T02:06:17.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.364+0000 7f3b8ce83640 1 -- 172.21.15.105:0/3307497556 wait complete. 2024-09-16T02:06:17.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.365+0000 7f3b8ce83640 1 Processor -- start 2024-09-16T02:06:17.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.365+0000 7f3b8ce83640 1 -- start start 2024-09-16T02:06:17.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.365+0000 7f3b8ce83640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b88101df0 0x7f3b8819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:17.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.365+0000 7f3b8ce83640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b881027a0 0x7f3b8819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:17.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.365+0000 7f3b8ce83640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b88106f20 0x7f3b8819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:17.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.365+0000 7f3b8ce83640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3b88111df0 con 0x7f3b881027a0 2024-09-16T02:06:17.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.365+0000 7f3b8ce83640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3b88111c70 con 0x7f3b88106f20 2024-09-16T02:06:17.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.365+0000 7f3b8ce83640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3b88111f70 con 0x7f3b88101df0 2024-09-16T02:06:17.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.365+0000 7f3b85d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b881027a0 0x7f3b8819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:17.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.365+0000 7f3b85d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b881027a0 0x7f3b8819aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36366/0 (socket says 172.21.15.105:36366) 2024-09-16T02:06:17.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.365+0000 7f3b85d9b640 1 -- 172.21.15.105:0/2563771184 learned_addr learned my addr 172.21.15.105:0/2563771184 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:06:17.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.365+0000 7f3b8659c640 1 --2- 172.21.15.105:0/2563771184 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b88101df0 0x7f3b8819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:17.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.365+0000 7f3b86d9d640 1 --2- 172.21.15.105:0/2563771184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b88106f20 0x7f3b8819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:17.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.365+0000 7f3b85d9b640 1 -- 172.21.15.105:0/2563771184 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b88101df0 msgr2=0x7f3b8819a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:17.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.365+0000 7f3b85d9b640 1 --2- 172.21.15.105:0/2563771184 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b88101df0 0x7f3b8819a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:17.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.366+0000 7f3b85d9b640 1 -- 172.21.15.105:0/2563771184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b88106f20 msgr2=0x7f3b8819eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:17.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.366+0000 7f3b85d9b640 1 --2- 172.21.15.105:0/2563771184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b88106f20 0x7f3b8819eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:17.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.366+0000 7f3b85d9b640 1 -- 172.21.15.105:0/2563771184 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b8819f480 con 0x7f3b881027a0 2024-09-16T02:06:17.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.366+0000 7f3b85d9b640 1 --2- 172.21.15.105:0/2563771184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b881027a0 0x7f3b8819aa10 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c005940 tx=0x7f3b7c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:17.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.366+0000 7f3b777fe640 1 -- 172.21.15.105:0/2563771184 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c03f030 con 0x7f3b881027a0 2024-09-16T02:06:17.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.366+0000 7f3b777fe640 1 -- 172.21.15.105:0/2563771184 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b7c003aa0 con 0x7f3b881027a0 2024-09-16T02:06:17.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.366+0000 7f3b8ce83640 1 -- 172.21.15.105:0/2563771184 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b8819f710 con 0x7f3b881027a0 2024-09-16T02:06:17.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.366+0000 7f3b777fe640 1 -- 172.21.15.105:0/2563771184 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c035070 con 0x7f3b881027a0 2024-09-16T02:06:17.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.366+0000 7f3b8ce83640 1 -- 172.21.15.105:0/2563771184 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b881a7050 con 0x7f3b881027a0 2024-09-16T02:06:17.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.367+0000 7f3b8ce83640 1 -- 172.21.15.105:0/2563771184 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b881032d0 con 0x7f3b881027a0 2024-09-16T02:06:17.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.368+0000 7f3b777fe640 1 -- 172.21.15.105:0/2563771184 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f3b7c042030 con 0x7f3b881027a0 2024-09-16T02:06:17.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.369+0000 7f3b777fe640 1 --2- 172.21.15.105:0/2563771184 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f3b700a3240 0x7f3b700a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:17.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.369+0000 7f3b8659c640 1 --2- 172.21.15.105:0/2563771184 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f3b700a3240 0x7f3b700a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:17.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.369+0000 7f3b777fe640 1 -- 172.21.15.105:0/2563771184 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f3b7c0f03d0 con 0x7f3b881027a0 2024-09-16T02:06:17.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.370+0000 7f3b8659c640 1 --2- 172.21.15.105:0/2563771184 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f3b700a3240 0x7f3b700a5700 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3b8000abe0 tx=0x7f3b8000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:17.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.373+0000 7f3b777fe640 1 -- 172.21.15.105:0/2563771184 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3b7c0bd030 con 0x7f3b881027a0 2024-09-16T02:06:17.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.550+0000 7f3b8ce83640 1 -- 172.21.15.105:0/2563771184 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3b881a7220 con 0x7f3b881027a0 2024-09-16T02:06:17.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.550+0000 7f3b777fe640 1 -- 172.21.15.105:0/2563771184 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v83) ==== 81+0+602 (secure 0 0 0) 0x7f3b7c0bc930 con 0x7f3b881027a0 2024-09-16T02:06:17.552 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:0/2453739657 2024-09-17T01:53:39.351314+0000 2024-09-16T02:06:17.552 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:6801/1469189484 2024-09-17T01:53:39.351314+0000 2024-09-16T02:06:17.552 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:0/3466166879 2024-09-17T01:53:39.351314+0000 2024-09-16T02:06:17.552 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:6801/1268348742 2024-09-17T01:53:15.522227+0000 2024-09-16T02:06:17.552 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:0/2897248478 2024-09-17T01:53:39.351314+0000 2024-09-16T02:06:17.552 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:0/3709363371 2024-09-17T01:53:15.522227+0000 2024-09-16T02:06:17.552 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:6800/1268348742 2024-09-17T01:53:15.522227+0000 2024-09-16T02:06:17.552 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:6800/1469189484 2024-09-17T01:53:39.351314+0000 2024-09-16T02:06:17.552 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:0/1198176299 2024-09-17T01:53:15.522227+0000 2024-09-16T02:06:17.553 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:0/3031917848 2024-09-17T01:53:15.522227+0000 2024-09-16T02:06:17.553 INFO:teuthology.orchestra.run.smithi105.stderr:listed 10 entries 2024-09-16T02:06:17.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.552+0000 7f3b8ce83640 1 -- 172.21.15.105:0/2563771184 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f3b700a3240 msgr2=0x7f3b700a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:17.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.552+0000 7f3b8ce83640 1 --2- 172.21.15.105:0/2563771184 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f3b700a3240 0x7f3b700a5700 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3b8000abe0 tx=0x7f3b8000a2d0 comp rx=0 tx=0).stop 2024-09-16T02:06:17.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.552+0000 7f3b8ce83640 1 -- 172.21.15.105:0/2563771184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b881027a0 msgr2=0x7f3b8819aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:17.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.552+0000 7f3b8ce83640 1 --2- 172.21.15.105:0/2563771184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b881027a0 0x7f3b8819aa10 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c005940 tx=0x7f3b7c004270 comp rx=0 tx=0).stop 2024-09-16T02:06:17.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.552+0000 7f3b8ce83640 1 -- 172.21.15.105:0/2563771184 shutdown_connections 2024-09-16T02:06:17.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.552+0000 7f3b8ce83640 1 --2- 172.21.15.105:0/2563771184 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f3b700a3240 0x7f3b700a5700 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:17.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.552+0000 7f3b8ce83640 1 --2- 172.21.15.105:0/2563771184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b88106f20 0x7f3b8819eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:17.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.552+0000 7f3b8ce83640 1 --2- 172.21.15.105:0/2563771184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b881027a0 0x7f3b8819aa10 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:17.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.552+0000 7f3b8ce83640 1 --2- 172.21.15.105:0/2563771184 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b88101df0 0x7f3b8819a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:17.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.552+0000 7f3b8ce83640 1 -- 172.21.15.105:0/2563771184 >> 172.21.15.105:0/2563771184 conn(0x7f3b880fd8b0 msgr2=0x7f3b880feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:17.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.553+0000 7f3b8ce83640 1 -- 172.21.15.105:0/2563771184 shutdown_connections 2024-09-16T02:06:17.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:17.553+0000 7f3b8ce83640 1 -- 172.21.15.105:0/2563771184 wait complete. 2024-09-16T02:06:17.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:17 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/2563771184' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T02:06:17.954 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:17 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2563771184' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T02:06:18.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:17 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/2563771184' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T02:06:18.419 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-16T02:06:18.419 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T02:06:18.437 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-16T02:06:18.728 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:06:18.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:18 smithi105 ceph-mon[31197]: pgmap v452: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-16T02:06:18.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:18 smithi137 ceph-mon[34099]: pgmap v452: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-16T02:06:19.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:18 smithi190 ceph-mon[34762]: pgmap v452: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-16T02:06:19.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.217+0000 7f436a10a640 1 -- 172.21.15.105:0/1433626888 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4364101420 msgr2=0x7f436410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:19.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.217+0000 7f436a10a640 1 --2- 172.21.15.105:0/1433626888 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4364101420 0x7f436410d950 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f435c009cd0 tx=0x7f435c02a8e0 comp rx=0 tx=0).stop 2024-09-16T02:06:19.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.217+0000 7f436a10a640 1 -- 172.21.15.105:0/1433626888 shutdown_connections 2024-09-16T02:06:19.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.217+0000 7f436a10a640 1 --2- 172.21.15.105:0/1433626888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4364100a80 0x7f4364100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:19.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.217+0000 7f436a10a640 1 --2- 172.21.15.105:0/1433626888 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4364101420 0x7f436410d950 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:19.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.217+0000 7f436a10a640 1 --2- 172.21.15.105:0/1433626888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4364106a80 0x7f4364106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:19.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.217+0000 7f436a10a640 1 -- 172.21.15.105:0/1433626888 >> 172.21.15.105:0/1433626888 conn(0x7f43640fc7a0 msgr2=0x7f43640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:19.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.217+0000 7f436a10a640 1 -- 172.21.15.105:0/1433626888 shutdown_connections 2024-09-16T02:06:19.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.217+0000 7f436a10a640 1 -- 172.21.15.105:0/1433626888 wait complete. 2024-09-16T02:06:19.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.217+0000 7f436a10a640 1 Processor -- start 2024-09-16T02:06:19.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.218+0000 7f436a10a640 1 -- start start 2024-09-16T02:06:19.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.218+0000 7f436a10a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4364100a80 0x7f4364198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:19.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.218+0000 7f436a10a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4364101420 0x7f4364198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:19.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.218+0000 7f436a10a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4364106a80 0x7f436419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:19.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.218+0000 7f436a10a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4364110d30 con 0x7f4364100a80 2024-09-16T02:06:19.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.218+0000 7f436a10a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4364110bb0 con 0x7f4364106a80 2024-09-16T02:06:19.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.218+0000 7f436a10a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4364110eb0 con 0x7f4364101420 2024-09-16T02:06:19.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.218+0000 7f43637fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4364100a80 0x7f4364198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:19.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.218+0000 7f43637fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4364100a80 0x7f4364198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57190/0 (socket says 172.21.15.105:57190) 2024-09-16T02:06:19.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.218+0000 7f43637fe640 1 -- 172.21.15.105:0/2273432699 learned_addr learned my addr 172.21.15.105:0/2273432699 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:06:19.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.218+0000 7f4362ffd640 1 --2- 172.21.15.105:0/2273432699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4364101420 0x7f4364198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:19.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.218+0000 7f4363fff640 1 --2- 172.21.15.105:0/2273432699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4364106a80 0x7f436419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:19.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.218+0000 7f43637fe640 1 -- 172.21.15.105:0/2273432699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4364101420 msgr2=0x7f4364198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:19.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.218+0000 7f43637fe640 1 --2- 172.21.15.105:0/2273432699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4364101420 0x7f4364198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:19.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.218+0000 7f43637fe640 1 -- 172.21.15.105:0/2273432699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4364106a80 msgr2=0x7f436419cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:19.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.218+0000 7f43637fe640 1 --2- 172.21.15.105:0/2273432699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4364106a80 0x7f436419cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:19.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.218+0000 7f43637fe640 1 -- 172.21.15.105:0/2273432699 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f436419d300 con 0x7f4364100a80 2024-09-16T02:06:19.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.218+0000 7f4362ffd640 1 --2- 172.21.15.105:0/2273432699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4364101420 0x7f4364198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:06:19.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.218+0000 7f43637fe640 1 --2- 172.21.15.105:0/2273432699 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4364100a80 0x7f4364198350 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f435400d950 tx=0x7f435400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:19.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.219+0000 7f4360ff9640 1 -- 172.21.15.105:0/2273432699 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4354017070 con 0x7f4364100a80 2024-09-16T02:06:19.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.219+0000 7f436a10a640 1 -- 172.21.15.105:0/2273432699 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f436419d5f0 con 0x7f4364100a80 2024-09-16T02:06:19.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.219+0000 7f4360ff9640 1 -- 172.21.15.105:0/2273432699 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4354009c30 con 0x7f4364100a80 2024-09-16T02:06:19.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.219+0000 7f4360ff9640 1 -- 172.21.15.105:0/2273432699 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435400f620 con 0x7f4364100a80 2024-09-16T02:06:19.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.219+0000 7f436a10a640 1 -- 172.21.15.105:0/2273432699 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43641a4ed0 con 0x7f4364100a80 2024-09-16T02:06:19.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.220+0000 7f436a10a640 1 -- 172.21.15.105:0/2273432699 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4364100ee0 con 0x7f4364100a80 2024-09-16T02:06:19.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.221+0000 7f4360ff9640 1 -- 172.21.15.105:0/2273432699 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f43540109a0 con 0x7f4364100a80 2024-09-16T02:06:19.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.222+0000 7f4360ff9640 1 --2- 172.21.15.105:0/2273432699 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f43500a3240 0x7f43500a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:19.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.222+0000 7f4362ffd640 1 --2- 172.21.15.105:0/2273432699 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f43500a3240 0x7f43500a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:19.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.222+0000 7f4360ff9640 1 -- 172.21.15.105:0/2273432699 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f43540cf600 con 0x7f4364100a80 2024-09-16T02:06:19.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.222+0000 7f4362ffd640 1 --2- 172.21.15.105:0/2273432699 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f43500a3240 0x7f43500a5700 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f434c0058c0 tx=0x7f434c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:19.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.226+0000 7f4360ff9640 1 -- 172.21.15.105:0/2273432699 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4354009de0 con 0x7f4364100a80 2024-09-16T02:06:19.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.406+0000 7f436a10a640 1 -- 172.21.15.105:0/2273432699 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f43641996b0 con 0x7f4364100a80 2024-09-16T02:06:19.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.406+0000 7f4360ff9640 1 -- 172.21.15.105:0/2273432699 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v83) ==== 81+0+602 (secure 0 0 0) 0x7f435409c2c0 con 0x7f4364100a80 2024-09-16T02:06:19.408 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:0/2453739657 2024-09-17T01:53:39.351314+0000 2024-09-16T02:06:19.408 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:6801/1469189484 2024-09-17T01:53:39.351314+0000 2024-09-16T02:06:19.408 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:0/3466166879 2024-09-17T01:53:39.351314+0000 2024-09-16T02:06:19.408 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:6801/1268348742 2024-09-17T01:53:15.522227+0000 2024-09-16T02:06:19.408 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:0/2897248478 2024-09-17T01:53:39.351314+0000 2024-09-16T02:06:19.408 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:0/3709363371 2024-09-17T01:53:15.522227+0000 2024-09-16T02:06:19.408 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:6800/1268348742 2024-09-17T01:53:15.522227+0000 2024-09-16T02:06:19.409 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:6800/1469189484 2024-09-17T01:53:39.351314+0000 2024-09-16T02:06:19.409 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:0/1198176299 2024-09-17T01:53:15.522227+0000 2024-09-16T02:06:19.409 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:0/3031917848 2024-09-17T01:53:15.522227+0000 2024-09-16T02:06:19.410 INFO:teuthology.orchestra.run.smithi105.stderr:listed 10 entries 2024-09-16T02:06:19.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.408+0000 7f436a10a640 1 -- 172.21.15.105:0/2273432699 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f43500a3240 msgr2=0x7f43500a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:19.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.408+0000 7f436a10a640 1 --2- 172.21.15.105:0/2273432699 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f43500a3240 0x7f43500a5700 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f434c0058c0 tx=0x7f434c005850 comp rx=0 tx=0).stop 2024-09-16T02:06:19.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.408+0000 7f436a10a640 1 -- 172.21.15.105:0/2273432699 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4364100a80 msgr2=0x7f4364198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:19.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.408+0000 7f436a10a640 1 --2- 172.21.15.105:0/2273432699 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4364100a80 0x7f4364198350 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f435400d950 tx=0x7f435400de10 comp rx=0 tx=0).stop 2024-09-16T02:06:19.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.408+0000 7f436a10a640 1 -- 172.21.15.105:0/2273432699 shutdown_connections 2024-09-16T02:06:19.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.408+0000 7f436a10a640 1 --2- 172.21.15.105:0/2273432699 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f43500a3240 0x7f43500a5700 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:19.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.408+0000 7f436a10a640 1 --2- 172.21.15.105:0/2273432699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4364101420 0x7f4364198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:19.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.408+0000 7f436a10a640 1 --2- 172.21.15.105:0/2273432699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4364106a80 0x7f436419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:19.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.408+0000 7f436a10a640 1 --2- 172.21.15.105:0/2273432699 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4364100a80 0x7f4364198350 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:19.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.408+0000 7f436a10a640 1 -- 172.21.15.105:0/2273432699 >> 172.21.15.105:0/2273432699 conn(0x7f43640fc7a0 msgr2=0x7f436406aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:19.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.408+0000 7f436a10a640 1 -- 172.21.15.105:0/2273432699 shutdown_connections 2024-09-16T02:06:19.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:19.408+0000 7f436a10a640 1 -- 172.21.15.105:0/2273432699 wait complete. 2024-09-16T02:06:19.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:19 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2273432699' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T02:06:19.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:19 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/2273432699' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T02:06:20.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:19 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/2273432699' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T02:06:20.228 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi105.front.sepia.ceph.com... 2024-09-16T02:06:20.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T02:06:20.228 DEBUG:teuthology.orchestra.run.smithi105:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-16T02:06:20.255 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T02:06:20.255 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:ceph-fuse[83307]: fuse finished with error 0 and tester_r 0 2024-09-16T02:06:20.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:20 smithi137 ceph-mon[34099]: pgmap v453: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-16T02:06:20.994 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:20 smithi190 ceph-mon[34762]: pgmap v453: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-16T02:06:21.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:20 smithi105 ceph-mon[31197]: pgmap v453: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-16T02:06:22.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:22 smithi137 ceph-mon[34099]: pgmap v454: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-16T02:06:23.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:22 smithi105 ceph-mon[31197]: pgmap v454: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-16T02:06:23.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:22 smithi190 ceph-mon[34762]: pgmap v454: 97 pgs: 97 active+clean; 966 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-16T02:06:24.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:24 smithi137 ceph-mon[34099]: pgmap v455: 97 pgs: 97 active+clean; 967 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-16T02:06:25.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:24 smithi105 ceph-mon[31197]: pgmap v455: 97 pgs: 97 active+clean; 967 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-16T02:06:25.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:24 smithi190 ceph-mon[34762]: pgmap v455: 97 pgs: 97 active+clean; 967 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-16T02:06:26.356 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2024-09-16T02:06:26.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T02:06:26.356 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T02:06:26.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T02:06:26.388 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-09-16T02:06:26.447 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T02:06:26.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T02:06:26.447 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T02:06:26.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T02:06:26.518 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link delete ceph-brx 2024-09-16T02:06:26.638 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-16T02:06:26.653 INFO:tasks.cephadm:Teardown begin 2024-09-16T02:06:26.653 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T02:06:26.653 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:06:26.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:26 smithi105 ceph-mon[31197]: pgmap v456: 97 pgs: 97 active+clean; 967 KiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-16T02:06:26.903 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:06:26.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:26 smithi137 ceph-mon[34099]: pgmap v456: 97 pgs: 97 active+clean; 967 KiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-16T02:06:27.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:26 smithi190 ceph-mon[34762]: pgmap v456: 97 pgs: 97 active+clean; 967 KiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-16T02:06:27.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.428+0000 7f7833e75640 1 -- 172.21.15.105:0/2459510519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f782c102ef0 msgr2=0x7f782c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:27.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.428+0000 7f7833e75640 1 --2- 172.21.15.105:0/2459510519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f782c102ef0 0x7f782c10da40 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f7824009ca0 tx=0x7f782402a950 comp rx=0 tx=0).stop 2024-09-16T02:06:27.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.428+0000 7f7833e75640 1 -- 172.21.15.105:0/2459510519 shutdown_connections 2024-09-16T02:06:27.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.428+0000 7f7833e75640 1 --2- 172.21.15.105:0/2459510519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f782c102ef0 0x7f782c10da40 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.428+0000 7f7833e75640 1 --2- 172.21.15.105:0/2459510519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f782c102570 0x7f782c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.428+0000 7f7833e75640 1 --2- 172.21.15.105:0/2459510519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f782c100e20 0x7f782c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.428+0000 7f7833e75640 1 -- 172.21.15.105:0/2459510519 >> 172.21.15.105:0/2459510519 conn(0x7f782c0fc7d0 msgr2=0x7f782c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:27.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.428+0000 7f7833e75640 1 -- 172.21.15.105:0/2459510519 shutdown_connections 2024-09-16T02:06:27.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.428+0000 7f7833e75640 1 -- 172.21.15.105:0/2459510519 wait complete. 2024-09-16T02:06:27.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.429+0000 7f7833e75640 1 Processor -- start 2024-09-16T02:06:27.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.429+0000 7f7833e75640 1 -- start start 2024-09-16T02:06:27.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.429+0000 7f7833e75640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f782c100e20 0x7f782c198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:27.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.429+0000 7f7833e75640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f782c102570 0x7f782c198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:27.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.429+0000 7f7833e75640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f782c102ef0 0x7f782c19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:27.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.429+0000 7f7833e75640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f782c110ed0 con 0x7f782c102570 2024-09-16T02:06:27.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.429+0000 7f7833e75640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f782c110d50 con 0x7f782c100e20 2024-09-16T02:06:27.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.429+0000 7f7833e75640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f782c111050 con 0x7f782c102ef0 2024-09-16T02:06:27.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.429+0000 7f7831410640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f782c102570 0x7f782c198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:27.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.429+0000 7f7831410640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f782c102570 0x7f782c198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57220/0 (socket says 172.21.15.105:57220) 2024-09-16T02:06:27.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.429+0000 7f7831410640 1 -- 172.21.15.105:0/3998251923 learned_addr learned my addr 172.21.15.105:0/3998251923 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:06:27.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.429+0000 7f7831c11640 1 --2- 172.21.15.105:0/3998251923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f782c100e20 0x7f782c198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:27.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.430+0000 7f7832412640 1 --2- 172.21.15.105:0/3998251923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f782c102ef0 0x7f782c19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:27.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.430+0000 7f7831410640 1 -- 172.21.15.105:0/3998251923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f782c102ef0 msgr2=0x7f782c19cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:27.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.430+0000 7f7831410640 1 --2- 172.21.15.105:0/3998251923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f782c102ef0 0x7f782c19cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.430+0000 7f7831410640 1 -- 172.21.15.105:0/3998251923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f782c100e20 msgr2=0x7f782c198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:27.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.430+0000 7f7831410640 1 --2- 172.21.15.105:0/3998251923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f782c100e20 0x7f782c198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.430+0000 7f7831410640 1 -- 172.21.15.105:0/3998251923 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f782c19d4d0 con 0x7f782c102570 2024-09-16T02:06:27.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.430+0000 7f7831410640 1 --2- 172.21.15.105:0/3998251923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f782c102570 0x7f782c198a60 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f781c00b4a0 tx=0x7f781c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:27.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.430+0000 7f7822ffd640 1 -- 172.21.15.105:0/3998251923 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781c014070 con 0x7f782c102570 2024-09-16T02:06:27.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.430+0000 7f7833e75640 1 -- 172.21.15.105:0/3998251923 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f782c19d7c0 con 0x7f782c102570 2024-09-16T02:06:27.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.430+0000 7f7822ffd640 1 -- 172.21.15.105:0/3998251923 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f781c010ac0 con 0x7f782c102570 2024-09-16T02:06:27.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.430+0000 7f7822ffd640 1 -- 172.21.15.105:0/3998251923 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781c017c30 con 0x7f782c102570 2024-09-16T02:06:27.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.430+0000 7f7833e75640 1 -- 172.21.15.105:0/3998251923 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f782c1a50a0 con 0x7f782c102570 2024-09-16T02:06:27.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.431+0000 7f7833e75640 1 -- 172.21.15.105:0/3998251923 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f782c101280 con 0x7f782c102570 2024-09-16T02:06:27.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.432+0000 7f7822ffd640 1 -- 172.21.15.105:0/3998251923 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f781c017070 con 0x7f782c102570 2024-09-16T02:06:27.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.433+0000 7f7822ffd640 1 --2- 172.21.15.105:0/3998251923 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f78180a3240 0x7f78180a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:27.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.433+0000 7f7831c11640 1 --2- 172.21.15.105:0/3998251923 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f78180a3240 0x7f78180a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:27.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.433+0000 7f7822ffd640 1 -- 172.21.15.105:0/3998251923 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f781c0cedc0 con 0x7f782c102570 2024-09-16T02:06:27.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.434+0000 7f7831c11640 1 --2- 172.21.15.105:0/3998251923 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f78180a3240 0x7f78180a5700 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7828009890 tx=0x7f7828008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:27.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.437+0000 7f7822ffd640 1 -- 172.21.15.105:0/3998251923 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f781c09ba80 con 0x7f782c102570 2024-09-16T02:06:27.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.641+0000 7f7833e75640 1 -- 172.21.15.105:0/3998251923 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f782c1a5390 con 0x7f782c102570 2024-09-16T02:06:27.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.642+0000 7f7822ffd640 1 -- 172.21.15.105:0/3998251923 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11991 (secure 0 0 0) 0x7f781c09b380 con 0x7f782c102570 2024-09-16T02:06:27.645 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:06:27.645 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":28,"btime":"2024-09-16T02:02:41:554178+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2057699920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2057699920},{"type":"v1","addr":"172.21.15.105:6835","nonce":2057699920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1764886948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1764886948},{"type":"v1","addr":"172.21.15.105:6837","nonce":1764886948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T02:00:35.112266+0000","modified":"2024-09-16T02:02:40.885614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24469,"mds_2":24461,"mds_3":24457,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/4112062361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":4112062361},{"type":"v1","addr":"172.21.15.105:6839","nonce":4112062361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6841/163156514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":163156514},{"type":"v1","addr":"172.21.15.105:6841","nonce":163156514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/683519097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":683519097},{"type":"v1","addr":"172.21.15.190:6833","nonce":683519097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3025442862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3025442862},{"type":"v1","addr":"172.21.15.137:6833","nonce":3025442862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/763783602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":763783602},{"type":"v1","addr":"172.21.15.137:6835","nonce":763783602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.190:6835/1851042281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1851042281},{"type":"v1","addr":"172.21.15.190:6835","nonce":1851042281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/3181257549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3181257549},{"type":"v1","addr":"172.21.15.190:6837","nonce":3181257549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6837/2788541030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2788541030},{"type":"v1","addr":"172.21.15.137:6837","nonce":2788541030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.190:6839/1222573537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":1222573537},{"type":"v1","addr":"172.21.15.190:6839","nonce":1222573537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/3454924009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3454924009},{"type":"v1","addr":"172.21.15.137:6839","nonce":3454924009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24445,24457,24461,24469,24482]},"id":1}]} 2024-09-16T02:06:27.645 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 28 2024-09-16T02:06:27.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.644+0000 7f7833e75640 1 -- 172.21.15.105:0/3998251923 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f78180a3240 msgr2=0x7f78180a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.644+0000 7f7833e75640 1 --2- 172.21.15.105:0/3998251923 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f78180a3240 0x7f78180a5700 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7828009890 tx=0x7f7828008040 comp rx=0 tx=0).stop 2024-09-16T02:06:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.644+0000 7f7833e75640 1 -- 172.21.15.105:0/3998251923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f782c102570 msgr2=0x7f782c198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.644+0000 7f7833e75640 1 --2- 172.21.15.105:0/3998251923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f782c102570 0x7f782c198a60 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f781c00b4a0 tx=0x7f781c00b960 comp rx=0 tx=0).stop 2024-09-16T02:06:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.644+0000 7f7833e75640 1 -- 172.21.15.105:0/3998251923 shutdown_connections 2024-09-16T02:06:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.644+0000 7f7833e75640 1 --2- 172.21.15.105:0/3998251923 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f78180a3240 0x7f78180a5700 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.644+0000 7f7833e75640 1 --2- 172.21.15.105:0/3998251923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f782c102ef0 0x7f782c19cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.644+0000 7f7833e75640 1 --2- 172.21.15.105:0/3998251923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f782c102570 0x7f782c198a60 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.644+0000 7f7833e75640 1 --2- 172.21.15.105:0/3998251923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f782c100e20 0x7f782c198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.644+0000 7f7833e75640 1 -- 172.21.15.105:0/3998251923 >> 172.21.15.105:0/3998251923 conn(0x7f782c0fc7d0 msgr2=0x7f782c069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:27.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.644+0000 7f7833e75640 1 -- 172.21.15.105:0/3998251923 shutdown_connections 2024-09-16T02:06:27.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:27.644+0000 7f7833e75640 1 -- 172.21.15.105:0/3998251923 wait complete. 2024-09-16T02:06:28.445 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:06:28.695 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:06:28.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:28 smithi105 ceph-mon[31197]: pgmap v457: 97 pgs: 97 active+clean; 967 KiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-16T02:06:28.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:28 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/3998251923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:06:28.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:28 smithi137 ceph-mon[34099]: pgmap v457: 97 pgs: 97 active+clean; 967 KiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-16T02:06:28.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:28 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/3998251923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:06:29.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:28 smithi190 ceph-mon[34762]: pgmap v457: 97 pgs: 97 active+clean; 967 KiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-16T02:06:29.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:28 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/3998251923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:06:29.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.209+0000 7fbf97f07640 1 -- 172.21.15.105:0/718639052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf90103780 msgr2=0x7fbf90107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:29.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.209+0000 7fbf97f07640 1 --2- 172.21.15.105:0/718639052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf90103780 0x7fbf90107bc0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fbf80009930 tx=0x7fbf8002a420 comp rx=0 tx=0).stop 2024-09-16T02:06:29.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.210+0000 7fbf97f07640 1 -- 172.21.15.105:0/718639052 shutdown_connections 2024-09-16T02:06:29.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.210+0000 7fbf97f07640 1 --2- 172.21.15.105:0/718639052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf90108300 0x7fbf9010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.210+0000 7fbf97f07640 1 --2- 172.21.15.105:0/718639052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf90103780 0x7fbf90107bc0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.210+0000 7fbf97f07640 1 --2- 172.21.15.105:0/718639052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf90102dd0 0x7fbf901031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.210+0000 7fbf97f07640 1 -- 172.21.15.105:0/718639052 >> 172.21.15.105:0/718639052 conn(0x7fbf900fc7d0 msgr2=0x7fbf900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:29.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.210+0000 7fbf97f07640 1 -- 172.21.15.105:0/718639052 shutdown_connections 2024-09-16T02:06:29.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.210+0000 7fbf97f07640 1 -- 172.21.15.105:0/718639052 wait complete. 2024-09-16T02:06:29.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.210+0000 7fbf97f07640 1 Processor -- start 2024-09-16T02:06:29.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.210+0000 7fbf97f07640 1 -- start start 2024-09-16T02:06:29.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.210+0000 7fbf97f07640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf90102dd0 0x7fbf90100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:29.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.210+0000 7fbf97f07640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf90103780 0x7fbf90100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:29.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.210+0000 7fbf97f07640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf90108300 0x7fbf901014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:29.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.210+0000 7fbf97f07640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbf90111f00 con 0x7fbf90103780 2024-09-16T02:06:29.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.210+0000 7fbf97f07640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbf90111d80 con 0x7fbf90102dd0 2024-09-16T02:06:29.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.210+0000 7fbf97f07640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbf90112080 con 0x7fbf90108300 2024-09-16T02:06:29.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.210+0000 7fbf954a2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf90103780 0x7fbf90100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:29.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.211+0000 7fbf954a2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf90103780 0x7fbf90100f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44664/0 (socket says 172.21.15.105:44664) 2024-09-16T02:06:29.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.211+0000 7fbf954a2640 1 -- 172.21.15.105:0/2350020152 learned_addr learned my addr 172.21.15.105:0/2350020152 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:06:29.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.211+0000 7fbf95ca3640 1 --2- 172.21.15.105:0/2350020152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf90102dd0 0x7fbf90100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:29.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.211+0000 7fbf964a4640 1 --2- 172.21.15.105:0/2350020152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf90108300 0x7fbf901014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:29.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.211+0000 7fbf954a2640 1 -- 172.21.15.105:0/2350020152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf90108300 msgr2=0x7fbf901014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:29.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.211+0000 7fbf954a2640 1 --2- 172.21.15.105:0/2350020152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf90108300 0x7fbf901014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.211+0000 7fbf954a2640 1 -- 172.21.15.105:0/2350020152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf90102dd0 msgr2=0x7fbf90100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:29.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.211+0000 7fbf954a2640 1 --2- 172.21.15.105:0/2350020152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf90102dd0 0x7fbf90100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.211+0000 7fbf954a2640 1 -- 172.21.15.105:0/2350020152 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf901a6940 con 0x7fbf90103780 2024-09-16T02:06:29.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.211+0000 7fbf964a4640 1 --2- 172.21.15.105:0/2350020152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf90108300 0x7fbf901014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:06:29.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.211+0000 7fbf954a2640 1 --2- 172.21.15.105:0/2350020152 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf90103780 0x7fbf90100f70 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fbf80031440 tx=0x7fbf80004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:29.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.211+0000 7fbf86ffd640 1 -- 172.21.15.105:0/2350020152 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8003f030 con 0x7fbf90103780 2024-09-16T02:06:29.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.211+0000 7fbf86ffd640 1 -- 172.21.15.105:0/2350020152 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf80003aa0 con 0x7fbf90103780 2024-09-16T02:06:29.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.211+0000 7fbf86ffd640 1 -- 172.21.15.105:0/2350020152 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf80035070 con 0x7fbf90103780 2024-09-16T02:06:29.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.211+0000 7fbf97f07640 1 -- 172.21.15.105:0/2350020152 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf901a6bd0 con 0x7fbf90103780 2024-09-16T02:06:29.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.211+0000 7fbf97f07640 1 -- 172.21.15.105:0/2350020152 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf901a7100 con 0x7fbf90103780 2024-09-16T02:06:29.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.212+0000 7fbf97f07640 1 -- 172.21.15.105:0/2350020152 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf901031b0 con 0x7fbf90103780 2024-09-16T02:06:29.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.213+0000 7fbf86ffd640 1 -- 172.21.15.105:0/2350020152 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fbf80042030 con 0x7fbf90103780 2024-09-16T02:06:29.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.214+0000 7fbf86ffd640 1 --2- 172.21.15.105:0/2350020152 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fbf7c0a32c0 0x7fbf7c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:29.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.214+0000 7fbf95ca3640 1 --2- 172.21.15.105:0/2350020152 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fbf7c0a32c0 0x7fbf7c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:29.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.214+0000 7fbf86ffd640 1 -- 172.21.15.105:0/2350020152 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fbf800efd70 con 0x7fbf90103780 2024-09-16T02:06:29.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.214+0000 7fbf95ca3640 1 --2- 172.21.15.105:0/2350020152 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fbf7c0a32c0 0x7fbf7c0a5780 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c009bd0 tx=0x7fbf8c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:29.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.218+0000 7fbf86ffd640 1 -- 172.21.15.105:0/2350020152 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbf800bca30 con 0x7fbf90103780 2024-09-16T02:06:29.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.422+0000 7fbf97f07640 1 -- 172.21.15.105:0/2350020152 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf9010a290 con 0x7fbf90103780 2024-09-16T02:06:29.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.422+0000 7fbf86ffd640 1 -- 172.21.15.105:0/2350020152 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11991 (secure 0 0 0) 0x7fbf800bc330 con 0x7fbf90103780 2024-09-16T02:06:29.425 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:06:29.425 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":28,"btime":"2024-09-16T02:02:41:554178+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2057699920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2057699920},{"type":"v1","addr":"172.21.15.105:6835","nonce":2057699920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1764886948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1764886948},{"type":"v1","addr":"172.21.15.105:6837","nonce":1764886948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T02:00:35.112266+0000","modified":"2024-09-16T02:02:40.885614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24469,"mds_2":24461,"mds_3":24457,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/4112062361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":4112062361},{"type":"v1","addr":"172.21.15.105:6839","nonce":4112062361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6841/163156514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":163156514},{"type":"v1","addr":"172.21.15.105:6841","nonce":163156514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/683519097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":683519097},{"type":"v1","addr":"172.21.15.190:6833","nonce":683519097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3025442862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3025442862},{"type":"v1","addr":"172.21.15.137:6833","nonce":3025442862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/763783602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":763783602},{"type":"v1","addr":"172.21.15.137:6835","nonce":763783602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.190:6835/1851042281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1851042281},{"type":"v1","addr":"172.21.15.190:6835","nonce":1851042281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/3181257549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3181257549},{"type":"v1","addr":"172.21.15.190:6837","nonce":3181257549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6837/2788541030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2788541030},{"type":"v1","addr":"172.21.15.137:6837","nonce":2788541030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.190:6839/1222573537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":1222573537},{"type":"v1","addr":"172.21.15.190:6839","nonce":1222573537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/3454924009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3454924009},{"type":"v1","addr":"172.21.15.137:6839","nonce":3454924009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24445,24457,24461,24469,24482]},"id":1}]} 2024-09-16T02:06:29.425 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 28 2024-09-16T02:06:29.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.425+0000 7fbf97f07640 1 -- 172.21.15.105:0/2350020152 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fbf7c0a32c0 msgr2=0x7fbf7c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:29.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.425+0000 7fbf97f07640 1 --2- 172.21.15.105:0/2350020152 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fbf7c0a32c0 0x7fbf7c0a5780 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c009bd0 tx=0x7fbf8c005f90 comp rx=0 tx=0).stop 2024-09-16T02:06:29.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.425+0000 7fbf97f07640 1 -- 172.21.15.105:0/2350020152 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf90103780 msgr2=0x7fbf90100f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:29.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.425+0000 7fbf97f07640 1 --2- 172.21.15.105:0/2350020152 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf90103780 0x7fbf90100f70 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fbf80031440 tx=0x7fbf80004270 comp rx=0 tx=0).stop 2024-09-16T02:06:29.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.425+0000 7fbf97f07640 1 -- 172.21.15.105:0/2350020152 shutdown_connections 2024-09-16T02:06:29.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.425+0000 7fbf97f07640 1 --2- 172.21.15.105:0/2350020152 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fbf7c0a32c0 0x7fbf7c0a5780 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.425+0000 7fbf97f07640 1 --2- 172.21.15.105:0/2350020152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf90108300 0x7fbf901014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.425+0000 7fbf97f07640 1 --2- 172.21.15.105:0/2350020152 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf90103780 0x7fbf90100f70 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.425+0000 7fbf97f07640 1 --2- 172.21.15.105:0/2350020152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf90102dd0 0x7fbf90100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.425+0000 7fbf97f07640 1 -- 172.21.15.105:0/2350020152 >> 172.21.15.105:0/2350020152 conn(0x7fbf900fc7d0 msgr2=0x7fbf90105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:29.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.425+0000 7fbf97f07640 1 -- 172.21.15.105:0/2350020152 shutdown_connections 2024-09-16T02:06:29.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:29.425+0000 7fbf97f07640 1 -- 172.21.15.105:0/2350020152 wait complete. 2024-09-16T02:06:29.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:29 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2350020152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:06:29.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:29 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/2350020152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:06:30.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:29 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/2350020152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:06:30.106 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T02:06:30.356 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:06:30.793 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:30 smithi105 ceph-mon[31197]: pgmap v458: 97 pgs: 97 active+clean; 967 KiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-16T02:06:30.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:30 smithi137 ceph-mon[34099]: pgmap v458: 97 pgs: 97 active+clean; 967 KiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-16T02:06:30.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.870+0000 7fb8a35a5640 1 -- 172.21.15.105:0/3694855481 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb89c102790 msgr2=0x7fb89c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:30.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.870+0000 7fb8a35a5640 1 --2- 172.21.15.105:0/3694855481 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb89c102790 0x7fb89c1067e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb898009900 tx=0x7fb89802a410 comp rx=0 tx=0).stop 2024-09-16T02:06:30.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.870+0000 7fb8a35a5640 1 -- 172.21.15.105:0/3694855481 shutdown_connections 2024-09-16T02:06:30.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.870+0000 7fb8a35a5640 1 --2- 172.21.15.105:0/3694855481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb89c106f10 0x7fb89c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:30.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.870+0000 7fb8a35a5640 1 --2- 172.21.15.105:0/3694855481 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb89c102790 0x7fb89c1067e0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:30.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.870+0000 7fb8a35a5640 1 --2- 172.21.15.105:0/3694855481 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb89c101de0 0x7fb89c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:30.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.870+0000 7fb8a35a5640 1 -- 172.21.15.105:0/3694855481 >> 172.21.15.105:0/3694855481 conn(0x7fb89c0fd8c0 msgr2=0x7fb89c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:30.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.870+0000 7fb8a35a5640 1 -- 172.21.15.105:0/3694855481 shutdown_connections 2024-09-16T02:06:30.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.870+0000 7fb8a35a5640 1 -- 172.21.15.105:0/3694855481 wait complete. 2024-09-16T02:06:30.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.871+0000 7fb8a35a5640 1 Processor -- start 2024-09-16T02:06:30.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.871+0000 7fb8a35a5640 1 -- start start 2024-09-16T02:06:30.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.871+0000 7fb8a35a5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb89c101de0 0x7fb89c19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:30.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.871+0000 7fb8a35a5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb89c102790 0x7fb89c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:30.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.871+0000 7fb8a35a5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb89c106f10 0x7fb89c19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:30.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.871+0000 7fb8a35a5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb89c111e40 con 0x7fb89c101de0 2024-09-16T02:06:30.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.871+0000 7fb8a35a5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb89c111cc0 con 0x7fb89c106f10 2024-09-16T02:06:30.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.871+0000 7fb8a35a5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb89c111fc0 con 0x7fb89c102790 2024-09-16T02:06:30.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.871+0000 7fb8a1341640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb89c101de0 0x7fb89c19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:30.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.871+0000 7fb8a1341640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb89c101de0 0x7fb89c19a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44678/0 (socket says 172.21.15.105:44678) 2024-09-16T02:06:30.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.871+0000 7fb8a1341640 1 -- 172.21.15.105:0/2400512216 learned_addr learned my addr 172.21.15.105:0/2400512216 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:06:30.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.871+0000 7fb8a0b40640 1 --2- 172.21.15.105:0/2400512216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb89c102790 0x7fb89c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:30.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.871+0000 7fb8a1341640 1 -- 172.21.15.105:0/2400512216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb89c102790 msgr2=0x7fb89c19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:30.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.871+0000 7fb8a1341640 1 --2- 172.21.15.105:0/2400512216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb89c102790 0x7fb89c19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:30.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.871+0000 7fb8a1341640 1 -- 172.21.15.105:0/2400512216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb89c106f10 msgr2=0x7fb89c19eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:30.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.871+0000 7fb8a1341640 1 --2- 172.21.15.105:0/2400512216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb89c106f10 0x7fb89c19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:30.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.871+0000 7fb8a1341640 1 -- 172.21.15.105:0/2400512216 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb89c19f5a0 con 0x7fb89c101de0 2024-09-16T02:06:30.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.871+0000 7fb8a0b40640 1 --2- 172.21.15.105:0/2400512216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb89c102790 0x7fb89c19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:06:30.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.872+0000 7fb8a1341640 1 --2- 172.21.15.105:0/2400512216 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb89c101de0 0x7fb89c19a5f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb88c00c920 tx=0x7fb88c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:30.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.872+0000 7fb8927fc640 1 -- 172.21.15.105:0/2400512216 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88c007c70 con 0x7fb89c101de0 2024-09-16T02:06:30.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.872+0000 7fb8927fc640 1 -- 172.21.15.105:0/2400512216 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb88c007e10 con 0x7fb89c101de0 2024-09-16T02:06:30.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.872+0000 7fb8a35a5640 1 -- 172.21.15.105:0/2400512216 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb89c19f890 con 0x7fb89c101de0 2024-09-16T02:06:30.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.872+0000 7fb8927fc640 1 -- 172.21.15.105:0/2400512216 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88c018820 con 0x7fb89c101de0 2024-09-16T02:06:30.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.872+0000 7fb8a35a5640 1 -- 172.21.15.105:0/2400512216 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb89c1a7170 con 0x7fb89c101de0 2024-09-16T02:06:30.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.873+0000 7fb8a35a5640 1 -- 172.21.15.105:0/2400512216 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb89c1021c0 con 0x7fb89c101de0 2024-09-16T02:06:30.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.874+0000 7fb8927fc640 1 -- 172.21.15.105:0/2400512216 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fb88c0189c0 con 0x7fb89c101de0 2024-09-16T02:06:30.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.874+0000 7fb8927fc640 1 --2- 172.21.15.105:0/2400512216 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb8880a3240 0x7fb8880a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:30.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.875+0000 7fb8a0b40640 1 --2- 172.21.15.105:0/2400512216 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb8880a3240 0x7fb8880a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:30.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.875+0000 7fb8927fc640 1 -- 172.21.15.105:0/2400512216 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7fb88c0cfb90 con 0x7fb89c101de0 2024-09-16T02:06:30.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.875+0000 7fb8a0b40640 1 --2- 172.21.15.105:0/2400512216 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb8880a3240 0x7fb8880a5700 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb898000c00 tx=0x7fb898031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:30.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:30.879+0000 7fb8927fc640 1 -- 172.21.15.105:0/2400512216 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb88c014030 con 0x7fb89c101de0 2024-09-16T02:06:31.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:31.047+0000 7fb8a35a5640 1 -- 172.21.15.105:0/2400512216 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb89c19b950 con 0x7fb89c101de0 2024-09-16T02:06:31.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:31.049+0000 7fb8927fc640 1 -- 172.21.15.105:0/2400512216 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22962 (secure 0 0 0) 0x7fb88c09c850 con 0x7fb89c101de0 2024-09-16T02:06:31.052 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-16T02:06:31.052 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":83,"fsid":"423b7a2a-73ce-11ef-bceb-c7b262605968","created":"2024-09-16T01:52:44.388033+0000","modified":"2024-09-16T02:02:41.553985+0000","last_up_change":"2024-09-16T01:59:42.218034+0000","last_in_change":"2024-09-16T01:59:18.785252+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T01:55:53.680808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T02:00:31.286117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T02:00:33.310045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a60d6c72-da8b-4b02-bb33-672be795e7f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6803","nonce":1226660290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6805","nonce":1226660290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6809","nonce":1226660290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1226660290},{"type":"v1","addr":"172.21.15.105:6807","nonce":1226660290}]},"public_addr":"172.21.15.105:6803/1226660290","cluster_addr":"172.21.15.105:6805/1226660290","heartbeat_back_addr":"172.21.15.105:6809/1226660290","heartbeat_front_addr":"172.21.15.105:6807/1226660290","state":["exists","up"]},{"osd":1,"uuid":"b1d9ae84-b9a0-4a7e-b0fe-c402d46809fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6801","nonce":34753956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6803","nonce":34753956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6807","nonce":34753956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":34753956},{"type":"v1","addr":"172.21.15.137:6805","nonce":34753956}]},"public_addr":"172.21.15.137:6801/34753956","cluster_addr":"172.21.15.137:6803/34753956","heartbeat_back_addr":"172.21.15.137:6807/34753956","heartbeat_front_addr":"172.21.15.137:6805/34753956","state":["exists","up"]},{"osd":2,"uuid":"137faae6-2e8b-44b5-8976-3d95a6eef1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6801","nonce":1159531918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6803","nonce":1159531918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6807","nonce":1159531918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1159531918},{"type":"v1","addr":"172.21.15.190:6805","nonce":1159531918}]},"public_addr":"172.21.15.190:6801/1159531918","cluster_addr":"172.21.15.190:6803/1159531918","heartbeat_back_addr":"172.21.15.190:6807/1159531918","heartbeat_front_addr":"172.21.15.190:6805/1159531918","state":["exists","up"]},{"osd":3,"uuid":"5743c37b-5cd8-42b6-81c8-5cf5afc71e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6811","nonce":604905679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6813","nonce":604905679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6817","nonce":604905679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":604905679},{"type":"v1","addr":"172.21.15.105:6815","nonce":604905679}]},"public_addr":"172.21.15.105:6811/604905679","cluster_addr":"172.21.15.105:6813/604905679","heartbeat_back_addr":"172.21.15.105:6817/604905679","heartbeat_front_addr":"172.21.15.105:6815/604905679","state":["exists","up"]},{"osd":4,"uuid":"e439d4a2-b54c-4de9-97f6-3510afe57f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6809","nonce":3758109169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6811","nonce":3758109169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6815","nonce":3758109169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3758109169},{"type":"v1","addr":"172.21.15.137:6813","nonce":3758109169}]},"public_addr":"172.21.15.137:6809/3758109169","cluster_addr":"172.21.15.137:6811/3758109169","heartbeat_back_addr":"172.21.15.137:6815/3758109169","heartbeat_front_addr":"172.21.15.137:6813/3758109169","state":["exists","up"]},{"osd":5,"uuid":"d1e2a7d5-99f8-40fa-af6e-0305cc0af677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6809","nonce":2086773754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6811","nonce":2086773754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6815","nonce":2086773754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2086773754},{"type":"v1","addr":"172.21.15.190:6813","nonce":2086773754}]},"public_addr":"172.21.15.190:6809/2086773754","cluster_addr":"172.21.15.190:6811/2086773754","heartbeat_back_addr":"172.21.15.190:6815/2086773754","heartbeat_front_addr":"172.21.15.190:6813/2086773754","state":["exists","up"]},{"osd":6,"uuid":"cd418159-a0f6-44c7-9213-c010b132fb56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6819","nonce":4012018725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6821","nonce":4012018725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6825","nonce":4012018725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":4012018725},{"type":"v1","addr":"172.21.15.105:6823","nonce":4012018725}]},"public_addr":"172.21.15.105:6819/4012018725","cluster_addr":"172.21.15.105:6821/4012018725","heartbeat_back_addr":"172.21.15.105:6825/4012018725","heartbeat_front_addr":"172.21.15.105:6823/4012018725","state":["exists","up"]},{"osd":7,"uuid":"76c15863-801e-4a72-be73-ba9df46c15b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6817","nonce":888678288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6819","nonce":888678288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6823","nonce":888678288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":888678288},{"type":"v1","addr":"172.21.15.137:6821","nonce":888678288}]},"public_addr":"172.21.15.137:6817/888678288","cluster_addr":"172.21.15.137:6819/888678288","heartbeat_back_addr":"172.21.15.137:6823/888678288","heartbeat_front_addr":"172.21.15.137:6821/888678288","state":["exists","up"]},{"osd":8,"uuid":"6718597c-6f42-48ee-9493-edb2396fe707","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6817","nonce":2132638341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6819","nonce":2132638341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6823","nonce":2132638341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2132638341},{"type":"v1","addr":"172.21.15.190:6821","nonce":2132638341}]},"public_addr":"172.21.15.190:6817/2132638341","cluster_addr":"172.21.15.190:6819/2132638341","heartbeat_back_addr":"172.21.15.190:6823/2132638341","heartbeat_front_addr":"172.21.15.190:6821/2132638341","state":["exists","up"]},{"osd":9,"uuid":"5178edc0-a2e8-491c-821c-6f41ec5aeb2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6827","nonce":3375034979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6829","nonce":3375034979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6833","nonce":3375034979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3375034979},{"type":"v1","addr":"172.21.15.105:6831","nonce":3375034979}]},"public_addr":"172.21.15.105:6827/3375034979","cluster_addr":"172.21.15.105:6829/3375034979","heartbeat_back_addr":"172.21.15.105:6833/3375034979","heartbeat_front_addr":"172.21.15.105:6831/3375034979","state":["exists","up"]},{"osd":10,"uuid":"ad145d3e-a6e2-4dd7-bea3-bcd01eb3413e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6825","nonce":318562234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6827","nonce":318562234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6831","nonce":318562234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":318562234},{"type":"v1","addr":"172.21.15.137:6829","nonce":318562234}]},"public_addr":"172.21.15.137:6825/318562234","cluster_addr":"172.21.15.137:6827/318562234","heartbeat_back_addr":"172.21.15.137:6831/318562234","heartbeat_front_addr":"172.21.15.137:6829/318562234","state":["exists","up"]},{"osd":11,"uuid":"6d44bced-3a8b-4763-a93e-0ea34acb4cb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6825","nonce":3820460689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6827","nonce":3820460689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6831","nonce":3820460689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3820460689},{"type":"v1","addr":"172.21.15.190:6829","nonce":3820460689}]},"public_addr":"172.21.15.190:6825/3820460689","cluster_addr":"172.21.15.190:6827/3820460689","heartbeat_back_addr":"172.21.15.190:6831/3820460689","heartbeat_front_addr":"172.21.15.190:6829/3820460689","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:55:02.759033+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:55:26.829142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:55:51.120200+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:56:17.277870+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:56:42.313701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:07.013416+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:32.477916+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:58.643764+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:23.454908+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:48.869082+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:14.921573+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:40.908993+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":7,"to":6}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.17","mappings":[{"from":5,"to":1}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":7}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:0/2453739657":"2024-09-17T01:53:39.351314+0000","172.21.15.105:6801/1469189484":"2024-09-17T01:53:39.351314+0000","172.21.15.105:0/3466166879":"2024-09-17T01:53:39.351314+0000","172.21.15.105:6801/1268348742":"2024-09-17T01:53:15.522227+0000","172.21.15.105:0/2897248478":"2024-09-17T01:53:39.351314+0000","172.21.15.105:0/3709363371":"2024-09-17T01:53:15.522227+0000","172.21.15.105:6800/1268348742":"2024-09-17T01:53:15.522227+0000","172.21.15.105:6800/1469189484":"2024-09-17T01:53:39.351314+0000","172.21.15.105:0/1198176299":"2024-09-17T01:53:15.522227+0000","172.21.15.105:0/3031917848":"2024-09-17T01:53:15.522227+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T02:06:31.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:31.051+0000 7fb8a35a5640 1 -- 172.21.15.105:0/2400512216 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb8880a3240 msgr2=0x7fb8880a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:31.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:31.051+0000 7fb8a35a5640 1 --2- 172.21.15.105:0/2400512216 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb8880a3240 0x7fb8880a5700 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb898000c00 tx=0x7fb898031400 comp rx=0 tx=0).stop 2024-09-16T02:06:31.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:31.051+0000 7fb8a35a5640 1 -- 172.21.15.105:0/2400512216 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb89c101de0 msgr2=0x7fb89c19a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:31.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:31.051+0000 7fb8a35a5640 1 --2- 172.21.15.105:0/2400512216 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb89c101de0 0x7fb89c19a5f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb88c00c920 tx=0x7fb88c00cde0 comp rx=0 tx=0).stop 2024-09-16T02:06:31.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:31.051+0000 7fb8a35a5640 1 -- 172.21.15.105:0/2400512216 shutdown_connections 2024-09-16T02:06:31.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:31.051+0000 7fb8a35a5640 1 --2- 172.21.15.105:0/2400512216 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fb8880a3240 0x7fb8880a5700 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:31.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:31.051+0000 7fb8a35a5640 1 --2- 172.21.15.105:0/2400512216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb89c102790 0x7fb89c19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:31.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:31.051+0000 7fb8a35a5640 1 --2- 172.21.15.105:0/2400512216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb89c106f10 0x7fb89c19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:31.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:31.051+0000 7fb8a35a5640 1 --2- 172.21.15.105:0/2400512216 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb89c101de0 0x7fb89c19a5f0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:31.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:31.051+0000 7fb8a35a5640 1 -- 172.21.15.105:0/2400512216 >> 172.21.15.105:0/2400512216 conn(0x7fb89c0fd8c0 msgr2=0x7fb89c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:31.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:31.051+0000 7fb8a35a5640 1 -- 172.21.15.105:0/2400512216 shutdown_connections 2024-09-16T02:06:31.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:31.051+0000 7fb8a35a5640 1 -- 172.21.15.105:0/2400512216 wait complete. 2024-09-16T02:06:31.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:30 smithi190 ceph-mon[34762]: pgmap v458: 97 pgs: 97 active+clean; 967 KiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-16T02:06:31.794 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:31 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/2400512216' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:06:31.795 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:31 smithi105 ceph-mon[31197]: pgmap v459: 97 pgs: 97 active+clean; 967 KiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-16T02:06:31.897 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph fs fail cephfs 2024-09-16T02:06:31.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:31 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/2400512216' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:06:31.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:31 smithi137 ceph-mon[34099]: pgmap v459: 97 pgs: 97 active+clean; 967 KiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-16T02:06:32.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:31 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/2400512216' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:06:32.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:31 smithi190 ceph-mon[34762]: pgmap v459: 97 pgs: 97 active+clean; 967 KiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-16T02:06:32.145 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:06:32.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.649+0000 7f832e0bb640 1 -- 172.21.15.105:0/188112518 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83281036e0 msgr2=0x7f832810bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:32.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.649+0000 7f832e0bb640 1 --2- 172.21.15.105:0/188112518 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83281036e0 0x7f832810bbd0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f831c009900 tx=0x7f831c02a480 comp rx=0 tx=0).stop 2024-09-16T02:06:32.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.649+0000 7f832e0bb640 1 -- 172.21.15.105:0/188112518 shutdown_connections 2024-09-16T02:06:32.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.649+0000 7f832e0bb640 1 --2- 172.21.15.105:0/188112518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f832810c1a0 0x7f832810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:32.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.649+0000 7f832e0bb640 1 --2- 172.21.15.105:0/188112518 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83281036e0 0x7f832810bbd0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:32.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.649+0000 7f832e0bb640 1 --2- 172.21.15.105:0/188112518 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8328102dc0 0x7f83281031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:32.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.649+0000 7f832e0bb640 1 -- 172.21.15.105:0/188112518 >> 172.21.15.105:0/188112518 conn(0x7f83280fc7a0 msgr2=0x7f83280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:32.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.649+0000 7f832e0bb640 1 -- 172.21.15.105:0/188112518 shutdown_connections 2024-09-16T02:06:32.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.649+0000 7f832e0bb640 1 -- 172.21.15.105:0/188112518 wait complete. 2024-09-16T02:06:32.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.650+0000 7f832e0bb640 1 Processor -- start 2024-09-16T02:06:32.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.650+0000 7f832e0bb640 1 -- start start 2024-09-16T02:06:32.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.650+0000 7f832e0bb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8328102dc0 0x7f8328198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:32.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.650+0000 7f832e0bb640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83281036e0 0x7f83281989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:32.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.650+0000 7f832e0bb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f832810c1a0 0x7f832819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:32.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.650+0000 7f832e0bb640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8328111e60 con 0x7f8328102dc0 2024-09-16T02:06:32.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.650+0000 7f832e0bb640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8328111ce0 con 0x7f832810c1a0 2024-09-16T02:06:32.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.650+0000 7f832e0bb640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8328111fe0 con 0x7f83281036e0 2024-09-16T02:06:32.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.650+0000 7f83277fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8328102dc0 0x7f8328198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:32.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.650+0000 7f83277fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8328102dc0 0x7f8328198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44690/0 (socket says 172.21.15.105:44690) 2024-09-16T02:06:32.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.650+0000 7f83277fe640 1 -- 172.21.15.105:0/4234539451 learned_addr learned my addr 172.21.15.105:0/4234539451 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:06:32.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.651+0000 7f8327fff640 1 --2- 172.21.15.105:0/4234539451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f832810c1a0 0x7f832819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:32.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.651+0000 7f8326ffd640 1 --2- 172.21.15.105:0/4234539451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83281036e0 0x7f83281989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:32.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.651+0000 7f83277fe640 1 -- 172.21.15.105:0/4234539451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83281036e0 msgr2=0x7f83281989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:32.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.651+0000 7f83277fe640 1 --2- 172.21.15.105:0/4234539451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83281036e0 0x7f83281989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:32.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.651+0000 7f83277fe640 1 -- 172.21.15.105:0/4234539451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f832810c1a0 msgr2=0x7f832819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:32.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.651+0000 7f83277fe640 1 --2- 172.21.15.105:0/4234539451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f832810c1a0 0x7f832819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:32.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.651+0000 7f83277fe640 1 -- 172.21.15.105:0/4234539451 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f832819d410 con 0x7f8328102dc0 2024-09-16T02:06:32.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.651+0000 7f83277fe640 1 --2- 172.21.15.105:0/4234539451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8328102dc0 0x7f8328198460 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f831800d950 tx=0x7f831800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:32.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.651+0000 7f8324ff9640 1 -- 172.21.15.105:0/4234539451 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8318017030 con 0x7f8328102dc0 2024-09-16T02:06:32.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.651+0000 7f832e0bb640 1 -- 172.21.15.105:0/4234539451 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f832819d700 con 0x7f8328102dc0 2024-09-16T02:06:32.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.651+0000 7f8324ff9640 1 -- 172.21.15.105:0/4234539451 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8318004020 con 0x7f8328102dc0 2024-09-16T02:06:32.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.651+0000 7f8324ff9640 1 -- 172.21.15.105:0/4234539451 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83180047a0 con 0x7f8328102dc0 2024-09-16T02:06:32.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.651+0000 7f832e0bb640 1 -- 172.21.15.105:0/4234539451 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83281a4fe0 con 0x7f8328102dc0 2024-09-16T02:06:32.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.652+0000 7f832e0bb640 1 -- 172.21.15.105:0/4234539451 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83281031a0 con 0x7f8328102dc0 2024-09-16T02:06:32.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.653+0000 7f8324ff9640 1 -- 172.21.15.105:0/4234539451 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f83180041d0 con 0x7f8328102dc0 2024-09-16T02:06:32.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.654+0000 7f8324ff9640 1 --2- 172.21.15.105:0/4234539451 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f83100a3240 0x7f83100a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:32.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.654+0000 7f8326ffd640 1 --2- 172.21.15.105:0/4234539451 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f83100a3240 0x7f83100a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:32.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.654+0000 7f8324ff9640 1 -- 172.21.15.105:0/4234539451 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8335+0+0 (secure 0 0 0) 0x7f83180cf410 con 0x7f8328102dc0 2024-09-16T02:06:32.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.654+0000 7f8326ffd640 1 --2- 172.21.15.105:0/4234539451 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f83100a3240 0x7f83100a5700 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f831c031440 tx=0x7f831c0095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:32.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.658+0000 7f8324ff9640 1 -- 172.21.15.105:0/4234539451 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f831809c0d0 con 0x7f8328102dc0 2024-09-16T02:06:32.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.858+0000 7f832e0bb640 1 -- 172.21.15.105:0/4234539451 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f8328107210 con 0x7f8328102dc0 2024-09-16T02:06:32.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.860+0000 7f8324ff9640 1 -- 172.21.15.105:0/4234539451 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v29) ==== 161+0+0 (secure 0 0 0) 0x7f831809b9d0 con 0x7f8328102dc0 2024-09-16T02:06:32.862 INFO:teuthology.orchestra.run.smithi105.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T02:06:32.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.865+0000 7f832e0bb640 1 -- 172.21.15.105:0/4234539451 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f83100a3240 msgr2=0x7f83100a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:32.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.865+0000 7f832e0bb640 1 --2- 172.21.15.105:0/4234539451 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f83100a3240 0x7f83100a5700 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f831c031440 tx=0x7f831c0095f0 comp rx=0 tx=0).stop 2024-09-16T02:06:32.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.865+0000 7f832e0bb640 1 -- 172.21.15.105:0/4234539451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8328102dc0 msgr2=0x7f8328198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:32.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.865+0000 7f832e0bb640 1 --2- 172.21.15.105:0/4234539451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8328102dc0 0x7f8328198460 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f831800d950 tx=0x7f831800de10 comp rx=0 tx=0).stop 2024-09-16T02:06:32.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.865+0000 7f832e0bb640 1 -- 172.21.15.105:0/4234539451 shutdown_connections 2024-09-16T02:06:32.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.865+0000 7f832e0bb640 1 --2- 172.21.15.105:0/4234539451 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f83100a3240 0x7f83100a5700 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:32.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.865+0000 7f832e0bb640 1 --2- 172.21.15.105:0/4234539451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83281036e0 0x7f83281989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:32.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.865+0000 7f832e0bb640 1 --2- 172.21.15.105:0/4234539451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f832810c1a0 0x7f832819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:32.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.865+0000 7f832e0bb640 1 --2- 172.21.15.105:0/4234539451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8328102dc0 0x7f8328198460 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:32.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.865+0000 7f832e0bb640 1 -- 172.21.15.105:0/4234539451 >> 172.21.15.105:0/4234539451 conn(0x7f83280fc7a0 msgr2=0x7f83280fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:32.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.865+0000 7f832e0bb640 1 -- 172.21.15.105:0/4234539451 shutdown_connections 2024-09-16T02:06:32.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:32.865+0000 7f832e0bb640 1 -- 172.21.15.105:0/4234539451 wait complete. 2024-09-16T02:06:33.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:32 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/4234539451' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-16T02:06:33.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:32 smithi137 ceph-mon[34099]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-16T02:06:33.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:32 smithi137 ceph-mon[34099]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T02:06:33.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:32 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/4234539451' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-16T02:06:33.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:32 smithi137 ceph-mon[34099]: osdmap e84: 12 total, 12 up, 12 in 2024-09-16T02:06:33.172 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:32 smithi137 ceph-mon[34099]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-16T02:06:33.172 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 16 02:06:32 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-b[64153]: ignoring --setuser ceph since I am not root 2024-09-16T02:06:33.172 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 16 02:06:32 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-b[64153]: ignoring --setgroup ceph since I am not root 2024-09-16T02:06:33.172 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 16 02:06:32 smithi137 ceph-mds[64157]: -- 172.21.15.137:0/2268409146 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c362448680 con 0x55c36241c800 2024-09-16T02:06:33.172 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 16 02:06:32 smithi137 ceph-mds[64157]: -- 172.21.15.137:0/2268409146 <== mon.0 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c36243f860 con 0x55c36241c800 2024-09-16T02:06:33.172 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 16 02:06:32 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-b[64153]: starting mds.b at 2024-09-16T02:06:33.173 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 16 02:06:32 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-e[63535]: ignoring --setuser ceph since I am not root 2024-09-16T02:06:33.173 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 16 02:06:32 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-e[63535]: ignoring --setgroup ceph since I am not root 2024-09-16T02:06:33.173 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 16 02:06:32 smithi137 ceph-mds[63539]: -- 172.21.15.137:0/462180052 <== mon.0 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d2de89f860 con 0x55d2de87c800 2024-09-16T02:06:33.173 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 16 02:06:32 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-e[63535]: starting mds.e at 2024-09-16T02:06:33.173 INFO:journalctl@ceph.mds.h.smithi137.stdout:Sep 16 02:06:32 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-h[63836]: ignoring --setuser ceph since I am not root 2024-09-16T02:06:33.173 INFO:journalctl@ceph.mds.h.smithi137.stdout:Sep 16 02:06:32 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-h[63836]: ignoring --setgroup ceph since I am not root 2024-09-16T02:06:33.174 INFO:journalctl@ceph.mds.h.smithi137.stdout:Sep 16 02:06:32 smithi137 ceph-mds[63840]: -- 172.21.15.137:0/3782589103 <== mon.0 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561cdcf03860 con 0x561cdcee1000 2024-09-16T02:06:33.174 INFO:journalctl@ceph.mds.h.smithi137.stdout:Sep 16 02:06:32 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-h[63836]: starting mds.h at 2024-09-16T02:06:33.174 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 16 02:06:32 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-k[63222]: ignoring --setuser ceph since I am not root 2024-09-16T02:06:33.174 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 16 02:06:32 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-k[63222]: ignoring --setgroup ceph since I am not root 2024-09-16T02:06:33.174 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 16 02:06:32 smithi137 ceph-mds[63226]: -- 172.21.15.137:0/3470741530 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55698c9cd860 con 0x55698c9aa800 2024-09-16T02:06:33.174 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 16 02:06:32 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-k[63222]: starting mds.k at 2024-09-16T02:06:33.183 INFO:journalctl@ceph.mds.a.smithi105.stdout:Sep 16 02:06:32 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-a[72108]: ignoring --setuser ceph since I am not root 2024-09-16T02:06:33.183 INFO:journalctl@ceph.mds.a.smithi105.stdout:Sep 16 02:06:32 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-a[72108]: ignoring --setgroup ceph since I am not root 2024-09-16T02:06:33.183 INFO:journalctl@ceph.mds.a.smithi105.stdout:Sep 16 02:06:32 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-a[72108]: starting mds.a at 2024-09-16T02:06:33.183 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:32 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mon-a[31193]: 2024-09-16T02:06:32.859+0000 7f9eadf3b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T02:06:33.183 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:32 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/4234539451' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-16T02:06:33.183 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:32 smithi105 ceph-mon[31197]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-16T02:06:33.183 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:32 smithi105 ceph-mon[31197]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T02:06:33.183 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:32 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/4234539451' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-16T02:06:33.183 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:32 smithi105 ceph-mon[31197]: osdmap e84: 12 total, 12 up, 12 in 2024-09-16T02:06:33.184 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:32 smithi105 ceph-mon[31197]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-16T02:06:33.184 INFO:journalctl@ceph.mds.g.smithi105.stdout:Sep 16 02:06:32 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-g[71712]: ignoring --setuser ceph since I am not root 2024-09-16T02:06:33.184 INFO:journalctl@ceph.mds.g.smithi105.stdout:Sep 16 02:06:32 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-g[71712]: ignoring --setgroup ceph since I am not root 2024-09-16T02:06:33.184 INFO:journalctl@ceph.mds.g.smithi105.stdout:Sep 16 02:06:32 smithi105 ceph-mds[71727]: -- 172.21.15.105:0/1762767770 <== mon.0 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557030de1860 con 0x557030dbe800 2024-09-16T02:06:33.184 INFO:journalctl@ceph.mds.g.smithi105.stdout:Sep 16 02:06:32 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-g[71712]: starting mds.g at 2024-09-16T02:06:33.343 INFO:journalctl@ceph.mds.l.smithi190.stdout:Sep 16 02:06:32 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-l[63700]: ignoring --setuser ceph since I am not root 2024-09-16T02:06:33.343 INFO:journalctl@ceph.mds.l.smithi190.stdout:Sep 16 02:06:32 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-l[63700]: ignoring --setgroup ceph since I am not root 2024-09-16T02:06:33.343 INFO:journalctl@ceph.mds.l.smithi190.stdout:Sep 16 02:06:32 smithi190 ceph-mds[63704]: -- 172.21.15.190:0/818440900 <== mon.0 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b8ecc77c20 con 0x55b8ecc55000 2024-09-16T02:06:33.343 INFO:journalctl@ceph.mds.l.smithi190.stdout:Sep 16 02:06:32 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-l[63700]: starting mds.l at 2024-09-16T02:06:33.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:32 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/4234539451' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-16T02:06:33.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:32 smithi190 ceph-mon[34762]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-16T02:06:33.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:32 smithi190 ceph-mon[34762]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T02:06:33.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:32 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/4234539451' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-16T02:06:33.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:32 smithi190 ceph-mon[34762]: osdmap e84: 12 total, 12 up, 12 in 2024-09-16T02:06:33.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:32 smithi190 ceph-mon[34762]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-16T02:06:33.344 INFO:journalctl@ceph.mds.c.smithi190.stdout:Sep 16 02:06:32 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-c[64307]: ignoring --setuser ceph since I am not root 2024-09-16T02:06:33.344 INFO:journalctl@ceph.mds.c.smithi190.stdout:Sep 16 02:06:32 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-c[64307]: ignoring --setgroup ceph since I am not root 2024-09-16T02:06:33.344 INFO:journalctl@ceph.mds.c.smithi190.stdout:Sep 16 02:06:32 smithi190 ceph-mds[64311]: -- 172.21.15.190:0/2075931087 <== mon.0 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560c11b93860 con 0x560c11b71000 2024-09-16T02:06:33.344 INFO:journalctl@ceph.mds.c.smithi190.stdout:Sep 16 02:06:32 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-c[64307]: starting mds.c at 2024-09-16T02:06:33.345 INFO:journalctl@ceph.mds.i.smithi190.stdout:Sep 16 02:06:32 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-i[64605]: ignoring --setuser ceph since I am not root 2024-09-16T02:06:33.345 INFO:journalctl@ceph.mds.i.smithi190.stdout:Sep 16 02:06:32 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-i[64605]: ignoring --setgroup ceph since I am not root 2024-09-16T02:06:33.345 INFO:journalctl@ceph.mds.i.smithi190.stdout:Sep 16 02:06:32 smithi190 ceph-mds[64609]: -- 172.21.15.190:0/2867474493 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5600a8f73860 con 0x5600a8f51400 2024-09-16T02:06:33.345 INFO:journalctl@ceph.mds.i.smithi190.stdout:Sep 16 02:06:32 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-i[64605]: starting mds.i at 2024-09-16T02:06:33.345 INFO:journalctl@ceph.mds.f.smithi190.stdout:Sep 16 02:06:32 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-f[64004]: ignoring --setuser ceph since I am not root 2024-09-16T02:06:33.345 INFO:journalctl@ceph.mds.f.smithi190.stdout:Sep 16 02:06:32 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-f[64004]: ignoring --setgroup ceph since I am not root 2024-09-16T02:06:33.345 INFO:journalctl@ceph.mds.f.smithi190.stdout:Sep 16 02:06:32 smithi190 ceph-mds[64008]: -- 172.21.15.190:0/2988733893 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55937f883860 con 0x55937f860800 2024-09-16T02:06:33.345 INFO:journalctl@ceph.mds.f.smithi190.stdout:Sep 16 02:06:32 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-f[64004]: starting mds.f at 2024-09-16T02:06:33.670 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T02:06:33.916 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:06:34.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:33 smithi137 ceph-mon[34099]: pgmap v461: 97 pgs: 97 active+clean; 967 KiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-16T02:06:34.179 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:33 smithi105 ceph-mon[31197]: pgmap v461: 97 pgs: 97 active+clean; 967 KiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-16T02:06:34.342 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:33 smithi190 ceph-mon[34762]: pgmap v461: 97 pgs: 97 active+clean; 967 KiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-16T02:06:34.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.409+0000 7fd2bf4ba640 1 -- 172.21.15.105:0/2048665287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2b8102ef0 msgr2=0x7fd2b810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:34.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.409+0000 7fd2bf4ba640 1 --2- 172.21.15.105:0/2048665287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2b8102ef0 0x7fd2b810da40 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0009ca0 tx=0x7fd2b002a950 comp rx=0 tx=0).stop 2024-09-16T02:06:34.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.409+0000 7fd2bf4ba640 1 -- 172.21.15.105:0/2048665287 shutdown_connections 2024-09-16T02:06:34.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.409+0000 7fd2bf4ba640 1 --2- 172.21.15.105:0/2048665287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2b8102570 0x7fd2b81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.409+0000 7fd2bf4ba640 1 --2- 172.21.15.105:0/2048665287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2b8102ef0 0x7fd2b810da40 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.409+0000 7fd2bf4ba640 1 --2- 172.21.15.105:0/2048665287 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2b8100e20 0x7fd2b8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.409+0000 7fd2bf4ba640 1 -- 172.21.15.105:0/2048665287 >> 172.21.15.105:0/2048665287 conn(0x7fd2b80fc7d0 msgr2=0x7fd2b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:34.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.409+0000 7fd2bf4ba640 1 -- 172.21.15.105:0/2048665287 shutdown_connections 2024-09-16T02:06:34.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.409+0000 7fd2bf4ba640 1 -- 172.21.15.105:0/2048665287 wait complete. 2024-09-16T02:06:34.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.409+0000 7fd2bf4ba640 1 Processor -- start 2024-09-16T02:06:34.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.409+0000 7fd2bf4ba640 1 -- start start 2024-09-16T02:06:34.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.410+0000 7fd2bf4ba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2b8100e20 0x7fd2b81992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:34.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.410+0000 7fd2bf4ba640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2b8102570 0x7fd2b8199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:34.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.410+0000 7fd2bf4ba640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2b8102ef0 0x7fd2b81a4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:34.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.410+0000 7fd2bf4ba640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b8110e30 con 0x7fd2b8100e20 2024-09-16T02:06:34.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.410+0000 7fd2bf4ba640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b8110cb0 con 0x7fd2b8102570 2024-09-16T02:06:34.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.410+0000 7fd2bf4ba640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b8110fb0 con 0x7fd2b8102ef0 2024-09-16T02:06:34.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.410+0000 7fd2bd256640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2b8100e20 0x7fd2b81992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:34.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.410+0000 7fd2bd256640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2b8100e20 0x7fd2b81992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44738/0 (socket says 172.21.15.105:44738) 2024-09-16T02:06:34.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.410+0000 7fd2bd256640 1 -- 172.21.15.105:0/1545722841 learned_addr learned my addr 172.21.15.105:0/1545722841 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:06:34.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.410+0000 7fd2bda57640 1 --2- 172.21.15.105:0/1545722841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2b8102ef0 0x7fd2b81a4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:34.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.410+0000 7fd2bca55640 1 --2- 172.21.15.105:0/1545722841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2b8102570 0x7fd2b8199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:34.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.410+0000 7fd2bd256640 1 -- 172.21.15.105:0/1545722841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2b8102ef0 msgr2=0x7fd2b81a4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:34.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.410+0000 7fd2bd256640 1 --2- 172.21.15.105:0/1545722841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2b8102ef0 0x7fd2b81a4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.410+0000 7fd2bd256640 1 -- 172.21.15.105:0/1545722841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2b8102570 msgr2=0x7fd2b8199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:34.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.410+0000 7fd2bd256640 1 --2- 172.21.15.105:0/1545722841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2b8102570 0x7fd2b8199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.410+0000 7fd2bd256640 1 -- 172.21.15.105:0/1545722841 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2b81a4880 con 0x7fd2b8100e20 2024-09-16T02:06:34.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.410+0000 7fd2bd256640 1 --2- 172.21.15.105:0/1545722841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2b8100e20 0x7fd2b81992c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd2b400d950 tx=0x7fd2b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:34.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.410+0000 7fd2ae7fc640 1 -- 172.21.15.105:0/1545722841 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b4017070 con 0x7fd2b8100e20 2024-09-16T02:06:34.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.410+0000 7fd2ae7fc640 1 -- 172.21.15.105:0/1545722841 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2b4009c30 con 0x7fd2b8100e20 2024-09-16T02:06:34.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.410+0000 7fd2bf4ba640 1 -- 172.21.15.105:0/1545722841 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2b81a4a50 con 0x7fd2b8100e20 2024-09-16T02:06:34.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.410+0000 7fd2ae7fc640 1 -- 172.21.15.105:0/1545722841 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b400f620 con 0x7fd2b8100e20 2024-09-16T02:06:34.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.410+0000 7fd2bf4ba640 1 -- 172.21.15.105:0/1545722841 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2b81a4f60 con 0x7fd2b8100e20 2024-09-16T02:06:34.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.411+0000 7fd2bf4ba640 1 -- 172.21.15.105:0/1545722841 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2b8101200 con 0x7fd2b8100e20 2024-09-16T02:06:34.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.412+0000 7fd2ae7fc640 1 -- 172.21.15.105:0/1545722841 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fd2b40109a0 con 0x7fd2b8100e20 2024-09-16T02:06:34.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.413+0000 7fd2ae7fc640 1 --2- 172.21.15.105:0/1545722841 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fd2a40a3240 0x7fd2a40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:34.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.413+0000 7fd2bca55640 1 --2- 172.21.15.105:0/1545722841 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fd2a40a3240 0x7fd2a40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:34.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.413+0000 7fd2ae7fc640 1 -- 172.21.15.105:0/1545722841 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8765+0+0 (secure 0 0 0) 0x7fd2b40cf820 con 0x7fd2b8100e20 2024-09-16T02:06:34.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.413+0000 7fd2bca55640 1 --2- 172.21.15.105:0/1545722841 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fd2a40a3240 0x7fd2a40a5700 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8005d70 tx=0x7fd2a8005d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:34.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.417+0000 7fd2ae7fc640 1 -- 172.21.15.105:0/1545722841 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd2b4009de0 con 0x7fd2b8100e20 2024-09-16T02:06:34.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.623+0000 7fd2bf4ba640 1 -- 172.21.15.105:0/1545722841 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd2b8109090 con 0x7fd2b8100e20 2024-09-16T02:06:34.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.925+0000 7fd2ae7fc640 1 -- 172.21.15.105:0/1545722841 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v31) ==== 104+0+0 (secure 0 0 0) 0x7fd2b409c330 con 0x7fd2b8100e20 2024-09-16T02:06:34.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.927+0000 7fd2bf4ba640 1 -- 172.21.15.105:0/1545722841 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fd2a40a3240 msgr2=0x7fd2a40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:34.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.927+0000 7fd2bf4ba640 1 --2- 172.21.15.105:0/1545722841 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fd2a40a3240 0x7fd2a40a5700 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8005d70 tx=0x7fd2a8005d00 comp rx=0 tx=0).stop 2024-09-16T02:06:34.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.927+0000 7fd2bf4ba640 1 -- 172.21.15.105:0/1545722841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2b8100e20 msgr2=0x7fd2b81992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:34.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.927+0000 7fd2bf4ba640 1 --2- 172.21.15.105:0/1545722841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2b8100e20 0x7fd2b81992c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd2b400d950 tx=0x7fd2b400de10 comp rx=0 tx=0).stop 2024-09-16T02:06:34.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.927+0000 7fd2bf4ba640 1 -- 172.21.15.105:0/1545722841 shutdown_connections 2024-09-16T02:06:34.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.927+0000 7fd2bf4ba640 1 --2- 172.21.15.105:0/1545722841 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7fd2a40a3240 0x7fd2a40a5700 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.927+0000 7fd2bf4ba640 1 --2- 172.21.15.105:0/1545722841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2b8102ef0 0x7fd2b81a4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.927+0000 7fd2bf4ba640 1 --2- 172.21.15.105:0/1545722841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2b8102570 0x7fd2b8199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.928+0000 7fd2bf4ba640 1 --2- 172.21.15.105:0/1545722841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2b8100e20 0x7fd2b81992c0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.928+0000 7fd2bf4ba640 1 -- 172.21.15.105:0/1545722841 >> 172.21.15.105:0/1545722841 conn(0x7fd2b80fc7d0 msgr2=0x7fd2b80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:34.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.928+0000 7fd2bf4ba640 1 -- 172.21.15.105:0/1545722841 shutdown_connections 2024-09-16T02:06:34.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:34.928+0000 7fd2bf4ba640 1 -- 172.21.15.105:0/1545722841 wait complete. 2024-09-16T02:06:35.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:34 smithi105 ceph-mon[31197]: mds.? [v2:172.21.15.190:6833/1268797858,v1:172.21.15.190:6835/1268797858] up:boot 2024-09-16T02:06:35.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:34 smithi105 ceph-mon[31197]: mds.? [v2:172.21.15.105:6838/4115235926,v1:172.21.15.105:6840/4115235926] up:boot 2024-09-16T02:06:35.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:34 smithi105 ceph-mon[31197]: mds.? [v2:172.21.15.105:6839/2780260285,v1:172.21.15.105:6841/2780260285] up:boot 2024-09-16T02:06:35.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:34 smithi105 ceph-mon[31197]: mds.? [v2:172.21.15.190:6832/3429545246,v1:172.21.15.190:6834/3429545246] up:boot 2024-09-16T02:06:35.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:34 smithi105 ceph-mon[31197]: mds.? [v2:172.21.15.190:6836/625491845,v1:172.21.15.190:6838/625491845] up:boot 2024-09-16T02:06:35.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:34 smithi105 ceph-mon[31197]: mds.? [v2:172.21.15.190:6837/304180479,v1:172.21.15.190:6839/304180479] up:boot 2024-09-16T02:06:35.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:34 smithi105 ceph-mon[31197]: mds.? [v2:172.21.15.137:6832/1700632397,v1:172.21.15.137:6833/1700632397] up:boot 2024-09-16T02:06:35.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:34 smithi105 ceph-mon[31197]: mds.? [v2:172.21.15.137:6834/2070321815,v1:172.21.15.137:6837/2070321815] up:boot 2024-09-16T02:06:35.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:34 smithi105 ceph-mon[31197]: mds.? [v2:172.21.15.137:6836/3947750573,v1:172.21.15.137:6839/3947750573] up:boot 2024-09-16T02:06:35.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:34 smithi105 ceph-mon[31197]: mds.? [v2:172.21.15.137:6835/3467111214,v1:172.21.15.137:6838/3467111214] up:boot 2024-09-16T02:06:35.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:34 smithi105 ceph-mon[31197]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-16T02:06:35.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:34 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-16T02:06:35.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:34 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-16T02:06:35.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:34 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T02:06:35.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:34 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T02:06:35.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:34 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T02:06:35.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:34 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-16T02:06:35.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:34 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T02:06:35.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:34 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T02:06:35.297 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:34 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T02:06:35.297 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:34 smithi105 ceph-mon[31197]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T02:06:35.297 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:34 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/1545722841' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T02:06:35.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:34 smithi190 ceph-mon[34762]: mds.? [v2:172.21.15.190:6833/1268797858,v1:172.21.15.190:6835/1268797858] up:boot 2024-09-16T02:06:35.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:34 smithi190 ceph-mon[34762]: mds.? [v2:172.21.15.105:6838/4115235926,v1:172.21.15.105:6840/4115235926] up:boot 2024-09-16T02:06:35.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:34 smithi190 ceph-mon[34762]: mds.? [v2:172.21.15.105:6839/2780260285,v1:172.21.15.105:6841/2780260285] up:boot 2024-09-16T02:06:35.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:34 smithi190 ceph-mon[34762]: mds.? [v2:172.21.15.190:6832/3429545246,v1:172.21.15.190:6834/3429545246] up:boot 2024-09-16T02:06:35.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:34 smithi190 ceph-mon[34762]: mds.? [v2:172.21.15.190:6836/625491845,v1:172.21.15.190:6838/625491845] up:boot 2024-09-16T02:06:35.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:34 smithi190 ceph-mon[34762]: mds.? [v2:172.21.15.190:6837/304180479,v1:172.21.15.190:6839/304180479] up:boot 2024-09-16T02:06:35.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:34 smithi190 ceph-mon[34762]: mds.? [v2:172.21.15.137:6832/1700632397,v1:172.21.15.137:6833/1700632397] up:boot 2024-09-16T02:06:35.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:34 smithi190 ceph-mon[34762]: mds.? [v2:172.21.15.137:6834/2070321815,v1:172.21.15.137:6837/2070321815] up:boot 2024-09-16T02:06:35.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:34 smithi190 ceph-mon[34762]: mds.? [v2:172.21.15.137:6836/3947750573,v1:172.21.15.137:6839/3947750573] up:boot 2024-09-16T02:06:35.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:34 smithi190 ceph-mon[34762]: mds.? [v2:172.21.15.137:6835/3467111214,v1:172.21.15.137:6838/3467111214] up:boot 2024-09-16T02:06:35.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:34 smithi190 ceph-mon[34762]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-16T02:06:35.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:34 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-16T02:06:35.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:34 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-16T02:06:35.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:34 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T02:06:35.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:34 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T02:06:35.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:34 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T02:06:35.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:34 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-16T02:06:35.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:34 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T02:06:35.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:34 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T02:06:35.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:34 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T02:06:35.345 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:34 smithi190 ceph-mon[34762]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T02:06:35.345 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:34 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/1545722841' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T02:06:35.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:34 smithi137 ceph-mon[34099]: mds.? [v2:172.21.15.190:6833/1268797858,v1:172.21.15.190:6835/1268797858] up:boot 2024-09-16T02:06:35.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:34 smithi137 ceph-mon[34099]: mds.? [v2:172.21.15.105:6838/4115235926,v1:172.21.15.105:6840/4115235926] up:boot 2024-09-16T02:06:35.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:34 smithi137 ceph-mon[34099]: mds.? [v2:172.21.15.105:6839/2780260285,v1:172.21.15.105:6841/2780260285] up:boot 2024-09-16T02:06:35.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:34 smithi137 ceph-mon[34099]: mds.? [v2:172.21.15.190:6832/3429545246,v1:172.21.15.190:6834/3429545246] up:boot 2024-09-16T02:06:35.422 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:34 smithi137 ceph-mon[34099]: mds.? [v2:172.21.15.190:6836/625491845,v1:172.21.15.190:6838/625491845] up:boot 2024-09-16T02:06:35.422 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:34 smithi137 ceph-mon[34099]: mds.? [v2:172.21.15.190:6837/304180479,v1:172.21.15.190:6839/304180479] up:boot 2024-09-16T02:06:35.422 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:34 smithi137 ceph-mon[34099]: mds.? [v2:172.21.15.137:6832/1700632397,v1:172.21.15.137:6833/1700632397] up:boot 2024-09-16T02:06:35.422 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:34 smithi137 ceph-mon[34099]: mds.? [v2:172.21.15.137:6834/2070321815,v1:172.21.15.137:6837/2070321815] up:boot 2024-09-16T02:06:35.422 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:34 smithi137 ceph-mon[34099]: mds.? [v2:172.21.15.137:6836/3947750573,v1:172.21.15.137:6839/3947750573] up:boot 2024-09-16T02:06:35.422 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:34 smithi137 ceph-mon[34099]: mds.? [v2:172.21.15.137:6835/3467111214,v1:172.21.15.137:6838/3467111214] up:boot 2024-09-16T02:06:35.422 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:34 smithi137 ceph-mon[34099]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-16T02:06:35.422 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:34 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-16T02:06:35.422 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:34 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-16T02:06:35.422 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:34 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T02:06:35.422 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:34 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T02:06:35.422 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:34 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T02:06:35.422 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:34 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-16T02:06:35.423 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:34 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T02:06:35.423 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:34 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T02:06:35.423 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:34 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T02:06:35.423 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:34 smithi137 ceph-mon[34099]: from='mgr.14150 172.21.15.105:0/2597342194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T02:06:35.423 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:34 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/1545722841' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T02:06:35.771 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T02:06:36.022 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:06:36.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:35 smithi105 ceph-mon[31197]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-16T02:06:36.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:35 smithi105 ceph-mon[31197]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-16T02:06:36.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:35 smithi105 ceph-mon[31197]: Cluster is now healthy 2024-09-16T02:06:36.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:35 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/1545722841' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-16T02:06:36.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:35 smithi105 ceph-mon[31197]: fsmap 12 up:standby 2024-09-16T02:06:36.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:35 smithi105 ceph-mon[31197]: pgmap v462: 97 pgs: 97 active+clean; 967 KiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-16T02:06:36.342 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:35 smithi190 ceph-mon[34762]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-16T02:06:36.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:35 smithi190 ceph-mon[34762]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-16T02:06:36.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:35 smithi190 ceph-mon[34762]: Cluster is now healthy 2024-09-16T02:06:36.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:35 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/1545722841' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-16T02:06:36.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:35 smithi190 ceph-mon[34762]: fsmap 12 up:standby 2024-09-16T02:06:36.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:35 smithi190 ceph-mon[34762]: pgmap v462: 97 pgs: 97 active+clean; 967 KiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-16T02:06:36.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:35 smithi137 ceph-mon[34099]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-16T02:06:36.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:35 smithi137 ceph-mon[34099]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-16T02:06:36.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:35 smithi137 ceph-mon[34099]: Cluster is now healthy 2024-09-16T02:06:36.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:35 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/1545722841' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-16T02:06:36.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:35 smithi137 ceph-mon[34099]: fsmap 12 up:standby 2024-09-16T02:06:36.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:35 smithi137 ceph-mon[34099]: pgmap v462: 97 pgs: 97 active+clean; 967 KiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-16T02:06:36.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.506+0000 7f7a79d6e640 1 -- 172.21.15.105:0/3468667561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a74103660 msgr2=0x7f7a7410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:36.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.506+0000 7f7a79d6e640 1 --2- 172.21.15.105:0/3468667561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a74103660 0x7f7a7410bb50 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f7a68009900 tx=0x7f7a6802a480 comp rx=0 tx=0).stop 2024-09-16T02:06:36.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.506+0000 7f7a79d6e640 1 -- 172.21.15.105:0/3468667561 shutdown_connections 2024-09-16T02:06:36.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.506+0000 7f7a79d6e640 1 --2- 172.21.15.105:0/3468667561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a7410c170 0x7f7a7410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:36.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.506+0000 7f7a79d6e640 1 --2- 172.21.15.105:0/3468667561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a74103660 0x7f7a7410bb50 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:36.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.506+0000 7f7a79d6e640 1 --2- 172.21.15.105:0/3468667561 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a74102d40 0x7f7a74103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:36.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.506+0000 7f7a79d6e640 1 -- 172.21.15.105:0/3468667561 >> 172.21.15.105:0/3468667561 conn(0x7f7a740fc7a0 msgr2=0x7f7a740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:36.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.506+0000 7f7a79d6e640 1 -- 172.21.15.105:0/3468667561 shutdown_connections 2024-09-16T02:06:36.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.506+0000 7f7a79d6e640 1 -- 172.21.15.105:0/3468667561 wait complete. 2024-09-16T02:06:36.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.507+0000 7f7a79d6e640 1 Processor -- start 2024-09-16T02:06:36.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.507+0000 7f7a79d6e640 1 -- start start 2024-09-16T02:06:36.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.507+0000 7f7a79d6e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a74102d40 0x7f7a741009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:36.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.507+0000 7f7a79d6e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a74103660 0x7f7a74100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:36.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.507+0000 7f7a79d6e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a7410c170 0x7f7a74101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:36.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.507+0000 7f7a79d6e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7a74111e60 con 0x7f7a74102d40 2024-09-16T02:06:36.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.507+0000 7f7a79d6e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7a74111ce0 con 0x7f7a74103660 2024-09-16T02:06:36.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.507+0000 7f7a79d6e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7a74111fe0 con 0x7f7a7410c170 2024-09-16T02:06:36.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.507+0000 7f7a737fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a74102d40 0x7f7a741009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:36.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.507+0000 7f7a737fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a74102d40 0x7f7a741009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44764/0 (socket says 172.21.15.105:44764) 2024-09-16T02:06:36.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.507+0000 7f7a737fe640 1 -- 172.21.15.105:0/4186736719 learned_addr learned my addr 172.21.15.105:0/4186736719 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:06:36.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.507+0000 7f7a73fff640 1 --2- 172.21.15.105:0/4186736719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a7410c170 0x7f7a74101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:36.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.508+0000 7f7a72ffd640 1 --2- 172.21.15.105:0/4186736719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a74103660 0x7f7a74100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:36.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.508+0000 7f7a737fe640 1 -- 172.21.15.105:0/4186736719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a7410c170 msgr2=0x7f7a74101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:36.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.508+0000 7f7a737fe640 1 --2- 172.21.15.105:0/4186736719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a7410c170 0x7f7a74101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:36.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.508+0000 7f7a737fe640 1 -- 172.21.15.105:0/4186736719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a74103660 msgr2=0x7f7a74100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:36.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.508+0000 7f7a737fe640 1 --2- 172.21.15.105:0/4186736719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a74103660 0x7f7a74100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:36.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.508+0000 7f7a737fe640 1 -- 172.21.15.105:0/4186736719 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a741a6890 con 0x7f7a74102d40 2024-09-16T02:06:36.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.508+0000 7f7a737fe640 1 --2- 172.21.15.105:0/4186736719 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a74102d40 0x7f7a741009a0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f7a6400cce0 tx=0x7f7a6400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:36.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.508+0000 7f7a70ff9640 1 -- 172.21.15.105:0/4186736719 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a64004b20 con 0x7f7a74102d40 2024-09-16T02:06:36.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.508+0000 7f7a79d6e640 1 -- 172.21.15.105:0/4186736719 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a741a6b80 con 0x7f7a74102d40 2024-09-16T02:06:36.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.508+0000 7f7a70ff9640 1 -- 172.21.15.105:0/4186736719 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a64004cc0 con 0x7f7a74102d40 2024-09-16T02:06:36.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.508+0000 7f7a70ff9640 1 -- 172.21.15.105:0/4186736719 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6400f6b0 con 0x7f7a74102d40 2024-09-16T02:06:36.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.508+0000 7f7a79d6e640 1 -- 172.21.15.105:0/4186736719 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a741a70e0 con 0x7f7a74102d40 2024-09-16T02:06:36.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.509+0000 7f7a79d6e640 1 -- 172.21.15.105:0/4186736719 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a74103120 con 0x7f7a74102d40 2024-09-16T02:06:36.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.510+0000 7f7a70ff9640 1 -- 172.21.15.105:0/4186736719 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f7a6400f850 con 0x7f7a74102d40 2024-09-16T02:06:36.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.511+0000 7f7a70ff9640 1 --2- 172.21.15.105:0/4186736719 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7a5c0a32c0 0x7f7a5c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:36.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.511+0000 7f7a72ffd640 1 --2- 172.21.15.105:0/4186736719 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7a5c0a32c0 0x7f7a5c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:36.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.511+0000 7f7a70ff9640 1 -- 172.21.15.105:0/4186736719 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8765+0+0 (secure 0 0 0) 0x7f7a640cfb80 con 0x7f7a74102d40 2024-09-16T02:06:36.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.511+0000 7f7a72ffd640 1 --2- 172.21.15.105:0/4186736719 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7a5c0a32c0 0x7f7a5c0a5780 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7a68031440 tx=0x7f7a680095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:36.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.515+0000 7f7a70ff9640 1 -- 172.21.15.105:0/4186736719 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7a64014030 con 0x7f7a74102d40 2024-09-16T02:06:36.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.684+0000 7f7a79d6e640 1 -- 172.21.15.105:0/4186736719 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7a74106c50 con 0x7f7a74102d40 2024-09-16T02:06:36.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.934+0000 7f7a70ff9640 1 -- 172.21.15.105:0/4186736719 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v85) ==== 181+0+0 (secure 0 0 0) 0x7f7a6409c690 con 0x7f7a74102d40 2024-09-16T02:06:36.937 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_metadata' removed 2024-09-16T02:06:36.937 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:36 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/4186736719' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T02:06:36.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.937+0000 7f7a79d6e640 1 -- 172.21.15.105:0/4186736719 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7a5c0a32c0 msgr2=0x7f7a5c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:36.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.937+0000 7f7a79d6e640 1 --2- 172.21.15.105:0/4186736719 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7a5c0a32c0 0x7f7a5c0a5780 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7a68031440 tx=0x7f7a680095f0 comp rx=0 tx=0).stop 2024-09-16T02:06:36.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.937+0000 7f7a79d6e640 1 -- 172.21.15.105:0/4186736719 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a74102d40 msgr2=0x7f7a741009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:36.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.937+0000 7f7a79d6e640 1 --2- 172.21.15.105:0/4186736719 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a74102d40 0x7f7a741009a0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f7a6400cce0 tx=0x7f7a6400a620 comp rx=0 tx=0).stop 2024-09-16T02:06:36.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.937+0000 7f7a79d6e640 1 -- 172.21.15.105:0/4186736719 shutdown_connections 2024-09-16T02:06:36.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.937+0000 7f7a79d6e640 1 --2- 172.21.15.105:0/4186736719 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f7a5c0a32c0 0x7f7a5c0a5780 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:36.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.937+0000 7f7a79d6e640 1 --2- 172.21.15.105:0/4186736719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a7410c170 0x7f7a74101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:36.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.937+0000 7f7a79d6e640 1 --2- 172.21.15.105:0/4186736719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a74103660 0x7f7a74100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:36.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.937+0000 7f7a79d6e640 1 --2- 172.21.15.105:0/4186736719 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a74102d40 0x7f7a741009a0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:36.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.937+0000 7f7a79d6e640 1 -- 172.21.15.105:0/4186736719 >> 172.21.15.105:0/4186736719 conn(0x7f7a740fc7a0 msgr2=0x7f7a740fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:36.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.938+0000 7f7a79d6e640 1 -- 172.21.15.105:0/4186736719 shutdown_connections 2024-09-16T02:06:36.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:36.938+0000 7f7a79d6e640 1 -- 172.21.15.105:0/4186736719 wait complete. 2024-09-16T02:06:37.342 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:36 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/4186736719' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T02:06:37.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:36 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/4186736719' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T02:06:37.769 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T02:06:38.018 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:06:38.044 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:37 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/4186736719' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T02:06:38.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:37 smithi105 ceph-mon[31197]: osdmap e85: 12 total, 12 up, 12 in 2024-09-16T02:06:38.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:37 smithi105 ceph-mon[31197]: pgmap v464: 65 pgs: 65 active+clean; 577 KiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:06:38.342 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:37 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/4186736719' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T02:06:38.342 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:37 smithi190 ceph-mon[34762]: osdmap e85: 12 total, 12 up, 12 in 2024-09-16T02:06:38.342 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:37 smithi190 ceph-mon[34762]: pgmap v464: 65 pgs: 65 active+clean; 577 KiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:06:38.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:37 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/4186736719' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T02:06:38.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:37 smithi137 ceph-mon[34099]: osdmap e85: 12 total, 12 up, 12 in 2024-09-16T02:06:38.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:37 smithi137 ceph-mon[34099]: pgmap v464: 65 pgs: 65 active+clean; 577 KiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:06:38.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.516+0000 7f5dd3c1d640 1 -- 172.21.15.105:0/160743313 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dcc1008c0 msgr2=0x7f5dcc100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:38.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.516+0000 7f5dd3c1d640 1 --2- 172.21.15.105:0/160743313 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dcc1008c0 0x7f5dcc100ca0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8009900 tx=0x7f5dc802a410 comp rx=0 tx=0).stop 2024-09-16T02:06:38.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.516+0000 7f5dd3c1d640 1 -- 172.21.15.105:0/160743313 shutdown_connections 2024-09-16T02:06:38.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.516+0000 7f5dd3c1d640 1 --2- 172.21.15.105:0/160743313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5dcc105ec0 0x7f5dcc106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:38.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.516+0000 7f5dd3c1d640 1 --2- 172.21.15.105:0/160743313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5dcc1011e0 0x7f5dcc105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:38.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.516+0000 7f5dd3c1d640 1 --2- 172.21.15.105:0/160743313 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dcc1008c0 0x7f5dcc100ca0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:38.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.516+0000 7f5dd3c1d640 1 -- 172.21.15.105:0/160743313 >> 172.21.15.105:0/160743313 conn(0x7f5dcc0fc7d0 msgr2=0x7f5dcc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:38.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.516+0000 7f5dd3c1d640 1 -- 172.21.15.105:0/160743313 shutdown_connections 2024-09-16T02:06:38.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.516+0000 7f5dd3c1d640 1 -- 172.21.15.105:0/160743313 wait complete. 2024-09-16T02:06:38.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.517+0000 7f5dd3c1d640 1 Processor -- start 2024-09-16T02:06:38.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.517+0000 7f5dd3c1d640 1 -- start start 2024-09-16T02:06:38.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.517+0000 7f5dd3c1d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dcc1008c0 0x7f5dcc1a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:38.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.517+0000 7f5dd3c1d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5dcc1011e0 0x7f5dcc1a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:38.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.517+0000 7f5dd3c1d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5dcc105ec0 0x7f5dcc19e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:38.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.517+0000 7f5dd3c1d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc118540 con 0x7f5dcc1008c0 2024-09-16T02:06:38.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.517+0000 7f5dd3c1d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc1183c0 con 0x7f5dcc105ec0 2024-09-16T02:06:38.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.517+0000 7f5dd3c1d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc1186c0 con 0x7f5dcc1011e0 2024-09-16T02:06:38.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.517+0000 7f5dd19b9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dcc1008c0 0x7f5dcc1a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:38.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.517+0000 7f5dd19b9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dcc1008c0 0x7f5dcc1a4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46620/0 (socket says 172.21.15.105:46620) 2024-09-16T02:06:38.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.517+0000 7f5dd19b9640 1 -- 172.21.15.105:0/116944992 learned_addr learned my addr 172.21.15.105:0/116944992 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-16T02:06:38.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.517+0000 7f5dd11b8640 1 --2- 172.21.15.105:0/116944992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5dcc1011e0 0x7f5dcc1a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:38.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.517+0000 7f5dd21ba640 1 --2- 172.21.15.105:0/116944992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5dcc105ec0 0x7f5dcc19e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:38.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.517+0000 7f5dd19b9640 1 -- 172.21.15.105:0/116944992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5dcc1011e0 msgr2=0x7f5dcc1a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:38.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.517+0000 7f5dd19b9640 1 --2- 172.21.15.105:0/116944992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5dcc1011e0 0x7f5dcc1a4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:38.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.517+0000 7f5dd19b9640 1 -- 172.21.15.105:0/116944992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5dcc105ec0 msgr2=0x7f5dcc19e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:38.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.517+0000 7f5dd19b9640 1 --2- 172.21.15.105:0/116944992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5dcc105ec0 0x7f5dcc19e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:38.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.517+0000 7f5dd19b9640 1 -- 172.21.15.105:0/116944992 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dcc19f020 con 0x7f5dcc1008c0 2024-09-16T02:06:38.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.518+0000 7f5dd19b9640 1 --2- 172.21.15.105:0/116944992 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dcc1008c0 0x7f5dcc1a4730 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8006010 tx=0x7f5dc80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:38.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.518+0000 7f5dc2ffd640 1 -- 172.21.15.105:0/116944992 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc803f070 con 0x7f5dcc1008c0 2024-09-16T02:06:38.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.518+0000 7f5dc2ffd640 1 -- 172.21.15.105:0/116944992 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dc8003aa0 con 0x7f5dcc1008c0 2024-09-16T02:06:38.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.518+0000 7f5dd3c1d640 1 -- 172.21.15.105:0/116944992 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dcc19f2b0 con 0x7f5dcc1008c0 2024-09-16T02:06:38.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.518+0000 7f5dc2ffd640 1 -- 172.21.15.105:0/116944992 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc8035030 con 0x7f5dcc1008c0 2024-09-16T02:06:38.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.518+0000 7f5dd3c1d640 1 -- 172.21.15.105:0/116944992 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dcc1ab600 con 0x7f5dcc1008c0 2024-09-16T02:06:38.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.519+0000 7f5dd3c1d640 1 -- 172.21.15.105:0/116944992 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dcc06a850 con 0x7f5dcc1008c0 2024-09-16T02:06:38.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.520+0000 7f5dc2ffd640 1 -- 172.21.15.105:0/116944992 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f5dc8042070 con 0x7f5dcc1008c0 2024-09-16T02:06:38.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.521+0000 7f5dc2ffd640 1 --2- 172.21.15.105:0/116944992 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5db80a3240 0x7f5db80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:38.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.521+0000 7f5dc2ffd640 1 -- 172.21.15.105:0/116944992 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8145+0+0 (secure 0 0 0) 0x7f5dc80f0410 con 0x7f5dcc1008c0 2024-09-16T02:06:38.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.521+0000 7f5dd11b8640 1 --2- 172.21.15.105:0/116944992 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5db80a3240 0x7f5db80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:38.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.521+0000 7f5dd11b8640 1 --2- 172.21.15.105:0/116944992 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5db80a3240 0x7f5db80a5700 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc100030 tx=0x7f5dbc00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:38.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.524+0000 7f5dc2ffd640 1 -- 172.21.15.105:0/116944992 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5dc80bd1d0 con 0x7f5dcc1008c0 2024-09-16T02:06:38.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.699+0000 7f5dd3c1d640 1 -- 172.21.15.105:0/116944992 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5dcc105980 con 0x7f5dcc1008c0 2024-09-16T02:06:38.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.948+0000 7f5dc2ffd640 1 -- 172.21.15.105:0/116944992 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v86) ==== 169+0+0 (secure 0 0 0) 0x7f5dcc105980 con 0x7f5dcc1008c0 2024-09-16T02:06:38.950 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data' removed 2024-09-16T02:06:38.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.951+0000 7f5dd3c1d640 1 -- 172.21.15.105:0/116944992 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5db80a3240 msgr2=0x7f5db80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:38.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.951+0000 7f5dd3c1d640 1 --2- 172.21.15.105:0/116944992 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5db80a3240 0x7f5db80a5700 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc100030 tx=0x7f5dbc00a2d0 comp rx=0 tx=0).stop 2024-09-16T02:06:38.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.951+0000 7f5dd3c1d640 1 -- 172.21.15.105:0/116944992 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dcc1008c0 msgr2=0x7f5dcc1a4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:38.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.951+0000 7f5dd3c1d640 1 --2- 172.21.15.105:0/116944992 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dcc1008c0 0x7f5dcc1a4730 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8006010 tx=0x7f5dc80042f0 comp rx=0 tx=0).stop 2024-09-16T02:06:38.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.951+0000 7f5dd3c1d640 1 -- 172.21.15.105:0/116944992 shutdown_connections 2024-09-16T02:06:38.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.951+0000 7f5dd3c1d640 1 --2- 172.21.15.105:0/116944992 >> [v2:172.21.15.105:6800/275626334,v1:172.21.15.105:6801/275626334] conn(0x7f5db80a3240 0x7f5db80a5700 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:38.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.951+0000 7f5dd3c1d640 1 --2- 172.21.15.105:0/116944992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5dcc1011e0 0x7f5dcc1a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:38.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.951+0000 7f5dd3c1d640 1 --2- 172.21.15.105:0/116944992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5dcc105ec0 0x7f5dcc19e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:38.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.951+0000 7f5dd3c1d640 1 --2- 172.21.15.105:0/116944992 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dcc1008c0 0x7f5dcc1a4730 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:38.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.951+0000 7f5dd3c1d640 1 -- 172.21.15.105:0/116944992 >> 172.21.15.105:0/116944992 conn(0x7f5dcc0fc7d0 msgr2=0x7f5dcc103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:38.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.951+0000 7f5dd3c1d640 1 -- 172.21.15.105:0/116944992 shutdown_connections 2024-09-16T02:06:38.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-16T02:06:38.952+0000 7f5dd3c1d640 1 -- 172.21.15.105:0/116944992 wait complete. 2024-09-16T02:06:39.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:38 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/116944992' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T02:06:39.351 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:38 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/116944992' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T02:06:39.421 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:38 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/116944992' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T02:06:39.692 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T02:06:39.725 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T02:06:39.760 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T02:06:39.791 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-16T02:06:39.792 DEBUG:teuthology.orchestra.run.smithi105:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T02:06:39.807 DEBUG:teuthology.orchestra.run.smithi137:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T02:06:39.825 DEBUG:teuthology.orchestra.run.smithi190:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T02:06:39.849 INFO:tasks.cephadm:Stopping all daemons... 2024-09-16T02:06:39.849 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-16T02:06:39.849 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mon.a 2024-09-16T02:06:39.955 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:39 smithi105 systemd[1]: Stopping Ceph mon.a for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:06:40.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:39 smithi190 ceph-mon[34762]: from='client.? 172.21.15.105:0/116944992' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T02:06:40.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:39 smithi190 ceph-mon[34762]: osdmap e86: 12 total, 12 up, 12 in 2024-09-16T02:06:40.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:39 smithi190 ceph-mon[34762]: pgmap v466: 1 pgs: 1 active+clean; 577 KiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:06:40.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:39 smithi137 ceph-mon[34099]: from='client.? 172.21.15.105:0/116944992' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T02:06:40.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:39 smithi137 ceph-mon[34099]: osdmap e86: 12 total, 12 up, 12 in 2024-09-16T02:06:40.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:39 smithi137 ceph-mon[34099]: pgmap v466: 1 pgs: 1 active+clean; 577 KiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:06:40.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:39 smithi105 ceph-mon[31197]: from='client.? 172.21.15.105:0/116944992' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T02:06:40.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:39 smithi105 ceph-mon[31197]: osdmap e86: 12 total, 12 up, 12 in 2024-09-16T02:06:40.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:39 smithi105 ceph-mon[31197]: pgmap v466: 1 pgs: 1 active+clean; 577 KiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:06:40.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:40 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mon-a[31193]: 2024-09-16T02:06:40.158+0000 7f9eb3746640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:06:40.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:40 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mon-a[31193]: 2024-09-16T02:06:40.158+0000 7f9eb3746640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-16T02:06:40.754 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:40 smithi105 podman[93067]: 2024-09-16 02:06:40.404764814 +0000 UTC m=+0.373547303 container died 6554b6c6100439457da03be64f7dfc7f6dad18f10370ba1b0dc3f80b10cc1cf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T02:06:41.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:40 smithi105 podman[93067]: 2024-09-16 02:06:40.961144464 +0000 UTC m=+0.929927006 container cleanup 6554b6c6100439457da03be64f7dfc7f6dad18f10370ba1b0dc3f80b10cc1cf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-16T02:06:41.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Sep 16 02:06:40 smithi105 bash[93067]: ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mon-a 2024-09-16T02:06:41.457 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mon.a.service' 2024-09-16T02:06:41.501 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:06:41.502 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-16T02:06:41.502 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-16T02:06:41.502 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mon.b 2024-09-16T02:06:41.819 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:41 smithi137 systemd[1]: Stopping Ceph mon.b for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:06:42.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:41 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mon-b[34095]: 2024-09-16T02:06:41.818+0000 7f22ab272640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:06:42.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:41 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mon-b[34095]: 2024-09-16T02:06:41.818+0000 7f22ab272640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-16T02:06:42.171 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:42 smithi137 podman[67064]: 2024-09-16 02:06:42.063442892 +0000 UTC m=+0.358677201 container died 9b210384a9410a43431c2b1b427350646ed78e87443667a39ccf077b93cde29a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-16T02:06:42.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:42 smithi137 podman[67064]: 2024-09-16 02:06:42.460007906 +0000 UTC m=+0.755242205 container cleanup 9b210384a9410a43431c2b1b427350646ed78e87443667a39ccf077b93cde29a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_CLEAN=True, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-16T02:06:42.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Sep 16 02:06:42 smithi137 bash[67064]: ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mon-b 2024-09-16T02:06:42.888 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mon.b.service' 2024-09-16T02:06:42.952 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:06:42.952 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-16T02:06:42.952 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-16T02:06:42.952 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mon.c 2024-09-16T02:06:43.342 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:42 smithi190 systemd[1]: Stopping Ceph mon.c for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:06:43.697 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:43 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mon-c[34739]: 2024-09-16T02:06:43.430+0000 7f3cb798e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:06:43.697 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:43 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mon-c[34739]: 2024-09-16T02:06:43.430+0000 7f3cb798e640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-16T02:06:44.008 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:43 smithi190 podman[67571]: 2024-09-16 02:06:43.696820233 +0000 UTC m=+0.548615738 container died ef4296c19219a3f982cbe0cbbf1726bea47db08be9cba32b98014a518474f61a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mon-c, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T02:06:44.325 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:44 smithi190 podman[67571]: 2024-09-16 02:06:44.106218727 +0000 UTC m=+0.958014233 container cleanup ef4296c19219a3f982cbe0cbbf1726bea47db08be9cba32b98014a518474f61a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mon-c, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T02:06:44.325 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:44 smithi190 bash[67571]: ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mon-c 2024-09-16T02:06:44.541 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mon.c.service' 2024-09-16T02:06:44.565 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:44 smithi190 podman[67584]: 2024-09-16 02:06:44.324457743 +0000 UTC m=+0.630008744 container remove ef4296c19219a3f982cbe0cbbf1726bea47db08be9cba32b98014a518474f61a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mon-c, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-16T02:06:44.566 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:44 smithi190 systemd[1]: ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mon.c.service: Deactivated successfully. 2024-09-16T02:06:44.566 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:44 smithi190 systemd[1]: Stopped Ceph mon.c for 423b7a2a-73ce-11ef-bceb-c7b262605968. 2024-09-16T02:06:44.566 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 02:06:44 smithi190 systemd[1]: ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mon.c.service: Consumed 8.495s CPU time. 2024-09-16T02:06:44.585 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:06:44.585 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-16T02:06:44.585 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-16T02:06:44.585 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mgr.x 2024-09-16T02:06:44.901 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 02:06:44 smithi105 systemd[1]: Stopping Ceph mgr.x for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:06:45.285 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 02:06:44 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x[31424]: 2024-09-16T02:06:44.898+0000 7f227db81640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:06:45.285 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 02:06:44 smithi105 podman[93189]: 2024-09-16 02:06:44.930104861 +0000 UTC m=+0.149817774 container died 774d5ede24dcf4af137ba7031885ecbbab9a6cd514d9b570d8328cae7ecac455 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x, GIT_CLEAN=True, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-16T02:06:45.544 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 02:06:45 smithi105 podman[93189]: 2024-09-16 02:06:45.49288335 +0000 UTC m=+0.712596308 container cleanup 774d5ede24dcf4af137ba7031885ecbbab9a6cd514d9b570d8328cae7ecac455 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-16T02:06:45.545 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Sep 16 02:06:45 smithi105 bash[93189]: ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-x 2024-09-16T02:06:45.974 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-16T02:06:46.011 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:06:46.011 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-16T02:06:46.011 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-16T02:06:46.012 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mgr.y 2024-09-16T02:06:46.345 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 16 02:06:46 smithi137 systemd[1]: Stopping Ceph mgr.y for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:06:46.634 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 16 02:06:46 smithi137 podman[67188]: 2024-09-16 02:06:46.344003774 +0000 UTC m=+0.135869596 container died 19c45ee05332cec61ec7f1d7926fcff04e954f8d922238a5ea0daa3345cb20b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-y, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-16T02:06:46.921 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 16 02:06:46 smithi137 podman[67188]: 2024-09-16 02:06:46.732407131 +0000 UTC m=+0.524272963 container cleanup 19c45ee05332cec61ec7f1d7926fcff04e954f8d922238a5ea0daa3345cb20b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-y, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T02:06:46.921 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Sep 16 02:06:46 smithi137 bash[67188]: ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-y 2024-09-16T02:06:47.155 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-16T02:06:47.189 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:06:47.190 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-16T02:06:47.190 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-16T02:06:47.190 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mgr.z 2024-09-16T02:06:47.532 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 02:06:47 smithi190 systemd[1]: Stopping Ceph mgr.z for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:06:47.842 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 02:06:47 smithi190 podman[67693]: 2024-09-16 02:06:47.531962253 +0000 UTC m=+0.149052957 container died ccc24b8c20c82dd74f477c5c0d212485719db2244c0b44208fbc39cccb1f354f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-16T02:06:48.197 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 02:06:47 smithi190 podman[67693]: 2024-09-16 02:06:47.953616149 +0000 UTC m=+0.570706852 container cleanup ccc24b8c20c82dd74f477c5c0d212485719db2244c0b44208fbc39cccb1f354f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, ceph=True) 2024-09-16T02:06:48.197 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 02:06:47 smithi190 bash[67693]: ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-z 2024-09-16T02:06:48.428 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mgr.z.service' 2024-09-16T02:06:48.449 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 02:06:48 smithi190 podman[67705]: 2024-09-16 02:06:48.196812609 +0000 UTC m=+0.660439327 container remove ccc24b8c20c82dd74f477c5c0d212485719db2244c0b44208fbc39cccb1f354f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mgr-z, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux ) 2024-09-16T02:06:48.449 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 02:06:48 smithi190 systemd[1]: ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-09-16T02:06:48.449 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 02:06:48 smithi190 systemd[1]: ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mgr.z.service: Failed with result 'exit-code'. 2024-09-16T02:06:48.449 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 02:06:48 smithi190 systemd[1]: Stopped Ceph mgr.z for 423b7a2a-73ce-11ef-bceb-c7b262605968. 2024-09-16T02:06:48.449 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Sep 16 02:06:48 smithi190 systemd[1]: ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mgr.z.service: Consumed 6.245s CPU time. 2024-09-16T02:06:48.462 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:06:48.462 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-16T02:06:48.462 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-16T02:06:48.462 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@osd.0 2024-09-16T02:06:48.782 INFO:journalctl@ceph.osd.0.smithi105.stdout:Sep 16 02:06:48 smithi105 systemd[1]: Stopping Ceph osd.0 for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:06:49.044 INFO:journalctl@ceph.osd.0.smithi105.stdout:Sep 16 02:06:48 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-0[44633]: 2024-09-16T02:06:48.780+0000 7f53a9703640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:06:49.045 INFO:journalctl@ceph.osd.0.smithi105.stdout:Sep 16 02:06:48 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-0[44633]: 2024-09-16T02:06:48.780+0000 7f53a9703640 -1 osd.0 86 *** Got signal Terminated *** 2024-09-16T02:06:49.045 INFO:journalctl@ceph.osd.0.smithi105.stdout:Sep 16 02:06:48 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-0[44633]: 2024-09-16T02:06:48.780+0000 7f53a9703640 -1 osd.0 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T02:06:54.491 INFO:journalctl@ceph.osd.0.smithi105.stdout:Sep 16 02:06:54 smithi105 podman[93308]: 2024-09-16 02:06:54.167247551 +0000 UTC m=+5.514145882 container died 4157ffd1183d7148129b3f2855173cb3d2392716a23c55646bb015628700f5cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-16T02:06:54.795 INFO:journalctl@ceph.osd.0.smithi105.stdout:Sep 16 02:06:54 smithi105 podman[93308]: 2024-09-16 02:06:54.690567079 +0000 UTC m=+6.037465413 container cleanup 4157ffd1183d7148129b3f2855173cb3d2392716a23c55646bb015628700f5cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-16T02:06:54.795 INFO:journalctl@ceph.osd.0.smithi105.stdout:Sep 16 02:06:54 smithi105 bash[93308]: ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-0 2024-09-16T02:06:55.295 INFO:journalctl@ceph.osd.0.smithi105.stdout:Sep 16 02:06:54 smithi105 podman[93323]: 2024-09-16 02:06:54.95907466 +0000 UTC m=+0.791550786 container remove 4157ffd1183d7148129b3f2855173cb3d2392716a23c55646bb015628700f5cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-16T02:06:55.739 INFO:journalctl@ceph.osd.0.smithi105.stdout:Sep 16 02:06:55 smithi105 podman[93417]: 2024-09-16 02:06:55.354071275 +0000 UTC m=+0.015769756 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:06:55.739 INFO:journalctl@ceph.osd.0.smithi105.stdout:Sep 16 02:06:55 smithi105 podman[93417]: 2024-09-16 02:06:55.493502311 +0000 UTC m=+0.155200792 container create ed4409820ed3d195fb8d7cc3fb909509f6d17154b375aa90ab57ae2e067bb624 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-0-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-16T02:06:56.045 INFO:journalctl@ceph.osd.0.smithi105.stdout:Sep 16 02:06:55 smithi105 podman[93417]: 2024-09-16 02:06:55.737866627 +0000 UTC m=+0.399565120 container init ed4409820ed3d195fb8d7cc3fb909509f6d17154b375aa90ab57ae2e067bb624 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-8921c8e) 2024-09-16T02:06:56.045 INFO:journalctl@ceph.osd.0.smithi105.stdout:Sep 16 02:06:55 smithi105 podman[93417]: 2024-09-16 02:06:55.741506248 +0000 UTC m=+0.403204729 container start ed4409820ed3d195fb8d7cc3fb909509f6d17154b375aa90ab57ae2e067bb624 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-16T02:06:56.045 INFO:journalctl@ceph.osd.0.smithi105.stdout:Sep 16 02:06:55 smithi105 podman[93417]: 2024-09-16 02:06:55.846279987 +0000 UTC m=+0.507978467 container attach ed4409820ed3d195fb8d7cc3fb909509f6d17154b375aa90ab57ae2e067bb624 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-0-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T02:06:56.045 INFO:journalctl@ceph.osd.0.smithi105.stdout:Sep 16 02:06:55 smithi105 podman[93417]: 2024-09-16 02:06:55.936750821 +0000 UTC m=+0.598449302 container died ed4409820ed3d195fb8d7cc3fb909509f6d17154b375aa90ab57ae2e067bb624 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T02:06:56.810 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@osd.0.service' 2024-09-16T02:06:56.854 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:06:56.854 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-16T02:06:56.854 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-16T02:06:56.854 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@osd.1 2024-09-16T02:06:57.163 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 16 02:06:56 smithi137 systemd[1]: Stopping Ceph osd.1 for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:06:57.421 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 16 02:06:57 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-1[38549]: 2024-09-16T02:06:57.160+0000 7f1362876640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:06:57.421 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 16 02:06:57 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-1[38549]: 2024-09-16T02:06:57.160+0000 7f1362876640 -1 osd.1 86 *** Got signal Terminated *** 2024-09-16T02:06:57.421 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 16 02:06:57 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-1[38549]: 2024-09-16T02:06:57.160+0000 7f1362876640 -1 osd.1 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T02:07:02.833 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 16 02:07:02 smithi137 podman[67324]: 2024-09-16 02:07:02.521603669 +0000 UTC m=+5.485754635 container died 557105c5c9c5abe08d7621fa9fe40cdb5cf417235456d153ff31bf7ed99e73ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-1, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T02:07:03.171 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 16 02:07:02 smithi137 podman[67324]: 2024-09-16 02:07:02.922507482 +0000 UTC m=+5.886658459 container cleanup 557105c5c9c5abe08d7621fa9fe40cdb5cf417235456d153ff31bf7ed99e73ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-1, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731) 2024-09-16T02:07:03.171 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 16 02:07:02 smithi137 bash[67324]: ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-1 2024-09-16T02:07:03.533 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 16 02:07:03 smithi137 podman[67340]: 2024-09-16 02:07:03.199073327 +0000 UTC m=+0.668668344 container remove 557105c5c9c5abe08d7621fa9fe40cdb5cf417235456d153ff31bf7ed99e73ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-1, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T02:07:03.811 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 16 02:07:03 smithi137 podman[67431]: 2024-09-16 02:07:03.434907533 +0000 UTC m=+0.015871918 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:07:03.812 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 16 02:07:03 smithi137 podman[67431]: 2024-09-16 02:07:03.558488452 +0000 UTC m=+0.139452879 container create 2e83e857ef22a8dfab81f10ab75907d9ededb0791e4918679208d87e8a022af1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-16T02:07:03.812 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 16 02:07:03 smithi137 podman[67431]: 2024-09-16 02:07:03.811118999 +0000 UTC m=+0.392083382 container init 2e83e857ef22a8dfab81f10ab75907d9ededb0791e4918679208d87e8a022af1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-16T02:07:04.171 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 16 02:07:03 smithi137 podman[67431]: 2024-09-16 02:07:03.814731474 +0000 UTC m=+0.395695859 container start 2e83e857ef22a8dfab81f10ab75907d9ededb0791e4918679208d87e8a022af1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-1-deactivate, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-16T02:07:04.171 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 16 02:07:03 smithi137 podman[67431]: 2024-09-16 02:07:03.902901918 +0000 UTC m=+0.483866309 container attach 2e83e857ef22a8dfab81f10ab75907d9ededb0791e4918679208d87e8a022af1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-16T02:07:04.171 INFO:journalctl@ceph.osd.1.smithi137.stdout:Sep 16 02:07:04 smithi137 podman[67431]: 2024-09-16 02:07:04.001306958 +0000 UTC m=+0.582271356 container died 2e83e857ef22a8dfab81f10ab75907d9ededb0791e4918679208d87e8a022af1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-16T02:07:04.681 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@osd.1.service' 2024-09-16T02:07:04.721 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:07:04.722 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-16T02:07:04.722 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-16T02:07:04.722 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@osd.2 2024-09-16T02:07:05.042 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 02:07:04 smithi190 systemd[1]: Stopping Ceph osd.2 for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:07:05.342 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 02:07:05 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-2[39695]: 2024-09-16T02:07:05.040+0000 7fba19e1e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:07:05.343 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 02:07:05 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-2[39695]: 2024-09-16T02:07:05.040+0000 7fba19e1e640 -1 osd.2 86 *** Got signal Terminated *** 2024-09-16T02:07:05.343 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 02:07:05 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-2[39695]: 2024-09-16T02:07:05.040+0000 7fba19e1e640 -1 osd.2 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T02:07:10.768 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 02:07:10 smithi190 podman[67830]: 2024-09-16 02:07:10.46101424 +0000 UTC m=+5.540406648 container died b1e8df4b4ee3acc84cf061abd24a147852016e5cf6795eda174a2adef5b5bca2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-16T02:07:11.092 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 02:07:10 smithi190 podman[67830]: 2024-09-16 02:07:10.84941961 +0000 UTC m=+5.928812017 container cleanup b1e8df4b4ee3acc84cf061abd24a147852016e5cf6795eda174a2adef5b5bca2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-16T02:07:11.093 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 02:07:10 smithi190 bash[67830]: ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-2 2024-09-16T02:07:11.470 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 02:07:11 smithi190 podman[67845]: 2024-09-16 02:07:11.108951882 +0000 UTC m=+0.643764326 container remove b1e8df4b4ee3acc84cf061abd24a147852016e5cf6795eda174a2adef5b5bca2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-2, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-16T02:07:11.763 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 02:07:11 smithi190 podman[67935]: 2024-09-16 02:07:11.37234345 +0000 UTC m=+0.015830344 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:07:11.763 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 02:07:11 smithi190 podman[67935]: 2024-09-16 02:07:11.493138321 +0000 UTC m=+0.136625200 container create 43277b841b0dd8ff53c2b16766b6ee21e1cbd6054ecc70701fa2671fcc96219a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-16T02:07:11.763 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 02:07:11 smithi190 podman[67935]: 2024-09-16 02:07:11.762661508 +0000 UTC m=+0.406148395 container init 43277b841b0dd8ff53c2b16766b6ee21e1cbd6054ecc70701fa2671fcc96219a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-16T02:07:12.093 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 02:07:11 smithi190 podman[67935]: 2024-09-16 02:07:11.76615028 +0000 UTC m=+0.409637157 container start 43277b841b0dd8ff53c2b16766b6ee21e1cbd6054ecc70701fa2671fcc96219a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-16T02:07:12.093 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 02:07:11 smithi190 podman[67935]: 2024-09-16 02:07:11.87085595 +0000 UTC m=+0.514342836 container attach 43277b841b0dd8ff53c2b16766b6ee21e1cbd6054ecc70701fa2671fcc96219a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-2-deactivate, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0) 2024-09-16T02:07:12.093 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 02:07:11 smithi190 podman[67935]: 2024-09-16 02:07:11.953981184 +0000 UTC m=+0.597468061 container died 43277b841b0dd8ff53c2b16766b6ee21e1cbd6054ecc70701fa2671fcc96219a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-2-deactivate, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-16T02:07:12.605 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@osd.2.service' 2024-09-16T02:07:12.644 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:07:12.645 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-16T02:07:12.645 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-16T02:07:12.645 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@osd.3 2024-09-16T02:07:12.964 INFO:journalctl@ceph.osd.3.smithi105.stdout:Sep 16 02:07:12 smithi105 systemd[1]: Stopping Ceph osd.3 for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:07:13.295 INFO:journalctl@ceph.osd.3.smithi105.stdout:Sep 16 02:07:12 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-3[51503]: 2024-09-16T02:07:12.962+0000 7f315ed21640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:07:13.295 INFO:journalctl@ceph.osd.3.smithi105.stdout:Sep 16 02:07:12 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-3[51503]: 2024-09-16T02:07:12.962+0000 7f315ed21640 -1 osd.3 86 *** Got signal Terminated *** 2024-09-16T02:07:13.295 INFO:journalctl@ceph.osd.3.smithi105.stdout:Sep 16 02:07:12 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-3[51503]: 2024-09-16T02:07:12.962+0000 7f315ed21640 -1 osd.3 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T02:07:15.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:15 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:15.034+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:16.294 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:15 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:15.992+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:16.592 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:16 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:16.207+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:16.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:16 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:16.402+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:16.794 INFO:journalctl@ceph.osd.3.smithi105.stdout:Sep 16 02:07:16 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-3[51503]: 2024-09-16T02:07:16.493+0000 7f315732f640 -1 osd.3 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.513195+0000 front 2024-09-16T02:06:50.513133+0000 (oldest deadline 2024-09-16T02:07:16.413309+0000) 2024-09-16T02:07:17.294 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:16 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:16.961+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:17.550 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:17 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:17.181+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:17.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:17 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:17.388+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:17.794 INFO:journalctl@ceph.osd.3.smithi105.stdout:Sep 16 02:07:17 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-3[51503]: 2024-09-16T02:07:17.451+0000 7f315732f640 -1 osd.3 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.513195+0000 front 2024-09-16T02:06:50.513133+0000 (oldest deadline 2024-09-16T02:07:16.413309+0000) 2024-09-16T02:07:17.842 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:17 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:17.548+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:18.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:17 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:17.959+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:18.531 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:18 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:18.152+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:18.724 INFO:journalctl@ceph.osd.3.smithi105.stdout:Sep 16 02:07:18 smithi105 podman[93533]: 2024-09-16 02:07:18.458181897 +0000 UTC m=+5.617478000 container died d961aa8fca889192412f6c5a6d77d53204ddd3543d23922faf127799412a17f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-3, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-16T02:07:18.842 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:18 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:18.530+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:18.843 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:18 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5[46111]: 2024-09-16T02:07:18.632+0000 7f074d88c640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:52.547933+0000 front 2024-09-16T02:06:52.548048+0000 (oldest deadline 2024-09-16T02:07:17.847923+0000) 2024-09-16T02:07:18.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:18 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:18.424+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:19.022 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:18 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:18.956+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:19.022 INFO:journalctl@ceph.osd.3.smithi105.stdout:Sep 16 02:07:18 smithi105 podman[93533]: 2024-09-16 02:07:18.947026317 +0000 UTC m=+6.106322400 container cleanup d961aa8fca889192412f6c5a6d77d53204ddd3543d23922faf127799412a17f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-3, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-16T02:07:19.022 INFO:journalctl@ceph.osd.3.smithi105.stdout:Sep 16 02:07:18 smithi105 bash[93533]: ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-3 2024-09-16T02:07:19.294 INFO:journalctl@ceph.osd.3.smithi105.stdout:Sep 16 02:07:19 smithi105 podman[93547]: 2024-09-16 02:07:19.215224397 +0000 UTC m=+0.755916962 container remove d961aa8fca889192412f6c5a6d77d53204ddd3543d23922faf127799412a17f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-16T02:07:19.295 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:19 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:19.019+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:53.757260+0000 front 2024-09-16T02:06:53.757170+0000 (oldest deadline 2024-09-16T02:07:18.457255+0000) 2024-09-16T02:07:19.511 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:19 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:19.164+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:19.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:19 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:19.412+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:19.794 INFO:journalctl@ceph.osd.3.smithi105.stdout:Sep 16 02:07:19 smithi105 podman[93639]: 2024-09-16 02:07:19.425034979 +0000 UTC m=+0.015541592 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:07:19.795 INFO:journalctl@ceph.osd.3.smithi105.stdout:Sep 16 02:07:19 smithi105 podman[93639]: 2024-09-16 02:07:19.566069477 +0000 UTC m=+0.156576091 container create 2823c1e02128ee947203a9971e6712c0c765a59550b1b399e8a7917af46156ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-16T02:07:19.842 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:19 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:19.510+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:19.842 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:19 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5[46111]: 2024-09-16T02:07:19.603+0000 7f074d88c640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:52.547933+0000 front 2024-09-16T02:06:52.548048+0000 (oldest deadline 2024-09-16T02:07:17.847923+0000) 2024-09-16T02:07:20.294 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:20 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:20.016+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:53.757260+0000 front 2024-09-16T02:06:53.757170+0000 (oldest deadline 2024-09-16T02:07:18.457255+0000) 2024-09-16T02:07:20.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:19 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:19.972+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:20.295 INFO:journalctl@ceph.osd.3.smithi105.stdout:Sep 16 02:07:19 smithi105 podman[93639]: 2024-09-16 02:07:19.818789973 +0000 UTC m=+0.409296593 container init 2823c1e02128ee947203a9971e6712c0c765a59550b1b399e8a7917af46156ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-16T02:07:20.295 INFO:journalctl@ceph.osd.3.smithi105.stdout:Sep 16 02:07:19 smithi105 podman[93639]: 2024-09-16 02:07:19.822259092 +0000 UTC m=+0.412765714 container start 2823c1e02128ee947203a9971e6712c0c765a59550b1b399e8a7917af46156ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-16T02:07:20.295 INFO:journalctl@ceph.osd.3.smithi105.stdout:Sep 16 02:07:19 smithi105 podman[93639]: 2024-09-16 02:07:19.902148042 +0000 UTC m=+0.492654667 container attach 2823c1e02128ee947203a9971e6712c0c765a59550b1b399e8a7917af46156ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T02:07:20.295 INFO:journalctl@ceph.osd.3.smithi105.stdout:Sep 16 02:07:20 smithi105 podman[93639]: 2024-09-16 02:07:20.017211355 +0000 UTC m=+0.607717977 container died 2823c1e02128ee947203a9971e6712c0c765a59550b1b399e8a7917af46156ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T02:07:20.556 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:20 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:20.141+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:20.836 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@osd.3.service' 2024-09-16T02:07:20.842 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:20 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5[46111]: 2024-09-16T02:07:20.600+0000 7f074d88c640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:52.547933+0000 front 2024-09-16T02:06:52.548048+0000 (oldest deadline 2024-09-16T02:07:17.847923+0000) 2024-09-16T02:07:20.843 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:20 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:20.555+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:20.882 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:07:20.882 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-16T02:07:20.882 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-16T02:07:20.883 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@osd.4 2024-09-16T02:07:20.909 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:20 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:20.426+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:21.171 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 16 02:07:20 smithi137 systemd[1]: Stopping Ceph osd.4 for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:07:21.295 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:20 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:20.969+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:53.757260+0000 front 2024-09-16T02:06:53.757170+0000 (oldest deadline 2024-09-16T02:07:18.457255+0000) 2024-09-16T02:07:21.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:20 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:20.992+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:21.564 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:21 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:21.153+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:21.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:21 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:21.421+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:21.671 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 16 02:07:21 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-4[45404]: 2024-09-16T02:07:21.226+0000 7f7a4b74e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:07:21.671 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 16 02:07:21 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-4[45404]: 2024-09-16T02:07:21.226+0000 7f7a4b74e640 -1 osd.4 86 *** Got signal Terminated *** 2024-09-16T02:07:21.671 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 16 02:07:21 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-4[45404]: 2024-09-16T02:07:21.226+0000 7f7a4b74e640 -1 osd.4 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T02:07:21.842 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:21 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5[46111]: 2024-09-16T02:07:21.627+0000 7f074d88c640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:52.547933+0000 front 2024-09-16T02:06:52.548048+0000 (oldest deadline 2024-09-16T02:07:17.847923+0000) 2024-09-16T02:07:21.843 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:21 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:21.562+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:22.294 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:21 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:21.979+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:53.757260+0000 front 2024-09-16T02:06:53.757170+0000 (oldest deadline 2024-09-16T02:07:18.457255+0000) 2024-09-16T02:07:22.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:21 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:21.952+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:22.528 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:22 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:22.134+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:22.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:22 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:22.412+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:22.842 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:22 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:22.526+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:22.843 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:22 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5[46111]: 2024-09-16T02:07:22.652+0000 7f074d88c640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:52.547933+0000 front 2024-09-16T02:06:52.548048+0000 (oldest deadline 2024-09-16T02:07:17.847923+0000) 2024-09-16T02:07:23.295 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:23 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:23.023+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:53.757260+0000 front 2024-09-16T02:06:53.757170+0000 (oldest deadline 2024-09-16T02:07:18.457255+0000) 2024-09-16T02:07:23.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:22 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:22.939+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:23.489 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:23 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:23.091+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:23.843 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:23 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5[46111]: 2024-09-16T02:07:23.693+0000 7f074d88c640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:52.547933+0000 front 2024-09-16T02:06:52.548048+0000 (oldest deadline 2024-09-16T02:07:17.847923+0000) 2024-09-16T02:07:23.843 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:23 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:23.487+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:23.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:23 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:23.453+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:24.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:23 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:23.963+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:24.295 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:24 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:24.026+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:53.757260+0000 front 2024-09-16T02:06:53.757170+0000 (oldest deadline 2024-09-16T02:07:18.457255+0000) 2024-09-16T02:07:24.342 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:24 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:24.085+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:24.421 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 16 02:07:23 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-4[45404]: 2024-09-16T02:07:23.995+0000 7f7a47d64640 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.987615+0000 front 2024-09-16T02:07:01.987639+0000 (oldest deadline 2024-09-16T02:07:23.588303+0000) 2024-09-16T02:07:24.842 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:24 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:24.494+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:24.842 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:24 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5[46111]: 2024-09-16T02:07:24.706+0000 7f074d88c640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:52.547933+0000 front 2024-09-16T02:06:52.548048+0000 (oldest deadline 2024-09-16T02:07:17.847923+0000) 2024-09-16T02:07:24.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:24 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:24.474+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:25.294 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:24 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:24.994+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:53.757260+0000 front 2024-09-16T02:06:53.757170+0000 (oldest deadline 2024-09-16T02:07:18.457255+0000) 2024-09-16T02:07:25.295 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:24 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:24.994+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.158189+0000 front 2024-09-16T02:07:00.158266+0000 (oldest deadline 2024-09-16T02:07:24.858027+0000) 2024-09-16T02:07:25.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:24 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:24.939+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:25.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:24 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:24.939+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:07:25.342 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:25 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:25.050+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:25.421 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 16 02:07:24 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-4[45404]: 2024-09-16T02:07:24.956+0000 7f7a47d64640 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.987615+0000 front 2024-09-16T02:07:01.987639+0000 (oldest deadline 2024-09-16T02:07:23.588303+0000) 2024-09-16T02:07:25.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:25 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:25.007+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:06:59.698219+0000 front 2024-09-16T02:06:59.698240+0000 (oldest deadline 2024-09-16T02:07:24.398448+0000) 2024-09-16T02:07:25.612 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:25 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:25.485+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:25.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:25 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:25.496+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:26.075 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:25 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5[46111]: 2024-09-16T02:07:25.752+0000 7f074d88c640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:52.547933+0000 front 2024-09-16T02:06:52.548048+0000 (oldest deadline 2024-09-16T02:07:17.847923+0000) 2024-09-16T02:07:26.075 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:25 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5[46111]: 2024-09-16T02:07:25.752+0000 7f074d88c640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.749025+0000 front 2024-09-16T02:07:00.748970+0000 (oldest deadline 2024-09-16T02:07:24.848818+0000) 2024-09-16T02:07:26.294 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:25 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:25.968+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:53.757260+0000 front 2024-09-16T02:06:53.757170+0000 (oldest deadline 2024-09-16T02:07:18.457255+0000) 2024-09-16T02:07:26.295 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:25 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:25.968+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.158189+0000 front 2024-09-16T02:07:00.158266+0000 (oldest deadline 2024-09-16T02:07:24.858027+0000) 2024-09-16T02:07:26.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:25 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:25.902+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:26.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:25 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:25.902+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:07:26.342 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:26 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:26.074+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:26.421 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 16 02:07:25 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-4[45404]: 2024-09-16T02:07:25.950+0000 7f7a47d64640 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.987615+0000 front 2024-09-16T02:07:01.987639+0000 (oldest deadline 2024-09-16T02:07:23.588303+0000) 2024-09-16T02:07:26.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:26 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:26.038+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:06:59.698219+0000 front 2024-09-16T02:06:59.698240+0000 (oldest deadline 2024-09-16T02:07:24.398448+0000) 2024-09-16T02:07:26.788 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:26 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:26.466+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:26.788 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:26 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:26.466+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:07:26.921 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 16 02:07:26 smithi137 podman[67534]: 2024-09-16 02:07:26.755246623 +0000 UTC m=+5.673161383 container died 4f5f277921f2c9e1962674ce55301abc9ddabbddbe1a504c9b425ae7a3e1f22d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-4, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T02:07:26.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:26 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:26.546+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:26.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:26 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:26.546+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:07:27.092 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:26 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5[46111]: 2024-09-16T02:07:26.787+0000 7f074d88c640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:52.547933+0000 front 2024-09-16T02:06:52.548048+0000 (oldest deadline 2024-09-16T02:07:17.847923+0000) 2024-09-16T02:07:27.093 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:26 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5[46111]: 2024-09-16T02:07:26.787+0000 7f074d88c640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.749025+0000 front 2024-09-16T02:07:00.748970+0000 (oldest deadline 2024-09-16T02:07:24.848818+0000) 2024-09-16T02:07:27.295 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:27 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:27.006+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:53.757260+0000 front 2024-09-16T02:06:53.757170+0000 (oldest deadline 2024-09-16T02:07:18.457255+0000) 2024-09-16T02:07:27.295 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:27 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:27.006+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.158189+0000 front 2024-09-16T02:07:00.158266+0000 (oldest deadline 2024-09-16T02:07:24.858027+0000) 2024-09-16T02:07:27.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:26 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:26.864+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:27.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:26 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:26.864+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:07:27.328 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:27 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:27.019+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:06:59.698219+0000 front 2024-09-16T02:06:59.698240+0000 (oldest deadline 2024-09-16T02:07:24.398448+0000) 2024-09-16T02:07:27.328 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 16 02:07:27 smithi137 podman[67534]: 2024-09-16 02:07:27.134420988 +0000 UTC m=+6.052335738 container cleanup 4f5f277921f2c9e1962674ce55301abc9ddabbddbe1a504c9b425ae7a3e1f22d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-4, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T02:07:27.329 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 16 02:07:27 smithi137 bash[67534]: ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-4 2024-09-16T02:07:27.479 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:27 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:27.094+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:27.671 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 16 02:07:27 smithi137 podman[67552]: 2024-09-16 02:07:27.327700235 +0000 UTC m=+0.569079015 container remove 4f5f277921f2c9e1962674ce55301abc9ddabbddbe1a504c9b425ae7a3e1f22d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-4, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2) 2024-09-16T02:07:27.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:27 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:27.552+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:27.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:27 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:27.552+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:07:27.822 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:27 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:27.478+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:27.822 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:27 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:27.478+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:07:28.023 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 16 02:07:27 smithi137 podman[67641]: 2024-09-16 02:07:27.629529519 +0000 UTC m=+0.016064712 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:07:28.023 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 16 02:07:27 smithi137 podman[67641]: 2024-09-16 02:07:27.770175837 +0000 UTC m=+0.156711017 container create f9f5030dc0ef07b3d6364bef642fdca513038e9acb5a83babb8c80f0180e3680 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T02:07:28.023 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 16 02:07:28 smithi137 podman[67641]: 2024-09-16 02:07:28.022626346 +0000 UTC m=+0.409161539 container init f9f5030dc0ef07b3d6364bef642fdca513038e9acb5a83babb8c80f0180e3680 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0) 2024-09-16T02:07:28.092 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:27 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5[46111]: 2024-09-16T02:07:27.821+0000 7f074d88c640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:52.547933+0000 front 2024-09-16T02:06:52.548048+0000 (oldest deadline 2024-09-16T02:07:17.847923+0000) 2024-09-16T02:07:28.093 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:27 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5[46111]: 2024-09-16T02:07:27.821+0000 7f074d88c640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.749025+0000 front 2024-09-16T02:07:00.748970+0000 (oldest deadline 2024-09-16T02:07:24.848818+0000) 2024-09-16T02:07:28.295 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:28 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:28.043+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:53.757260+0000 front 2024-09-16T02:06:53.757170+0000 (oldest deadline 2024-09-16T02:07:18.457255+0000) 2024-09-16T02:07:28.295 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:28 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:28.043+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.158189+0000 front 2024-09-16T02:07:00.158266+0000 (oldest deadline 2024-09-16T02:07:24.858027+0000) 2024-09-16T02:07:28.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:27 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:27.840+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:28.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:27 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:27.840+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:07:28.421 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 16 02:07:28 smithi137 podman[67641]: 2024-09-16 02:07:28.026006633 +0000 UTC m=+0.412541814 container start f9f5030dc0ef07b3d6364bef642fdca513038e9acb5a83babb8c80f0180e3680 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-16T02:07:28.421 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 16 02:07:28 smithi137 podman[67641]: 2024-09-16 02:07:28.097692869 +0000 UTC m=+0.484228069 container attach f9f5030dc0ef07b3d6364bef642fdca513038e9acb5a83babb8c80f0180e3680 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-16T02:07:28.421 INFO:journalctl@ceph.osd.4.smithi137.stdout:Sep 16 02:07:28 smithi137 podman[67641]: 2024-09-16 02:07:28.206908782 +0000 UTC m=+0.593443963 container died f9f5030dc0ef07b3d6364bef642fdca513038e9acb5a83babb8c80f0180e3680 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-16T02:07:28.422 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:28 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:28.023+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:06:59.698219+0000 front 2024-09-16T02:06:59.698240+0000 (oldest deadline 2024-09-16T02:07:24.398448+0000) 2024-09-16T02:07:28.492 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:28 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:28.134+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:28.828 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:28 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:28.491+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:28.828 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:28 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:28.491+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:07:28.852 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:28 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:28.557+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:28.852 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:28 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:28.557+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:07:28.874 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@osd.4.service' 2024-09-16T02:07:28.955 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:07:28.955 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-16T02:07:28.955 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-16T02:07:28.955 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@osd.5 2024-09-16T02:07:29.068 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:28 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:28.801+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:29.068 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:28 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:28.801+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:07:29.092 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:28 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5[46111]: 2024-09-16T02:07:28.827+0000 7f074d88c640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:52.547933+0000 front 2024-09-16T02:06:52.548048+0000 (oldest deadline 2024-09-16T02:07:17.847923+0000) 2024-09-16T02:07:29.092 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:28 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5[46111]: 2024-09-16T02:07:28.827+0000 7f074d88c640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.749025+0000 front 2024-09-16T02:07:00.748970+0000 (oldest deadline 2024-09-16T02:07:24.848818+0000) 2024-09-16T02:07:29.093 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:29 smithi190 systemd[1]: Stopping Ceph osd.5 for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:07:29.171 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:29 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:29.003+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:06:59.698219+0000 front 2024-09-16T02:06:59.698240+0000 (oldest deadline 2024-09-16T02:07:24.398448+0000) 2024-09-16T02:07:29.459 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:29 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5[46111]: 2024-09-16T02:07:29.285+0000 7f0751276640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:07:29.459 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:29 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5[46111]: 2024-09-16T02:07:29.285+0000 7f0751276640 -1 osd.5 86 *** Got signal Terminated *** 2024-09-16T02:07:29.459 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:29 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5[46111]: 2024-09-16T02:07:29.285+0000 7f0751276640 -1 osd.5 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T02:07:29.459 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:29 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:29.137+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:29.545 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:29 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:29.066+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:53.757260+0000 front 2024-09-16T02:06:53.757170+0000 (oldest deadline 2024-09-16T02:07:18.457255+0000) 2024-09-16T02:07:29.545 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:29 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:29.066+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.158189+0000 front 2024-09-16T02:07:00.158266+0000 (oldest deadline 2024-09-16T02:07:24.858027+0000) 2024-09-16T02:07:29.797 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:29 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:29.458+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:29.797 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:29 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:29.458+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:07:29.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:29 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:29.595+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:29.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:29 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:29.595+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:07:30.092 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:29 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5[46111]: 2024-09-16T02:07:29.795+0000 7f074d88c640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:52.547933+0000 front 2024-09-16T02:06:52.548048+0000 (oldest deadline 2024-09-16T02:07:17.847923+0000) 2024-09-16T02:07:30.092 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:29 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5[46111]: 2024-09-16T02:07:29.795+0000 7f074d88c640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.749025+0000 front 2024-09-16T02:07:00.748970+0000 (oldest deadline 2024-09-16T02:07:24.848818+0000) 2024-09-16T02:07:30.294 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:29 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:29.800+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:30.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:29 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:29.800+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:07:30.295 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:30 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:30.034+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:53.757260+0000 front 2024-09-16T02:06:53.757170+0000 (oldest deadline 2024-09-16T02:07:18.457255+0000) 2024-09-16T02:07:30.295 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:30 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:30.034+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.158189+0000 front 2024-09-16T02:07:00.158266+0000 (oldest deadline 2024-09-16T02:07:24.858027+0000) 2024-09-16T02:07:30.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:30 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:30.007+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:06:59.698219+0000 front 2024-09-16T02:06:59.698240+0000 (oldest deadline 2024-09-16T02:07:24.398448+0000) 2024-09-16T02:07:30.474 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:30 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:30.160+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:30.799 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:30 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:30.473+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:30.799 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:30 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:30.473+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:07:30.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:30 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:30.593+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:30.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:30 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:30.593+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:07:30.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:30 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:30.593+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:07:31.085 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:30 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:30.801+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:31.085 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:30 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:30.801+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:07:31.092 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:30 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5[46111]: 2024-09-16T02:07:30.796+0000 7f074d88c640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:52.547933+0000 front 2024-09-16T02:06:52.548048+0000 (oldest deadline 2024-09-16T02:07:17.847923+0000) 2024-09-16T02:07:31.092 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:30 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5[46111]: 2024-09-16T02:07:30.796+0000 7f074d88c640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.749025+0000 front 2024-09-16T02:07:00.748970+0000 (oldest deadline 2024-09-16T02:07:24.848818+0000) 2024-09-16T02:07:31.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:30 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:30.988+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:06:59.698219+0000 front 2024-09-16T02:06:59.698240+0000 (oldest deadline 2024-09-16T02:07:24.398448+0000) 2024-09-16T02:07:31.521 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:31 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:31.111+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:31.549 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:31 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:31.083+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:53.757260+0000 front 2024-09-16T02:06:53.757170+0000 (oldest deadline 2024-09-16T02:07:18.457255+0000) 2024-09-16T02:07:31.549 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:31 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:31.083+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.158189+0000 front 2024-09-16T02:07:00.158266+0000 (oldest deadline 2024-09-16T02:07:24.858027+0000) 2024-09-16T02:07:31.549 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:31 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:31.083+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.960085+0000 front 2024-09-16T02:07:09.959999+0000 (oldest deadline 2024-09-16T02:07:30.459889+0000) 2024-09-16T02:07:31.821 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:31 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:31.518+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:31.821 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:31 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:31.518+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:07:31.922 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:31 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:31.577+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:31.922 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:31 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:31.577+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:07:31.923 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:31 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:31.577+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:07:32.092 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:31 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5[46111]: 2024-09-16T02:07:31.818+0000 7f074d88c640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:52.547933+0000 front 2024-09-16T02:06:52.548048+0000 (oldest deadline 2024-09-16T02:07:17.847923+0000) 2024-09-16T02:07:32.093 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:31 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5[46111]: 2024-09-16T02:07:31.818+0000 7f074d88c640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.749025+0000 front 2024-09-16T02:07:00.748970+0000 (oldest deadline 2024-09-16T02:07:24.848818+0000) 2024-09-16T02:07:32.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:31 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:31.831+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:32.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:31 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:31.831+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:07:32.295 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:32 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:32.057+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:53.757260+0000 front 2024-09-16T02:06:53.757170+0000 (oldest deadline 2024-09-16T02:07:18.457255+0000) 2024-09-16T02:07:32.295 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:32 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:32.057+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.158189+0000 front 2024-09-16T02:07:00.158266+0000 (oldest deadline 2024-09-16T02:07:24.858027+0000) 2024-09-16T02:07:32.295 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:32 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:32.057+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.960085+0000 front 2024-09-16T02:07:09.959999+0000 (oldest deadline 2024-09-16T02:07:30.459889+0000) 2024-09-16T02:07:32.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:31 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:31.978+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:06:59.698219+0000 front 2024-09-16T02:06:59.698240+0000 (oldest deadline 2024-09-16T02:07:24.398448+0000) 2024-09-16T02:07:32.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:31 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:31.978+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.099524+0000 front 2024-09-16T02:07:06.099616+0000 (oldest deadline 2024-09-16T02:07:31.399445+0000) 2024-09-16T02:07:32.540 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:32 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:32.095+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:32.540 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:32 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:32.095+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:07.645729+0000 front 2024-09-16T02:07:07.645761+0000 (oldest deadline 2024-09-16T02:07:31.145992+0000) 2024-09-16T02:07:32.843 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:32 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:32.538+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:32.843 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:32 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:32.538+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:07:32.843 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:32 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5[46111]: 2024-09-16T02:07:32.774+0000 7f074d88c640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:52.547933+0000 front 2024-09-16T02:06:52.548048+0000 (oldest deadline 2024-09-16T02:07:17.847923+0000) 2024-09-16T02:07:32.843 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:32 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5[46111]: 2024-09-16T02:07:32.774+0000 7f074d88c640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.749025+0000 front 2024-09-16T02:07:00.748970+0000 (oldest deadline 2024-09-16T02:07:24.848818+0000) 2024-09-16T02:07:32.844 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:32 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5[46111]: 2024-09-16T02:07:32.774+0000 7f074d88c640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:08.849993+0000 front 2024-09-16T02:07:08.849916+0000 (oldest deadline 2024-09-16T02:07:32.349999+0000) 2024-09-16T02:07:32.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:32 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:32.551+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:32.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:32 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:32.551+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:07:32.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:32 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:32.551+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:07:33.045 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:32 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:32.788+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:33.045 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:32 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:32.788+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:07:33.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:32 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:32.956+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:06:59.698219+0000 front 2024-09-16T02:06:59.698240+0000 (oldest deadline 2024-09-16T02:07:24.398448+0000) 2024-09-16T02:07:33.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:32 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:32.956+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.099524+0000 front 2024-09-16T02:07:06.099616+0000 (oldest deadline 2024-09-16T02:07:31.399445+0000) 2024-09-16T02:07:33.545 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:33 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:33.046+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:53.757260+0000 front 2024-09-16T02:06:53.757170+0000 (oldest deadline 2024-09-16T02:07:18.457255+0000) 2024-09-16T02:07:33.545 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:33 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:33.046+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.158189+0000 front 2024-09-16T02:07:00.158266+0000 (oldest deadline 2024-09-16T02:07:24.858027+0000) 2024-09-16T02:07:33.545 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:33 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:33.046+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.960085+0000 front 2024-09-16T02:07:09.959999+0000 (oldest deadline 2024-09-16T02:07:30.459889+0000) 2024-09-16T02:07:33.549 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:33 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:33.133+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:33.549 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:33 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:33.133+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:07.645729+0000 front 2024-09-16T02:07:07.645761+0000 (oldest deadline 2024-09-16T02:07:31.145992+0000) 2024-09-16T02:07:33.842 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:33 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5[46111]: 2024-09-16T02:07:33.738+0000 7f074d88c640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:52.547933+0000 front 2024-09-16T02:06:52.548048+0000 (oldest deadline 2024-09-16T02:07:17.847923+0000) 2024-09-16T02:07:33.843 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:33 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5[46111]: 2024-09-16T02:07:33.738+0000 7f074d88c640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.749025+0000 front 2024-09-16T02:07:00.748970+0000 (oldest deadline 2024-09-16T02:07:24.848818+0000) 2024-09-16T02:07:33.843 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:33 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5[46111]: 2024-09-16T02:07:33.738+0000 7f074d88c640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:08.849993+0000 front 2024-09-16T02:07:08.849916+0000 (oldest deadline 2024-09-16T02:07:32.349999+0000) 2024-09-16T02:07:33.843 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:33 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:33.547+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:33.843 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:33 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:33.547+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:07:33.843 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:33 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:33.547+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:07:33.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:33 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:33.507+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:33.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:33 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:33.507+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:07:33.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:33 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:33.507+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:07:34.087 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:33 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:33.798+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:34.087 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:33 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:33.798+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:07:34.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:33 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:33.996+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:06:59.698219+0000 front 2024-09-16T02:06:59.698240+0000 (oldest deadline 2024-09-16T02:07:24.398448+0000) 2024-09-16T02:07:34.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:33 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:33.996+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.099524+0000 front 2024-09-16T02:07:06.099616+0000 (oldest deadline 2024-09-16T02:07:31.399445+0000) 2024-09-16T02:07:34.542 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:34 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:34.136+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:34.543 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:34 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:34.136+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:07.645729+0000 front 2024-09-16T02:07:07.645761+0000 (oldest deadline 2024-09-16T02:07:31.145992+0000) 2024-09-16T02:07:34.544 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:34 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:34.085+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:53.757260+0000 front 2024-09-16T02:06:53.757170+0000 (oldest deadline 2024-09-16T02:07:18.457255+0000) 2024-09-16T02:07:34.544 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:34 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:34.085+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.158189+0000 front 2024-09-16T02:07:00.158266+0000 (oldest deadline 2024-09-16T02:07:24.858027+0000) 2024-09-16T02:07:34.544 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:34 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:34.085+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.960085+0000 front 2024-09-16T02:07:09.959999+0000 (oldest deadline 2024-09-16T02:07:30.459889+0000) 2024-09-16T02:07:34.843 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:34 smithi190 podman[68050]: 2024-09-16 02:07:34.678884936 +0000 UTC m=+5.530851452 container died 5f4f7269ef89c6fc6717570d56304e95c683ee4a4bc41d25b3636897a7e42da1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-16T02:07:34.843 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:34 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:34.541+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:34.843 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:34 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:34.541+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:07:34.843 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:34 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:34.541+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:07:34.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:34 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:34.504+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:34.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:34 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:34.504+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:07:34.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:34 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:34.504+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:07:35.117 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:34 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:34.808+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:35.117 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:34 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:34.808+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:07:35.321 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:35 smithi190 podman[68050]: 2024-09-16 02:07:35.077860632 +0000 UTC m=+5.929827136 container cleanup 5f4f7269ef89c6fc6717570d56304e95c683ee4a4bc41d25b3636897a7e42da1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-16T02:07:35.321 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:35 smithi190 bash[68050]: ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5 2024-09-16T02:07:35.322 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:35 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:35.087+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:35.322 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:35 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:35.087+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:07.645729+0000 front 2024-09-16T02:07:07.645761+0000 (oldest deadline 2024-09-16T02:07:31.145992+0000) 2024-09-16T02:07:35.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:35 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:35.013+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:06:59.698219+0000 front 2024-09-16T02:06:59.698240+0000 (oldest deadline 2024-09-16T02:07:24.398448+0000) 2024-09-16T02:07:35.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:35 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:35.013+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.099524+0000 front 2024-09-16T02:07:06.099616+0000 (oldest deadline 2024-09-16T02:07:31.399445+0000) 2024-09-16T02:07:35.545 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:35 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:35.116+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:53.757260+0000 front 2024-09-16T02:06:53.757170+0000 (oldest deadline 2024-09-16T02:07:18.457255+0000) 2024-09-16T02:07:35.545 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:35 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:35.116+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.158189+0000 front 2024-09-16T02:07:00.158266+0000 (oldest deadline 2024-09-16T02:07:24.858027+0000) 2024-09-16T02:07:35.545 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:35 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:35.116+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.960085+0000 front 2024-09-16T02:07:09.959999+0000 (oldest deadline 2024-09-16T02:07:30.459889+0000) 2024-09-16T02:07:35.592 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:35 smithi190 podman[68065]: 2024-09-16 02:07:35.32089267 +0000 UTC m=+0.636399539 container remove 5f4f7269ef89c6fc6717570d56304e95c683ee4a4bc41d25b3636897a7e42da1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-16T02:07:35.593 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:35 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:35.501+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:35.593 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:35 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:35.501+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:07:35.593 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:35 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:35.501+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:07:35.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:35 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:35.513+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:35.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:35 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:35.513+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:07:35.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:35 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:35.513+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:07:35.951 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:35 smithi190 podman[68153]: 2024-09-16 02:07:35.560217836 +0000 UTC m=+0.016448100 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:07:35.951 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:35 smithi190 podman[68153]: 2024-09-16 02:07:35.680241136 +0000 UTC m=+0.136471440 container create 90e39895b9c7bbc54e3e48485dc7bb6eeb8d0425fca0a3008a2b1504947282c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T02:07:36.144 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:35 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:35.855+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:36.144 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:35 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:35.855+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:07:36.144 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:35 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:35.855+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.948571+0000 front 2024-09-16T02:07:09.948470+0000 (oldest deadline 2024-09-16T02:07:35.248301+0000) 2024-09-16T02:07:36.342 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:36 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:36.050+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:36.343 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:36 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:36.050+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:07.645729+0000 front 2024-09-16T02:07:07.645761+0000 (oldest deadline 2024-09-16T02:07:31.145992+0000) 2024-09-16T02:07:36.343 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:35 smithi190 podman[68153]: 2024-09-16 02:07:35.950509798 +0000 UTC m=+0.406740062 container init 90e39895b9c7bbc54e3e48485dc7bb6eeb8d0425fca0a3008a2b1504947282c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T02:07:36.343 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:35 smithi190 podman[68153]: 2024-09-16 02:07:35.95393255 +0000 UTC m=+0.410162811 container start 90e39895b9c7bbc54e3e48485dc7bb6eeb8d0425fca0a3008a2b1504947282c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-16T02:07:36.343 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:36 smithi190 podman[68153]: 2024-09-16 02:07:36.066069949 +0000 UTC m=+0.522300222 container attach 90e39895b9c7bbc54e3e48485dc7bb6eeb8d0425fca0a3008a2b1504947282c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-16T02:07:36.343 INFO:journalctl@ceph.osd.5.smithi190.stdout:Sep 16 02:07:36 smithi190 podman[68153]: 2024-09-16 02:07:36.143149221 +0000 UTC m=+0.599379488 container died 90e39895b9c7bbc54e3e48485dc7bb6eeb8d0425fca0a3008a2b1504947282c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-16T02:07:36.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:36 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:36.012+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:06:59.698219+0000 front 2024-09-16T02:06:59.698240+0000 (oldest deadline 2024-09-16T02:07:24.398448+0000) 2024-09-16T02:07:36.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:36 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:36.012+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.099524+0000 front 2024-09-16T02:07:06.099616+0000 (oldest deadline 2024-09-16T02:07:31.399445+0000) 2024-09-16T02:07:36.545 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:36 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:36.142+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:53.757260+0000 front 2024-09-16T02:06:53.757170+0000 (oldest deadline 2024-09-16T02:07:18.457255+0000) 2024-09-16T02:07:36.545 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:36 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:36.142+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.158189+0000 front 2024-09-16T02:07:00.158266+0000 (oldest deadline 2024-09-16T02:07:24.858027+0000) 2024-09-16T02:07:36.545 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:36 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:36.142+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.960085+0000 front 2024-09-16T02:07:09.959999+0000 (oldest deadline 2024-09-16T02:07:30.459889+0000) 2024-09-16T02:07:36.842 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:36 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:36.545+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:36.843 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:36 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:36.545+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:07:36.843 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:36 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:36.545+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:07:36.915 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@osd.5.service' 2024-09-16T02:07:36.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:36 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:36.551+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:36.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:36 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:36.551+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:07:36.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:36 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:36.551+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:07:36.954 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:07:36.954 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-16T02:07:36.954 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-16T02:07:36.955 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@osd.6 2024-09-16T02:07:37.167 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:36 smithi105 systemd[1]: Stopping Ceph osd.6 for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:07:37.167 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:36 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:36.889+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:37.167 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:36 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:36.889+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:07:37.167 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:36 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:36.889+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.948571+0000 front 2024-09-16T02:07:09.948470+0000 (oldest deadline 2024-09-16T02:07:35.248301+0000) 2024-09-16T02:07:37.342 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:37 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:37.033+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:37.342 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:37 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:37.033+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:07.645729+0000 front 2024-09-16T02:07:07.645761+0000 (oldest deadline 2024-09-16T02:07:31.145992+0000) 2024-09-16T02:07:37.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:37 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:37.033+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:06:59.698219+0000 front 2024-09-16T02:06:59.698240+0000 (oldest deadline 2024-09-16T02:07:24.398448+0000) 2024-09-16T02:07:37.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:37 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:37.033+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.099524+0000 front 2024-09-16T02:07:06.099616+0000 (oldest deadline 2024-09-16T02:07:31.399445+0000) 2024-09-16T02:07:37.545 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:37 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:37.165+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:53.757260+0000 front 2024-09-16T02:06:53.757170+0000 (oldest deadline 2024-09-16T02:07:18.457255+0000) 2024-09-16T02:07:37.545 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:37 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:37.165+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.158189+0000 front 2024-09-16T02:07:00.158266+0000 (oldest deadline 2024-09-16T02:07:24.858027+0000) 2024-09-16T02:07:37.545 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:37 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:37.165+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.960085+0000 front 2024-09-16T02:07:09.959999+0000 (oldest deadline 2024-09-16T02:07:30.459889+0000) 2024-09-16T02:07:37.545 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:37 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:37.276+0000 7f248f503640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:07:37.545 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:37 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:37.276+0000 7f248f503640 -1 osd.6 86 *** Got signal Terminated *** 2024-09-16T02:07:37.545 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:37 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:37.276+0000 7f248f503640 -1 osd.6 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T02:07:37.842 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:37 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:37.564+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:37.842 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:37 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:37.565+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:07:37.843 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:37 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:37.565+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:07:37.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:37 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:37.582+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:37.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:37 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:37.582+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:07:37.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:37 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:37.582+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:07:38.294 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:38 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:38.115+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:53.757260+0000 front 2024-09-16T02:06:53.757170+0000 (oldest deadline 2024-09-16T02:07:18.457255+0000) 2024-09-16T02:07:38.295 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:38 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:38.115+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.158189+0000 front 2024-09-16T02:07:00.158266+0000 (oldest deadline 2024-09-16T02:07:24.858027+0000) 2024-09-16T02:07:38.295 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:38 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:38.115+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.960085+0000 front 2024-09-16T02:07:09.959999+0000 (oldest deadline 2024-09-16T02:07:30.459889+0000) 2024-09-16T02:07:38.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:37 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:37.901+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:38.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:37 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:37.901+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:07:38.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:37 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:37.901+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.948571+0000 front 2024-09-16T02:07:09.948470+0000 (oldest deadline 2024-09-16T02:07:35.248301+0000) 2024-09-16T02:07:38.342 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:37 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:37.989+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:38.343 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:37 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:37.989+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:07.645729+0000 front 2024-09-16T02:07:07.645761+0000 (oldest deadline 2024-09-16T02:07:31.145992+0000) 2024-09-16T02:07:38.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:38 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:38.044+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:06:59.698219+0000 front 2024-09-16T02:06:59.698240+0000 (oldest deadline 2024-09-16T02:07:24.398448+0000) 2024-09-16T02:07:38.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:38 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:38.044+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.099524+0000 front 2024-09-16T02:07:06.099616+0000 (oldest deadline 2024-09-16T02:07:31.399445+0000) 2024-09-16T02:07:38.842 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:38 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:38.583+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:38.842 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:38 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:38.583+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:07:38.843 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:38 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:38.583+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:07:38.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:38 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:38.562+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:38.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:38 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:38.562+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:07:38.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:38 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:38.562+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:07:39.294 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:39 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:39.111+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:53.757260+0000 front 2024-09-16T02:06:53.757170+0000 (oldest deadline 2024-09-16T02:07:18.457255+0000) 2024-09-16T02:07:39.295 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:39 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:39.111+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.158189+0000 front 2024-09-16T02:07:00.158266+0000 (oldest deadline 2024-09-16T02:07:24.858027+0000) 2024-09-16T02:07:39.295 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:39 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:39.111+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.960085+0000 front 2024-09-16T02:07:09.959999+0000 (oldest deadline 2024-09-16T02:07:30.459889+0000) 2024-09-16T02:07:39.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:38 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:38.925+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:39.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:38 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:38.925+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:07:39.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:38 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:38.925+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.948571+0000 front 2024-09-16T02:07:09.948470+0000 (oldest deadline 2024-09-16T02:07:35.248301+0000) 2024-09-16T02:07:39.342 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:38 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:38.996+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:39.343 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:38 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:38.996+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:07.645729+0000 front 2024-09-16T02:07:07.645761+0000 (oldest deadline 2024-09-16T02:07:31.145992+0000) 2024-09-16T02:07:39.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:39 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:39.043+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:06:59.698219+0000 front 2024-09-16T02:06:59.698240+0000 (oldest deadline 2024-09-16T02:07:24.398448+0000) 2024-09-16T02:07:39.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:39 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:39.043+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.099524+0000 front 2024-09-16T02:07:06.099616+0000 (oldest deadline 2024-09-16T02:07:31.399445+0000) 2024-09-16T02:07:39.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:39 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:39.522+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:39.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:39 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:39.522+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:07:39.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:39 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:39.522+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:07:40.017 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:39 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:39.624+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:40.017 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:39 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:39.624+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:07:40.017 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:39 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:39.624+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:07:40.295 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:40 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:40.128+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:53.757260+0000 front 2024-09-16T02:06:53.757170+0000 (oldest deadline 2024-09-16T02:07:18.457255+0000) 2024-09-16T02:07:40.295 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:40 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:40.128+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.158189+0000 front 2024-09-16T02:07:00.158266+0000 (oldest deadline 2024-09-16T02:07:24.858027+0000) 2024-09-16T02:07:40.295 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:40 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:40.128+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.960085+0000 front 2024-09-16T02:07:09.959999+0000 (oldest deadline 2024-09-16T02:07:30.459889+0000) 2024-09-16T02:07:40.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:39 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:39.909+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:40.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:39 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:39.909+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:07:40.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:39 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:39.909+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.948571+0000 front 2024-09-16T02:07:09.948470+0000 (oldest deadline 2024-09-16T02:07:35.248301+0000) 2024-09-16T02:07:40.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:39 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:39.909+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.449541+0000 front 2024-09-16T02:07:17.449605+0000 (oldest deadline 2024-09-16T02:07:39.749594+0000) 2024-09-16T02:07:40.342 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:40 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:40.015+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:40.342 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:40 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:40.015+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:07.645729+0000 front 2024-09-16T02:07:07.645761+0000 (oldest deadline 2024-09-16T02:07:31.145992+0000) 2024-09-16T02:07:40.342 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:40 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:40.015+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:15.846928+0000 front 2024-09-16T02:07:15.847025+0000 (oldest deadline 2024-09-16T02:07:39.346979+0000) 2024-09-16T02:07:40.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:40 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:40.092+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:06:59.698219+0000 front 2024-09-16T02:06:59.698240+0000 (oldest deadline 2024-09-16T02:07:24.398448+0000) 2024-09-16T02:07:40.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:40 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:40.092+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.099524+0000 front 2024-09-16T02:07:06.099616+0000 (oldest deadline 2024-09-16T02:07:31.399445+0000) 2024-09-16T02:07:40.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:40 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:40.479+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:40.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:40 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:40.479+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:07:40.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:40 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:40.479+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:07:41.014 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:40 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:40.615+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:41.014 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:40 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:40.615+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:07:41.014 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:40 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:40.615+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:07:41.294 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:41 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:41.082+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:53.757260+0000 front 2024-09-16T02:06:53.757170+0000 (oldest deadline 2024-09-16T02:07:18.457255+0000) 2024-09-16T02:07:41.295 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:41 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:41.082+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.158189+0000 front 2024-09-16T02:07:00.158266+0000 (oldest deadline 2024-09-16T02:07:24.858027+0000) 2024-09-16T02:07:41.295 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:41 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:41.082+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.960085+0000 front 2024-09-16T02:07:09.959999+0000 (oldest deadline 2024-09-16T02:07:30.459889+0000) 2024-09-16T02:07:41.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:40 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:40.878+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:41.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:40 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:40.878+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:07:41.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:40 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:40.878+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.948571+0000 front 2024-09-16T02:07:09.948470+0000 (oldest deadline 2024-09-16T02:07:35.248301+0000) 2024-09-16T02:07:41.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:40 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:40.878+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.449541+0000 front 2024-09-16T02:07:17.449605+0000 (oldest deadline 2024-09-16T02:07:39.749594+0000) 2024-09-16T02:07:41.342 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:41 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:41.012+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:41.342 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:41 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:41.012+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:07.645729+0000 front 2024-09-16T02:07:07.645761+0000 (oldest deadline 2024-09-16T02:07:31.145992+0000) 2024-09-16T02:07:41.343 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:41 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:41.012+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:15.846928+0000 front 2024-09-16T02:07:15.847025+0000 (oldest deadline 2024-09-16T02:07:39.346979+0000) 2024-09-16T02:07:41.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:41 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:41.105+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:06:59.698219+0000 front 2024-09-16T02:06:59.698240+0000 (oldest deadline 2024-09-16T02:07:24.398448+0000) 2024-09-16T02:07:41.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:41 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:41.105+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.099524+0000 front 2024-09-16T02:07:06.099616+0000 (oldest deadline 2024-09-16T02:07:31.399445+0000) 2024-09-16T02:07:41.842 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:41 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:41.588+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:42.038 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:41 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:41.588+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:07:42.038 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:41 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:41.588+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:07:42.039 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:41 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:41.429+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:42.039 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:41 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:41.429+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:07:42.039 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:41 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:41.429+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:07:42.039 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:41 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:41.429+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.570558+0000 front 2024-09-16T02:07:17.570576+0000 (oldest deadline 2024-09-16T02:07:41.070561+0000) 2024-09-16T02:07:42.295 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:42 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:42.044+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:53.757260+0000 front 2024-09-16T02:06:53.757170+0000 (oldest deadline 2024-09-16T02:07:18.457255+0000) 2024-09-16T02:07:42.295 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:42 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:42.044+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.158189+0000 front 2024-09-16T02:07:00.158266+0000 (oldest deadline 2024-09-16T02:07:24.858027+0000) 2024-09-16T02:07:42.295 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:42 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:42.044+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.960085+0000 front 2024-09-16T02:07:09.959999+0000 (oldest deadline 2024-09-16T02:07:30.459889+0000) 2024-09-16T02:07:42.295 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:42 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6[57977]: 2024-09-16T02:07:42.044+0000 7f2487b11640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:16.861366+0000 front 2024-09-16T02:07:16.861338+0000 (oldest deadline 2024-09-16T02:07:41.561526+0000) 2024-09-16T02:07:42.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:41 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:41.891+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:42.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:41 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:41.891+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:07:42.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:41 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:41.891+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.948571+0000 front 2024-09-16T02:07:09.948470+0000 (oldest deadline 2024-09-16T02:07:35.248301+0000) 2024-09-16T02:07:42.296 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:41 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:41.891+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.449541+0000 front 2024-09-16T02:07:17.449605+0000 (oldest deadline 2024-09-16T02:07:39.749594+0000) 2024-09-16T02:07:42.343 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:42 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:42.008+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:42.343 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:42 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:42.008+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:07.645729+0000 front 2024-09-16T02:07:07.645761+0000 (oldest deadline 2024-09-16T02:07:31.145992+0000) 2024-09-16T02:07:42.343 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:42 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:42.008+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:15.846928+0000 front 2024-09-16T02:07:15.847025+0000 (oldest deadline 2024-09-16T02:07:39.346979+0000) 2024-09-16T02:07:42.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:42 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:42.061+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:06:59.698219+0000 front 2024-09-16T02:06:59.698240+0000 (oldest deadline 2024-09-16T02:07:24.398448+0000) 2024-09-16T02:07:42.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:42 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:42.061+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.099524+0000 front 2024-09-16T02:07:06.099616+0000 (oldest deadline 2024-09-16T02:07:31.399445+0000) 2024-09-16T02:07:42.886 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:42 smithi105 podman[93741]: 2024-09-16 02:07:42.59145157 +0000 UTC m=+5.450703991 container died 01ed28b823f4a7a234a0c103ced18498c3cf9ac1b64bea3048adbeca60f63b74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-16T02:07:42.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:42 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:42.449+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:42.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:42 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:42.449+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:07:42.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:42 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:42.449+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:07:42.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:42 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:42.449+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.570558+0000 front 2024-09-16T02:07:17.570576+0000 (oldest deadline 2024-09-16T02:07:41.070561+0000) 2024-09-16T02:07:43.045 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:42 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:42.613+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:43.045 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:42 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:42.613+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:07:43.045 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:42 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:42.613+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:07:43.294 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:43 smithi105 podman[93741]: 2024-09-16 02:07:43.076849089 +0000 UTC m=+5.936101546 container cleanup 01ed28b823f4a7a234a0c103ced18498c3cf9ac1b64bea3048adbeca60f63b74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e) 2024-09-16T02:07:43.295 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:43 smithi105 bash[93741]: ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6 2024-09-16T02:07:43.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:42 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:42.940+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:43.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:42 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:42.940+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:07:43.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:42 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:42.940+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.948571+0000 front 2024-09-16T02:07:09.948470+0000 (oldest deadline 2024-09-16T02:07:35.248301+0000) 2024-09-16T02:07:43.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:42 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:42.940+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.449541+0000 front 2024-09-16T02:07:17.449605+0000 (oldest deadline 2024-09-16T02:07:39.749594+0000) 2024-09-16T02:07:43.342 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:43 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:43.044+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:43.342 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:43 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:43.044+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:07.645729+0000 front 2024-09-16T02:07:07.645761+0000 (oldest deadline 2024-09-16T02:07:31.145992+0000) 2024-09-16T02:07:43.343 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:43 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:43.044+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:15.846928+0000 front 2024-09-16T02:07:15.847025+0000 (oldest deadline 2024-09-16T02:07:39.346979+0000) 2024-09-16T02:07:43.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:43 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:43.092+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:06:59.698219+0000 front 2024-09-16T02:06:59.698240+0000 (oldest deadline 2024-09-16T02:07:24.398448+0000) 2024-09-16T02:07:43.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:43 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:43.092+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.099524+0000 front 2024-09-16T02:07:06.099616+0000 (oldest deadline 2024-09-16T02:07:31.399445+0000) 2024-09-16T02:07:43.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:43 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:43.092+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:15.802100+0000 front 2024-09-16T02:07:15.802150+0000 (oldest deadline 2024-09-16T02:07:42.802526+0000) 2024-09-16T02:07:43.694 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:43 smithi105 podman[93769]: 2024-09-16 02:07:43.386797699 +0000 UTC m=+0.788739023 container remove 01ed28b823f4a7a234a0c103ced18498c3cf9ac1b64bea3048adbeca60f63b74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-16T02:07:43.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:43 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:43.460+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:43.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:43 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:43.460+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:07:43.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:43 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:43.460+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:07:43.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:43 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:43.460+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.570558+0000 front 2024-09-16T02:07:17.570576+0000 (oldest deadline 2024-09-16T02:07:41.070561+0000) 2024-09-16T02:07:43.974 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:43 smithi105 podman[93856]: 2024-09-16 02:07:43.596228276 +0000 UTC m=+0.015663774 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:07:43.975 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:43 smithi105 podman[93856]: 2024-09-16 02:07:43.712701602 +0000 UTC m=+0.132137095 container create e287b97781704c8194c0771b6b885ede12b38ef1f52aa4580bbde34c8cf5790d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T02:07:44.086 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:43 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:43.592+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:44.086 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:43 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:43.592+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:07:44.086 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:43 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:43.592+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:07:44.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:43 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:43.985+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:44.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:43 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:43.985+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:07:44.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:43 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:43.985+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.948571+0000 front 2024-09-16T02:07:09.948470+0000 (oldest deadline 2024-09-16T02:07:35.248301+0000) 2024-09-16T02:07:44.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:43 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:43.985+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.449541+0000 front 2024-09-16T02:07:17.449605+0000 (oldest deadline 2024-09-16T02:07:39.749594+0000) 2024-09-16T02:07:44.295 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:43 smithi105 podman[93856]: 2024-09-16 02:07:43.973795485 +0000 UTC m=+0.393230990 container init e287b97781704c8194c0771b6b885ede12b38ef1f52aa4580bbde34c8cf5790d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-16T02:07:44.295 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:43 smithi105 podman[93856]: 2024-09-16 02:07:43.977487697 +0000 UTC m=+0.396923193 container start e287b97781704c8194c0771b6b885ede12b38ef1f52aa4580bbde34c8cf5790d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-16T02:07:44.295 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:44 smithi105 podman[93856]: 2024-09-16 02:07:44.082062247 +0000 UTC m=+0.501497749 container attach e287b97781704c8194c0771b6b885ede12b38ef1f52aa4580bbde34c8cf5790d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-16T02:07:44.296 INFO:journalctl@ceph.osd.6.smithi105.stdout:Sep 16 02:07:44 smithi105 podman[93856]: 2024-09-16 02:07:44.162987734 +0000 UTC m=+0.582423231 container died e287b97781704c8194c0771b6b885ede12b38ef1f52aa4580bbde34c8cf5790d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-16T02:07:44.342 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:44 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:44.084+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:44.343 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:44 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:44.084+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:07.645729+0000 front 2024-09-16T02:07:07.645761+0000 (oldest deadline 2024-09-16T02:07:31.145992+0000) 2024-09-16T02:07:44.343 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:44 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:44.084+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:15.846928+0000 front 2024-09-16T02:07:15.847025+0000 (oldest deadline 2024-09-16T02:07:39.346979+0000) 2024-09-16T02:07:44.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:44 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:44.066+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:06:59.698219+0000 front 2024-09-16T02:06:59.698240+0000 (oldest deadline 2024-09-16T02:07:24.398448+0000) 2024-09-16T02:07:44.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:44 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:44.066+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.099524+0000 front 2024-09-16T02:07:06.099616+0000 (oldest deadline 2024-09-16T02:07:31.399445+0000) 2024-09-16T02:07:44.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:44 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:44.067+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:15.802100+0000 front 2024-09-16T02:07:15.802150+0000 (oldest deadline 2024-09-16T02:07:42.802526+0000) 2024-09-16T02:07:44.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:44 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:44.508+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:44.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:44 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:44.508+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:07:44.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:44 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:44.508+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:07:44.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:44 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:44.508+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.570558+0000 front 2024-09-16T02:07:17.570576+0000 (oldest deadline 2024-09-16T02:07:41.070561+0000) 2024-09-16T02:07:44.955 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@osd.6.service' 2024-09-16T02:07:45.008 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:07:45.008 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-16T02:07:45.009 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-16T02:07:45.009 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@osd.7 2024-09-16T02:07:45.092 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:44 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:44.623+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:45.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:44 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:44.623+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:07:45.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:44 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:44.623+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:07:45.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:44 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:44.997+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:45.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:44 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:44.997+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:07:45.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:44 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:44.997+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.948571+0000 front 2024-09-16T02:07:09.948470+0000 (oldest deadline 2024-09-16T02:07:35.248301+0000) 2024-09-16T02:07:45.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:44 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:44.997+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.449541+0000 front 2024-09-16T02:07:17.449605+0000 (oldest deadline 2024-09-16T02:07:39.749594+0000) 2024-09-16T02:07:45.304 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:45 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:45.027+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:06:59.698219+0000 front 2024-09-16T02:06:59.698240+0000 (oldest deadline 2024-09-16T02:07:24.398448+0000) 2024-09-16T02:07:45.304 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:45 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:45.027+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.099524+0000 front 2024-09-16T02:07:06.099616+0000 (oldest deadline 2024-09-16T02:07:31.399445+0000) 2024-09-16T02:07:45.305 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:45 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:45.027+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:15.802100+0000 front 2024-09-16T02:07:15.802150+0000 (oldest deadline 2024-09-16T02:07:42.802526+0000) 2024-09-16T02:07:45.305 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:45 smithi137 systemd[1]: Stopping Ceph osd.7 for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:07:45.305 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:45 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:45.303+0000 7fcd41d95640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:07:45.305 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:45 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:45.303+0000 7fcd41d95640 -1 osd.7 86 *** Got signal Terminated *** 2024-09-16T02:07:45.305 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:45 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:45.303+0000 7fcd41d95640 -1 osd.7 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T02:07:45.593 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:45 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:45.132+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:45.593 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:45 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:45.132+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:07.645729+0000 front 2024-09-16T02:07:07.645761+0000 (oldest deadline 2024-09-16T02:07:31.145992+0000) 2024-09-16T02:07:45.594 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:45 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:45.132+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:15.846928+0000 front 2024-09-16T02:07:15.847025+0000 (oldest deadline 2024-09-16T02:07:39.346979+0000) 2024-09-16T02:07:45.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:45 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:45.471+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:45.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:45 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:45.471+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:07:45.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:45 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:45.471+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:07:45.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:45 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:45.471+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.570558+0000 front 2024-09-16T02:07:17.570576+0000 (oldest deadline 2024-09-16T02:07:41.070561+0000) 2024-09-16T02:07:46.092 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:45 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:45.654+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:46.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:45 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:45.654+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:07:46.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:45 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:45.654+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:07:46.294 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:46 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:46.037+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:46.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:46 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:46.037+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:07:46.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:46 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:46.037+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.948571+0000 front 2024-09-16T02:07:09.948470+0000 (oldest deadline 2024-09-16T02:07:35.248301+0000) 2024-09-16T02:07:46.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:46 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:46.037+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.449541+0000 front 2024-09-16T02:07:17.449605+0000 (oldest deadline 2024-09-16T02:07:39.749594+0000) 2024-09-16T02:07:46.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:46 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:46.076+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:06:59.698219+0000 front 2024-09-16T02:06:59.698240+0000 (oldest deadline 2024-09-16T02:07:24.398448+0000) 2024-09-16T02:07:46.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:46 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:46.076+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.099524+0000 front 2024-09-16T02:07:06.099616+0000 (oldest deadline 2024-09-16T02:07:31.399445+0000) 2024-09-16T02:07:46.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:46 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:46.076+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:15.802100+0000 front 2024-09-16T02:07:15.802150+0000 (oldest deadline 2024-09-16T02:07:42.802526+0000) 2024-09-16T02:07:46.593 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:46 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:46.122+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:46.593 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:46 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:46.122+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:07.645729+0000 front 2024-09-16T02:07:07.645761+0000 (oldest deadline 2024-09-16T02:07:31.145992+0000) 2024-09-16T02:07:46.593 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:46 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:46.122+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:15.846928+0000 front 2024-09-16T02:07:15.847025+0000 (oldest deadline 2024-09-16T02:07:39.346979+0000) 2024-09-16T02:07:46.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:46 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:46.518+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:46.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:46 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:46.518+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:07:46.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:46 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:46.518+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:07:46.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:46 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:46.518+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.570558+0000 front 2024-09-16T02:07:17.570576+0000 (oldest deadline 2024-09-16T02:07:41.070561+0000) 2024-09-16T02:07:47.092 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:46 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:46.640+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:47.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:46 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:46.640+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:07:47.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:46 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:46.640+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:07:47.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:47 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:47.101+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:06:59.698219+0000 front 2024-09-16T02:06:59.698240+0000 (oldest deadline 2024-09-16T02:07:24.398448+0000) 2024-09-16T02:07:47.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:47 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:47.101+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.099524+0000 front 2024-09-16T02:07:06.099616+0000 (oldest deadline 2024-09-16T02:07:31.399445+0000) 2024-09-16T02:07:47.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:47 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:47.101+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:15.802100+0000 front 2024-09-16T02:07:15.802150+0000 (oldest deadline 2024-09-16T02:07:42.802526+0000) 2024-09-16T02:07:47.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:47 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:47.081+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:47.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:47 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:47.081+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:07:47.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:47 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:47.081+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.948571+0000 front 2024-09-16T02:07:09.948470+0000 (oldest deadline 2024-09-16T02:07:35.248301+0000) 2024-09-16T02:07:47.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:47 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:47.081+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.449541+0000 front 2024-09-16T02:07:17.449605+0000 (oldest deadline 2024-09-16T02:07:39.749594+0000) 2024-09-16T02:07:47.592 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:47 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:47.118+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:47.592 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:47 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:47.118+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:07.645729+0000 front 2024-09-16T02:07:07.645761+0000 (oldest deadline 2024-09-16T02:07:31.145992+0000) 2024-09-16T02:07:47.593 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:47 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:47.118+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:15.846928+0000 front 2024-09-16T02:07:15.847025+0000 (oldest deadline 2024-09-16T02:07:39.346979+0000) 2024-09-16T02:07:47.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:47 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:47.524+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:47.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:47 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:47.524+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:07:47.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:47 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:47.524+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:07:47.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:47 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:47.524+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.570558+0000 front 2024-09-16T02:07:17.570576+0000 (oldest deadline 2024-09-16T02:07:41.070561+0000) 2024-09-16T02:07:47.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:47 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:47.524+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.471846+0000 front 2024-09-16T02:07:24.471866+0000 (oldest deadline 2024-09-16T02:07:46.771852+0000) 2024-09-16T02:07:48.092 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:47 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:47.638+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:48.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:47 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:47.638+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:07:48.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:47 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:47.638+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:07:48.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:48 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:48.102+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:06:59.698219+0000 front 2024-09-16T02:06:59.698240+0000 (oldest deadline 2024-09-16T02:07:24.398448+0000) 2024-09-16T02:07:48.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:48 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:48.102+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.099524+0000 front 2024-09-16T02:07:06.099616+0000 (oldest deadline 2024-09-16T02:07:31.399445+0000) 2024-09-16T02:07:48.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:48 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:48.102+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:15.802100+0000 front 2024-09-16T02:07:15.802150+0000 (oldest deadline 2024-09-16T02:07:42.802526+0000) 2024-09-16T02:07:48.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:48 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:48.128+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:48.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:48 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:48.128+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:07:48.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:48 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:48.128+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.948571+0000 front 2024-09-16T02:07:09.948470+0000 (oldest deadline 2024-09-16T02:07:35.248301+0000) 2024-09-16T02:07:48.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:48 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:48.128+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.449541+0000 front 2024-09-16T02:07:17.449605+0000 (oldest deadline 2024-09-16T02:07:39.749594+0000) 2024-09-16T02:07:48.592 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:48 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:48.101+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:48.592 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:48 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:48.101+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:07.645729+0000 front 2024-09-16T02:07:07.645761+0000 (oldest deadline 2024-09-16T02:07:31.145992+0000) 2024-09-16T02:07:48.592 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:48 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:48.101+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:15.846928+0000 front 2024-09-16T02:07:15.847025+0000 (oldest deadline 2024-09-16T02:07:39.346979+0000) 2024-09-16T02:07:48.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:48 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:48.494+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:48.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:48 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:48.494+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:07:48.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:48 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:48.494+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:07:48.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:48 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:48.494+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.570558+0000 front 2024-09-16T02:07:17.570576+0000 (oldest deadline 2024-09-16T02:07:41.070561+0000) 2024-09-16T02:07:48.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:48 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:48.494+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.471846+0000 front 2024-09-16T02:07:24.471866+0000 (oldest deadline 2024-09-16T02:07:46.771852+0000) 2024-09-16T02:07:49.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:48 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:48.665+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:49.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:48 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:48.665+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:07:49.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:48 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:48.665+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:07:49.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:48 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:48.665+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.065430+0000 front 2024-09-16T02:07:24.065531+0000 (oldest deadline 2024-09-16T02:07:48.165250+0000) 2024-09-16T02:07:49.294 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:49 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:49.135+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:49.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:49 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:49.135+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:07:49.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:49 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:49.135+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.948571+0000 front 2024-09-16T02:07:09.948470+0000 (oldest deadline 2024-09-16T02:07:35.248301+0000) 2024-09-16T02:07:49.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:49 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:49.135+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.449541+0000 front 2024-09-16T02:07:17.449605+0000 (oldest deadline 2024-09-16T02:07:39.749594+0000) 2024-09-16T02:07:49.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:49 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:49.064+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:06:59.698219+0000 front 2024-09-16T02:06:59.698240+0000 (oldest deadline 2024-09-16T02:07:24.398448+0000) 2024-09-16T02:07:49.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:49 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:49.064+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.099524+0000 front 2024-09-16T02:07:06.099616+0000 (oldest deadline 2024-09-16T02:07:31.399445+0000) 2024-09-16T02:07:49.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:49 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:49.064+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:15.802100+0000 front 2024-09-16T02:07:15.802150+0000 (oldest deadline 2024-09-16T02:07:42.802526+0000) 2024-09-16T02:07:49.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:49 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:49.064+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:22.802834+0000 front 2024-09-16T02:07:22.802866+0000 (oldest deadline 2024-09-16T02:07:48.702982+0000) 2024-09-16T02:07:49.592 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:49 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:49.146+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:49.592 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:49 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:49.146+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:07.645729+0000 front 2024-09-16T02:07:07.645761+0000 (oldest deadline 2024-09-16T02:07:31.145992+0000) 2024-09-16T02:07:49.593 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:49 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:49.146+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:15.846928+0000 front 2024-09-16T02:07:15.847025+0000 (oldest deadline 2024-09-16T02:07:39.346979+0000) 2024-09-16T02:07:49.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:49 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:49.484+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:49.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:49 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:49.484+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:07:49.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:49 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:49.484+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:07:49.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:49 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:49.484+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.570558+0000 front 2024-09-16T02:07:17.570576+0000 (oldest deadline 2024-09-16T02:07:41.070561+0000) 2024-09-16T02:07:49.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:49 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:49.484+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.471846+0000 front 2024-09-16T02:07:24.471866+0000 (oldest deadline 2024-09-16T02:07:46.771852+0000) 2024-09-16T02:07:50.092 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:49 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:49.696+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:50.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:49 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:49.696+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:07:50.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:49 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:49.696+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:07:50.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:49 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:49.696+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.065430+0000 front 2024-09-16T02:07:24.065531+0000 (oldest deadline 2024-09-16T02:07:48.165250+0000) 2024-09-16T02:07:50.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:50 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:50.056+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:06:59.698219+0000 front 2024-09-16T02:06:59.698240+0000 (oldest deadline 2024-09-16T02:07:24.398448+0000) 2024-09-16T02:07:50.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:50 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:50.056+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.099524+0000 front 2024-09-16T02:07:06.099616+0000 (oldest deadline 2024-09-16T02:07:31.399445+0000) 2024-09-16T02:07:50.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:50 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:50.056+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:15.802100+0000 front 2024-09-16T02:07:15.802150+0000 (oldest deadline 2024-09-16T02:07:42.802526+0000) 2024-09-16T02:07:50.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:50 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7[52002]: 2024-09-16T02:07:50.056+0000 7fcd3a3a3640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:22.802834+0000 front 2024-09-16T02:07:22.802866+0000 (oldest deadline 2024-09-16T02:07:48.702982+0000) 2024-09-16T02:07:50.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:50 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:50.166+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:50.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:50 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:50.166+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:07:50.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:50 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:50.166+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.948571+0000 front 2024-09-16T02:07:09.948470+0000 (oldest deadline 2024-09-16T02:07:35.248301+0000) 2024-09-16T02:07:50.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:50 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:50.166+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.449541+0000 front 2024-09-16T02:07:17.449605+0000 (oldest deadline 2024-09-16T02:07:39.749594+0000) 2024-09-16T02:07:50.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:50 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:50.166+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:23.850643+0000 front 2024-09-16T02:07:23.850510+0000 (oldest deadline 2024-09-16T02:07:49.150405+0000) 2024-09-16T02:07:50.593 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:50 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:50.186+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:50.593 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:50 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:50.186+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:07.645729+0000 front 2024-09-16T02:07:07.645761+0000 (oldest deadline 2024-09-16T02:07:31.145992+0000) 2024-09-16T02:07:50.593 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:50 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:50.186+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:15.846928+0000 front 2024-09-16T02:07:15.847025+0000 (oldest deadline 2024-09-16T02:07:39.346979+0000) 2024-09-16T02:07:50.846 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:50 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:50.525+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:50.846 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:50 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:50.525+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:07:50.846 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:50 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:50.525+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:07:50.846 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:50 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:50.525+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.570558+0000 front 2024-09-16T02:07:17.570576+0000 (oldest deadline 2024-09-16T02:07:41.070561+0000) 2024-09-16T02:07:50.846 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:50 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:50.525+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.471846+0000 front 2024-09-16T02:07:24.471866+0000 (oldest deadline 2024-09-16T02:07:46.771852+0000) 2024-09-16T02:07:51.092 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:50 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:50.717+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:51.092 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:50 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:50.718+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:07:51.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:50 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:50.718+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:07:51.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:50 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:50.718+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.065430+0000 front 2024-09-16T02:07:24.065531+0000 (oldest deadline 2024-09-16T02:07:48.165250+0000) 2024-09-16T02:07:51.096 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:50 smithi137 podman[67746]: 2024-09-16 02:07:50.845213299 +0000 UTC m=+5.640249725 container died 714626cf7745a7b98c9869e1a47f10fde27b61a4c480059a294daf38a3a0340e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD) 2024-09-16T02:07:51.380 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:51 smithi137 podman[67746]: 2024-09-16 02:07:51.186859672 +0000 UTC m=+5.981896097 container cleanup 714626cf7745a7b98c9869e1a47f10fde27b61a4c480059a294daf38a3a0340e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T02:07:51.381 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:51 smithi137 bash[67746]: ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7 2024-09-16T02:07:51.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:51 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:51.165+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:51.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:51 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:51.165+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:07:51.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:51 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:51.165+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.948571+0000 front 2024-09-16T02:07:09.948470+0000 (oldest deadline 2024-09-16T02:07:35.248301+0000) 2024-09-16T02:07:51.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:51 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:51.165+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.449541+0000 front 2024-09-16T02:07:17.449605+0000 (oldest deadline 2024-09-16T02:07:39.749594+0000) 2024-09-16T02:07:51.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:51 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:51.165+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:23.850643+0000 front 2024-09-16T02:07:23.850510+0000 (oldest deadline 2024-09-16T02:07:49.150405+0000) 2024-09-16T02:07:51.592 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:51 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:51.150+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:51.592 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:51 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:51.150+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:07.645729+0000 front 2024-09-16T02:07:07.645761+0000 (oldest deadline 2024-09-16T02:07:31.145992+0000) 2024-09-16T02:07:51.593 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:51 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:51.150+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:15.846928+0000 front 2024-09-16T02:07:15.847025+0000 (oldest deadline 2024-09-16T02:07:39.346979+0000) 2024-09-16T02:07:51.671 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:51 smithi137 podman[67762]: 2024-09-16 02:07:51.380091144 +0000 UTC m=+0.528825151 container remove 714626cf7745a7b98c9869e1a47f10fde27b61a4c480059a294daf38a3a0340e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T02:07:51.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:51 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:51.478+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:51.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:51 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:51.478+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:07:51.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:51 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:51.478+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:07:51.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:51 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:51.478+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.570558+0000 front 2024-09-16T02:07:17.570576+0000 (oldest deadline 2024-09-16T02:07:41.070561+0000) 2024-09-16T02:07:51.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:51 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:51.478+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.471846+0000 front 2024-09-16T02:07:24.471866+0000 (oldest deadline 2024-09-16T02:07:46.771852+0000) 2024-09-16T02:07:52.050 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:51 smithi137 podman[67853]: 2024-09-16 02:07:51.699394073 +0000 UTC m=+0.015840409 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:07:52.050 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:51 smithi137 podman[67853]: 2024-09-16 02:07:51.82250375 +0000 UTC m=+0.138950126 container create 33de13a2049eb6db7ebded02f3f0e69558238c0a93a8c5c976aae11081b4ed31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2) 2024-09-16T02:07:52.051 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:52 smithi137 podman[67853]: 2024-09-16 02:07:52.049970076 +0000 UTC m=+0.366416407 container init 33de13a2049eb6db7ebded02f3f0e69558238c0a93a8c5c976aae11081b4ed31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7-deactivate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, ceph=True) 2024-09-16T02:07:52.092 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:51 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:51.748+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:52.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:51 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:51.748+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:07:52.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:51 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:51.748+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:07:52.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:51 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:51.748+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.065430+0000 front 2024-09-16T02:07:24.065531+0000 (oldest deadline 2024-09-16T02:07:48.165250+0000) 2024-09-16T02:07:52.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:52 smithi137 podman[67853]: 2024-09-16 02:07:52.053550608 +0000 UTC m=+0.369996939 container start 33de13a2049eb6db7ebded02f3f0e69558238c0a93a8c5c976aae11081b4ed31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731) 2024-09-16T02:07:52.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:52 smithi137 podman[67853]: 2024-09-16 02:07:52.149822187 +0000 UTC m=+0.466268524 container attach 33de13a2049eb6db7ebded02f3f0e69558238c0a93a8c5c976aae11081b4ed31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-16T02:07:52.421 INFO:journalctl@ceph.osd.7.smithi137.stdout:Sep 16 02:07:52 smithi137 podman[67853]: 2024-09-16 02:07:52.241491167 +0000 UTC m=+0.557937497 container died 33de13a2049eb6db7ebded02f3f0e69558238c0a93a8c5c976aae11081b4ed31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-8921c8e, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-16T02:07:52.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:52 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:52.188+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:52.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:52 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:52.188+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:07:52.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:52 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:52.188+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.948571+0000 front 2024-09-16T02:07:09.948470+0000 (oldest deadline 2024-09-16T02:07:35.248301+0000) 2024-09-16T02:07:52.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:52 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:52.188+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.449541+0000 front 2024-09-16T02:07:17.449605+0000 (oldest deadline 2024-09-16T02:07:39.749594+0000) 2024-09-16T02:07:52.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:52 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:52.188+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:23.850643+0000 front 2024-09-16T02:07:23.850510+0000 (oldest deadline 2024-09-16T02:07:49.150405+0000) 2024-09-16T02:07:52.592 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:52 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:52.120+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:52.593 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:52 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:52.120+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:07.645729+0000 front 2024-09-16T02:07:07.645761+0000 (oldest deadline 2024-09-16T02:07:31.145992+0000) 2024-09-16T02:07:52.593 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:52 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:52.120+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:15.846928+0000 front 2024-09-16T02:07:15.847025+0000 (oldest deadline 2024-09-16T02:07:39.346979+0000) 2024-09-16T02:07:52.593 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:52 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:52.120+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:25.148535+0000 front 2024-09-16T02:07:25.148677+0000 (oldest deadline 2024-09-16T02:07:51.548956+0000) 2024-09-16T02:07:52.829 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:52 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:52.452+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:52.829 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:52 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:52.452+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:07:52.829 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:52 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:52.452+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:07:52.829 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:52 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:52.452+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.570558+0000 front 2024-09-16T02:07:17.570576+0000 (oldest deadline 2024-09-16T02:07:41.070561+0000) 2024-09-16T02:07:52.829 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:52 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:52.452+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.471846+0000 front 2024-09-16T02:07:24.471866+0000 (oldest deadline 2024-09-16T02:07:46.771852+0000) 2024-09-16T02:07:52.852 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@osd.7.service' 2024-09-16T02:07:52.890 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:07:52.890 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-16T02:07:52.890 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-16T02:07:52.891 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@osd.8 2024-09-16T02:07:53.089 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:52 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:52.711+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:53.089 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:52 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:52.711+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:07:53.089 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:52 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:52.711+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:07:53.089 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:52 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:52.711+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.065430+0000 front 2024-09-16T02:07:24.065531+0000 (oldest deadline 2024-09-16T02:07:48.165250+0000) 2024-09-16T02:07:53.089 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:52 smithi190 systemd[1]: Stopping Ceph osd.8 for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:07:53.089 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:53 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:53.087+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:53.090 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:53 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:53.087+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:07.645729+0000 front 2024-09-16T02:07:07.645761+0000 (oldest deadline 2024-09-16T02:07:31.145992+0000) 2024-09-16T02:07:53.090 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:53 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:53.087+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:15.846928+0000 front 2024-09-16T02:07:15.847025+0000 (oldest deadline 2024-09-16T02:07:39.346979+0000) 2024-09-16T02:07:53.090 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:53 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:53.087+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:25.148535+0000 front 2024-09-16T02:07:25.148677+0000 (oldest deadline 2024-09-16T02:07:51.548956+0000) 2024-09-16T02:07:53.342 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:53 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:53.196+0000 7fada80f9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:07:53.342 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:53 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:53.196+0000 7fada80f9640 -1 osd.8 86 *** Got signal Terminated *** 2024-09-16T02:07:53.342 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:53 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:53.196+0000 7fada80f9640 -1 osd.8 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T02:07:53.544 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:53 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:53.168+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:53.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:53 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:53.168+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:07:53.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:53 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:53.168+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.948571+0000 front 2024-09-16T02:07:09.948470+0000 (oldest deadline 2024-09-16T02:07:35.248301+0000) 2024-09-16T02:07:53.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:53 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:53.168+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.449541+0000 front 2024-09-16T02:07:17.449605+0000 (oldest deadline 2024-09-16T02:07:39.749594+0000) 2024-09-16T02:07:53.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:53 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:53.168+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:23.850643+0000 front 2024-09-16T02:07:23.850510+0000 (oldest deadline 2024-09-16T02:07:49.150405+0000) 2024-09-16T02:07:53.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:53 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:53.468+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:53.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:53 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:53.468+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:07:53.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:53 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:53.468+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:07:53.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:53 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:53.468+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.570558+0000 front 2024-09-16T02:07:17.570576+0000 (oldest deadline 2024-09-16T02:07:41.070561+0000) 2024-09-16T02:07:53.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:53 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:53.468+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.471846+0000 front 2024-09-16T02:07:24.471866+0000 (oldest deadline 2024-09-16T02:07:46.771852+0000) 2024-09-16T02:07:54.092 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:53 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:53.696+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:54.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:53 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:53.696+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:07:54.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:53 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:53.696+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:07:54.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:53 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:53.696+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.065430+0000 front 2024-09-16T02:07:24.065531+0000 (oldest deadline 2024-09-16T02:07:48.165250+0000) 2024-09-16T02:07:54.544 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:54 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:54.206+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:54.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:54 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:54.206+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:07:54.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:54 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:54.206+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.948571+0000 front 2024-09-16T02:07:09.948470+0000 (oldest deadline 2024-09-16T02:07:35.248301+0000) 2024-09-16T02:07:54.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:54 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:54.206+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.449541+0000 front 2024-09-16T02:07:17.449605+0000 (oldest deadline 2024-09-16T02:07:39.749594+0000) 2024-09-16T02:07:54.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:54 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:54.206+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:23.850643+0000 front 2024-09-16T02:07:23.850510+0000 (oldest deadline 2024-09-16T02:07:49.150405+0000) 2024-09-16T02:07:54.592 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:54 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:54.118+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:54.593 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:54 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:54.118+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:07.645729+0000 front 2024-09-16T02:07:07.645761+0000 (oldest deadline 2024-09-16T02:07:31.145992+0000) 2024-09-16T02:07:54.593 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:54 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:54.118+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:15.846928+0000 front 2024-09-16T02:07:15.847025+0000 (oldest deadline 2024-09-16T02:07:39.346979+0000) 2024-09-16T02:07:54.593 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:54 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:54.118+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:25.148535+0000 front 2024-09-16T02:07:25.148677+0000 (oldest deadline 2024-09-16T02:07:51.548956+0000) 2024-09-16T02:07:54.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:54 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:54.450+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:54.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:54 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:54.450+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:07:54.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:54 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:54.450+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:07:54.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:54 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:54.450+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.570558+0000 front 2024-09-16T02:07:17.570576+0000 (oldest deadline 2024-09-16T02:07:41.070561+0000) 2024-09-16T02:07:54.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:54 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:54.450+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.471846+0000 front 2024-09-16T02:07:24.471866+0000 (oldest deadline 2024-09-16T02:07:46.771852+0000) 2024-09-16T02:07:55.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:54 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:54.687+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:55.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:54 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:54.687+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:07:55.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:54 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:54.687+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:07:55.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:54 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:54.687+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.065430+0000 front 2024-09-16T02:07:24.065531+0000 (oldest deadline 2024-09-16T02:07:48.165250+0000) 2024-09-16T02:07:55.544 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:55 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:55.229+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:55.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:55 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:55.229+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:07:55.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:55 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:55.229+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.948571+0000 front 2024-09-16T02:07:09.948470+0000 (oldest deadline 2024-09-16T02:07:35.248301+0000) 2024-09-16T02:07:55.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:55 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:55.229+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.449541+0000 front 2024-09-16T02:07:17.449605+0000 (oldest deadline 2024-09-16T02:07:39.749594+0000) 2024-09-16T02:07:55.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:55 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:55.229+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:23.850643+0000 front 2024-09-16T02:07:23.850510+0000 (oldest deadline 2024-09-16T02:07:49.150405+0000) 2024-09-16T02:07:55.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:55 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:55.229+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:33.751859+0000 front 2024-09-16T02:07:33.751963+0000 (oldest deadline 2024-09-16T02:07:54.851785+0000) 2024-09-16T02:07:55.593 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:55 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:55.104+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:55.593 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:55 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:55.104+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:07.645729+0000 front 2024-09-16T02:07:07.645761+0000 (oldest deadline 2024-09-16T02:07:31.145992+0000) 2024-09-16T02:07:55.593 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:55 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:55.104+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:15.846928+0000 front 2024-09-16T02:07:15.847025+0000 (oldest deadline 2024-09-16T02:07:39.346979+0000) 2024-09-16T02:07:55.593 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:55 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:55.104+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:25.148535+0000 front 2024-09-16T02:07:25.148677+0000 (oldest deadline 2024-09-16T02:07:51.548956+0000) 2024-09-16T02:07:55.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:55 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:55.419+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:55.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:55 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:55.419+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:07:55.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:55 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:55.419+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:07:55.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:55 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:55.419+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.570558+0000 front 2024-09-16T02:07:17.570576+0000 (oldest deadline 2024-09-16T02:07:41.070561+0000) 2024-09-16T02:07:55.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:55 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:55.419+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.471846+0000 front 2024-09-16T02:07:24.471866+0000 (oldest deadline 2024-09-16T02:07:46.771852+0000) 2024-09-16T02:07:56.092 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:55 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:55.734+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:56.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:55 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:55.734+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:07:56.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:55 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:55.734+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:07:56.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:55 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:55.734+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.065430+0000 front 2024-09-16T02:07:24.065531+0000 (oldest deadline 2024-09-16T02:07:48.165250+0000) 2024-09-16T02:07:56.360 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:56 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:56.098+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:56.361 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:56 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:56.098+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:07.645729+0000 front 2024-09-16T02:07:07.645761+0000 (oldest deadline 2024-09-16T02:07:31.145992+0000) 2024-09-16T02:07:56.361 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:56 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:56.098+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:15.846928+0000 front 2024-09-16T02:07:15.847025+0000 (oldest deadline 2024-09-16T02:07:39.346979+0000) 2024-09-16T02:07:56.361 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:56 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:56.098+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:25.148535+0000 front 2024-09-16T02:07:25.148677+0000 (oldest deadline 2024-09-16T02:07:51.548956+0000) 2024-09-16T02:07:56.544 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:56 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:56.204+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:56.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:56 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:56.204+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:07:56.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:56 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:56.204+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.948571+0000 front 2024-09-16T02:07:09.948470+0000 (oldest deadline 2024-09-16T02:07:35.248301+0000) 2024-09-16T02:07:56.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:56 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:56.204+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.449541+0000 front 2024-09-16T02:07:17.449605+0000 (oldest deadline 2024-09-16T02:07:39.749594+0000) 2024-09-16T02:07:56.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:56 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:56.204+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:23.850643+0000 front 2024-09-16T02:07:23.850510+0000 (oldest deadline 2024-09-16T02:07:49.150405+0000) 2024-09-16T02:07:56.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:56 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:56.204+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:33.751859+0000 front 2024-09-16T02:07:33.751963+0000 (oldest deadline 2024-09-16T02:07:54.851785+0000) 2024-09-16T02:07:56.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:56 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:56.412+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:56.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:56 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:56.412+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:07:56.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:56 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:56.412+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:07:56.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:56 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:56.412+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.570558+0000 front 2024-09-16T02:07:17.570576+0000 (oldest deadline 2024-09-16T02:07:41.070561+0000) 2024-09-16T02:07:56.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:56 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:56.412+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.471846+0000 front 2024-09-16T02:07:24.471866+0000 (oldest deadline 2024-09-16T02:07:46.771852+0000) 2024-09-16T02:07:56.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:56 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:56.412+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:32.573523+0000 front 2024-09-16T02:07:32.573580+0000 (oldest deadline 2024-09-16T02:07:55.473409+0000) 2024-09-16T02:07:57.073 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:56 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:56.778+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:57.073 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:56 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:56.778+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:07:57.073 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:56 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:56.778+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:07:57.073 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:56 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:56.778+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.065430+0000 front 2024-09-16T02:07:24.065531+0000 (oldest deadline 2024-09-16T02:07:48.165250+0000) 2024-09-16T02:07:57.329 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:57 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:57.071+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:57.329 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:57 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:57.071+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:07.645729+0000 front 2024-09-16T02:07:07.645761+0000 (oldest deadline 2024-09-16T02:07:31.145992+0000) 2024-09-16T02:07:57.330 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:57 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:57.071+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:15.846928+0000 front 2024-09-16T02:07:15.847025+0000 (oldest deadline 2024-09-16T02:07:39.346979+0000) 2024-09-16T02:07:57.330 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:57 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:57.071+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:25.148535+0000 front 2024-09-16T02:07:25.148677+0000 (oldest deadline 2024-09-16T02:07:51.548956+0000) 2024-09-16T02:07:57.330 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:57 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:57.071+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:31.549325+0000 front 2024-09-16T02:07:31.549365+0000 (oldest deadline 2024-09-16T02:07:56.849522+0000) 2024-09-16T02:07:57.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:57 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:57.171+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:57.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:57 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:57.171+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:07:57.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:57 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:57.171+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.948571+0000 front 2024-09-16T02:07:09.948470+0000 (oldest deadline 2024-09-16T02:07:35.248301+0000) 2024-09-16T02:07:57.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:57 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:57.171+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.449541+0000 front 2024-09-16T02:07:17.449605+0000 (oldest deadline 2024-09-16T02:07:39.749594+0000) 2024-09-16T02:07:57.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:57 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:57.171+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:23.850643+0000 front 2024-09-16T02:07:23.850510+0000 (oldest deadline 2024-09-16T02:07:49.150405+0000) 2024-09-16T02:07:57.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:57 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:57.171+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:33.751859+0000 front 2024-09-16T02:07:33.751963+0000 (oldest deadline 2024-09-16T02:07:54.851785+0000) 2024-09-16T02:07:57.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:57 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:57.442+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:57.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:57 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:57.442+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:07:57.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:57 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:57.442+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:07:57.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:57 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:57.442+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.570558+0000 front 2024-09-16T02:07:17.570576+0000 (oldest deadline 2024-09-16T02:07:41.070561+0000) 2024-09-16T02:07:57.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:57 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:57.442+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.471846+0000 front 2024-09-16T02:07:24.471866+0000 (oldest deadline 2024-09-16T02:07:46.771852+0000) 2024-09-16T02:07:57.921 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:57 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:57.442+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:32.573523+0000 front 2024-09-16T02:07:32.573580+0000 (oldest deadline 2024-09-16T02:07:55.473409+0000) 2024-09-16T02:07:58.092 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:57 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:57.792+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:58.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:57 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:57.792+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:07:58.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:57 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:57.792+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:07:58.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:57 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:57.792+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.065430+0000 front 2024-09-16T02:07:24.065531+0000 (oldest deadline 2024-09-16T02:07:48.165250+0000) 2024-09-16T02:07:58.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:58 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:58.121+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:58.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:58 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:58.121+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:07:58.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:58 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:58.121+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.948571+0000 front 2024-09-16T02:07:09.948470+0000 (oldest deadline 2024-09-16T02:07:35.248301+0000) 2024-09-16T02:07:58.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:58 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:58.121+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.449541+0000 front 2024-09-16T02:07:17.449605+0000 (oldest deadline 2024-09-16T02:07:39.749594+0000) 2024-09-16T02:07:58.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:58 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:58.121+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:23.850643+0000 front 2024-09-16T02:07:23.850510+0000 (oldest deadline 2024-09-16T02:07:49.150405+0000) 2024-09-16T02:07:58.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:58 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:58.121+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:33.751859+0000 front 2024-09-16T02:07:33.751963+0000 (oldest deadline 2024-09-16T02:07:54.851785+0000) 2024-09-16T02:07:58.593 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:58 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:58.103+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.844259+0000 front 2024-09-16T02:06:51.844474+0000 (oldest deadline 2024-09-16T02:07:15.944212+0000) 2024-09-16T02:07:58.593 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:58 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:58.103+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:07.645729+0000 front 2024-09-16T02:07:07.645761+0000 (oldest deadline 2024-09-16T02:07:31.145992+0000) 2024-09-16T02:07:58.593 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:58 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:58.103+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:15.846928+0000 front 2024-09-16T02:07:15.847025+0000 (oldest deadline 2024-09-16T02:07:39.346979+0000) 2024-09-16T02:07:58.593 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:58 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:58.103+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:25.148535+0000 front 2024-09-16T02:07:25.148677+0000 (oldest deadline 2024-09-16T02:07:51.548956+0000) 2024-09-16T02:07:58.593 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:58 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8[52577]: 2024-09-16T02:07:58.103+0000 7fada470f640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:31.549325+0000 front 2024-09-16T02:07:31.549365+0000 (oldest deadline 2024-09-16T02:07:56.849522+0000) 2024-09-16T02:07:58.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:58 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:58.416+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:58.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:58 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:58.416+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:07:58.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:58 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:58.416+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:07:58.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:58 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:58.416+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.570558+0000 front 2024-09-16T02:07:17.570576+0000 (oldest deadline 2024-09-16T02:07:41.070561+0000) 2024-09-16T02:07:58.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:58 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:58.416+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.471846+0000 front 2024-09-16T02:07:24.471866+0000 (oldest deadline 2024-09-16T02:07:46.771852+0000) 2024-09-16T02:07:58.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:58 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:58.416+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:32.573523+0000 front 2024-09-16T02:07:32.573580+0000 (oldest deadline 2024-09-16T02:07:55.473409+0000) 2024-09-16T02:07:58.969 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:58 smithi190 podman[68255]: 2024-09-16 02:07:58.647680916 +0000 UTC m=+5.554116932 container died 0aa5ec2b62135a27fba0d7384932b8f5a09cb342388aa4c357d4a908cb74f114 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-16T02:07:58.970 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:58 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:58.790+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:07:58.970 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:58 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:58.790+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:07:58.970 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:58 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:58.790+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:07:58.970 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:58 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:58.790+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.065430+0000 front 2024-09-16T02:07:24.065531+0000 (oldest deadline 2024-09-16T02:07:48.165250+0000) 2024-09-16T02:07:58.970 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:58 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:58.790+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:33.366463+0000 front 2024-09-16T02:07:33.366510+0000 (oldest deadline 2024-09-16T02:07:58.666604+0000) 2024-09-16T02:07:59.299 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:59 smithi190 podman[68255]: 2024-09-16 02:07:59.072189233 +0000 UTC m=+5.978625294 container cleanup 0aa5ec2b62135a27fba0d7384932b8f5a09cb342388aa4c357d4a908cb74f114 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e) 2024-09-16T02:07:59.299 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:59 smithi190 bash[68255]: ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8 2024-09-16T02:07:59.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:59 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:59.171+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:07:59.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:59 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:59.171+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:07:59.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:59 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:59.171+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.948571+0000 front 2024-09-16T02:07:09.948470+0000 (oldest deadline 2024-09-16T02:07:35.248301+0000) 2024-09-16T02:07:59.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:59 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:59.171+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.449541+0000 front 2024-09-16T02:07:17.449605+0000 (oldest deadline 2024-09-16T02:07:39.749594+0000) 2024-09-16T02:07:59.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:59 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:59.171+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:23.850643+0000 front 2024-09-16T02:07:23.850510+0000 (oldest deadline 2024-09-16T02:07:49.150405+0000) 2024-09-16T02:07:59.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:07:59 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:07:59.171+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:33.751859+0000 front 2024-09-16T02:07:33.751963+0000 (oldest deadline 2024-09-16T02:07:54.851785+0000) 2024-09-16T02:07:59.592 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:59 smithi190 podman[68283]: 2024-09-16 02:07:59.298408718 +0000 UTC m=+0.651861108 container remove 0aa5ec2b62135a27fba0d7384932b8f5a09cb342388aa4c357d4a908cb74f114 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T02:07:59.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:59 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:59.394+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:07:59.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:59 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:59.394+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:07:59.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:59 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:59.394+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:07:59.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:59 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:59.394+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.570558+0000 front 2024-09-16T02:07:17.570576+0000 (oldest deadline 2024-09-16T02:07:41.070561+0000) 2024-09-16T02:07:59.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:59 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:59.394+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.471846+0000 front 2024-09-16T02:07:24.471866+0000 (oldest deadline 2024-09-16T02:07:46.771852+0000) 2024-09-16T02:07:59.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:07:59 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:07:59.394+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:32.573523+0000 front 2024-09-16T02:07:32.573580+0000 (oldest deadline 2024-09-16T02:07:55.473409+0000) 2024-09-16T02:08:00.012 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:59 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:59.830+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:08:00.012 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:59 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:59.830+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:08:00.012 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:59 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:59.830+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:08:00.012 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:59 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:59.830+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.065430+0000 front 2024-09-16T02:07:24.065531+0000 (oldest deadline 2024-09-16T02:07:48.165250+0000) 2024-09-16T02:08:00.012 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:07:59 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:07:59.830+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:33.366463+0000 front 2024-09-16T02:07:33.366510+0000 (oldest deadline 2024-09-16T02:07:58.666604+0000) 2024-09-16T02:08:00.013 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:59 smithi190 podman[68372]: 2024-09-16 02:07:59.64172863 +0000 UTC m=+0.015840606 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:08:00.013 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:07:59 smithi190 podman[68372]: 2024-09-16 02:07:59.765777267 +0000 UTC m=+0.139889237 container create 0c571b60171245f71157295fbfcd4a78e987c5497f32f5708ee378134c676dd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0) 2024-09-16T02:08:00.013 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:08:00 smithi190 podman[68372]: 2024-09-16 02:08:00.011093874 +0000 UTC m=+0.385205858 container init 0c571b60171245f71157295fbfcd4a78e987c5497f32f5708ee378134c676dd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e) 2024-09-16T02:08:00.342 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:08:00 smithi190 podman[68372]: 2024-09-16 02:08:00.014880819 +0000 UTC m=+0.388992785 container start 0c571b60171245f71157295fbfcd4a78e987c5497f32f5708ee378134c676dd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-16T02:08:00.343 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:08:00 smithi190 podman[68372]: 2024-09-16 02:08:00.118444033 +0000 UTC m=+0.492556001 container attach 0c571b60171245f71157295fbfcd4a78e987c5497f32f5708ee378134c676dd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-16T02:08:00.343 INFO:journalctl@ceph.osd.8.smithi190.stdout:Sep 16 02:08:00 smithi190 podman[68372]: 2024-09-16 02:08:00.209934413 +0000 UTC m=+0.584046382 container died 0c571b60171245f71157295fbfcd4a78e987c5497f32f5708ee378134c676dd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-8-deactivate, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-16T02:08:00.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:00 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:00.126+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:08:00.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:00 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:00.126+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:08:00.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:00 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:00.126+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.948571+0000 front 2024-09-16T02:07:09.948470+0000 (oldest deadline 2024-09-16T02:07:35.248301+0000) 2024-09-16T02:08:00.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:00 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:00.126+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.449541+0000 front 2024-09-16T02:07:17.449605+0000 (oldest deadline 2024-09-16T02:07:39.749594+0000) 2024-09-16T02:08:00.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:00 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:00.126+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:23.850643+0000 front 2024-09-16T02:07:23.850510+0000 (oldest deadline 2024-09-16T02:07:49.150405+0000) 2024-09-16T02:08:00.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:00 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:00.126+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:33.751859+0000 front 2024-09-16T02:07:33.751963+0000 (oldest deadline 2024-09-16T02:07:54.851785+0000) 2024-09-16T02:08:00.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:00 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:00.365+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:08:00.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:00 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:00.365+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:08:00.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:00 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:00.365+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:08:00.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:00 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:00.365+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.570558+0000 front 2024-09-16T02:07:17.570576+0000 (oldest deadline 2024-09-16T02:07:41.070561+0000) 2024-09-16T02:08:00.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:00 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:00.365+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.471846+0000 front 2024-09-16T02:07:24.471866+0000 (oldest deadline 2024-09-16T02:07:46.771852+0000) 2024-09-16T02:08:00.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:00 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:00.365+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:32.573523+0000 front 2024-09-16T02:07:32.573580+0000 (oldest deadline 2024-09-16T02:07:55.473409+0000) 2024-09-16T02:08:00.874 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@osd.8.service' 2024-09-16T02:08:00.914 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:08:00.914 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-16T02:08:00.915 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-16T02:08:00.915 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@osd.9 2024-09-16T02:08:01.273 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:00 smithi105 systemd[1]: Stopping Ceph osd.9 for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:08:01.273 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:01 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:01.102+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:08:01.273 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:01 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:01.102+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:08:01.274 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:01 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:01.102+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.948571+0000 front 2024-09-16T02:07:09.948470+0000 (oldest deadline 2024-09-16T02:07:35.248301+0000) 2024-09-16T02:08:01.274 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:01 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:01.102+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.449541+0000 front 2024-09-16T02:07:17.449605+0000 (oldest deadline 2024-09-16T02:07:39.749594+0000) 2024-09-16T02:08:01.274 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:01 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:01.102+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:23.850643+0000 front 2024-09-16T02:07:23.850510+0000 (oldest deadline 2024-09-16T02:07:49.150405+0000) 2024-09-16T02:08:01.274 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:01 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:01.102+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:33.751859+0000 front 2024-09-16T02:07:33.751963+0000 (oldest deadline 2024-09-16T02:07:54.851785+0000) 2024-09-16T02:08:01.274 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:01 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:01.271+0000 7f330c2de640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:08:01.274 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:01 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:01.271+0000 7f330c2de640 -1 osd.9 86 *** Got signal Terminated *** 2024-09-16T02:08:01.274 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:01 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:01.271+0000 7f330c2de640 -1 osd.9 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T02:08:01.342 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:00 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:00.860+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:08:01.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:00 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:00.860+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:08:01.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:00 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:00.860+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:08:01.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:00 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:00.860+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.065430+0000 front 2024-09-16T02:07:24.065531+0000 (oldest deadline 2024-09-16T02:07:48.165250+0000) 2024-09-16T02:08:01.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:00 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:00.860+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:33.366463+0000 front 2024-09-16T02:07:33.366510+0000 (oldest deadline 2024-09-16T02:07:58.666604+0000) 2024-09-16T02:08:01.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:01 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:01.338+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:08:01.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:01 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:01.338+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:08:01.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:01 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:01.338+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:08:01.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:01 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:01.338+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.570558+0000 front 2024-09-16T02:07:17.570576+0000 (oldest deadline 2024-09-16T02:07:41.070561+0000) 2024-09-16T02:08:01.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:01 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:01.338+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.471846+0000 front 2024-09-16T02:07:24.471866+0000 (oldest deadline 2024-09-16T02:07:46.771852+0000) 2024-09-16T02:08:01.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:01 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:01.338+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:32.573523+0000 front 2024-09-16T02:07:32.573580+0000 (oldest deadline 2024-09-16T02:07:55.473409+0000) 2024-09-16T02:08:02.092 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:01 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:01.822+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:08:02.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:01 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:01.822+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:08:02.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:01 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:01.822+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:08:02.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:01 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:01.822+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.065430+0000 front 2024-09-16T02:07:24.065531+0000 (oldest deadline 2024-09-16T02:07:48.165250+0000) 2024-09-16T02:08:02.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:01 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:01.822+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:33.366463+0000 front 2024-09-16T02:07:33.366510+0000 (oldest deadline 2024-09-16T02:07:58.666604+0000) 2024-09-16T02:08:02.544 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:02 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:02.135+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:08:02.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:02 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:02.135+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:08:02.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:02 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:02.135+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.948571+0000 front 2024-09-16T02:07:09.948470+0000 (oldest deadline 2024-09-16T02:07:35.248301+0000) 2024-09-16T02:08:02.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:02 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:02.135+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.449541+0000 front 2024-09-16T02:07:17.449605+0000 (oldest deadline 2024-09-16T02:07:39.749594+0000) 2024-09-16T02:08:02.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:02 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:02.135+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:23.850643+0000 front 2024-09-16T02:07:23.850510+0000 (oldest deadline 2024-09-16T02:07:49.150405+0000) 2024-09-16T02:08:02.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:02 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:02.135+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:33.751859+0000 front 2024-09-16T02:07:33.751963+0000 (oldest deadline 2024-09-16T02:07:54.851785+0000) 2024-09-16T02:08:02.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:02 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:02.356+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:08:02.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:02 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:02.356+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:08:02.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:02 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:02.356+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:08:02.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:02 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:02.356+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.570558+0000 front 2024-09-16T02:07:17.570576+0000 (oldest deadline 2024-09-16T02:07:41.070561+0000) 2024-09-16T02:08:02.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:02 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:02.356+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.471846+0000 front 2024-09-16T02:07:24.471866+0000 (oldest deadline 2024-09-16T02:07:46.771852+0000) 2024-09-16T02:08:02.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:02 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:02.356+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:32.573523+0000 front 2024-09-16T02:07:32.573580+0000 (oldest deadline 2024-09-16T02:07:55.473409+0000) 2024-09-16T02:08:03.342 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:02 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:02.870+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:08:03.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:02 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:02.870+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:08:03.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:02 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:02.870+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:08:03.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:02 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:02.870+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.065430+0000 front 2024-09-16T02:07:24.065531+0000 (oldest deadline 2024-09-16T02:07:48.165250+0000) 2024-09-16T02:08:03.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:02 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:02.870+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:33.366463+0000 front 2024-09-16T02:07:33.366510+0000 (oldest deadline 2024-09-16T02:07:58.666604+0000) 2024-09-16T02:08:03.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:03 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:03.096+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:08:03.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:03 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:03.096+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:08:03.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:03 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:03.096+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.948571+0000 front 2024-09-16T02:07:09.948470+0000 (oldest deadline 2024-09-16T02:07:35.248301+0000) 2024-09-16T02:08:03.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:03 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:03.096+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.449541+0000 front 2024-09-16T02:07:17.449605+0000 (oldest deadline 2024-09-16T02:07:39.749594+0000) 2024-09-16T02:08:03.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:03 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:03.096+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:23.850643+0000 front 2024-09-16T02:07:23.850510+0000 (oldest deadline 2024-09-16T02:07:49.150405+0000) 2024-09-16T02:08:03.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:03 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:03.096+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:33.751859+0000 front 2024-09-16T02:07:33.751963+0000 (oldest deadline 2024-09-16T02:07:54.851785+0000) 2024-09-16T02:08:03.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:03 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:03.354+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:08:03.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:03 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:03.354+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:08:03.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:03 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:03.354+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:08:03.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:03 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:03.354+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.570558+0000 front 2024-09-16T02:07:17.570576+0000 (oldest deadline 2024-09-16T02:07:41.070561+0000) 2024-09-16T02:08:03.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:03 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:03.354+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.471846+0000 front 2024-09-16T02:07:24.471866+0000 (oldest deadline 2024-09-16T02:07:46.771852+0000) 2024-09-16T02:08:03.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:03 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:03.354+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:32.573523+0000 front 2024-09-16T02:07:32.573580+0000 (oldest deadline 2024-09-16T02:07:55.473409+0000) 2024-09-16T02:08:03.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:03 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:03.354+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-09-16T02:07:40.675134+0000 front 2024-09-16T02:07:40.675181+0000 (oldest deadline 2024-09-16T02:08:02.974962+0000) 2024-09-16T02:08:04.342 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:03 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:03.894+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:08:04.342 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:03 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:03.894+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:08:04.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:03 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:03.894+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:08:04.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:03 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:03.894+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.065430+0000 front 2024-09-16T02:07:24.065531+0000 (oldest deadline 2024-09-16T02:07:48.165250+0000) 2024-09-16T02:08:04.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:03 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:03.894+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:33.366463+0000 front 2024-09-16T02:07:33.366510+0000 (oldest deadline 2024-09-16T02:07:58.666604+0000) 2024-09-16T02:08:04.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:04 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:04.050+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:08:04.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:04 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:04.050+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:08:04.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:04 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:04.050+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.948571+0000 front 2024-09-16T02:07:09.948470+0000 (oldest deadline 2024-09-16T02:07:35.248301+0000) 2024-09-16T02:08:04.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:04 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:04.050+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.449541+0000 front 2024-09-16T02:07:17.449605+0000 (oldest deadline 2024-09-16T02:07:39.749594+0000) 2024-09-16T02:08:04.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:04 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:04.050+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:23.850643+0000 front 2024-09-16T02:07:23.850510+0000 (oldest deadline 2024-09-16T02:07:49.150405+0000) 2024-09-16T02:08:04.546 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:04 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:04.050+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:33.751859+0000 front 2024-09-16T02:07:33.751963+0000 (oldest deadline 2024-09-16T02:07:54.851785+0000) 2024-09-16T02:08:04.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:04 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:04.333+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:08:04.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:04 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:04.333+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:08:04.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:04 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:04.333+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:08:04.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:04 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:04.333+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.570558+0000 front 2024-09-16T02:07:17.570576+0000 (oldest deadline 2024-09-16T02:07:41.070561+0000) 2024-09-16T02:08:04.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:04 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:04.333+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.471846+0000 front 2024-09-16T02:07:24.471866+0000 (oldest deadline 2024-09-16T02:07:46.771852+0000) 2024-09-16T02:08:04.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:04 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:04.333+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:32.573523+0000 front 2024-09-16T02:07:32.573580+0000 (oldest deadline 2024-09-16T02:07:55.473409+0000) 2024-09-16T02:08:04.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:04 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:04.333+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-09-16T02:07:40.675134+0000 front 2024-09-16T02:07:40.675181+0000 (oldest deadline 2024-09-16T02:08:02.974962+0000) 2024-09-16T02:08:05.294 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:05 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:05.016+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:08:05.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:05 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:05.016+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:08:05.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:05 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:05.016+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.948571+0000 front 2024-09-16T02:07:09.948470+0000 (oldest deadline 2024-09-16T02:07:35.248301+0000) 2024-09-16T02:08:05.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:05 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:05.016+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.449541+0000 front 2024-09-16T02:07:17.449605+0000 (oldest deadline 2024-09-16T02:07:39.749594+0000) 2024-09-16T02:08:05.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:05 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:05.016+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:23.850643+0000 front 2024-09-16T02:07:23.850510+0000 (oldest deadline 2024-09-16T02:07:49.150405+0000) 2024-09-16T02:08:05.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:05 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:05.016+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:33.751859+0000 front 2024-09-16T02:07:33.751963+0000 (oldest deadline 2024-09-16T02:07:54.851785+0000) 2024-09-16T02:08:05.342 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:04 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:04.869+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:08:05.342 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:04 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:04.869+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:08:05.342 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:04 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:04.869+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:08:05.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:04 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:04.869+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.065430+0000 front 2024-09-16T02:07:24.065531+0000 (oldest deadline 2024-09-16T02:07:48.165250+0000) 2024-09-16T02:08:05.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:04 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:04.869+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:33.366463+0000 front 2024-09-16T02:07:33.366510+0000 (oldest deadline 2024-09-16T02:07:58.666604+0000) 2024-09-16T02:08:05.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:04 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:04.869+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-09-16T02:07:40.268148+0000 front 2024-09-16T02:07:40.268206+0000 (oldest deadline 2024-09-16T02:08:04.368006+0000) 2024-09-16T02:08:05.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:05 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:05.328+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:08:05.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:05 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:05.328+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:08:05.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:05 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:05.328+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:08:05.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:05 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:05.328+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.570558+0000 front 2024-09-16T02:07:17.570576+0000 (oldest deadline 2024-09-16T02:07:41.070561+0000) 2024-09-16T02:08:05.672 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:05 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:05.328+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.471846+0000 front 2024-09-16T02:07:24.471866+0000 (oldest deadline 2024-09-16T02:07:46.771852+0000) 2024-09-16T02:08:05.672 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:05 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:05.328+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:32.573523+0000 front 2024-09-16T02:07:32.573580+0000 (oldest deadline 2024-09-16T02:07:55.473409+0000) 2024-09-16T02:08:05.672 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:05 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:05.328+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-09-16T02:07:40.675134+0000 front 2024-09-16T02:07:40.675181+0000 (oldest deadline 2024-09-16T02:08:02.974962+0000) 2024-09-16T02:08:06.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:06 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:06.042+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:50.745039+0000 front 2024-09-16T02:06:50.745163+0000 (oldest deadline 2024-09-16T02:07:14.845182+0000) 2024-09-16T02:08:06.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:06 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:06.042+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.046602+0000 front 2024-09-16T02:07:00.046711+0000 (oldest deadline 2024-09-16T02:07:24.746680+0000) 2024-09-16T02:08:06.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:06 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:06.042+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.948571+0000 front 2024-09-16T02:07:09.948470+0000 (oldest deadline 2024-09-16T02:07:35.248301+0000) 2024-09-16T02:08:06.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:06 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:06.042+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.449541+0000 front 2024-09-16T02:07:17.449605+0000 (oldest deadline 2024-09-16T02:07:39.749594+0000) 2024-09-16T02:08:06.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:06 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:06.042+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:23.850643+0000 front 2024-09-16T02:07:23.850510+0000 (oldest deadline 2024-09-16T02:07:49.150405+0000) 2024-09-16T02:08:06.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:06 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:06.042+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:33.751859+0000 front 2024-09-16T02:07:33.751963+0000 (oldest deadline 2024-09-16T02:07:54.851785+0000) 2024-09-16T02:08:06.295 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:06 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9[64957]: 2024-09-16T02:08:06.042+0000 7f33048ec640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-09-16T02:07:40.653228+0000 front 2024-09-16T02:07:40.653202+0000 (oldest deadline 2024-09-16T02:08:05.353352+0000) 2024-09-16T02:08:06.342 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:05 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:05.901+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:08:06.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:05 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:05.901+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:08:06.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:05 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:05.901+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:08:06.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:05 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:05.901+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.065430+0000 front 2024-09-16T02:07:24.065531+0000 (oldest deadline 2024-09-16T02:07:48.165250+0000) 2024-09-16T02:08:06.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:05 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:05.901+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:33.366463+0000 front 2024-09-16T02:07:33.366510+0000 (oldest deadline 2024-09-16T02:07:58.666604+0000) 2024-09-16T02:08:06.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:05 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:05.901+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-09-16T02:07:40.268148+0000 front 2024-09-16T02:07:40.268206+0000 (oldest deadline 2024-09-16T02:08:04.368006+0000) 2024-09-16T02:08:06.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:06 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:06.366+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:08:06.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:06 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:06.366+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:08:06.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:06 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:06.366+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:08:06.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:06 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:06.366+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.570558+0000 front 2024-09-16T02:07:17.570576+0000 (oldest deadline 2024-09-16T02:07:41.070561+0000) 2024-09-16T02:08:06.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:06 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:06.366+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.471846+0000 front 2024-09-16T02:07:24.471866+0000 (oldest deadline 2024-09-16T02:07:46.771852+0000) 2024-09-16T02:08:06.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:06 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:06.366+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:32.573523+0000 front 2024-09-16T02:07:32.573580+0000 (oldest deadline 2024-09-16T02:07:55.473409+0000) 2024-09-16T02:08:06.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:06 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:06.366+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-09-16T02:07:40.675134+0000 front 2024-09-16T02:07:40.675181+0000 (oldest deadline 2024-09-16T02:08:02.974962+0000) 2024-09-16T02:08:07.081 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:06 smithi105 podman[93958]: 2024-09-16 02:08:06.80755427 +0000 UTC m=+5.701543061 container died 8e431fc270a4000b7e07bbef4cfabdf81dd6c064d9b4be528330428a6f0d59f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-16T02:08:07.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:06 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:06.903+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:08:07.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:06 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:06.903+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:08:07.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:06 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:06.903+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:08:07.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:06 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:06.903+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.065430+0000 front 2024-09-16T02:07:24.065531+0000 (oldest deadline 2024-09-16T02:07:48.165250+0000) 2024-09-16T02:08:07.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:06 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:06.903+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:33.366463+0000 front 2024-09-16T02:07:33.366510+0000 (oldest deadline 2024-09-16T02:07:58.666604+0000) 2024-09-16T02:08:07.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:06 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:06.903+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-09-16T02:07:40.268148+0000 front 2024-09-16T02:07:40.268206+0000 (oldest deadline 2024-09-16T02:08:04.368006+0000) 2024-09-16T02:08:07.544 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:07 smithi105 podman[93958]: 2024-09-16 02:08:07.255241192 +0000 UTC m=+6.149230014 container cleanup 8e431fc270a4000b7e07bbef4cfabdf81dd6c064d9b4be528330428a6f0d59f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD) 2024-09-16T02:08:07.545 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:07 smithi105 bash[93958]: ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9 2024-09-16T02:08:07.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:07 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:07.339+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:08:07.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:07 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:07.339+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:08:07.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:07 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:07.339+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:08:07.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:07 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:07.339+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.570558+0000 front 2024-09-16T02:07:17.570576+0000 (oldest deadline 2024-09-16T02:07:41.070561+0000) 2024-09-16T02:08:07.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:07 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:07.339+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.471846+0000 front 2024-09-16T02:07:24.471866+0000 (oldest deadline 2024-09-16T02:07:46.771852+0000) 2024-09-16T02:08:07.672 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:07 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:07.339+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:32.573523+0000 front 2024-09-16T02:07:32.573580+0000 (oldest deadline 2024-09-16T02:07:55.473409+0000) 2024-09-16T02:08:07.672 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:07 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:07.339+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-09-16T02:07:40.675134+0000 front 2024-09-16T02:07:40.675181+0000 (oldest deadline 2024-09-16T02:08:02.974962+0000) 2024-09-16T02:08:07.915 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:07 smithi105 podman[93974]: 2024-09-16 02:08:07.556630806 +0000 UTC m=+0.745247491 container remove 8e431fc270a4000b7e07bbef4cfabdf81dd6c064d9b4be528330428a6f0d59f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-16T02:08:08.202 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:07 smithi105 podman[94062]: 2024-09-16 02:08:07.817433845 +0000 UTC m=+0.015610476 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:08:08.203 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:07 smithi105 podman[94062]: 2024-09-16 02:08:07.924258608 +0000 UTC m=+0.122435281 container create b26894b7248a0bb8a8739d44600355a6611933a3ab56e918aea8134dabb9a5da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T02:08:08.203 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:08 smithi105 podman[94062]: 2024-09-16 02:08:08.20197139 +0000 UTC m=+0.400148021 container init b26894b7248a0bb8a8739d44600355a6611933a3ab56e918aea8134dabb9a5da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e) 2024-09-16T02:08:08.342 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:07 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:07.895+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:08:08.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:07 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:07.895+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:08:08.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:07 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:07.895+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:08:08.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:07 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:07.895+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.065430+0000 front 2024-09-16T02:07:24.065531+0000 (oldest deadline 2024-09-16T02:07:48.165250+0000) 2024-09-16T02:08:08.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:07 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:07.895+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:33.366463+0000 front 2024-09-16T02:07:33.366510+0000 (oldest deadline 2024-09-16T02:07:58.666604+0000) 2024-09-16T02:08:08.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:07 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:07.895+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-09-16T02:07:40.268148+0000 front 2024-09-16T02:07:40.268206+0000 (oldest deadline 2024-09-16T02:08:04.368006+0000) 2024-09-16T02:08:08.472 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:08 smithi105 podman[94062]: 2024-09-16 02:08:08.205605824 +0000 UTC m=+0.403782465 container start b26894b7248a0bb8a8739d44600355a6611933a3ab56e918aea8134dabb9a5da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-16T02:08:08.472 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:08 smithi105 podman[94062]: 2024-09-16 02:08:08.302003034 +0000 UTC m=+0.500179694 container attach b26894b7248a0bb8a8739d44600355a6611933a3ab56e918aea8134dabb9a5da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T02:08:08.472 INFO:journalctl@ceph.osd.9.smithi105.stdout:Sep 16 02:08:08 smithi105 podman[94062]: 2024-09-16 02:08:08.398496507 +0000 UTC m=+0.596673141 container died b26894b7248a0bb8a8739d44600355a6611933a3ab56e918aea8134dabb9a5da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-9-deactivate, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-16T02:08:08.673 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:08 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:08.325+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:08:08.673 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:08 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:08.325+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:08:08.673 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:08 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:08.325+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:08:08.673 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:08 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:08.325+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.570558+0000 front 2024-09-16T02:07:17.570576+0000 (oldest deadline 2024-09-16T02:07:41.070561+0000) 2024-09-16T02:08:08.673 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:08 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:08.325+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.471846+0000 front 2024-09-16T02:07:24.471866+0000 (oldest deadline 2024-09-16T02:07:46.771852+0000) 2024-09-16T02:08:08.673 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:08 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:08.325+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:32.573523+0000 front 2024-09-16T02:07:32.573580+0000 (oldest deadline 2024-09-16T02:07:55.473409+0000) 2024-09-16T02:08:08.673 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:08 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:08.325+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-09-16T02:07:40.675134+0000 front 2024-09-16T02:07:40.675181+0000 (oldest deadline 2024-09-16T02:08:02.974962+0000) 2024-09-16T02:08:09.039 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@osd.9.service' 2024-09-16T02:08:09.082 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:08:09.082 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-16T02:08:09.082 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-16T02:08:09.082 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@osd.10 2024-09-16T02:08:09.342 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:08 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:08.891+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:08:09.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:08 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:08.891+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:08:09.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:08 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:08.891+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:08:09.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:08 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:08.891+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.065430+0000 front 2024-09-16T02:07:24.065531+0000 (oldest deadline 2024-09-16T02:07:48.165250+0000) 2024-09-16T02:08:09.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:08 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:08.891+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:33.366463+0000 front 2024-09-16T02:07:33.366510+0000 (oldest deadline 2024-09-16T02:07:58.666604+0000) 2024-09-16T02:08:09.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:08 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:08.891+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-09-16T02:07:40.268148+0000 front 2024-09-16T02:07:40.268206+0000 (oldest deadline 2024-09-16T02:08:04.368006+0000) 2024-09-16T02:08:09.390 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:09 smithi137 systemd[1]: Stopping Ceph osd.10 for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:08:09.390 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:09 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:09.311+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:08:09.390 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:09 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:09.311+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:08:09.390 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:09 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:09.311+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:08:09.390 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:09 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:09.311+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.570558+0000 front 2024-09-16T02:07:17.570576+0000 (oldest deadline 2024-09-16T02:07:41.070561+0000) 2024-09-16T02:08:09.391 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:09 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:09.311+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.471846+0000 front 2024-09-16T02:07:24.471866+0000 (oldest deadline 2024-09-16T02:07:46.771852+0000) 2024-09-16T02:08:09.391 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:09 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:09.311+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:32.573523+0000 front 2024-09-16T02:07:32.573580+0000 (oldest deadline 2024-09-16T02:07:55.473409+0000) 2024-09-16T02:08:09.391 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:09 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:09.311+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-09-16T02:07:40.675134+0000 front 2024-09-16T02:07:40.675181+0000 (oldest deadline 2024-09-16T02:08:02.974962+0000) 2024-09-16T02:08:09.672 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:09 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:09.388+0000 7fa7ee270640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:08:09.672 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:09 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:09.388+0000 7fa7ee270640 -1 osd.10 86 *** Got signal Terminated *** 2024-09-16T02:08:09.672 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:09 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:09.388+0000 7fa7ee270640 -1 osd.10 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T02:08:10.342 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:09 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:09.929+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:08:10.342 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:09 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:09.929+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:08:10.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:09 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:09.929+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:08:10.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:09 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:09.929+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.065430+0000 front 2024-09-16T02:07:24.065531+0000 (oldest deadline 2024-09-16T02:07:48.165250+0000) 2024-09-16T02:08:10.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:09 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:09.929+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:33.366463+0000 front 2024-09-16T02:07:33.366510+0000 (oldest deadline 2024-09-16T02:07:58.666604+0000) 2024-09-16T02:08:10.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:09 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:09.929+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-09-16T02:07:40.268148+0000 front 2024-09-16T02:07:40.268206+0000 (oldest deadline 2024-09-16T02:08:04.368006+0000) 2024-09-16T02:08:10.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:10 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:10.316+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:08:10.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:10 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:10.316+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:08:10.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:10 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:10.316+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:08:10.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:10 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:10.316+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.570558+0000 front 2024-09-16T02:07:17.570576+0000 (oldest deadline 2024-09-16T02:07:41.070561+0000) 2024-09-16T02:08:10.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:10 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:10.316+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.471846+0000 front 2024-09-16T02:07:24.471866+0000 (oldest deadline 2024-09-16T02:07:46.771852+0000) 2024-09-16T02:08:10.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:10 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:10.316+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:32.573523+0000 front 2024-09-16T02:07:32.573580+0000 (oldest deadline 2024-09-16T02:07:55.473409+0000) 2024-09-16T02:08:10.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:10 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:10.316+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-09-16T02:07:40.675134+0000 front 2024-09-16T02:07:40.675181+0000 (oldest deadline 2024-09-16T02:08:02.974962+0000) 2024-09-16T02:08:11.342 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:10 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:10.885+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:08:11.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:10 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:10.885+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:08:11.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:10 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:10.886+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:08:11.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:10 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:10.886+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.065430+0000 front 2024-09-16T02:07:24.065531+0000 (oldest deadline 2024-09-16T02:07:48.165250+0000) 2024-09-16T02:08:11.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:10 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:10.886+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:33.366463+0000 front 2024-09-16T02:07:33.366510+0000 (oldest deadline 2024-09-16T02:07:58.666604+0000) 2024-09-16T02:08:11.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:10 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:10.886+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-09-16T02:07:40.268148+0000 front 2024-09-16T02:07:40.268206+0000 (oldest deadline 2024-09-16T02:08:04.368006+0000) 2024-09-16T02:08:11.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:11 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:11.359+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:08:11.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:11 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:11.359+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:08:11.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:11 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:11.359+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:08:11.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:11 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:11.359+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.570558+0000 front 2024-09-16T02:07:17.570576+0000 (oldest deadline 2024-09-16T02:07:41.070561+0000) 2024-09-16T02:08:11.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:11 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:11.359+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.471846+0000 front 2024-09-16T02:07:24.471866+0000 (oldest deadline 2024-09-16T02:07:46.771852+0000) 2024-09-16T02:08:11.672 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:11 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:11.359+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:32.573523+0000 front 2024-09-16T02:07:32.573580+0000 (oldest deadline 2024-09-16T02:07:55.473409+0000) 2024-09-16T02:08:11.672 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:11 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:11.359+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-09-16T02:07:40.675134+0000 front 2024-09-16T02:07:40.675181+0000 (oldest deadline 2024-09-16T02:08:02.974962+0000) 2024-09-16T02:08:12.345 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:11 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:11.843+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:08:12.345 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:11 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:11.843+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:08:12.345 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:11 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:11.843+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:08:12.345 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:11 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:11.843+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.065430+0000 front 2024-09-16T02:07:24.065531+0000 (oldest deadline 2024-09-16T02:07:48.165250+0000) 2024-09-16T02:08:12.345 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:11 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:11.843+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:33.366463+0000 front 2024-09-16T02:07:33.366510+0000 (oldest deadline 2024-09-16T02:07:58.666604+0000) 2024-09-16T02:08:12.345 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:11 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:11.843+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-09-16T02:07:40.268148+0000 front 2024-09-16T02:07:40.268206+0000 (oldest deadline 2024-09-16T02:08:04.368006+0000) 2024-09-16T02:08:12.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:12 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:12.363+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:08:12.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:12 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:12.363+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:08:12.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:12 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:12.363+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:08:12.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:12 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:12.363+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.570558+0000 front 2024-09-16T02:07:17.570576+0000 (oldest deadline 2024-09-16T02:07:41.070561+0000) 2024-09-16T02:08:12.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:12 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:12.363+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.471846+0000 front 2024-09-16T02:07:24.471866+0000 (oldest deadline 2024-09-16T02:07:46.771852+0000) 2024-09-16T02:08:12.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:12 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:12.363+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:32.573523+0000 front 2024-09-16T02:07:32.573580+0000 (oldest deadline 2024-09-16T02:07:55.473409+0000) 2024-09-16T02:08:12.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:12 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:12.363+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-09-16T02:07:40.675134+0000 front 2024-09-16T02:07:40.675181+0000 (oldest deadline 2024-09-16T02:08:02.974962+0000) 2024-09-16T02:08:13.092 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:12 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:12.812+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:08:13.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:12 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:12.812+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:08:13.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:12 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:12.812+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:08:13.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:12 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:12.812+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.065430+0000 front 2024-09-16T02:07:24.065531+0000 (oldest deadline 2024-09-16T02:07:48.165250+0000) 2024-09-16T02:08:13.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:12 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:12.812+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:33.366463+0000 front 2024-09-16T02:07:33.366510+0000 (oldest deadline 2024-09-16T02:07:58.666604+0000) 2024-09-16T02:08:13.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:12 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:12.812+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-09-16T02:07:40.268148+0000 front 2024-09-16T02:07:40.268206+0000 (oldest deadline 2024-09-16T02:08:04.368006+0000) 2024-09-16T02:08:13.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:12 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:12.812+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-09-16T02:07:49.068908+0000 front 2024-09-16T02:07:49.069010+0000 (oldest deadline 2024-09-16T02:08:12.568848+0000) 2024-09-16T02:08:13.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:13 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:13.343+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:08:13.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:13 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:13.343+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:08:13.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:13 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:13.343+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:08:13.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:13 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:13.343+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.570558+0000 front 2024-09-16T02:07:17.570576+0000 (oldest deadline 2024-09-16T02:07:41.070561+0000) 2024-09-16T02:08:13.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:13 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:13.343+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.471846+0000 front 2024-09-16T02:07:24.471866+0000 (oldest deadline 2024-09-16T02:07:46.771852+0000) 2024-09-16T02:08:13.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:13 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:13.343+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:32.573523+0000 front 2024-09-16T02:07:32.573580+0000 (oldest deadline 2024-09-16T02:07:55.473409+0000) 2024-09-16T02:08:13.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:13 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:13.343+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-09-16T02:07:40.675134+0000 front 2024-09-16T02:07:40.675181+0000 (oldest deadline 2024-09-16T02:08:02.974962+0000) 2024-09-16T02:08:14.342 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:13 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:13.850+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:08:14.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:13 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:13.850+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:08:14.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:13 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:13.850+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:08:14.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:13 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:13.850+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.065430+0000 front 2024-09-16T02:07:24.065531+0000 (oldest deadline 2024-09-16T02:07:48.165250+0000) 2024-09-16T02:08:14.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:13 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:13.850+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:33.366463+0000 front 2024-09-16T02:07:33.366510+0000 (oldest deadline 2024-09-16T02:07:58.666604+0000) 2024-09-16T02:08:14.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:13 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:13.850+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-09-16T02:07:40.268148+0000 front 2024-09-16T02:07:40.268206+0000 (oldest deadline 2024-09-16T02:08:04.368006+0000) 2024-09-16T02:08:14.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:13 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:13.850+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-09-16T02:07:49.068908+0000 front 2024-09-16T02:07:49.069010+0000 (oldest deadline 2024-09-16T02:08:12.568848+0000) 2024-09-16T02:08:14.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:14 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:14.311+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:49.967789+0000 front 2024-09-16T02:06:49.967751+0000 (oldest deadline 2024-09-16T02:07:15.867542+0000) 2024-09-16T02:08:14.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:14 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:14.311+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:01.168491+0000 front 2024-09-16T02:07:01.168649+0000 (oldest deadline 2024-09-16T02:07:26.468468+0000) 2024-09-16T02:08:14.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:14 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:14.311+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:06.469064+0000 front 2024-09-16T02:07:06.469107+0000 (oldest deadline 2024-09-16T02:07:30.569062+0000) 2024-09-16T02:08:14.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:14 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:14.311+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-09-16T02:07:17.570558+0000 front 2024-09-16T02:07:17.570576+0000 (oldest deadline 2024-09-16T02:07:41.070561+0000) 2024-09-16T02:08:14.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:14 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:14.311+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.471846+0000 front 2024-09-16T02:07:24.471866+0000 (oldest deadline 2024-09-16T02:07:46.771852+0000) 2024-09-16T02:08:14.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:14 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:14.311+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:32.573523+0000 front 2024-09-16T02:07:32.573580+0000 (oldest deadline 2024-09-16T02:07:55.473409+0000) 2024-09-16T02:08:14.671 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:14 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10[58661]: 2024-09-16T02:08:14.311+0000 7fa7e687e640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-09-16T02:07:40.675134+0000 front 2024-09-16T02:07:40.675181+0000 (oldest deadline 2024-09-16T02:08:02.974962+0000) 2024-09-16T02:08:15.014 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:14 smithi137 podman[67967]: 2024-09-16 02:08:14.709868002 +0000 UTC m=+5.427810058 container died 1d8ba7b3745ed02b8ca3544780090781d5e6c1f9d6c1e98d9c5bb37546b83e38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-16T02:08:15.014 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:15 smithi137 podman[67967]: 2024-09-16 02:08:15.013641508 +0000 UTC m=+5.731583596 container cleanup 1d8ba7b3745ed02b8ca3544780090781d5e6c1f9d6c1e98d9c5bb37546b83e38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-16T02:08:15.342 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:14 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:14.851+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:08:15.342 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:14 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:14.851+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:08:15.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:14 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:14.851+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:08:15.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:14 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:14.851+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.065430+0000 front 2024-09-16T02:07:24.065531+0000 (oldest deadline 2024-09-16T02:07:48.165250+0000) 2024-09-16T02:08:15.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:14 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:14.851+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:33.366463+0000 front 2024-09-16T02:07:33.366510+0000 (oldest deadline 2024-09-16T02:07:58.666604+0000) 2024-09-16T02:08:15.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:14 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:14.851+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-09-16T02:07:40.268148+0000 front 2024-09-16T02:07:40.268206+0000 (oldest deadline 2024-09-16T02:08:04.368006+0000) 2024-09-16T02:08:15.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:14 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:14.851+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-09-16T02:07:49.068908+0000 front 2024-09-16T02:07:49.069010+0000 (oldest deadline 2024-09-16T02:08:12.568848+0000) 2024-09-16T02:08:15.421 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:15 smithi137 bash[67967]: ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10 2024-09-16T02:08:15.421 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:15 smithi137 podman[67983]: 2024-09-16 02:08:15.181453936 +0000 UTC m=+0.469273945 container remove 1d8ba7b3745ed02b8ca3544780090781d5e6c1f9d6c1e98d9c5bb37546b83e38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10, RELEASE=squid-8921c8e, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS) 2024-09-16T02:08:15.893 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:15 smithi137 podman[68073]: 2024-09-16 02:08:15.52067879 +0000 UTC m=+0.015571366 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:08:15.894 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:15 smithi137 podman[68073]: 2024-09-16 02:08:15.640468789 +0000 UTC m=+0.135361393 container create 4214b46845e5f902c2f7ae1feb71048d3cafab79c29dec4fedf147e9ab715fdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux ) 2024-09-16T02:08:15.894 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:15 smithi137 podman[68073]: 2024-09-16 02:08:15.892631828 +0000 UTC m=+0.387524390 container init 4214b46845e5f902c2f7ae1feb71048d3cafab79c29dec4fedf147e9ab715fdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T02:08:16.171 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:15 smithi137 podman[68073]: 2024-09-16 02:08:15.89592914 +0000 UTC m=+0.390821711 container start 4214b46845e5f902c2f7ae1feb71048d3cafab79c29dec4fedf147e9ab715fdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e) 2024-09-16T02:08:16.171 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:15 smithi137 podman[68073]: 2024-09-16 02:08:15.976132829 +0000 UTC m=+0.471025393 container attach 4214b46845e5f902c2f7ae1feb71048d3cafab79c29dec4fedf147e9ab715fdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-16T02:08:16.171 INFO:journalctl@ceph.osd.10.smithi137.stdout:Sep 16 02:08:16 smithi137 podman[68073]: 2024-09-16 02:08:16.087573826 +0000 UTC m=+0.582466399 container died 4214b46845e5f902c2f7ae1feb71048d3cafab79c29dec4fedf147e9ab715fdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-16T02:08:16.342 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:15 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:15.886+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:08:16.342 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:15 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:15.886+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:08:16.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:15 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:15.886+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:08:16.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:15 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:15.886+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.065430+0000 front 2024-09-16T02:07:24.065531+0000 (oldest deadline 2024-09-16T02:07:48.165250+0000) 2024-09-16T02:08:16.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:15 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:15.886+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:33.366463+0000 front 2024-09-16T02:07:33.366510+0000 (oldest deadline 2024-09-16T02:07:58.666604+0000) 2024-09-16T02:08:16.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:15 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:15.886+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-09-16T02:07:40.268148+0000 front 2024-09-16T02:07:40.268206+0000 (oldest deadline 2024-09-16T02:08:04.368006+0000) 2024-09-16T02:08:16.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:15 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:15.886+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-09-16T02:07:49.068908+0000 front 2024-09-16T02:07:49.069010+0000 (oldest deadline 2024-09-16T02:08:12.568848+0000) 2024-09-16T02:08:16.671 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@osd.10.service' 2024-09-16T02:08:16.708 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:08:16.709 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-16T02:08:16.709 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-16T02:08:16.709 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@osd.11 2024-09-16T02:08:17.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:16 smithi190 systemd[1]: Stopping Ceph osd.11 for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:08:17.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:16 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:16.922+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:08:17.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:16 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:16.922+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:08:17.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:16 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:16.922+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:08:17.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:16 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:16.922+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.065430+0000 front 2024-09-16T02:07:24.065531+0000 (oldest deadline 2024-09-16T02:07:48.165250+0000) 2024-09-16T02:08:17.093 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:16 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:16.922+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:33.366463+0000 front 2024-09-16T02:07:33.366510+0000 (oldest deadline 2024-09-16T02:07:58.666604+0000) 2024-09-16T02:08:17.094 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:16 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:16.922+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-09-16T02:07:40.268148+0000 front 2024-09-16T02:07:40.268206+0000 (oldest deadline 2024-09-16T02:08:04.368006+0000) 2024-09-16T02:08:17.094 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:16 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:16.922+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-09-16T02:07:49.068908+0000 front 2024-09-16T02:07:49.069010+0000 (oldest deadline 2024-09-16T02:08:12.568848+0000) 2024-09-16T02:08:17.592 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:17 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:17.340+0000 7fddd7b00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:08:17.593 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:17 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:17.340+0000 7fddd7b00640 -1 osd.11 86 *** Got signal Terminated *** 2024-09-16T02:08:17.593 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:17 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:17.340+0000 7fddd7b00640 -1 osd.11 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T02:08:18.342 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:17 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:17.935+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:08:18.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:17 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:17.935+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:08:18.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:17 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:17.935+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:08:18.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:17 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:17.935+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.065430+0000 front 2024-09-16T02:07:24.065531+0000 (oldest deadline 2024-09-16T02:07:48.165250+0000) 2024-09-16T02:08:18.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:17 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:17.935+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:33.366463+0000 front 2024-09-16T02:07:33.366510+0000 (oldest deadline 2024-09-16T02:07:58.666604+0000) 2024-09-16T02:08:18.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:17 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:17.935+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-09-16T02:07:40.268148+0000 front 2024-09-16T02:07:40.268206+0000 (oldest deadline 2024-09-16T02:08:04.368006+0000) 2024-09-16T02:08:18.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:17 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:17.935+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-09-16T02:07:49.068908+0000 front 2024-09-16T02:07:49.069010+0000 (oldest deadline 2024-09-16T02:08:12.568848+0000) 2024-09-16T02:08:19.342 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:18 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:18.967+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:08:19.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:18 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:18.967+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:08:19.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:18 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:18.967+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:08:19.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:18 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:18.967+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.065430+0000 front 2024-09-16T02:07:24.065531+0000 (oldest deadline 2024-09-16T02:07:48.165250+0000) 2024-09-16T02:08:19.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:18 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:18.967+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:33.366463+0000 front 2024-09-16T02:07:33.366510+0000 (oldest deadline 2024-09-16T02:07:58.666604+0000) 2024-09-16T02:08:19.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:18 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:18.967+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-09-16T02:07:40.268148+0000 front 2024-09-16T02:07:40.268206+0000 (oldest deadline 2024-09-16T02:08:04.368006+0000) 2024-09-16T02:08:19.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:18 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:18.967+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-09-16T02:07:49.068908+0000 front 2024-09-16T02:07:49.069010+0000 (oldest deadline 2024-09-16T02:08:12.568848+0000) 2024-09-16T02:08:20.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:19 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:19.997+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:08:20.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:19 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:19.997+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:08:20.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:19 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:19.997+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:08:20.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:19 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:19.997+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.065430+0000 front 2024-09-16T02:07:24.065531+0000 (oldest deadline 2024-09-16T02:07:48.165250+0000) 2024-09-16T02:08:20.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:19 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:19.997+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:33.366463+0000 front 2024-09-16T02:07:33.366510+0000 (oldest deadline 2024-09-16T02:07:58.666604+0000) 2024-09-16T02:08:20.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:19 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:19.997+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-09-16T02:07:40.268148+0000 front 2024-09-16T02:07:40.268206+0000 (oldest deadline 2024-09-16T02:08:04.368006+0000) 2024-09-16T02:08:20.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:19 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:19.997+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-09-16T02:07:49.068908+0000 front 2024-09-16T02:07:49.069010+0000 (oldest deadline 2024-09-16T02:08:12.568848+0000) 2024-09-16T02:08:20.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:19 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:19.997+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6820 osd.8 since back 2024-09-16T02:07:53.670115+0000 front 2024-09-16T02:07:53.670136+0000 (oldest deadline 2024-09-16T02:08:19.470617+0000) 2024-09-16T02:08:21.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:20 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:20.977+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:08:21.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:20 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:20.977+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:08:21.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:20 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:20.977+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:08:21.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:20 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:20.977+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.065430+0000 front 2024-09-16T02:07:24.065531+0000 (oldest deadline 2024-09-16T02:07:48.165250+0000) 2024-09-16T02:08:21.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:20 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:20.977+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:33.366463+0000 front 2024-09-16T02:07:33.366510+0000 (oldest deadline 2024-09-16T02:07:58.666604+0000) 2024-09-16T02:08:21.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:20 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:20.977+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-09-16T02:07:40.268148+0000 front 2024-09-16T02:07:40.268206+0000 (oldest deadline 2024-09-16T02:08:04.368006+0000) 2024-09-16T02:08:21.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:20 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:20.977+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-09-16T02:07:49.068908+0000 front 2024-09-16T02:07:49.069010+0000 (oldest deadline 2024-09-16T02:08:12.568848+0000) 2024-09-16T02:08:21.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:20 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:20.977+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6820 osd.8 since back 2024-09-16T02:07:53.670115+0000 front 2024-09-16T02:07:53.670136+0000 (oldest deadline 2024-09-16T02:08:19.470617+0000) 2024-09-16T02:08:22.342 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:22 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:22.012+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-09-16T02:06:51.559064+0000 front 2024-09-16T02:06:51.559095+0000 (oldest deadline 2024-09-16T02:07:16.859188+0000) 2024-09-16T02:08:22.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:22 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:22.012+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-09-16T02:07:00.360153+0000 front 2024-09-16T02:07:00.360210+0000 (oldest deadline 2024-09-16T02:07:25.660341+0000) 2024-09-16T02:08:22.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:22 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:22.012+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-09-16T02:07:09.661888+0000 front 2024-09-16T02:07:09.662120+0000 (oldest deadline 2024-09-16T02:07:33.162028+0000) 2024-09-16T02:08:22.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:22 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:22.012+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-09-16T02:07:24.065430+0000 front 2024-09-16T02:07:24.065531+0000 (oldest deadline 2024-09-16T02:07:48.165250+0000) 2024-09-16T02:08:22.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:22 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:22.012+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-09-16T02:07:33.366463+0000 front 2024-09-16T02:07:33.366510+0000 (oldest deadline 2024-09-16T02:07:58.666604+0000) 2024-09-16T02:08:22.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:22 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:22.012+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-09-16T02:07:40.268148+0000 front 2024-09-16T02:07:40.268206+0000 (oldest deadline 2024-09-16T02:08:04.368006+0000) 2024-09-16T02:08:22.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:22 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:22.012+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-09-16T02:07:49.068908+0000 front 2024-09-16T02:07:49.069010+0000 (oldest deadline 2024-09-16T02:08:12.568848+0000) 2024-09-16T02:08:22.343 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:22 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11[59087]: 2024-09-16T02:08:22.012+0000 7fddd4116640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.190:6820 osd.8 since back 2024-09-16T02:07:53.670115+0000 front 2024-09-16T02:07:53.670136+0000 (oldest deadline 2024-09-16T02:08:19.470617+0000) 2024-09-16T02:08:23.142 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:22 smithi190 podman[68475]: 2024-09-16 02:08:22.858673868 +0000 UTC m=+5.951817361 container died 0710c35df955c9136a01fca54a5fedb2d1bf899ee7dee1969f1fdfeedddf90e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=) 2024-09-16T02:08:23.492 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:23 smithi190 podman[68475]: 2024-09-16 02:08:23.240198734 +0000 UTC m=+6.333342228 container cleanup 0710c35df955c9136a01fca54a5fedb2d1bf899ee7dee1969f1fdfeedddf90e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T02:08:23.492 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:23 smithi190 bash[68475]: ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11 2024-09-16T02:08:23.842 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:23 smithi190 podman[68492]: 2024-09-16 02:08:23.49129453 +0000 UTC m=+0.632799355 container remove 0710c35df955c9136a01fca54a5fedb2d1bf899ee7dee1969f1fdfeedddf90e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-16T02:08:24.128 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:23 smithi190 podman[68581]: 2024-09-16 02:08:23.753411485 +0000 UTC m=+0.015893214 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:08:24.128 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:23 smithi190 podman[68581]: 2024-09-16 02:08:23.858583284 +0000 UTC m=+0.121065047 container create d9b811e65b5980c1ac6b08f7b7b35e30abb62fe92bb1463243b273e7856bd061 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e) 2024-09-16T02:08:24.128 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:24 smithi190 podman[68581]: 2024-09-16 02:08:24.127773806 +0000 UTC m=+0.390255534 container init d9b811e65b5980c1ac6b08f7b7b35e30abb62fe92bb1463243b273e7856bd061 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11-deactivate, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-16T02:08:24.593 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:24 smithi190 podman[68581]: 2024-09-16 02:08:24.131225503 +0000 UTC m=+0.393707227 container start d9b811e65b5980c1ac6b08f7b7b35e30abb62fe92bb1463243b273e7856bd061 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-16T02:08:24.593 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:24 smithi190 podman[68581]: 2024-09-16 02:08:24.219298243 +0000 UTC m=+0.481779980 container attach d9b811e65b5980c1ac6b08f7b7b35e30abb62fe92bb1463243b273e7856bd061 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T02:08:24.593 INFO:journalctl@ceph.osd.11.smithi190.stdout:Sep 16 02:08:24 smithi190 podman[68581]: 2024-09-16 02:08:24.314168446 +0000 UTC m=+0.576650165 container died d9b811e65b5980c1ac6b08f7b7b35e30abb62fe92bb1463243b273e7856bd061 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-16T02:08:24.905 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@osd.11.service' 2024-09-16T02:08:24.947 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:08:24.947 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-16T02:08:24.947 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-16T02:08:24.948 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mds.a 2024-09-16T02:08:25.259 INFO:journalctl@ceph.mds.a.smithi105.stdout:Sep 16 02:08:24 smithi105 systemd[1]: Stopping Ceph mds.a for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:08:25.544 INFO:journalctl@ceph.mds.a.smithi105.stdout:Sep 16 02:08:25 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-a[72108]: 2024-09-16T02:08:25.256+0000 7f67986c5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:08:25.545 INFO:journalctl@ceph.mds.a.smithi105.stdout:Sep 16 02:08:25 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-a[72108]: 2024-09-16T02:08:25.256+0000 7f67986c5640 -1 mds.a *** got signal Terminated *** 2024-09-16T02:08:29.217 INFO:journalctl@ceph.mds.a.smithi105.stdout:Sep 16 02:08:28 smithi105 podman[94177]: 2024-09-16 02:08:28.934017635 +0000 UTC m=+3.792555747 container died eeeb2888b1f44a64f60a5a78c19e5a4fa53ffe72f0a045c951f1651736f7f493 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-a, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e) 2024-09-16T02:08:29.545 INFO:journalctl@ceph.mds.a.smithi105.stdout:Sep 16 02:08:29 smithi105 podman[94177]: 2024-09-16 02:08:29.398904709 +0000 UTC m=+4.257442862 container cleanup eeeb2888b1f44a64f60a5a78c19e5a4fa53ffe72f0a045c951f1651736f7f493 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-a, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-16T02:08:29.545 INFO:journalctl@ceph.mds.a.smithi105.stdout:Sep 16 02:08:29 smithi105 bash[94177]: ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-a 2024-09-16T02:08:29.870 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mds.a.service' 2024-09-16T02:08:29.905 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:08:29.905 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-16T02:08:29.905 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-16T02:08:29.905 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mds.d 2024-09-16T02:08:30.044 INFO:journalctl@ceph.mds.d.smithi105.stdout:Sep 16 02:08:29 smithi105 systemd[1]: Stopping Ceph mds.d for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:08:30.545 INFO:journalctl@ceph.mds.d.smithi105.stdout:Sep 16 02:08:30 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-d[70927]: 2024-09-16T02:08:30.219+0000 7f8477d47640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:08:30.545 INFO:journalctl@ceph.mds.d.smithi105.stdout:Sep 16 02:08:30 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-d[70927]: 2024-09-16T02:08:30.219+0000 7f8477d47640 -1 mds.d *** got signal Terminated *** 2024-09-16T02:08:34.205 INFO:journalctl@ceph.mds.d.smithi105.stdout:Sep 16 02:08:33 smithi105 podman[94298]: 2024-09-16 02:08:33.908778984 +0000 UTC m=+3.790209825 container died 66e3dc8a813a465223b7a118077795a450d6e13ed6290100fe8b6fbf8fab85ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-d, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T02:08:34.545 INFO:journalctl@ceph.mds.d.smithi105.stdout:Sep 16 02:08:34 smithi105 podman[94298]: 2024-09-16 02:08:34.361638478 +0000 UTC m=+4.243069377 container cleanup 66e3dc8a813a465223b7a118077795a450d6e13ed6290100fe8b6fbf8fab85ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True) 2024-09-16T02:08:34.545 INFO:journalctl@ceph.mds.d.smithi105.stdout:Sep 16 02:08:34 smithi105 bash[94298]: ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-d 2024-09-16T02:08:34.784 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mds.d.service' 2024-09-16T02:08:34.805 INFO:journalctl@ceph.mds.d.smithi105.stdout:Sep 16 02:08:34 smithi105 podman[94310]: 2024-09-16 02:08:34.554511919 +0000 UTC m=+0.643556054 container remove 66e3dc8a813a465223b7a118077795a450d6e13ed6290100fe8b6fbf8fab85ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e) 2024-09-16T02:08:34.805 INFO:journalctl@ceph.mds.d.smithi105.stdout:Sep 16 02:08:34 smithi105 systemd[1]: ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mds.d.service: Deactivated successfully. 2024-09-16T02:08:34.806 INFO:journalctl@ceph.mds.d.smithi105.stdout:Sep 16 02:08:34 smithi105 systemd[1]: Stopped Ceph mds.d for 423b7a2a-73ce-11ef-bceb-c7b262605968. 2024-09-16T02:08:34.817 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:08:34.817 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-16T02:08:34.818 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-16T02:08:34.818 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mds.g 2024-09-16T02:08:35.141 INFO:journalctl@ceph.mds.g.smithi105.stdout:Sep 16 02:08:34 smithi105 systemd[1]: Stopping Ceph mds.g for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:08:35.545 INFO:journalctl@ceph.mds.g.smithi105.stdout:Sep 16 02:08:35 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-g[71712]: 2024-09-16T02:08:35.140+0000 7fdd4a294640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:08:35.545 INFO:journalctl@ceph.mds.g.smithi105.stdout:Sep 16 02:08:35 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-g[71712]: 2024-09-16T02:08:35.140+0000 7fdd4a294640 -1 mds.g *** got signal Terminated *** 2024-09-16T02:08:37.208 INFO:journalctl@ceph.mds.g.smithi105.stdout:Sep 16 02:08:36 smithi105 podman[94419]: 2024-09-16 02:08:36.929274215 +0000 UTC m=+1.886522238 container died f203576a25fd05c61304502f4ac5101646f379d73872e065c6e254437cf40890 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-g, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True) 2024-09-16T02:08:37.545 INFO:journalctl@ceph.mds.g.smithi105.stdout:Sep 16 02:08:37 smithi105 podman[94419]: 2024-09-16 02:08:37.382214414 +0000 UTC m=+2.339462433 container cleanup f203576a25fd05c61304502f4ac5101646f379d73872e065c6e254437cf40890 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-g, RELEASE=squid-8921c8e, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-16T02:08:37.545 INFO:journalctl@ceph.mds.g.smithi105.stdout:Sep 16 02:08:37 smithi105 bash[94419]: ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-g 2024-09-16T02:08:37.820 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mds.g.service' 2024-09-16T02:08:37.853 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:08:37.853 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-16T02:08:37.853 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-16T02:08:37.853 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mds.j 2024-09-16T02:08:38.295 INFO:journalctl@ceph.mds.j.smithi105.stdout:Sep 16 02:08:37 smithi105 systemd[1]: Stopping Ceph mds.j for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:08:38.295 INFO:journalctl@ceph.mds.j.smithi105.stdout:Sep 16 02:08:38 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-j[70522]: 2024-09-16T02:08:38.186+0000 7f8608190640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:08:38.295 INFO:journalctl@ceph.mds.j.smithi105.stdout:Sep 16 02:08:38 smithi105 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-j[70522]: 2024-09-16T02:08:38.186+0000 7f8608190640 -1 mds.j *** got signal Terminated *** 2024-09-16T02:08:42.497 INFO:journalctl@ceph.mds.j.smithi105.stdout:Sep 16 02:08:42 smithi105 podman[94542]: 2024-09-16 02:08:42.200860015 +0000 UTC m=+4.123169151 container died d83fe50c53e3dcfc02a657e3a118ae57199201748919679b21196b49b6f9094c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-j, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T02:08:42.794 INFO:journalctl@ceph.mds.j.smithi105.stdout:Sep 16 02:08:42 smithi105 podman[94542]: 2024-09-16 02:08:42.678380338 +0000 UTC m=+4.600689516 container cleanup d83fe50c53e3dcfc02a657e3a118ae57199201748919679b21196b49b6f9094c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-j, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T02:08:42.795 INFO:journalctl@ceph.mds.j.smithi105.stdout:Sep 16 02:08:42 smithi105 bash[94542]: ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-j 2024-09-16T02:08:43.058 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mds.j.service' 2024-09-16T02:08:43.093 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:08:43.093 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-16T02:08:43.094 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-16T02:08:43.094 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mds.b 2024-09-16T02:08:43.421 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 16 02:08:43 smithi137 systemd[1]: Stopping Ceph mds.b for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:08:43.678 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 16 02:08:43 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-b[64153]: 2024-09-16T02:08:43.428+0000 7f7eae452640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:08:43.679 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 16 02:08:43 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-b[64153]: 2024-09-16T02:08:43.428+0000 7f7eae452640 -1 mds.b *** got signal Terminated *** 2024-09-16T02:08:45.205 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 16 02:08:44 smithi137 podman[68175]: 2024-09-16 02:08:44.933528122 +0000 UTC m=+1.642112154 container died d260d3616179444105cafc91f4de590945867af01fe919edae1771484df65124 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-b, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-16T02:08:45.564 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 16 02:08:45 smithi137 podman[68175]: 2024-09-16 02:08:45.295344207 +0000 UTC m=+2.003928239 container cleanup d260d3616179444105cafc91f4de590945867af01fe919edae1771484df65124 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-b, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T02:08:45.564 INFO:journalctl@ceph.mds.b.smithi137.stdout:Sep 16 02:08:45 smithi137 bash[68175]: ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-b 2024-09-16T02:08:45.723 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mds.b.service' 2024-09-16T02:08:45.758 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:08:45.759 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-16T02:08:45.759 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-16T02:08:45.759 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mds.e 2024-09-16T02:08:46.084 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 16 02:08:45 smithi137 systemd[1]: Stopping Ceph mds.e for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:08:46.421 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 16 02:08:46 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-e[63535]: 2024-09-16T02:08:46.081+0000 7ff1b0510640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:08:46.421 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 16 02:08:46 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-e[63535]: 2024-09-16T02:08:46.081+0000 7ff1b0510640 -1 mds.e *** got signal Terminated *** 2024-09-16T02:08:49.258 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 16 02:08:48 smithi137 podman[68309]: 2024-09-16 02:08:48.939048953 +0000 UTC m=+2.958575888 container died c81646a1b2254ea2204c8e7be9573c1ddf3c7a337add7d284731c309e2632ea4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-16T02:08:49.258 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 16 02:08:49 smithi137 podman[68309]: 2024-09-16 02:08:49.257282614 +0000 UTC m=+3.276809549 container cleanup c81646a1b2254ea2204c8e7be9573c1ddf3c7a337add7d284731c309e2632ea4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-e, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-16T02:08:49.669 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 16 02:08:49 smithi137 bash[68309]: ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-e 2024-09-16T02:08:49.669 INFO:journalctl@ceph.mds.e.smithi137.stdout:Sep 16 02:08:49 smithi137 podman[68323]: 2024-09-16 02:08:49.4751505 +0000 UTC m=+0.536422631 container remove c81646a1b2254ea2204c8e7be9573c1ddf3c7a337add7d284731c309e2632ea4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-16T02:08:49.684 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mds.e.service' 2024-09-16T02:08:49.717 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:08:49.718 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-16T02:08:49.718 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-16T02:08:49.718 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mds.h 2024-09-16T02:08:50.037 INFO:journalctl@ceph.mds.h.smithi137.stdout:Sep 16 02:08:49 smithi137 systemd[1]: Stopping Ceph mds.h for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:08:50.421 INFO:journalctl@ceph.mds.h.smithi137.stdout:Sep 16 02:08:50 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-h[63836]: 2024-09-16T02:08:50.034+0000 7f692b07c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:08:50.421 INFO:journalctl@ceph.mds.h.smithi137.stdout:Sep 16 02:08:50 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-h[63836]: 2024-09-16T02:08:50.034+0000 7f692b07c640 -1 mds.h *** got signal Terminated *** 2024-09-16T02:08:53.196 INFO:journalctl@ceph.mds.h.smithi137.stdout:Sep 16 02:08:52 smithi137 podman[68430]: 2024-09-16 02:08:52.93422941 +0000 UTC m=+2.991684955 container died 39d3393c8f3e0536c64a84380a679453bcb8e95386e38e0aec9ed5a5bad089af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-h, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-16T02:08:53.644 INFO:journalctl@ceph.mds.h.smithi137.stdout:Sep 16 02:08:53 smithi137 podman[68430]: 2024-09-16 02:08:53.268717315 +0000 UTC m=+3.326172847 container cleanup 39d3393c8f3e0536c64a84380a679453bcb8e95386e38e0aec9ed5a5bad089af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e) 2024-09-16T02:08:53.645 INFO:journalctl@ceph.mds.h.smithi137.stdout:Sep 16 02:08:53 smithi137 bash[68430]: ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-h 2024-09-16T02:08:53.645 INFO:journalctl@ceph.mds.h.smithi137.stdout:Sep 16 02:08:53 smithi137 podman[68442]: 2024-09-16 02:08:53.428333314 +0000 UTC m=+0.487549244 container remove 39d3393c8f3e0536c64a84380a679453bcb8e95386e38e0aec9ed5a5bad089af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-h, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-16T02:08:53.666 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mds.h.service' 2024-09-16T02:08:53.697 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:08:53.698 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-16T02:08:53.698 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-16T02:08:53.698 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mds.k 2024-09-16T02:08:53.921 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 16 02:08:53 smithi137 systemd[1]: Stopping Ceph mds.k for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:08:54.421 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 16 02:08:54 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-k[63222]: 2024-09-16T02:08:54.013+0000 7f3b28a2d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:08:54.421 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 16 02:08:54 smithi137 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-k[63222]: 2024-09-16T02:08:54.013+0000 7f3b28a2d640 -1 mds.k *** got signal Terminated *** 2024-09-16T02:08:57.248 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 16 02:08:56 smithi137 podman[68553]: 2024-09-16 02:08:56.932091898 +0000 UTC m=+3.013013716 container died defb1dff59d08dc90807bcab5501b595a4da78b794ae5f88a622368317c4cac6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-k, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T02:08:57.248 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 16 02:08:57 smithi137 podman[68553]: 2024-09-16 02:08:57.247763208 +0000 UTC m=+3.328685026 container cleanup defb1dff59d08dc90807bcab5501b595a4da78b794ae5f88a622368317c4cac6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-k, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T02:08:57.576 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 16 02:08:57 smithi137 bash[68553]: ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-k 2024-09-16T02:08:57.577 INFO:journalctl@ceph.mds.k.smithi137.stdout:Sep 16 02:08:57 smithi137 podman[68566]: 2024-09-16 02:08:57.457056154 +0000 UTC m=+0.513452641 container remove defb1dff59d08dc90807bcab5501b595a4da78b794ae5f88a622368317c4cac6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-k, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T02:08:57.597 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mds.k.service' 2024-09-16T02:08:57.629 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:08:57.629 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-16T02:08:57.629 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-16T02:08:57.629 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mds.c 2024-09-16T02:08:57.913 INFO:journalctl@ceph.mds.c.smithi190.stdout:Sep 16 02:08:57 smithi190 systemd[1]: Stopping Ceph mds.c for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:08:58.342 INFO:journalctl@ceph.mds.c.smithi190.stdout:Sep 16 02:08:57 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-c[64307]: 2024-09-16T02:08:57.921+0000 7f9ab9444640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:08:58.342 INFO:journalctl@ceph.mds.c.smithi190.stdout:Sep 16 02:08:57 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-c[64307]: 2024-09-16T02:08:57.921+0000 7f9ab9444640 -1 mds.c *** got signal Terminated *** 2024-09-16T02:09:01.231 INFO:journalctl@ceph.mds.c.smithi190.stdout:Sep 16 02:09:00 smithi190 podman[68699]: 2024-09-16 02:09:00.933263766 +0000 UTC m=+3.112585014 container died 9877b492490bb82c4ae16ea67e2bb10da7054e81b9e4e6378320be6e8b3e5b88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T02:09:01.565 INFO:journalctl@ceph.mds.c.smithi190.stdout:Sep 16 02:09:01 smithi190 podman[68699]: 2024-09-16 02:09:01.329989653 +0000 UTC m=+3.509310894 container cleanup 9877b492490bb82c4ae16ea67e2bb10da7054e81b9e4e6378320be6e8b3e5b88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-c, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-16T02:09:01.565 INFO:journalctl@ceph.mds.c.smithi190.stdout:Sep 16 02:09:01 smithi190 bash[68699]: ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-c 2024-09-16T02:09:01.713 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mds.c.service' 2024-09-16T02:09:01.749 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:09:01.750 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-16T02:09:01.750 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-16T02:09:01.750 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mds.f 2024-09-16T02:09:02.092 INFO:journalctl@ceph.mds.f.smithi190.stdout:Sep 16 02:09:01 smithi190 systemd[1]: Stopping Ceph mds.f for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:09:02.092 INFO:journalctl@ceph.mds.f.smithi190.stdout:Sep 16 02:09:02 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-f[64004]: 2024-09-16T02:09:02.075+0000 7fd5e4dc0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:09:02.093 INFO:journalctl@ceph.mds.f.smithi190.stdout:Sep 16 02:09:02 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-f[64004]: 2024-09-16T02:09:02.075+0000 7fd5e4dc0640 -1 mds.f *** got signal Terminated *** 2024-09-16T02:09:05.219 INFO:journalctl@ceph.mds.f.smithi190.stdout:Sep 16 02:09:04 smithi190 podman[68832]: 2024-09-16 02:09:04.931552353 +0000 UTC m=+2.957945873 container died f2747f302c47ada4495cbf1caafde855f9202762f5c208a9fa65f72d179c07cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-f, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2) 2024-09-16T02:09:05.585 INFO:journalctl@ceph.mds.f.smithi190.stdout:Sep 16 02:09:05 smithi190 podman[68832]: 2024-09-16 02:09:05.316834552 +0000 UTC m=+3.343228065 container cleanup f2747f302c47ada4495cbf1caafde855f9202762f5c208a9fa65f72d179c07cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-f, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux ) 2024-09-16T02:09:05.585 INFO:journalctl@ceph.mds.f.smithi190.stdout:Sep 16 02:09:05 smithi190 bash[68832]: ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-f 2024-09-16T02:09:05.752 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mds.f.service' 2024-09-16T02:09:05.784 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:09:05.785 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-16T02:09:05.785 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-16T02:09:05.785 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mds.i 2024-09-16T02:09:06.247 INFO:journalctl@ceph.mds.i.smithi190.stdout:Sep 16 02:09:05 smithi190 systemd[1]: Stopping Ceph mds.i for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:09:06.592 INFO:journalctl@ceph.mds.i.smithi190.stdout:Sep 16 02:09:06 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-i[64605]: 2024-09-16T02:09:06.245+0000 7f226f1d8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:09:06.592 INFO:journalctl@ceph.mds.i.smithi190.stdout:Sep 16 02:09:06 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-i[64605]: 2024-09-16T02:09:06.245+0000 7f226f1d8640 -1 mds.i *** got signal Terminated *** 2024-09-16T02:09:09.221 INFO:journalctl@ceph.mds.i.smithi190.stdout:Sep 16 02:09:08 smithi190 podman[68953]: 2024-09-16 02:09:08.929989785 +0000 UTC m=+2.934095144 container died 6ac8de37e3cc5191a62453fb360495e567ffe1cd3f468693f25bec08193ded58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-i, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-16T02:09:09.566 INFO:journalctl@ceph.mds.i.smithi190.stdout:Sep 16 02:09:09 smithi190 podman[68953]: 2024-09-16 02:09:09.320312598 +0000 UTC m=+3.324417965 container cleanup 6ac8de37e3cc5191a62453fb360495e567ffe1cd3f468693f25bec08193ded58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-i, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-8921c8e) 2024-09-16T02:09:09.566 INFO:journalctl@ceph.mds.i.smithi190.stdout:Sep 16 02:09:09 smithi190 bash[68953]: ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-i 2024-09-16T02:09:09.705 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mds.i.service' 2024-09-16T02:09:09.737 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:09:09.737 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-16T02:09:09.737 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-16T02:09:09.738 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mds.l 2024-09-16T02:09:10.092 INFO:journalctl@ceph.mds.l.smithi190.stdout:Sep 16 02:09:09 smithi190 systemd[1]: Stopping Ceph mds.l for 423b7a2a-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:09:10.093 INFO:journalctl@ceph.mds.l.smithi190.stdout:Sep 16 02:09:10 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-l[63700]: 2024-09-16T02:09:10.056+0000 7f25a0b4c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:09:10.093 INFO:journalctl@ceph.mds.l.smithi190.stdout:Sep 16 02:09:10 smithi190 ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-l[63700]: 2024-09-16T02:09:10.056+0000 7f25a0b4c640 -1 mds.l *** got signal Terminated *** 2024-09-16T02:09:13.217 INFO:journalctl@ceph.mds.l.smithi190.stdout:Sep 16 02:09:12 smithi190 podman[69074]: 2024-09-16 02:09:12.938149195 +0000 UTC m=+2.984047395 container died 24708c1f20096d70b332937f7b552076a8b6ba54b6b6dfa0980e901c9ac40e9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T02:09:13.550 INFO:journalctl@ceph.mds.l.smithi190.stdout:Sep 16 02:09:13 smithi190 podman[69074]: 2024-09-16 02:09:13.315887147 +0000 UTC m=+3.361785364 container cleanup 24708c1f20096d70b332937f7b552076a8b6ba54b6b6dfa0980e901c9ac40e9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-16T02:09:13.550 INFO:journalctl@ceph.mds.l.smithi190.stdout:Sep 16 02:09:13 smithi190 bash[69074]: ceph-423b7a2a-73ce-11ef-bceb-c7b262605968-mds-l 2024-09-16T02:09:13.691 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-423b7a2a-73ce-11ef-bceb-c7b262605968@mds.l.service' 2024-09-16T02:09:13.723 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:09:13.723 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-16T02:09:13.724 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T02:09:13.925 INFO:teuthology.orchestra.run.smithi105.stdout:Deleting cluster with fsid: 423b7a2a-73ce-11ef-bceb-c7b262605968 2024-09-16T02:09:16.400 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T02:09:16.603 INFO:teuthology.orchestra.run.smithi137.stdout:Deleting cluster with fsid: 423b7a2a-73ce-11ef-bceb-c7b262605968 2024-09-16T02:09:19.024 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T02:09:19.227 INFO:teuthology.orchestra.run.smithi190.stdout:Deleting cluster with fsid: 423b7a2a-73ce-11ef-bceb-c7b262605968 2024-09-16T02:09:21.669 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T02:09:21.705 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T02:09:21.740 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T02:09:21.770 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-16T02:09:22.098 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906494/remote/smithi105/crash 2024-09-16T02:09:22.108 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/crash -- . 2024-09-16T02:09:22.156 INFO:teuthology.orchestra.run.smithi105.stderr:tar: /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T02:09:22.156 INFO:teuthology.orchestra.run.smithi105.stderr:tar: Error is not recoverable: exiting now 2024-09-16T02:09:22.158 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906494/remote/smithi137/crash 2024-09-16T02:09:22.159 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/crash -- . 2024-09-16T02:09:22.219 INFO:teuthology.orchestra.run.smithi137.stderr:tar: /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T02:09:22.219 INFO:teuthology.orchestra.run.smithi137.stderr:tar: Error is not recoverable: exiting now 2024-09-16T02:09:22.222 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906494/remote/smithi190/crash 2024-09-16T02:09:22.222 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/crash -- . 2024-09-16T02:09:22.262 INFO:teuthology.orchestra.run.smithi190.stderr:tar: /var/lib/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T02:09:22.262 INFO:teuthology.orchestra.run.smithi190.stderr:tar: Error is not recoverable: exiting now 2024-09-16T02:09:22.263 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-16T02:09:22.264 DEBUG:teuthology.orchestra.run.smithi105:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-16T02:09:22.309 INFO:tasks.cephadm:Compressing logs... 2024-09-16T02:09:22.310 DEBUG:teuthology.orchestra.run.smithi105:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T02:09:22.312 DEBUG:teuthology.orchestra.run.smithi137:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T02:09:22.315 DEBUG:teuthology.orchestra.run.smithi190:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T02:09:22.336 INFO:teuthology.orchestra.run.smithi105.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-osd.0.log 2024-09-16T02:09:22.336 INFO:teuthology.orchestra.run.smithi105.stderr:: No such file or directory 2024-09-16T02:09:22.336 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-osd.9.log 2024-09-16T02:09:22.337 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-16T02:09:22.337 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-16T02:09:22.337 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mds.a.log 2024-09-16T02:09:22.338 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-16T02:09:22.338 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph.audit.log: /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mon.a.log 2024-09-16T02:09:22.338 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mds.g.log 2024-09-16T02:09:22.338 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-osd.6.log 2024-09-16T02:09:22.338 INFO:teuthology.orchestra.run.smithi105.stderr: 83.0% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-16T02:09:22.340 INFO:teuthology.orchestra.run.smithi190.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-16T02:09:22.341 INFO:teuthology.orchestra.run.smithi190.stderr:: No such file or directory 2024-09-16T02:09:22.341 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-16T02:09:22.341 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mds.i.log 2024-09-16T02:09:22.341 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-volume.log: /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mon.c.log 2024-09-16T02:09:22.341 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-16T02:09:22.341 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-osd.11.log 2024-09-16T02:09:22.341 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mon.c.log: /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-osd.2.log 2024-09-16T02:09:22.341 INFO:teuthology.orchestra.run.smithi190.stderr: 83.1% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-16T02:09:22.341 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-osd.5.log 2024-09-16T02:09:22.342 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mds.g.log: 90.3% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-16T02:09:22.343 INFO:teuthology.orchestra.run.smithi137.stderr:find: ‘/var/log/rbd-target-api’gzip: No such file or directory 2024-09-16T02:09:22.343 INFO:teuthology.orchestra.run.smithi137.stderr: -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-osd.10.log 2024-09-16T02:09:22.343 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-osd.1.log 2024-09-16T02:09:22.343 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-osd.7.log 2024-09-16T02:09:22.343 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-16T02:09:22.343 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mds.b.log 2024-09-16T02:09:22.345 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mds.k.log 2024-09-16T02:09:22.345 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-osd.4.log 2024-09-16T02:09:22.346 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-16T02:09:22.346 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph.log 2024-09-16T02:09:22.347 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mds.l.log 2024-09-16T02:09:22.347 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph.log: 90.5% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-16T02:09:22.347 INFO:teuthology.orchestra.run.smithi190.stderr: 86.3% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-16T02:09:22.347 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-osd.8.log 2024-09-16T02:09:22.347 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mgr.z.log 2024-09-16T02:09:22.347 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-16T02:09:22.348 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph.audit.log: 90.5% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-16T02:09:22.348 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mgr.y.log 2024-09-16T02:09:22.349 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph.cephadm.log: 83.2% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-16T02:09:22.349 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph.log 2024-09-16T02:09:22.351 INFO:teuthology.orchestra.run.smithi105.stderr: 91.1% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-16T02:09:22.352 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph.log 2024-09-16T02:09:22.353 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mds.d.log 2024-09-16T02:09:22.353 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mgr.x.log 2024-09-16T02:09:22.355 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-osd.8.log: 90.9% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-16T02:09:22.355 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mds.c.log 2024-09-16T02:09:22.356 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mds.d.log: 86.3% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-16T02:09:22.356 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mds.h.log 2024-09-16T02:09:22.356 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph.log: 92.3% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mgr.y.log.gz 2024-09-16T02:09:22.356 INFO:teuthology.orchestra.run.smithi137.stderr: 86.3% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-16T02:09:22.358 INFO:teuthology.orchestra.run.smithi105.stderr: 94.7% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mds.d.log.gz 2024-09-16T02:09:22.358 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mds.j.log 2024-09-16T02:09:22.358 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-osd.3.log 2024-09-16T02:09:22.358 INFO:teuthology.orchestra.run.smithi137.stderr: 91.0% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-16T02:09:22.359 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mon.b.log 2024-09-16T02:09:22.360 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mgr.z.log: 92.4% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mgr.z.log.gz 2024-09-16T02:09:22.360 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mds.f.log 2024-09-16T02:09:22.361 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mds.e.log 2024-09-16T02:09:22.362 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mds.j.log: 94.7% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mds.j.log.gz 2024-09-16T02:09:22.362 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-09-16T02:09:22.372 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-16T02:09:22.374 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-16T02:09:22.377 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.0.log: 94.0% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mds.a.log.gz 2024-09-16T02:09:22.379 INFO:teuthology.orchestra.run.smithi105.stderr: 94.1% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mds.g.log.gz 2024-09-16T02:09:22.381 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-16T02:09:22.385 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mds.e.log: 94.1% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mds.k.log.gz 2024-09-16T02:09:22.386 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.3% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mds.i.log.gz 2024-09-16T02:09:22.387 INFO:teuthology.orchestra.run.smithi190.stderr: 92.2% 94.0% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mds.l.log.gz 2024-09-16T02:09:22.387 INFO:teuthology.orchestra.run.smithi190.stderr: -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-16T02:09:22.388 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-16T02:09:22.398 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/cephadm.log: 93.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-16T02:09:22.399 INFO:teuthology.orchestra.run.smithi190.stderr: 92.3% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mds.c.log.gz 2024-09-16T02:09:22.407 INFO:teuthology.orchestra.run.smithi137.stderr: 92.2% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mds.h.log.gz 2024-09-16T02:09:22.417 INFO:teuthology.orchestra.run.smithi190.stderr: 92.4% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mds.f.log.gz 2024-09-16T02:09:22.454 INFO:teuthology.orchestra.run.smithi137.stderr: 93.0% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mds.e.log.gz 2024-09-16T02:09:22.553 INFO:teuthology.orchestra.run.smithi190.stderr: 91.8% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mon.c.log.gz 2024-09-16T02:09:22.561 INFO:teuthology.orchestra.run.smithi105.stderr: 90.3% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mgr.x.log.gz 2024-09-16T02:09:22.573 INFO:teuthology.orchestra.run.smithi137.stderr: 92.6% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mds.b.log.gz 2024-09-16T02:09:22.613 INFO:teuthology.orchestra.run.smithi137.stderr: 91.8% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mon.b.log.gz 2024-09-16T02:09:23.164 INFO:teuthology.orchestra.run.smithi105.stderr: 90.6% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-mon.a.log.gz 2024-09-16T02:09:24.934 INFO:teuthology.orchestra.run.smithi190.stderr: 93.9% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-osd.8.log.gz 2024-09-16T02:09:25.030 INFO:teuthology.orchestra.run.smithi105.stderr: 93.9% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-osd.9.log.gz 2024-09-16T02:09:25.369 INFO:teuthology.orchestra.run.smithi190.stderr: 94.0% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-osd.11.log.gz 2024-09-16T02:09:25.617 INFO:teuthology.orchestra.run.smithi190.stderr: 93.9% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-osd.5.log.gz 2024-09-16T02:09:25.648 INFO:teuthology.orchestra.run.smithi137.stderr: 94.0% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-osd.4.log.gz 2024-09-16T02:09:25.760 INFO:teuthology.orchestra.run.smithi137.stderr: 93.8% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-osd.7.log.gz 2024-09-16T02:09:25.880 INFO:teuthology.orchestra.run.smithi190.stderr: 93.9% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-osd.2.log.gz 2024-09-16T02:09:25.882 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-09-16T02:09:25.882 INFO:teuthology.orchestra.run.smithi190.stderr:real 0m3.553s 2024-09-16T02:09:25.882 INFO:teuthology.orchestra.run.smithi190.stderr:user 0m12.336s 2024-09-16T02:09:25.882 INFO:teuthology.orchestra.run.smithi190.stderr:sys 0m0.496s 2024-09-16T02:09:25.998 INFO:teuthology.orchestra.run.smithi137.stderr: 93.9% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-osd.10.log.gz 2024-09-16T02:09:26.184 INFO:teuthology.orchestra.run.smithi105.stderr: 93.9% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-osd.6.log.gz 2024-09-16T02:09:26.371 INFO:teuthology.orchestra.run.smithi137.stderr: 93.9% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-osd.1.log.gz 2024-09-16T02:09:26.373 INFO:teuthology.orchestra.run.smithi137.stderr: 2024-09-16T02:09:26.373 INFO:teuthology.orchestra.run.smithi137.stderr:real 0m4.043s 2024-09-16T02:09:26.373 INFO:teuthology.orchestra.run.smithi137.stderr:user 0m14.496s 2024-09-16T02:09:26.373 INFO:teuthology.orchestra.run.smithi137.stderr:sys 0m0.561s 2024-09-16T02:09:26.667 INFO:teuthology.orchestra.run.smithi105.stderr: 93.9% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-osd.0.log.gz 2024-09-16T02:09:27.023 INFO:teuthology.orchestra.run.smithi105.stderr: 93.9% -- replaced with /var/log/ceph/423b7a2a-73ce-11ef-bceb-c7b262605968/ceph-osd.3.log.gz 2024-09-16T02:09:30.006 INFO:teuthology.orchestra.run.smithi105.stderr: 96.7% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-09-16T02:09:30.008 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-16T02:09:30.008 INFO:teuthology.orchestra.run.smithi105.stderr:real 0m7.683s 2024-09-16T02:09:30.008 INFO:teuthology.orchestra.run.smithi105.stderr:user 0m23.367s 2024-09-16T02:09:30.008 INFO:teuthology.orchestra.run.smithi105.stderr:sys 0m0.906s 2024-09-16T02:09:30.009 INFO:tasks.cephadm:Archiving logs... 2024-09-16T02:09:30.009 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906494/remote/smithi105/log 2024-09-16T02:09:30.010 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T02:09:32.939 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906494/remote/smithi137/log 2024-09-16T02:09:32.940 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T02:09:35.057 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906494/remote/smithi190/log 2024-09-16T02:09:35.058 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T02:09:36.957 INFO:tasks.cephadm:Removing cluster... 2024-09-16T02:09:36.957 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 --force 2024-09-16T02:09:37.159 INFO:teuthology.orchestra.run.smithi105.stdout:Deleting cluster with fsid: 423b7a2a-73ce-11ef-bceb-c7b262605968 2024-09-16T02:09:37.430 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 --force 2024-09-16T02:09:37.631 INFO:teuthology.orchestra.run.smithi137.stdout:Deleting cluster with fsid: 423b7a2a-73ce-11ef-bceb-c7b262605968 2024-09-16T02:09:37.959 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 423b7a2a-73ce-11ef-bceb-c7b262605968 --force 2024-09-16T02:09:38.160 INFO:teuthology.orchestra.run.smithi190.stdout:Deleting cluster with fsid: 423b7a2a-73ce-11ef-bceb-c7b262605968 2024-09-16T02:09:38.491 INFO:tasks.cephadm:Removing cephadm ... 2024-09-16T02:09:38.491 DEBUG:teuthology.orchestra.run.smithi105:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-16T02:09:38.510 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-16T02:09:38.528 DEBUG:teuthology.orchestra.run.smithi190:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-16T02:09:38.543 INFO:tasks.cephadm:Teardown complete 2024-09-16T02:09:38.544 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-16T02:09:38.559 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-16T02:09:38.559 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T02:09:38.562 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T02:09:38.570 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T02:09:38.679 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-16T02:09:38.689 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-16T02:09:38.689 DEBUG:teuthology.orchestra.run.smithi105:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T02:09:38.692 DEBUG:teuthology.orchestra.run.smithi137:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T02:09:38.695 DEBUG:teuthology.orchestra.run.smithi190:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T02:09:38.706 INFO:teuthology.orchestra.run.smithi105.stderr:bash: line 1: ntpq: command not found 2024-09-16T02:09:38.707 INFO:teuthology.orchestra.run.smithi137.stderr:bash: line 1: ntpq: command not found 2024-09-16T02:09:38.709 INFO:teuthology.orchestra.run.smithi190.stderr:bash: line 1: ntpq: command not found 2024-09-16T02:09:38.711 INFO:teuthology.orchestra.run.smithi137.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T02:09:38.711 INFO:teuthology.orchestra.run.smithi137.stdout:=============================================================================== 2024-09-16T02:09:38.711 INFO:teuthology.orchestra.run.smithi137.stdout:^* hv01.front.sepia.ceph.com 3 6 377 63 -460us[ -448us] +/- 44ms 2024-09-16T02:09:38.711 INFO:teuthology.orchestra.run.smithi137.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 62 +428us[ +428us] +/- 56ms 2024-09-16T02:09:38.711 INFO:teuthology.orchestra.run.smithi137.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 59 +164us[ +164us] +/- 69ms 2024-09-16T02:09:38.711 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T02:09:38.713 INFO:teuthology.orchestra.run.smithi105.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T02:09:38.713 INFO:teuthology.orchestra.run.smithi105.stdout:=============================================================================== 2024-09-16T02:09:38.713 INFO:teuthology.orchestra.run.smithi105.stdout:^* hv01.front.sepia.ceph.com 3 6 377 63 -444us[ -473us] +/- 44ms 2024-09-16T02:09:38.713 INFO:teuthology.orchestra.run.smithi105.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 63 +430us[ +430us] +/- 56ms 2024-09-16T02:09:38.713 INFO:teuthology.orchestra.run.smithi105.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 63 +173us[ +143us] +/- 69ms 2024-09-16T02:09:38.713 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T02:09:38.714 INFO:teuthology.orchestra.run.smithi190.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T02:09:38.714 INFO:teuthology.orchestra.run.smithi190.stdout:=============================================================================== 2024-09-16T02:09:38.714 INFO:teuthology.orchestra.run.smithi190.stdout:^* hv01.front.sepia.ceph.com 3 6 377 63 -441us[ -446us] +/- 44ms 2024-09-16T02:09:38.714 INFO:teuthology.orchestra.run.smithi190.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 64 +435us[ +430us] +/- 56ms 2024-09-16T02:09:38.714 INFO:teuthology.orchestra.run.smithi190.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 60 +173us[ +173us] +/- 69ms 2024-09-16T02:09:38.714 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T02:09:38.715 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-16T02:09:38.726 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-16T02:09:38.727 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-16T02:09:38.742 DEBUG:teuthology.orchestra.run.smithi105:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T02:09:38.836 DEBUG:teuthology.orchestra.run.smithi137:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T02:09:38.903 DEBUG:teuthology.orchestra.run.smithi190:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T02:09:38.970 DEBUG:teuthology.orchestra.run.smithi105:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T02:09:39.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T02:09:39.001 DEBUG:teuthology.orchestra.run.smithi137:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T02:09:39.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T02:09:39.028 DEBUG:teuthology.orchestra.run.smithi190:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T02:09:39.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T02:09:39.056 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-16T02:09:39.066 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-16T02:09:39.079 INFO:teuthology.task.internal:Duration was 1412.426224 seconds 2024-09-16T02:09:39.080 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-16T02:09:39.094 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-16T02:09:39.094 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T02:09:39.097 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T02:09:39.099 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T02:09:39.136 INFO:teuthology.orchestra.run.smithi137.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T02:09:39.136 INFO:teuthology.orchestra.run.smithi105.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T02:09:39.138 INFO:teuthology.orchestra.run.smithi190.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T02:09:39.454 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-16T02:09:39.454 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi105.front.sepia.ceph.com 2024-09-16T02:09:39.455 DEBUG:teuthology.orchestra.run.smithi105:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T02:09:39.476 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi137.front.sepia.ceph.com 2024-09-16T02:09:39.476 DEBUG:teuthology.orchestra.run.smithi137:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T02:09:39.517 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi190.front.sepia.ceph.com 2024-09-16T02:09:39.517 DEBUG:teuthology.orchestra.run.smithi190:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T02:09:39.543 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-16T02:09:39.543 DEBUG:teuthology.orchestra.run.smithi105:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T02:09:39.546 DEBUG:teuthology.orchestra.run.smithi137:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T02:09:39.559 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T02:09:39.609 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-16T02:09:39.609 DEBUG:teuthology.orchestra.run.smithi105:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T02:09:39.612 DEBUG:teuthology.orchestra.run.smithi137:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T02:09:39.625 DEBUG:teuthology.orchestra.run.smithi190:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T02:09:40.314 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-16T02:09:40.325 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-16T02:09:40.325 DEBUG:teuthology.orchestra.run.smithi105:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T02:09:40.350 DEBUG:teuthology.orchestra.run.smithi137:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T02:09:40.376 DEBUG:teuthology.orchestra.run.smithi190:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T02:09:40.402 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-16T02:09:40.414 DEBUG:teuthology.orchestra.run.smithi105:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T02:09:40.417 DEBUG:teuthology.orchestra.run.smithi137:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T02:09:40.419 DEBUG:teuthology.orchestra.run.smithi190:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T02:09:40.437 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern = core 2024-09-16T02:09:40.441 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern = core 2024-09-16T02:09:40.466 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = core 2024-09-16T02:09:40.506 DEBUG:teuthology.orchestra.run.smithi105:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T02:09:40.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T02:09:40.520 DEBUG:teuthology.orchestra.run.smithi137:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T02:09:40.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T02:09:40.535 DEBUG:teuthology.orchestra.run.smithi190:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T02:09:40.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T02:09:40.560 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-16T02:09:40.574 INFO:teuthology.task.internal:Transferring archived files... 2024-09-16T02:09:40.575 DEBUG:teuthology.misc:Transferring archived files from smithi105:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906494/remote/smithi105 2024-09-16T02:09:40.575 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T02:09:40.625 DEBUG:teuthology.misc:Transferring archived files from smithi137:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906494/remote/smithi137 2024-09-16T02:09:40.625 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T02:09:40.660 DEBUG:teuthology.misc:Transferring archived files from smithi190:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906494/remote/smithi190 2024-09-16T02:09:40.660 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T02:09:40.701 INFO:teuthology.task.internal:Removing archive directory... 2024-09-16T02:09:40.701 DEBUG:teuthology.orchestra.run.smithi105:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T02:09:40.704 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T02:09:40.706 DEBUG:teuthology.orchestra.run.smithi190:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T02:09:40.750 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-16T02:09:40.763 INFO:teuthology.task.internal:Not uploading archives. 2024-09-16T02:09:40.763 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-16T02:09:40.774 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-16T02:09:40.774 DEBUG:teuthology.orchestra.run.smithi105:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T02:09:40.777 DEBUG:teuthology.orchestra.run.smithi137:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T02:09:40.780 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T02:09:40.789 INFO:teuthology.orchestra.run.smithi105.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 02:09 /home/ubuntu/cephtest 2024-09-16T02:09:40.792 INFO:teuthology.orchestra.run.smithi137.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 02:09 /home/ubuntu/cephtest 2024-09-16T02:09:40.805 INFO:teuthology.orchestra.run.smithi190.stdout: 265907 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 02:09 /home/ubuntu/cephtest 2024-09-16T02:09:40.806 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-16T02:09:40.817 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-16T02:09:40.844 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/suites/iozone}} duration: 1412.4262237548828 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-16T02:09:40.845 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T02:09:40.999 INFO:teuthology.run:pass